Cloud GPU services vs having your own, which is better for AI?

Howdy howdy y’all,

I’ve recently been thinking of building a desktop workstation, and have been attempting to find information on the cost and performance of buying your own GPU, vs renting one from a service like Lambda Labs.

The cheapest option that Lambda Labs provides is their RTX 6000 (I realize it’s an older card, but for the sake of argument), at $0.50 / hr, if you were to buy an RTX 6000, the average price I’ve seen is around $4000.

It seems like renting a the GPU from Lambda Labs is cheaper for the first 8000 hours of renting (nearly a year).

That means buying your own RTX 6000 is perhaps not the best choice unless you’re running it continuously for over a year.

However, (and this is where my confusion comes in) I’ve read some articles saying that cloud GPU performance is bottlenecked, and falls behind compared to the theoretical performance of the GPU.

In which case it’d be better to compare the Cloud RTX 6000 with a personal 3090 that’s 1/4 the cost.

Is it true that cloud GPUs are significantly bottlenecked enough to that purchasing a cheaper and theoretically less powerful GPU is better in the long run?

Thanks!
Dylan

Hi Dylan

Off topic from your question but there is also the cost of electricity, the heat it generates, the possibility of damage or theft. Also the potential loss if you resell. There are also contract rules when using GPU for fun and academic work.

I have noticed Colab Pro is much faster than Microsoft Azure for the same notebook when I tried it the other day (a sample of one is not really a sample). Also there is the TPU if you are using TensorFlow rather than PyTorch.

So it depends on how frequently and for how long because some cloud providers do have limits.

Regards Conwyn