Cloud GPUs: Comparison and Free Credits

Thought this might be useful to some of you here who like me only have a small local GPU and rely on cloud GPUs for larger training runs!

I created a page that compares Cloud GPU Services. You can look for specific GPU models, pricing (static at the moment but hoping to make this dynamic soon), and features like notebooks/SSH/data persistence. It also lists the amount available in free credits for each service, and some of the links have custom discount codes to get additional free credits.

Hope it’s useful, let me know what you think!

4 Likes

Very useful comparison. The Google Cloud spot price for V100 is $0.74. I’m curious as to why you use the non-preemptible one.

2 Likes

Thanks! That’s a really good point. We’re making a lot of simplifications on the page - e.g. in reality pricing is different per region, pricing isn’t actually static over time, and we’re not accounting for the value of other characteristics of the VM it’s attached to. (e.g. pricing per vCPU, SSD vs HDD, RAM…).

But you’re right - maybe the fact that we’re only looking at on-demand VMs and excluding preemptible ones makes GCP and AWS look more expensive than they would be in practice. We picked the on-demand ones because that’s most similar to what competitors offer (apart from maybe vast.ai).

Right now we’re trying to see how much interest there is in a comparison service like this, and if it’s sufficient then we’d spend time developing it further.

To address your point we could add a filter at the top to say “include preemptible instances” - do you think that would work well?

1 Like

Interesting. I opened an issue recently, https://github.com/fastai/course20/issues/66, to add our service.

I wonder how it would fit in this list because it allows you to use your compute from GCP, AWS, Azure, DigitalOcean directly to run notebooks on iko.ai.

For example, someone might apply for free credits on GCP, create a Kubernetes cluster (GKE), then get the config and use it to run the fast.ai notebooks. Given the resources are expensive, we automatically shut down idle notebook servers (that is: when there neither is user activity, nor computation or a busy cell).

The long running-notebooks can also run in the background in an ephemeral fashion: we use the compute just for the duration of the notebook job and that represents quite large savings as well.

I’d appreciate feedback on the issue as well.

Are you using this for the Google pricing: GPU pricing  |  Compute Engine: Virtual Machines (VMs)  |  Google Cloud

I second the need for spot pricing to be included - for anyone who is at all price-sensitive, that’s the only relevant metric. Your proposed service basically only appeals to people who are price-sensitive, so you’re not doing anyone a service by ignoring that.

This site is super useful for finding the best spot-price of any AWS instance globally: Cheapest Amazon EC2 Spot Price Region - this shows p3.2xlarge (so a single V100) at $0.92/hr currently, which takes AWS from the most expensive to the second-cheapest in your list (well, third cheapest if the google spot price from the earlier post is added). That makes your list largely irrelevant without taking spot pricing into account.

1 Like

Fair point, thanks! I’ll have a think about this, but based on what both of you have said it probably makes sense to showing spot/preemptible by default, and adding a caveat about that for anyone who’s looking for on-demand specifically.

@hushitz @jules43 I’ve updated it to use spot/preemptible prices for AWS and GCP GPUs, thanks again for the feedback.

I’ve also moved this page to its own domain now, to better represent that it’s focused on cloud GPU services: cloud-gpus.com.

Any further feedback please keep it coming :slight_smile:

Oh and it’s all open source - code here in case you’re interested: https://github.com/cloud-gpus/cloud-gpus.github.io

@jhadjar looks interesting. Reminds me a bit of this open source library, Spotty, which allows you to make use of spot instances effectively: https://github.com/spotty-cloud/spotty

1 Like

Putting in the spot price as the default is fantastic, thanks. It’s also really interesting to see how much the price varies between the different companies.