Thought this might be useful to some of you here who like me only have a small local GPU and rely on cloud GPUs for larger training runs!
I created a page that compares Cloud GPU Services. You can look for specific GPU models, pricing (static at the moment but hoping to make this dynamic soon), and features like notebooks/SSH/data persistence. It also lists the amount available in free credits for each service, and some of the links have custom discount codes to get additional free credits.
Thanks! That’s a really good point. We’re making a lot of simplifications on the page - e.g. in reality pricing is different per region, pricing isn’t actually static over time, and we’re not accounting for the value of other characteristics of the VM it’s attached to. (e.g. pricing per vCPU, SSD vs HDD, RAM…).
But you’re right - maybe the fact that we’re only looking at on-demand VMs and excluding preemptible ones makes GCP and AWS look more expensive than they would be in practice. We picked the on-demand ones because that’s most similar to what competitors offer (apart from maybe vast.ai).
Right now we’re trying to see how much interest there is in a comparison service like this, and if it’s sufficient then we’d spend time developing it further.
To address your point we could add a filter at the top to say “include preemptible instances” - do you think that would work well?
I wonder how it would fit in this list because it allows you to use your compute from GCP, AWS, Azure, DigitalOcean directly to run notebooks on iko.ai.
For example, someone might apply for free credits on GCP, create a Kubernetes cluster (GKE), then get the config and use it to run the fast.ai notebooks. Given the resources are expensive, we automatically shut down idle notebook servers (that is: when there neither is user activity, nor computation or a busy cell).
The long running-notebooks can also run in the background in an ephemeral fashion: we use the compute just for the duration of the notebook job and that represents quite large savings as well.
I second the need for spot pricing to be included - for anyone who is at all price-sensitive, that’s the only relevant metric. Your proposed service basically only appeals to people who are price-sensitive, so you’re not doing anyone a service by ignoring that.
This site is super useful for finding the best spot-price of any AWS instance globally: Cheapest Amazon EC2 Spot Price Region - this shows p3.2xlarge (so a single V100) at $0.92/hr currently, which takes AWS from the most expensive to the second-cheapest in your list (well, third cheapest if the google spot price from the earlier post is added). That makes your list largely irrelevant without taking spot pricing into account.
Fair point, thanks! I’ll have a think about this, but based on what both of you have said it probably makes sense to showing spot/preemptible by default, and adding a caveat about that for anyone who’s looking for on-demand specifically.
Putting in the spot price as the default is fantastic, thanks. It’s also really interesting to see how much the price varies between the different companies.
I’d like to add another platform to this awesome list! It’s Q Blocks - decentralized GPU computing platform.
They have community and data center nodes with on-demand global access. 3090s are available on it around $0.75/hr and A100s around $2.5/hr. I’ve seen the prices to be stable from a long period on it.
I know it’s an old post but this is a useful comparison thanks!
Another provider that might be worth adding is OVHcloud. They offer dedicated NVIDIA GPUs with flat pricing. Might be a good option for people who want cost stability without worrying about instances getting interrupted.
