I actually have quiet a good experience with vast.ai. But it took me some time to appreciate it. I have been using for a bit longer than a year now. And in mean time, I spend about 3150 usd for my AI projects. (wow, is it that much).
I needed vast.ai, because I have a website that makes real time predictions for the data of users. These predictions use AI, so I was in need of strong GPUs to create these predictions. The application was written in C++, which does not really matter now, but the front-end and back-end was written in javascript (angular and nodejs). So, I didn’t just need vast.ai, I needed an automated way to create instances as well !
Fortunately vast.ai has a pretty good cli tool, which uses webservices (the project is also on github) So, my back-end basically had to communicate with vast.ai services to create instances. (multiple instances, and depending on the number of active visitors). So, I wrote a kind of “load balancer” in node.js which spawns new vast.ai instances when necessary.
It seemed easy, but it was difficult to make it reliable. It took me about 4 months to get it stable. And on many occasions I was actually totally frustrated, sometimes angry, sometimes almost crying. It was a nightmare at first.
The problem is that there are many different suppliers, and many different computer configurations, and probably many different geographical locations. And while you can filter on some specifications, you cannot always predict whether a configuration will actually work, be reliable, be compatible, …
I tried many solutions, but in the end, it was impossible for me (a human) to tell which servers were good ones and which were bad ones. Finally I found a solution: I created software to do this for me.
So, as mentioned, I created a kind of “rating” for vast instances, and I created a microservice which keeps track of this in a database. So, each time the node.js script would spawn a new instances, it would also store statistics about it, and it would store statistics to keep track of the reliability. Once all of that started to work, my vast experience totally changed. --> now when new instances are spawned, it verifies the offers using that rating service, and I have about >80% assurance that a service will actually boot and work as expected. Without this service, it was <50%. Services can run for days sometimes, but sometimes they just run for hours.
The next thing I noticed, was that sometimes instances kept hanging. Their installation would fail and the load balancer had issues to detect this and clean it. Sometimes I would be charged for instances that weren’t even running. (a failed installation) And that’s also when I started to doubt the invoices. Sometimes prices didn’t match with the statistics of my services. In my opinion, this is the weakest point of vast.ai. - There is absolutely no historic data about launch-time/active-time/… and that makes it hard to trust the invoices. I wrote about this to them, but in the last year nothing changed at all. You’re really on your own here.
Which brings us to the next point. Are these invoices even invoices ? - There is no address on them, the date and number of the invoice changes each time you download it. There’s no VAT on them. There’s also no company name, phone number, VAT number, business number, … By the way, there is not even an address on the website. The terms of service and privacy declaration contain no address neither. - I don’t care if vast.ai is trying to escape from taxes or whatever. But I after a year of using vast.ai, I am now starting to realize, that I may have unwillingly committed a federal crime by using this service. Just HOW am I supposed to explain this to my tax officer ? I fear that there could be legal consequences for using vast.ai as a business.
Of course the prices of vast.ai are super low. And that is basically the strongest point of vast.ai. (but again, only if you find a way - like I have - to make it reliable)
- But I can imagine that this can be important for other applications.