Platform: GCP ✅

Just sharing my feedback so far on GCP. The first few times I used it everything ran smoothly. This morning, I cannot start the VM because “The zone {…us-west1-b} does not have enough resources available to fulfill the request. Try a different zone, or try again later.”

My availability’s been down for 30 min and I’ve seen 3 hours downtime for preemptibles on online forums.

I’ve run into the same thing on other providers. So in this case I just wanted to share that I’m glad to have a second instance set up that I can switch to when resources aren’t available. Currently have 1 GCP and 1 Paperspace. You could also have 2 GCPs in different regions.

5 Likes

I’m just now trying to start an instance, it will let me do it with a p4 but not with a p100.

The resource 'projects/xxxxx/zones/us-west2-b/acceleratorTypes/nvidia-tesla-p100' was not found

So that means if all fastai students use the preemptible instances, they are overbooked :wink:

I have now checked 5 different zones, with or without --preemptible, p100 and v100.
I can create p4 instances, but not the better ones
Is it possible that there are restrictions on those, like you have to apply for limit increases in aws for p2.xl etc. first?! If yes, what do I have to do to get access to these GPUs? The docs just say there is a limit of 1 gpu (quotas page), but no restriciton on gpu type?!

I’ve got a p100 VM in europe-west1-d that I’m currently using, so I don’t think there are restrictions. I’ve created it on Tuesday morning right after the live course. Hope they will solve the congestion problem soon.

Try doing it thru the GUI - it’ll make sure you don’t pick GPU types that aren’t available in that zone. Or check the GCP docs that also have availability tables.

1 Like

Mark, you might want to check a few things in that case.

First thing is that not all GPUs are available in all region/zones. Make sure you’re requesting GPUs on a region that has it available. (You can check the listing here https://cloud.google.com/compute/docs/gpus/ )

Another thing to think about is that, it’s usually the pre-emptible instances itself that are not available, not necessarily the GPU, because the GPU is a ‘guest accelerator’ attachment to the instance.
So, just switch the instance type to something else like n1-standard-4/n1-standard-8/n1-standard-16 or even the highmem / highcpu varieties. ( Listing here https://cloud.google.com/compute/docs/machine-types)

In short

  • make sure your region/zone provides the GPU you’re requesting
  • try switching the instance type if not available
  • try switching the zone/region itself
  • try switching the gpu type (since they’re all using tesla 396 drivers, should work fine with the same standard fastai image)

I’m pretty sure you’ll find a pre-emptible instance available with the combinations above. Good luck !

5 Likes

Like jeremy also mentioned above, you’re requesting P100 from us-west2-b, which is not available at all. As you can see below, P100 is only available in zone 1 in us-west. Hope that clarifies.

2 Likes

I’m trying to connect to my instance using

gcloud compute ssh --zone=$ZONE jupyter@$INSTANCE_NAME – -L 8080:localhost:8080

But I keep getting the error: could not parse resource []

I even created a new instance but still get the same error.

Aahh, sorry for being so stupid. I didn’ realize they were available only selectively. But it was also bad luck, because as I said above, I tried in 6! different zones incl. Europe but managed to by chance only take those that didn’t have p100s :see_no_evil:
Now at least I get a different error message (not available!) :joy:

The zone 'projects/xxxxx/zones/us-west1-a' does not have enough resources available to fulfill the request.  Try a different zone, or try again later.

Thanks for your help!!

1 Like

No worries ! The array of combinations can be confusing in the beginning. Hope you get a pre-emptible instance with a gpu soon :beers:

1 Like

It looks like the command you’re running is not correct starting at -- -L ............. You probably want to pass ssh arguments using --ssh-flag="......." ! Docs here https://cloud.google.com/sdk/gcloud/reference/compute/ssh

2 Likes

Hi guys! Sorry if this has been asked already yet I searched the forum for the subject with no success. Trying to create GCP account I found out there is no way of setting up individual (non business) account for European location (Belarus if being precise). Official support states that in Russia the service shouldn’t be used at all if no potential economic benefit is expected from the usage. This is all a bit confusing since I just don’t have individual account option even though I’m not in Russia, but neither do I intend to use GCP for business purposes. Can anyone suggest? Thanks!

@marcmuc If you try connecting to us-west2-b but you get resource not available error you could try another zone. I found this link useful to choose an optimal zone depending the GPU I want to use:
https://cloud.google.com/compute/docs/gpus/
https://cloud.google.com/compute/docs/regions-zones/

3 Likes

I changed the command to this:

gcloud compute ssh --zone us-west1-b jupyter@$fastai --ssh-flag="-L 8080:localhost:8080"

And I’m still getting

ERROR: (gcloud.compute.ssh) could not parse resource []

1 Like

Hello! Anybody interested in free $500 Google Cloud credits? As Google employee, I can refer other people for free $500 (this is in addition to $300 for a new account, so $800 in total). I have checked that it is ok to refer fast.ai students, even a lot of them, and already tested the process on three colleagues from Czech&Slovak study group. To make it easier for me, please, do not send me messages but fill in your information into this form:

http://bit.ly/gcpextracredit

Why am I doing this? I remember myself starting with the cloud (AWS) in late 2010, in Eastern Europe, on a tight budget, detecting polymorphisms in mouse DNA. I know very well what is it to forget switching off your machines or screwing something and getting an extra $100 billed to your supervisor’s credit card. I was fortunate not to be fired and want to help people in a similar situation.

UPDATE: I am going to delete the form on Tuesday, November 6, in the evening. (I have already sent over 200 referrals and I will refer everybody who applies before that)

64 Likes

Wonderful! :slight_smile: I filled in your form. Thanks for taking the time to help the community in this way.

8 Likes

Thank you for offering this, Petr. This is such a great help.

1 Like

That will be super helpful for a lot of folks. Appreciate your time and effort!

1 Like

Hello Mac,

I was going through the same problem, the idea here is to atleast try 3 - 4 times in the same region. I try in europe-west1-b and have got it all the time after trying 3 - 4 times.

After the new message from Jeremy, I did conda update conda but conda install -c fastai fastai fails and gives out this error: