@floristco - I tried without the premptible and still get the same error. Any other ideas? Thanks!
I was trying to boot my gcloud preemptible VM, but before the VM properly boots up and I start my Jupyter Notebook, the VM gets auto shut down due to the preemptibe option. Did anyone else feel the same? My VM is in the US West region.
I am seeing an issue with preemptible instances as well. As soon as I add the --preemptible flag, the instance does not start
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Instance failed to start due to preemption.
If I remove the flag, the instance starts ok. Its more expensive though. Why does the preemptible instance not start up? Any help is appreciated. Thanks
Preemptible instance means it will use whatever resources are available in that area but priority is placed on non-preemptible resources. It’s effectively what other extra resources that are available are given to you. But sometimes there are no extra idle resources:
A preemptible VM is an instance that you can create and run at a much lower price than normal instances. However, Compute Engine might terminate (preempt) these instances if it requires access to those resources for other tasks. Preemptible instances are excess Compute Engine capacity, so their availability varies with usage.
(From GCP help page)
1 Like
I have same problem.
I think it is because google doesn’t allow us to do so with free tier. See below:
Preemptible instance limitations
Preemptible instances function like normal instances but have the following limitations:
- The Google Cloud Free Tier credits for Compute Engine do not apply to preemptible instances.
From Google Website: https://cloud.google.com/compute/docs/instances/preemptible
Preemptible instance limitations
Preemptible instances function like normal instances but have the following limitations:
- The Google Cloud Free Tier credits for Compute Engine do not apply to preemptible instances.
Thank you @Harvey. I was trying to use my free credits. It seems this condition was added recently bcz for Part 1 2019, I was able to run a preemptible instance with the free credits.
Thanks again.
I’ve been having problems creating a vm instance using the recommended machine type and GPU as described here in the tutorial:
However I get the following error:
n2-highmem-8, nvidia-tesla-p100] features are not compatible for creating instance.
I cannot find any documentation on GCP about what machine types and GPUs are compatible. Has any one else encountered this problem?
1 Like
Hi ,
Creating using the blow command doesn’t work, because :
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Invalid value for field ‘resource.machineType’: ‘https://compute.googleapis.com/compute/v1/projects/testnafiseh/zones/us-west1-b/machineTypes/n2d-highmem-8’. Machine type with name ‘n2d-highmem-8’ does not exist in zone ‘us-west1-b’.
.
Any idea if there is a change since the document is created and what should I change the Zone to now?
export IMAGE_FAMILY="pytorch-latest-gpu" # or "pytorch-latest-cpu" for non-GPU instances
export ZONE="us-west1-b"
export INSTANCE_NAME="my-fastai-instance"
export INSTANCE_TYPE="n2d-highmem-8" # budget: "n2d-highmem-4"
# budget: 'type=nvidia-tesla-T4,count=1'
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator="type=nvidia-tesla-p100,count=1" \
--machine-type=$INSTANCE_TYPE \
--boot-disk-size=200GB \
--metadata="install-nvidia-driver=True" \
--preemptible
1 Like
Hey Nick - did you find a solution for this? I’m running into the same problem now…
@steef yes, a bit of a workaround, really, just used the parameters from @futbol10 above. Worked for me for training the model in the first lesson.
export IMAGE_FAMILY="pytorch-latest-gpu"
export ZONE="us-west1-b"
export INSTANCE_NAME="fastai-instance"
export INSTANCE_TYPE="n1-highmem-4"
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator="type=nvidia-tesla-k80,count=1" \
--machine-type=$INSTANCE_TYPE \
--boot-disk-size=200GB \
--metadata="install-nvidia-driver=True" \
--preemptible
export ZONE="us-west1-b"
export INSTANCE_NAME="fastai-instance"
gcloud compute ssh --zone=$ZONE jupyter@$INSTANCE_NAME -- -L 8080:localhost:8080
2 Likes
@salmaniai @steef if you can’t set it up from the CLI you can also set up vm instances with the through the google cloud console, see below for more info.
https://cloud.google.com/ai-platform/notebooks/docs/create-new
Same here – I also found a workaround.
After reading some GCP docs I realized that N2D machines are in beta and they are no longer supported in the west zone + they no longer support the p100 GPU.
I got the following setup to work which has a little more memory than the recommended setup but has the same GPU as recommended.
@jeremy FYI that N2D machines no longer support the west zone nor the p100 GPU. You might want to update your documentation. @rachel FYI too.
export IMAGE_FAMILY="pytorch-latest-gpu"
export ZONE="us-west1-b"
export INSTANCE_NAME="my-fastai-instance"
export INSTANCE_TYPE="n1-highmem-16" # It seems like the N2D machines are in beta and are no longer available in all zones + not working with p100 anymore.
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator="type=nvidia-tesla-p100,count=1" \
--machine-type=$INSTANCE_TYPE \
--boot-disk-size=200GB \
--metadata="install-nvidia-driver=True" \
#--preemptible # Don’t use preemptible as it gave me issues before
@salmaniai I ran into the same existence issue that you described and that issue is also resolved with this solution.
4 Likes
Hi everyone!
i was using paperspace earlier, had some issues with space management in paperspace, decide to switch to GCP.
Got this error today, need help!
Thanks
I think you made a mistake copying INSTANCE_TYPE
try
export INSTANCE_TYPE=“n2d-highmem-8”
instead. It should work fine. Also i suggest not using us-west1-b as your zone. It is quite a busy server, and your instance frequently gets preempted . I suggest ‘europe-west1-b’ or something else.
All the best
Hello 
I need some help related to setup. I tried to increase my quota of GPUs to 1. I have followed all the steps as mentioned in the server setup for google cloud & made a request to increase quota. I got a confirmation email saying that the request to increase quota was successfully received. But within few seconds, I get another email quoting:
Unfortunately, we are unable to grant you additional quota at this time. If
this is a new project please wait 48h until you resubmit the request or
until your Billing account has additional history.Your Sales Rep is a good Escalation Path for these requests, and we highly
recommend you to reach out to them.
My project is new and I have waited for weeks, yet the quota didn’t change. I have tried a lot of times, yet I get the same email within few seconds after the request confirmation email every single time. Can someone help me resolve the issue?
Note: I have upgraded my account, my project is linked to a billing account.
Any help is appreciated
Thanks!
Hi Palaash, Are you sure n2d-highmem-8 is compatible with a GPU. I believe things have changed in GCP.
Have a look at the screenshot Caution note from GCP documentation.
I am at the exact problem. Any other help is appreciated.
Dear All, I think there are n2d-highmem-8 in some zone, and there are p100 GPU in other zones. I think there were machine n2d-highmem-8 with P100 GPU in US-West-??somewhere before. Thus, I would like to see what other budget options of machine/GPU combinations available. Anyone can share their success story and info about the cost. BTW, I am using Colab for fastai2 as a free option.
Thanks, @duerrseb for his post: FastAI2 notebooks in Kaggle which were updated March 22.
https://github.com/seduerr91/fastAI_v4/blob/master/fastai2%20on%20colab.md