(NB: this is incomplete. It’s a wiki post - please contribute!)
Basic steps
Step 1: Creating your account
When running gcloud init, you may get the following warning: WARNING: Listing available projects failed: There was a problem refreshing your current auth tokens: invalid_grant: Bad Request
in this case:
interrupt the execution of gcloud init by clicking Ctrl +C
run gcloud auth login: a browser session will open
run again gcloud init: now you won’t get the warning anymore
As explained here, the first command of Step 3 must be modified as follows:
export IMAGE_FAMILY="pytorch-latest-gpu"
export ZONE="us-west1-b"
export INSTANCE_NAME="my-fastai-instance"
export INSTANCE_TYPE="n1-highmem-16" # It seems like the N2D machines are in beta and are no longer available in all zones + not working with p100 anymore.
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator="type=nvidia-tesla-p100,count=1" \
--machine-type=$INSTANCE_TYPE \
--boot-disk-size=200GB \
--metadata="install-nvidia-driver=True"
Follow the rest of Step 3 as is, but note that since the instance is no more preemptible, it doesn’t stop automatically after 24 hours. Thus, you must stop it every time you’re not using it (e.g.,if you’re not training a network). See Step 5 to learn how to stop an instance.
As explained here, Step 4 must be substituted by the following commands:
git clone https://github.com/fastai/course-v4
git clone https://github.com/fastai/fastbook.git
Note: A git pull within the course-v4 folder or the ‘fastbook’ folder keeps them up to date.
pip install fastai2 in the command line and its dependencies with:
pip install -r requirements.txt after navigating to the course-v4 folder.
Please note that the initial 300$ credit can NOT be applied towards a GPU…
Also they refused to upgrade my account with a GPU because I did not have enough of a billing history.
I ended up setting up an AWS instance in 15 minutes.
We have received your quota request for fastaiserver. Unfortunately, we are unable to grant your quota increase due to insufficient service usage history.
Our suggestion would be for you to make usage of your current quota and if further resources are needed then please go ahead and contact[1] our Sales Team in order to discuss further options on higher quota eligibility.
Regarding your concerns, for now since this is a new project you are not eligible for increase yet so you have to wait for 48 hours until your Billing account has additional history. You cannot use the free 300$ credit for GPUs, it is only limited for some resources. You may visit: https://cloud.google.com/free/docs/gcp-free-tier#limitations to get more information on where you can use the $300 free credit.
Ah OK @sylvaint I think I see the issue. You need to upgrade your account - which means you need to provide a credit card. At that point, you still have the $300 credits, and should be able to use them for GPUs (if you’re able to get a quota request approved).
Account is upgraded with a valid Credit card. Maybe this issue is specific to my account.
However, they specifically stated that the 300$ credit could not be used towards a GPU.
so indeed free tier needs to be upgraded to paid account in order to use GPU!
You must upgrade to a paid account to use Google Cloud after the free trial ends. To take advantage of the features of a paid account (using GPUs, for example), you can upgrade before the trial ends. When you upgrade, the following conditions apply:
Any remaining, unexpired free trial credit remains in your account.
Your credit card on file is charged for resources you use in excess of what’s covered by any remaining credit.
My experience on this matter is that GPU’s are allowed for free tier(credit card added). However, when you’re building a preemptible instance, your VM just does not start or it starts and by the time you SSH into it, it just stops. This was not my experience last year. I’ve not tried non-premptible instance yet. I’ll update once I try it.
It seems preemptible instances are not allowed on the free tier. Thanks to @Harvey for the info. Leaving a link from another thread.
Make sure you STOP your instance especially now that it seems only non preemptible instances work. Your are charged as long as the instance is running.
I find using the GOOGLE CLI is easier and safer for me than the GCP web console.
You can setup aliases in bash.
gcloud compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
fastai us-west1-b n1-highmem-8 1.1.1.1 TERMINATED
I’m having trouble getting off the ground with GCP on MacOS High Sierra, despite having successfully used it on several machines previously. When I attempt to run gcloud init, I get the following error: zsh: command not found: gcloud. Upon Googling, I found this page and I’ve tried adding the following to my .zshrc file:
# The next line updates PATH for the Google Cloud SDK.
source /Users/dwchiang/google-cloud-sdk/path.zsh.inc
# The next line enables zsh completion for gcloud.
source /Users/dwchiang/google-cloud-sdk/completion.zsh.inc
And the following to my path.zsh.inc file:
script_link="$( readlink "$0" )" || script_link="$0"
apparent_sdk_dir="${script_link%/*}"
if [ "$apparent_sdk_dir" == "$script_link" ]; then
apparent_sdk_dir=.
fi
sdk_dir="$( cd -P "$apparent_sdk_dir" && pwd -P )"
bin_path="$sdk_dir/bin"
export PATH=$bin_path:$PATH
That didn’t work, so I tried another suggestion on the same page, which was to add gcloud to the list of plugins on my ~/.zshrc file, but this also hasn’t worked, and I’m still getting the same error.
Thanks! That looks like the same code in the tutorial Jeremy linked, which I used originally. I also tried copying and pasting from your post, and still have the same error.
I need some help in trying to SCP some of the notebooks I have created while following along the fastai v2 docs on GCP.
It looks like the setup guide for GCP puts me on the jupyter instance where I have access to the tutorials and the gcloud ssh code puts me in a different place. So when I try to use the provided SCP command, I get an error saying the folder/file I’m trying to transfer doesn’t exist…
