Introducing fastai-shell

Part 1 (2019)
1

This is the continuation of my guide to create a fastai setup.

Now this is a workflow to use fastai on Google Cloud Platform.

This project features:

  • Use preemptible instances to get the minimum price possible
  • Switch between different GPUs based on your requirement and cost
  • Keep data and tools you installed even when switching between GPUs
  • No need to install anything locally, you just need a web browser
  • Fully automated process, no SSH or complex commands required
  • Switch your instance between different availability zones

Start here: https://github.com/arunoda/fastai-shell

Watch how to use fastai-shell

25 Likes
2

Very Nice :grinning: …!!

1 Like
3

Hi Arunoda, the project looks very promising. Unfortunately I have run into a problem while trying it out. I already have a vm which was made according to the official fast.ai documentation for this course, and it’s worked just fine. I tried running the fastai create command and got the following

:~$ fastai create
Ensure fastai network
ERROR: (gcloud.compute.networks.list) The project property is set to the empty string, which is invalid.
To set your project, run:

  $ gcloud config set project PROJECT_ID

or to unset it, run:

  $ gcloud config unset project
ERROR: (gcloud.compute.networks.create) The project property is set to the empty string, which is invalid.
To set your project, run:

  $ gcloud config set project PROJECT_ID

or to unset it, run:

  $ gcloud config unset project

I don´t think I missed any of your steps. Is there a conflict with my current VM?

4

Are you using the cloud shell?
This is currently supported via the Cloud Shell only.

5

Thanks for including a video, it made all the difference. Can I suggest a feature? It would be good to have the option to mount a cloud backed drive such as Google Drive, Dropbox, OneDrive or just GoodSync.

1 Like
6

That’s a good suggestion. Thanks.

7

Yes I used the cloud shell. It looks like I just need to set the PROJECT_ID but I assumed it would set itself. Have I perhaps missed something?

8

That’s pretty weird.
It automatically set the DEVSHELL_PROJECT_ID env variable.
Try setting that env variable and run the script:

export DEVSHELL_PROJECT_ID="<your project id>"
fastai create
9

Logged in this morning againand checked the PROJECT_ID variable and it had a value. I set it to a new name (the project name not ID) which failed. I then reset it to it’s original value and the script now runs just fine. I now have a gleaming new VM waiting to be started. No idea what could hav egon wrong last time and why the PROJECT_ID was unavailable.
Thanks for the great work @arunoda!

1 Like
10

Glad to see everything working as expected.

11

The option to mount a Dropbox share into the instance would be super cool. Is this possible?

12

After you start the instance, you could do it yourself by opening the Terminal from the Jupyter.
All the changes you made will be saved.

I hope you can find those commands to mount the instance.

13

@arunoda, I’m currently checking out your fastai shell. However, whichever region I select, I always get:

ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - The zone 'projects/fastai-222019/zones/us-west1-b' does not have enough resources available to fulfill the request.  Try a different zon
e, or try again later.

Now, is this just bad luck and there are no preemptive instances available, or is this something with my account? I had two previous projects which I just deleted if this matters in any way…

I’d prefer a Europe instance (but I see they are not listed in the script), but US would be fine, too…

14

You can switch do a different zone.
See: https://github.com/arunoda/fastai-shell#switching-zones

15

I do get it for all and also added some European ones to your script…

1 Like
16

Feel free to send me a PR.

17

Will do. But does this look funny to you? Apart from he error itself?

$ fastai create
Ensure fastai network
Creating the boot instance
Listed 0 items.
Listed 0 items.
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - The zone 'projects/fastai-222019/zones/europe-west1-b' does not have enough resources available to fulfill the request.  Try a different
 zone, or try again later.
18

This might be a very busy time for them.

19

Seems like it :wink: Cheers… will try later. I was just wondering if there is something wrong with my project/ account etc.

20

Still not working. Maybe it’s with my account? I deleted two projects before creating a fresh one.