Colaboratory and Fastai


(Atul Krishna Singh) #81

Hello intheroom,
In the top left corner, there is file menu from where you can select the option to “upload notebook”.
for more information go through this:
http://forums.fast.ai/t/colaboratory-and-fastai/10122/76


(Ronaldo da Silva Alves Batista) #82

Hey guys,

Check this out:

This article lists down steps for setting up fast.ai course on Colab, but similar steps can be done for any ML project. http://bit.ly/2DyqMYd

This automates env setup, dataset and code download on Colab.

This actually solves the problem that the environment in Colab is not persistent ( i.e. every time you come back you must to install the libraries and download the datasets again )

Cheers


(Arnav) #83

I don’t think you need to do so much. I’ve done lessons 1-3 on Colab and sure, many problems do occur in the process but once I figured it out for lesson 1, it wasn’t very hard. Besides, the troubleshooting helped me learn a lot.
It can be a bit intimidating for beginners and take up some extra time as well but it’s worth it.
Everyone on this thread is very helpful as well.


(yohei) #84

When I try pip install fastai on colab,

it always hang up or too much slow to download.
More precisely it always stops at

Collecting bcolz (from fastai)
Using cached bcolz-1.2.0.tar.gz

Other library installation such as pytorch or kaggle I have no problem


(Arnav) #85

@yohei Try !pip install --upgrade git+https://github.com/fastai/fastai.git


(yohei) #86

Thank you so much


(Santiago Gonzalez) #87

Hello,

Did you try running lesson1_rxt50.ipynb notebook in Google Colab? I’m getting the following error:

FileNotFoundError: [Errno 2] No such file or directory: ‘/usr/local/lib/python3.6/dist-packages/fastai/weights/resnext_50_32x4d.pth’

I know that we need to download the weights.tgz file from http://files.fast.ai/models/weights.tgz but I don’t know exactly where to put it, as in paperspace/crestle/AWS we work with the notebook under the cloned repository folder, but in the case of Google Colab, the notebook is loaded from my Google Driver and not from the fastai local git repo.


(Cedric Chee) #88

Extra Google Colab Tips:

  1. As you know, every few hours, the kernel gets disconnected and local files removed. So you need to automatically backup the models during training to your Google Drive to save the progress and be able to re-use the model later. Here’s a script that helps us do that if you are using Keras: https://github.com/Zahlii/colab-tf-utils

  2. A repository of useful scripts for adding common services to non-persistent Colab VM sessions: https://github.com/mixuala/colab_utils


(Pierre Guillou) #89

If you use Clouderizer (notebooks) with Google Colab (GPU), your notebooks + data are saved in your Clouderizer drive.


(Cedric Chee) #90

Thank you for sharing. This is the first time I heard about Clouderizer. Have you try using it? Is there a guide somewhere which we can refer to to get started with Clouderizer + Google Colab? I couldn’t find it in their docs and blog posts. Thank you in advanced.


(Pierre Guillou) #91

Hi @cedric,

the use of Clouderizer helps a lot for running (for free) the Fastai notebooks using Google Colab GPU.

Best blog post : Fastest way to setup Fast.ai course notebooks , for free — using Google Colab and Clouderizer from @prakashgupta

… and “FastAI with Clouderizer – Get started in minutes”.

Have fun :slight_smile:


(Maria P) #92

How can I upload custom data set using this setup (like additional set, so I won’t have to build it twice, I’ve figured that I can provide link to data in the project settings)? I’ve tried to look for a data folder in fast.ai clouderizer drive, but I don’t see any. Also when I’m trying to log into the console I get the permission denied (using either empty password and my clouderizer password), so I can’t look it up either.


(Pierre Guillou) #93

There are at least 2 possibilities I think (cc @prakashgupta):

  1. Open a Terminal window from the leaderbord of your project in Clouderizer, use the clouderizer (with small c) password and use for example the wget command to download your dataset file.
  2. Open a jupyter notebook in your Clouderizer and use !wget (with exclamation point in front of wget) to download your dataset file.

If it does not work, you can ask in the Clouderizer forum as well.