Google Colab Setup for FastAI Part 2 v2

With my experience, it disappears unless saved explicitelty.

Ok Vikrant, good to know, thanks!

Hi naman. Thanks a lot for putting it all together. I am relatively new to colab. So if the steps you mentioned are followed, it gives us the fas.ai environment which lasts for 12 hours, right? I should take the notebook you shared that contains the installation scripts, run the scripts and then import fast.ai notebooks and start executing them, is it? I tried following the steps and then imported pascal notebook. when i run the first cell ,it says it can’t find fast.ai module. Am i missing anything here?

@avinash3593 Every notebook you start up in Colab provides you with a different Virtual Machine. Make sure the you add the installation scripts on top of the notebook you’re going to work with.
You could always !ls any_directory and poke around to see what is installed/downloaded and what’s not.
Also, since the VM resets every 12 hours or 1.5 idle hours, always keep track of the time your VM has been active using !uptime or other similar commands.
Hope this helps.

2 Likes

Thank you @keratin. I did try looking into what !ls shows. I was expecting the downloaded data and installed libs to be available for 12 hours. That wasn’t the case when I opened a new notebook. So I was wondering if it’s just that my connection to existing VM was reset or if every new notebook needs to be setup. Thanks for the clarification.

Hi !
Make sure you have changed the runtime type of the pascal.ipynb also to GPU. In case you initialized the Setup notebook in a GPU backend, the files and other repos are installed for that particular backend. By default, a new notebook opens in a CPU only backend.

1 Like

The data is actually never downloaded to your Drive but to the VM instance. So, it disappears and doesn’t take your Drive storage.

This worked. When I opened a new notebook, it’s runtime was not GPU. Once I changed it, the downloaded data and installed packages are working fine in the new notebook. Thank you!

thanks, I am downloading it now.

Hello Naman,

Do you also have the updated FastAI_v2_setup.ipynb for pascal-multi.ipynb?

Thanks

how can i download from here : http://files.fast.ai/data/ in colab notebook? wget extension isn’t wirking for this.

You should add a few more details. Which dataset are you trying to download exactly (or the corresponding notebook)?

I use colab quite frequently and yet to come across any wget issues.

Best,
A

I’m running lesson4 part1 v2 on colab and came across imdb dataset. How can I download it from the given link into the notebook.

I see. Looking at my old notes, these seemed to have done the trick for me. Give it a go …

!mkdir -p data/imdb/  
!wget http://files.fast.ai/data/aclImdb.tgz -P data/imdb/
!tar -xzf data/imdb/aclImdb.tgz -C data/imdb/
!rm -rf data/imdb/aclImdb.tgz

Good luck studying
A

2 Likes

Has anyone been able to successfully reproduce the results of Lesson 11 on Colab?
For me, the kernel restarts when I load the fasttext embeddings presumably because it ran out of memory. Any tips to fix this?

@naman-bhalla Thanks for this! I used this to run a practical session for a one-day conference I helped organize on 7th April.

2 Likes

My experience with Google Colab so far:

I started using Colab when it first came out during Beta stage. At that point, I have no confidence to continue using it for mildly serious work due to its stability, the show stopper issues encountered and zero ease-of-use design in mind.

  • connections: been quite good but not perfect. I think it really depends on your ISP?
  • file management:
    • Last I heard, the major issue is regarding file management as all data is lost after a session (VM instance) is destroyed.
  • GPU not available or fresh GPU with less than ~700 MB CUDA memory left.

Recently, I managed to reproduced and complete the training for these notebooks:

  • the beginning lessons (those smaller models and smaller datasets) of part1v2 within the 12 hours limit (or 1.5 idle hours).
  • artistic style transfer, super resolution, DCGAN and Wasserstein GAN with LSUN dataset (big 51 GB) of part2v1 with multiple (2 to 3) Colab sessions.

The solutions shared and discussed in this thread are good. However, these days I pretty much use the solution from Clouderizer to solve pretty much most of the problems and issues I have. Note: I am not affiliated with them. It’s the easiest & fastest way to get your notebook up & running on Colab (if you don’t mind giving your Google Drive access permission to Clouderizer). Clouderizer’s automatic 2 way data sync to Google Drive is nice despite it’s not real-time. So, to prevent training progress loss, I save the model state (checkpointing) more often. It’s a better solution but not perfect.

Go check out this thread that me and the rest have been discussing some tips & tricks & workaround when using Colab:

My early conclusion: Nevertheless, Colab is a great service for learners who can’t afford paid cloud computing services like AWS/GCP/Paperspace/etc.

3 Likes

What with the option: colabolatory to prototyping, preparing the model, following lesson, etc. Paperspace/AWS, whatever, to evaluate serious datasets?

Hi All,

I have tried most of the code in colab, it is having some or other error.
Anyone having working code in colab for part 2 full course.

Thanks in advance.

Thanks and Regards,
Subho

1 Like