Beginner: Using Colab or Kaggle ✅

Customising the Default Root and Project Folders for Google Colab
So I think I’ve figured out how to specify your Google Drive folder as the location for your root directory before loading data and creating models with fastai. I’m very happy to share my notebook describing how I am doing this here.

Hopefully you find this helpful @sambit @bencoman @mike.moloch.
Please let me know if you have any problems accessing the notebook or getting it to work for you.

5 Likes

There’s an easier way, which you can see here:

5 Likes

I have been looking how to use collapsible headings and navigation feature which Jeremy mentioned at start of lesson in JupyterLabs and in colab/Kaggle?

On running the command jeremy pointed out it looks like it doesn’t work in JupyterLabs.
I found an answer in stackoverflow which mentions jupyter notebook extension are not supported.

Although it seems jupyter labs has it’s own navigate menu when I used it in Jarvislabs, couldn’t find how to use collapsible heading there.

I want to use wget from WSL on my home laptop to download this model.pkl file from Kaggle. Normally I would expect to be able to right-click on a button or link and choose Copy link address
image

but although kaggle has a very pretty download screen, I can’t find what the url is to the file. Please advise how to deetermine the url for this file

1 Like

I used this method:

# make sure you're in the right directory
# should be /kaggle/working

!pwd

#zip it up
!zip -r kaggle_working_050122.zip 

# see if the file is there
!ls -lrt | tail 

# get a link to it
from IPython.display import FileLink
FileLink(r'kaggle_working_050122.zip')

If you have an exported pickle file in the kaggle home directory, you can get a downloadable link to it in your kaggle notebook using the above method as well.

At this point, you should see an html link in the output of the last command , if you click on it, you can download it to your local filesystem.

HTH

5 Likes

You can install kaggle python package.

Then you can download the kernel output like in the below screenshot.

2 Likes

@mike.moloch, thanks, this looks like a useful approach, looking into it.

@kurianbenoy, thats awesomesauce! even better than I was looking for.

[How to] deploy Kaggle trained model on Hugging Face

For my own later reference, and maybe help others unfamiliar with git,
here is my action log…

  1. Created a new space at hugging space

  2. Cloned that space to my local machine.

$ git clone https://huggingface.co/spaces/bencoman/cats-v-dogs
$ cd cats-v-dogs
  1. Added Git Large File Storage.
# This needs to do this once per machine
$ curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
$ sudo apt-get install git-lfs

# This needs to be done once per repo
$ git lfs install
$ git lfs track "*.pkl"
$ git add .gitattributes
$ git commit -m "added git large file storage"
  1. Copied random test pics dog.jpg and cat.jpg files into this folder

  2. Edited the application code app.py as follows…

from fastai.vision.all import *
import gradio as gr

def is_cat(x): return x[0].isupper()

def classify_image(img):
    pred,idx,probs = learn.predict(img)
    return( dict(zip(categories, map(float,probs)))

learn = load_learner('model.pkl')
categories = ('Dog', 'Cat')
image = gr.inputs.Image(shape=(192,192))
label = gr.outputs.Label()
examples = [ 'dog.jpg', 'cat.jpg' ]
iface = gr.Interface(fn=classify_image, inputs=image, outputs=label, examples=examples)
iface.launch(inline=False)
  1. Confirmed the trained model.pkl file was available here…
    https://www.kaggle.com/code/bencoman/cats-v-dogs-saving-a-basic-fastai-model/data. Note: it took me a while to realise this file was disappearing because I was doing a Quick Save rather than Save And Run All.

  2. Downloaded and installed API key in kaggle.json file created per API Credentials section at https://github.com/Kaggle/kaggle-api#api-credentials

  3. Reviewed kaggle list and output examples, then downloaded model…

$ pip install kaggle

# Downloaded API key to ~/.kaggle/kaggle.json

$ kaggle list kernels --mine | grep -i cat
code/bencoman/cats-v-dogs-inference-gradio                      Cats v Dogs - inference gradio                     Ben Coman  2022-05-08 13:39:35           0
code/bencoman/cats-v-dogs-saving-a-basic-fastai-model           Cats v Dogs - saving a basic fastai model          Ben Coman  2022-05-08 16:02:11           0

$ ls
README.md  app.py  cat.jpg   dog.jpg

$ kaggle kernels output bencoman/cats-v-dogs-saving-a-basic-fastai-model --wp

$ ls
README.md  app.py  cat.jpg   dog.jpg  
cats-v-dogs-saving-a-basic-fastai-model.log  model.pkl
  1. Pushed to hugging-space
$ git add -A
$ git status
        new file:   app.py
        new file:   cat.jpg
        new file:   cats-v-dogs-saving-a-basic-fastai-model.log
        new file:   dog.jpg
        new file:   model.pkl
$ git commit -m "First app & model"
$ git push
Username for 'https://huggingface.co': bencoman
Password for 'https://bencoman@huggingface.co': ******
Uploading LFS objects:   0% (0/1), 22 MB | 113 KB/s                                                                                 
Uploading LFS objects: 100% (1/1), 47 MB | 114 KB/s, done.
Enumerating objects: 12, done.
Counting objects: 100% (12/12), done.
Delta compression using up to 12 threads
Compressing objects: 100% (10/10), done.
Writing objects: 100% (10/10), 65.79 KiB | 21.93 MiB/s, done.
Total 10 (delta 2), reused 0 (delta 0)
remote: Enforcing permissions...
remote: Allowed refs: all
To https://huggingface.co/spaces/bencoman/cats-v-dogs
   ae86db9..813dfce  main -> main  
  1. Tested application by clicking…
    https://huggingface.co/spaces/bencoman/cats-v-dogs

Note: After completing this local jupyter from scratch setup,
the app can be run locally by creating a new notebook applocal.ipynb
next to app.py and copying the contents of app.py into the notebook.

9 Likes

@kurianbenoy

I went through the same exercise as you did.
Use the Extension Manager in case of JupyterLab. I searched for collapse and found the collapsible headings and select Install.

I was able to use Collapsible headings in JupyterLab

19 Best JupyterLab Extensions for Machine Learning - neptune.ai (See extension 17)

3 Likes

Thanks for sharing about it

hi @bencoman, thank you for these helpful tips.
I however don’t know if you have tried the ImageClassifierCleaner on Kaggle. I am currently trying to get the widgets to work but it is not producing the result. The pictures are shown and then the end produces

Error displaying widget: model not found

1 Like

I’ve only used it once while following the “bear” lesson. I don’t remember getting that error.

Maybe you will find something useful in this search: kaggle Error displaying widget

1 Like

Thank you @bencoman.
I already fixed the widget issue, I had to

pip install ipywidgets

and

import widget from ipywidget

and it is now working and interactive.

Thanks so much for your response.

4 Likes

@deelight_del, Thanks for reporting back!!! That will help the next person.

1 Like

Whenever using the fastbook search_images_ddg function in Colab I get the following error:

@LeoJuhlin This error pops up when interacting with a folder as if it’s a file.

I believe you have your results and dest flipped around, it should be something more akin to:

download_images(dest, urls=results.attrgot('contentURL')

You can take a look at this code from chapter 2 of the fastbook as an example (use a ctrl + f search for “download_images” if you wanna find the specific code block):

bear_types = 'grizzly','black','teddy'
path = Path('bears')

if not path.exists():
    path.mkdir()
    for o in bear_types:
        dest = (path/o)
        dest.mkdir(exist_ok=True)
        results = search_images_bing(key, f'{o} bear')
        download_images(dest, urls=results.attrgot('contentUrl'))
1 Like

I’m having trouble with autocomplete suggestion in Kaggle. I’m getting no suggestions, even when I hit tab, for both native Python functions and functions included in packages that I imported. Any advice on how to get autocomplete suggestions in Kaggle?

1 Like

It worked now. Thank you!

hey guys!

started lesson 1 yesterday, however when i copy the notebook “is it a bird?” in kaggle or re-write it from scratch in colab, i get this error when trying to run the fourth cell:

# searching for a batman photo
urls = search_images('batman photos', max_images = 1)
urls[0]

it comes with HTTP Error 403: Forbidden raised.

however, when I change the url of the previous cell from ‘duckduckgo.com’ to ‘google.com’, i got another type of error: HTTP Error 405: Method not allowed.

and when changed from ‘google.com’ to ‘yandex.com’ (russian-based search engine) it throws another error: AttributeError: 'NoneType' object has no attribute 'group'.

is something wrong with my colab environment?

3 Likes

Hey everyone

I too am getting a HTTP Error 403: Forbidden in lesson 1. Appreciate any help on this

/opt/conda/lib/python3.7/site-packages/fastcore/net.py in urlread(url, data, headers, decode, return_json, return_headers, timeout, **kwargs)
    113         with urlopen(url, data=data, headers=headers, timeout=timeout, **kwargs) as u: res,hdrs = u.read(),u.headers
    114     except HTTPError as e:
--> 115         if 400 <= e.code < 500: raise ExceptionsHTTP[e.code](e.url, e.hdrs, e.fp) from None
    116         else: raise
    117 

HTTP403ForbiddenError: HTTP Error 403: Forbidden
4 Likes

@svrmnnk @seiger I’m also getting these issues inside the Kaggle notebook itself, even when running it all vanilla, definitely not an issue with Colab.

Not exactly sure what the issue is, perhaps the notebook needs an update.