Lesson 1 official topic

Good luck with the issue. That is a strange one …

Generally, the first lesson should work on an M1 device since I did get it to work after making some changes. That was a few months back and PyTorch was at a different point then. They’ve fixed a bunch of things since then to work on M1 macs and so hopefully, it works better. If you do run into issues, check the FastAI GitHub repo — I opened an issue way back when listing all the changes that had to be made to get things working for lesson 1. Hope that helps!

1 Like

Thanks Fahim!

I think I’ve discovered the issue → the conda install fastbook is different to the git install…

However I’ve setup my env to use conda, so going to have to figure out how to get the git install working.

Note the different underlying code for the util function.

Will checkout your github issue to see if you answered how to get your MacM1 setup for working with the fastai library!

Thanks again!

As far as I can tell, what’s on the GitHub repo is version 0.0.19 whereas what you get via pip/conda is 0.0.29. So that would seem to indicate that what’s you get via pip/conda is a later version … You should be able to simply add the code from the GitHub repo to your Jupyter notebook and call it from within your notebook itself to see if that works when the other version does not …

That might help you figure out if the issue is withe one version of the code or the other, or something else …

Hi,
I’m trying to build my own classifier based on my pet cat (named Poopy). I’m running the code from lesson 1 on Kaggle, but the only difference is I’m not using DuckDuckGo and instead using my own JPG files. I uploaded a bunch of jpg files and then tried to use the DataBlock, but I’m running into errors:

ValueError: This DataLoader does not contain any batches

Can anyone help me? I’ve been stuck on this for a while.

Thanks!

Also I know its reading in data into the datablock because when I switch it to a dataset, it shows the file:

If you want to, you could make your notebook public and share a link here. It makes debuging easier especially since there is no obvious mistake in your code-snippet, at least that I can see.

Anyways, the dblock in your datasets example doesn’t contain the blocks argument, this could be the reason why your dsets work but the dls wont.

You can have a detailed look at how the dblock is generated with:

dblock = DataBlock(
    blocks=(ImageBlock, CategoryBlock),
    get_items=get_image_files,
    get_y=lambda x: x.name[0] == "I",
    item_tfms=[Resize(192), method='squish')],
)

dblock.summary(path)

Hopefully that gives more detail on whats going wrong.

Thanks for the response!

Here’s a link to the public notebook:

And actually I just removed that blocks section in dblocks so it would display the filename, otherwise it would display random parameters, but I can see why that’s confusing. I tried the summary, but it still doesn’t make sense to me:

Thanks!

Shure :slight_smile:

I still can’t see anything that shouldn’t work code-wise, so my guess is that the problem is in the data itself.
Your dataset is private, so I can’t check that :frowning: . You could make it public or add me as a contributor (“benkarr” - in the datasets Settings Tab under Sharing). If you want to keep your Poopy private (:face_with_hand_over_mouth:) a thing I would try is to load all images, see if there are broken ones and remove them, eg:

fns = get_image_files(path)
for fn in fns:
    try:
        Image.open(fn) 
    except:
        print(fn)

@blin17 Ok, the issue is that the dataset has 15 images, you use RandomSplitter to save some of them for validation so your training set has less than 15 images. Since the batch size is also 15, the dataloader tries to pull 15 images when creating the first batch and fails. Using a batch size that’s at most (15*0.8=) 12 should work!

Besides that: You can attach your dataset to a kaggle notebook via the right panel: (+Add Data) > (Your Datasets) > ''Poopy Train" and don’t have to use the opendatasets library. You can then find the data at ../input/poopy-train.

Also: Poopy!! :heart_eyes_cat: :heart_eyes_cat:

@benkarr haha no I don’t mind sharing picture of the Poops-- made you a collaborator on the dataset. But I tried loading every single image in that train folder and it still doesn’t work either-- I hand pasted every file name in the training set into the top cell and the pictures all load fine:

:face_with_peeking_eye:

1 Like

it works!! thank you @benkarr , but now it recognizes everything as a Poopy, is that because there’s not enough training data?

I was distracted by Poopys cuteness, so I didn’t think of it before :laughing:
Right now (at least it seems so in the notebook/datasets) you only provide data for your “True” label (images of Poopy) during training.
In order to make the model learn to predict “False” for other things you also have to provide the “other things” to the Learner. Your last screenshot suggest that you already have a ‘notpoopy’ folder, so to keep things simple I would do the following:

Locally create a single poopy_dataset folder. This folder contains poopy and notpoopy subfolders (we’re going to derive the labels from the folder names) and put all the images (so no split into train and valid as in your kaggle datasets before) into the according folders. The folder structure should look something like:

poopy_dataset
├── notpoopy
│   ├── IMG_2314.jpg <-- images of something else
│   ├── IMG_3142.jpg
│   └── etc…
└── poopy
    ├── IMG_1234.jpg <-- images of Poopy
    ├── IMG_4321.jpg
    └── etc…

Zip poopy_dataset and upload the .zip file to kaggle, creating a new Dataset (kaggle automatically unzips it). Say you use the method to add a dataset to a notebook that I mentioned before, you will have the same folder structure as above at path = Path('..input/poopy_dataset') (depending on the name you gave the dataset).

For training you only have to adjust the dblock in one place, namely the get_y function that generates the labels for each image. You can use parent_label which fastai provides. It returns the folder name each image is contained in as the label, so notpoopy for parent_label('../input/poopy_dataset/notpoopy/IMG_2314.jpg') for example.

A functioning result might look like this:

from fastai.vision.all import *
path = Path('../input/poopy_dataset')
dblock = DataBlock(
    get_items=get_image_files, 
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    get_y=parent_label,
    item_tfms=[Resize(192, method='squish')]
)
dls = dblock.dataloaders(path, bs=8)
dls.show_batch() ## Check if the labels match the images.
learn = vision_learner(dls, resnet18, metrics=error_rate)
learn.fine_tune(3)

Let me know if it worked or if you need any other help :slight_smile:.

3 Likes

Does the “Weights” in the above image refers to the parameter of the model/architecture?

P.S: Image is from Chapter 1 of the book under the section “How Our Image Recognizer Works”.

yes - the weights of the model are the parameters of the model

Hi all, I’m getting a strange output for the IMDB sentiment analysis model, I tried looking up the output to see if anyone posted something similar but I couldn’t find anything. The same output occurred in Kaggle when I fit the model there

edit: In addition, is it normal for each epoch in training this model to take 7 minutes on Collab when using the free GPU?

Could you be more specific? Its not clear from your screen grab what you are referring to.

It seems you’ve boxed “learn.predict” in red, but that seems less important when you’ve not shared your own research on that, having already read through its docs & search results and be asking specific clarifications about these:

1 Like

During Lesson 1 I encountered an issue regarding the IMDB sentiment analysis (running on Windows)
It wouldn’t allow me to run the TextDataLoader as it constantly got stuck when trying to retrieve the elements that go into the imdb_tok folder (imdb folder worked fine)
Figured to post the steps I followed to fix the issue in case more people are having this issue:

  1. Delete the IMDB/IMDB_tok folder located at ~\.fastai\data\imdb_tok
  2. Restart the kernel and run the following cells one by one:
import fastbook
fastbook.setup_book()
from fastai.text.all import *
path= untar_data(URLs.IMDB)
get_imdb = partial(get_text_files, folders=['train', 'test', 'unsup'])
dls_lm = DataBlock(
    blocks=TextBlock.from_folder(path, is_lm=True, n_workers=0),
    get_items=get_imdb, splitter=RandomSplitter(0.1)
).dataloaders(path, path=path, bs=16, n_workers=0)

Please note this last execution took about 4-5 minutes to finish but it finally grabbed all the elements needed and put into the IMDB_Tok folder ( including counter.pkl )

  1. Restart the kernel
  2. Run the original cell as demonstrated in the notebook:
from fastai.text.all import *

dls = TextDataLoaders.from_folder(untar_data(URLs.IMDB), valid='test', bs=32)
learn = text_classifier_learner(dls, AWD_LSTM, drop_mult=0.5, metrics=accuracy)
learn.fine_tune(4, 1e-2)

Hope this helps anyone running into the same issue :slight_smile:

2 Likes

n_workers=0 would be slow. Which notebook is this from?

(Setting to 0, is generally only needed for running in Jupiter in Windows instead of Linux (or wsl).

@AllenK - Yes sir, I’m running Jupyter in an Anaconda environment on Windows 10.

1 Like

Cool. Thanks for showing how you got around the problems, I’m sure it will be useful for the next person who encounters it.

Windows is not great for this.
I’d recommend using Colab or Kaggle to run the notebooks initially while learning.
(or wsl + ubuntu to run locally, but it adds an additional learning curve if not familiar with linux yet.)

1 Like