ImageDataLoaders always fails

[Heluca (Robert)]

I haven’t used fastai in a bit, so I am still new to figuring out the new library, but I keep receiving “Could not do one pass in your dataloader, there is something wrong in it”

I tried the most basic examples:
01_intro from fastbook and the vision tutorial at docs.fast.ai, show_batch gives me:
RuntimeError: Could not infer dtype of PILImage

04_mnist, show_batch gives me:
RuntimeError: Could not infer dtype of PngImageFile

DataLoaders.from_dsets works normally.

I’ve tried recreating the virtual env and a different machine. Am I using the wrong version of software or is there a bug?
Ubuntu 20.04
Python 3.8.10
CUDA 11.4
These are my pip packages:
torch & torchvision 1.9.0+cu111
fastai 2.4
fastbook 0.0.16
pillow 8.3.0

[idraja (Idris)]

See this post for what additional details you should add to your question so that someone can reasonably help you

How to debug your code and ask for help with fastai v2

We understand you might get frustrated when debugging your code and try to figure out if there is something wrong with it or if there is a bug in the library, so I wanted to highlight in a post the first steps to debug your model training with fastai v2 and how to best ask for help on the forum, to make sure you get quick answers from experienced users. What not to do when asking for help Here is an example post you should absolutely never write: When I tried learn.fit_one_cycle(1), my model …

[Heluca (Robert)]

I started over with conda and everything works fine now. Thanks.

[chelsea (Chelsea T)]

Hi Robert,

I seem to be getting the same problems that you had. Could you be a bit more elaborate with your solution of ‘I started over with conda’. Apologies if this seems a silly question. I’m very (very!) new to deep learning. I’m also using an AWS SageMaker notebook instance that tends to complicated things more.

Thanks,
Chelsea

[always-learning (Sam)]

I am also using sage maker as my GPU server and getting the same error. I believe it has something to do with Pillow version 8.3. I tried downgrading to Pillow version 8.2 with no luck however…

[vivekharshey1 (Vivek Harshey)]

Hi,
I am also getting the same error in paperspace gradient notebook when I create the data loader.
However the error disappeared magically when I followed the following steps.
!pip install fastai -q --upgrade
from fastai.basics import *
from fastai.vision.all import *
from fastai.callback.all import *
path = untar_data(URLs.PETS)
fnames = get_image_files(path/‘images’)
pat = r’(.+)_\d+.jpg$’

pets = DataBlock(blocks=(ImageBlock, CategoryBlock),
get_items=get_image_files,
splitter=RandomSplitter(),
get_y=RegexLabeller(pat = r’/([^/]+)_\d+.*’),
item_tfms=RandomResizedCrop(128, min_scale=0.35),
batch_tfms= [*aug_transforms(size=224, max_warp=0)])

Now the error disappeared.

[Heluca (Robert)]

Hi Chelsea,

The installation instructions at GitHub - fastai/fastai: The fastai deep learning library provide instructions to install fastai with either the pip or conda software package managers. In the past I had no difficulty with pip, but it doesn’t seem to work out of the box anymore. There is another response from Vivek with the steps he used to get pip to work.

I installed conda (it may already be installed on your machine, just try typing ‘conda info’ and it will work if it is installed. Otherwise you can install miniconda on a typical machine with ‘wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh’, then ‘bash Miniconda3-latest-Linux-x86_64.sh’. You need to agree to the T&Cs and then it will ask if you want to initialize, type yes.

The rest of the instructions are on the fastai github page: run ‘conda install -c fastchan fastai’

And voila, the error is gone and I am back to work.

[scottab (Scott)]

I was facing a similar issue when running on AWS for the first time. I found downgrading Pillow to 8.2.0 per the instructions here resolved the issue

mambda install Pillow=8.2.0

[chelsea (Chelsea T)]

I’ve been at this now for days and I was in the same boat. It seemed that downgrading to Pillow version 8.2.0 was working for most people but it just wasn’t fixing the ImageDataLoader error for me. UNTIL, I realised that I was downgrading Pillow in the wrong environment. Again, like I’ve mentioned, I’m very new to all of this so apologies if this seems obvious to some. But I found that I needed to specify which environment I needed Pillow to be version 8.2.0 in.

I don’t know the exact steps I did because I kept running different iteration but I think this may help:

1. In my AWS notebook instance I ran the following to check the version:
   Import PIL
print(PIL.__version__)

2. I then rang the below to see where the module was housed:
   ?? PIL
   The location is ~/SageMaker/.env/fastai/lib/python3.6/site-packages/PIL/init.py

3. I found then ran the following and one of these seemed to work:
   !conda install -p /home/ec2-user/SageMaker/.env/fastai -c pypi pillow==8.2.0
!conda install -p /home/ec2-user/SageMaker/.env/fastai -c conda-forge pillow==8.2.0

I hope this helps someone out there!

Chelsea

[Heluca (Robert)]

I think conda happened to install an older version of pillow at the moment I used it. A few days later, it too failed. Downgrading to 8.2.0 solved the problem with both package managers.

Thanks Chelsea for raising the point about checking the version of pillow being used in the current environment.

[Ernie]

Thank you all so much for these discussions. I just started this course (March 2022) and have been following along with an AWS SageMaker instance and have run into many issues so far due to environment issues. I just started Ch 4 of the book and was blocked on this. Downgrading to Pillow 8.2.0 solved my issues. Back to work…