Lesson 1 throwing error in ImageDataBunch stage in windows

I tried installing Pytorch and then torchvision and then fastai developer version.

Tested importing torch and torchvision and then ran the Lesson 1 notebook.

But it fails in this method.

‘’‘data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224)
data.normalize(imagenet_stats)’’’

In one thread found putting no_worker=0 in windows works, so i changed the above as.

data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224,num_workers=0)

But the error remains same…Please find the message below.


AttributeError Traceback (most recent call last)
in
----> 1 data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224,num_workers=0)
2 data.normalize(imagenet_stats)

D:\MyWork\Document\GitHub\course-v3\nbs\dl1\fastai\vision\data.py in from_name_re(cls, path, fnames, pat, valid_pct, test, **kwargs)
330 pat = re.compile(pat)
331 def _get_label(fn): return pat.search(str(fn)).group(1)
–> 332 return cls.from_name_func(path, fnames, _get_label, valid_pct=valid_pct, test=test, **kwargs)
333
334 def batch_stats(self, funcs:Collection[Callable]=None)->Tensor:

D:\MyWork\Document\GitHub\course-v3\nbs\dl1\fastai\vision\data.py in from_name_func(cls, path, fnames, label_func, valid_pct, test, **kwargs)
323 @classmethod
324 def from_name_func(cls, path:PathOrStr, fnames:FilePathList, label_func:Callable, valid_pct:int=0.2, test:str=None, **kwargs):
–> 325 labels = [label_func(o) for o in fnames]
326 return cls.from_lists(path, fnames, labels, valid_pct=valid_pct, test=test, **kwargs)
327

D:\MyWork\Document\GitHub\course-v3\nbs\dl1\fastai\vision\data.py in (.0)
323 @classmethod
324 def from_name_func(cls, path:PathOrStr, fnames:FilePathList, label_func:Callable, valid_pct:int=0.2, test:str=None, **kwargs):
–> 325 labels = [label_func(o) for o in fnames]
326 return cls.from_lists(path, fnames, labels, valid_pct=valid_pct, test=test, **kwargs)
327

D:\MyWork\Document\GitHub\course-v3\nbs\dl1\fastai\vision\data.py in _get_label(fn)
329 def from_name_re(cls, path:PathOrStr, fnames:FilePathList, pat:str, valid_pct:int=0.2, test:str=None, **kwargs):
330 pat = re.compile(pat)
–> 331 def _get_label(fn): return pat.search(str(fn)).group(1)
332 return cls.from_name_func(path, fnames, _get_label, valid_pct=valid_pct, test=test, **kwargs)
333

AttributeError: ‘NoneType’ object has no attribute ‘group’

3 Likes

I get the same error.

When performing regular expressions through importing re I retrieve the right files. However the ImageDataBunch.from_name_re function fails.

Example:
Retrieve all files ending in red.png ->
for n in range(len(files)):
if re.findall(’.
[red]+.png$’, files[n]):
colored_files.append(files[n])
print(colored_files[:10])

This same regex returns the above failure on my machine.

Were you guys able to fix this issue ?

2 Likes

I just hit this - and it is the Windows path with \ instead of / - even though it looks like it has / from the output of fnames. The regular expression neds to change - and the \ also needs escaping with another \

pat = re.compile(r’\\([^\\]+)_\d+.jpg$’)

And it took me a few edits so you could see two \ characters by entering 4…

5 Likes

Thanks a lot @brismith. You saved the Windows fastai v1 users :slight_smile:

1 Like

I am a Windows user, and I get the same error - AttributeError: ‘NoneType’ object has no attribute ‘group’

After changing the regular expression for variable (from: pat = r’/([^/]+)_\d+.jpg$’ ) to
pat = re.compile(r'\\([^\\]+)_\d+.jpg$')

I get the following error instead:

C:\Users\lenovo\Anaconda3\lib\site-packages\fastai\basic_data.py:201: UserWarning: There seems to be something wrong with your dataset, can't access self.train_ds[i] for all i in [1283, 2149, 4603, 3572, 43, 4427, 2195, 2059, 3249, 5795, 2803, 1988, 1149, 1505, 2894, 2226]
  warn(f"There seems to be something wrong with your dataset, can't access self.train_ds[i] for all i in {idx}")

You can deactivate this warning by passing `no_check=True`.

Grateful for any advise. Thank you. :slight_smile:

Hi - I get the same issue with windows (It did work on Crestle but was trying to use Windows).

I has change the pat directory to load the files and they all seem to be there.

line response says…

C:\Users\micha\Anaconda3\envs\fastai\lib\site-packages\fastai\basic_data.py:201: UserWarning: There seems to be something wrong with your dataset, can’t access self.train_ds[i] for all i in [5122, 1932, 202, 432, 2201, 2306, 290, 552, 2982, 5703, 294, 5631, 2849, 3584, 1904, 1584]
warn(f"There seems to be something wrong with your dataset, can’t access self.train_ds[i] for all i in {idx}")
You can deactivate this warning by passing no_check=True.

this is what i use and the notebook runs for me:

pat = r'([^\\]+)_\d+.jpg$'

and this is everything i ran to get it to train:

from fastai import *
from fastai.vision import *

data_path = Path('C:/Users/jpatn/data/oxford-iiit-pet/')
weights = Path('D:/projects/fastai-nbs/PETS/')

ims = data_path/'images'

def get_data(size,bs):
return (ImageItemList.from_folder(ims)
.random_split_by_pct(0.2, seed=2)
.label_from_re(r'([^\\]+)_\d+.jpg$')
.transform(get_transforms(), size=size)
.databunch(bs=bs).normalize(imagenet_stats))

(not sure how to get this to format correctly)

learn1 = create_cnn(data, models.resnet18, metrics=accuracy, path=weights)

learn1.fit_one_cycle(5, 1e-3)

I am also running into some issues on Windows (It is fairly easy to get it working on Google Colab and other services, mostly just wanted to take advantage of my 2080ti I have on my gaming computer). I got past the
“There seems to be something wrong with your dataset…” errors by upgrading the version of fastai I was using. I think it was fixed in 1.0.41, but I just installed the latest from master.

Now that I’ve gotten past that part, it just hangs when I call create_cnn. The furthest down I’ve been able to debug is in m.eval()(dummy_batch(m, size)) in callbacks/hooks.py. It looks as though it is stuck in libifcoremd.dll, which appears to be an Intel Fortran library makes me think one of my dependencies (MKL maybe?) wasn’t installed correctly, but I haven’t been able to figure out exactly what is going on yet.

1 Like

Hi - thanks for the replies, I still struggled to make things work in a Windows but then also decided that the speed may be better in crestle so I have decided to embrace that environment.

I am also having this same issue. Very disappointed that I can’t actually run this on Windows, and WSL support doesn’t work because of no GPU.

I got the same problem here, and I tried all the solutions on the forum.
None of them works for my computer so far.
I’ve stuck for 3 days, kind of frustrating, hope someone can solve this problem.

1 Like

So I get the same error, but I continued on and tried running pulling up an item with data.train_ds[0] and got an error RuntimeError: b should have at least 2 dimensions, but has 1 dimensions instead. I searched for this and found another thread that was seeing the same issue. Apparently it has been fixed, see here. All they had to do was update to fastai version 1.0.41. I’ve been trying to do that but my version is stuck on 1.0.38 and conda says there’s nothing to update when I tell it to update. I try to update using pip but it says it gives an error

Could not find a version that satisfies the requirement torch>=1.0.0 (from fastai) (from versions: 0.1.2, 0.1.2.post1)
No matching distribution found for torch>=1.0.0 (from fastai)

Not sure how I can update it at this point.

Edit:
So after some trying I finally managed to get fastai updated to version 1.0.45 after manually specifying that version and upgrading some other packages but now my conda installation is broken. When I try to start it up it says failed to create process in the console. This is the second time it’s happened(I had to re-install with miniconda) so I’m giving up until someone else figures out what’s going on.

1 Like

I uninstall Anaconda, and install both Python and Anaconda, then use pip to install fastai.
Then it suddenly works like a charm!!!

And I seriously don’t know why, probably because I only install fastai on Python instead of Anaconda, and I usually coding in Spyder not Jupyter Notebook.

Probably this solution only works on my shitty laptop

When I create a fresh virtualenv (Python 3.7) and run pip install fastai I get
No matching distribution found for torch>=1.0.0 (from fastai)

Because there are only a bunch of wheels for torch, none of which work on Windows. and no source. release Cool.

I had this problem too but I got it to work(to a point). You need to install torch from the website and fastai/jupyter notebook using pip and then it worked for me.

However now when I try to create_cnn with resnet34 it seems to hang, or is taking very very long(>10mins). It worked with vgg16 in under a minute though. Not sure what’s going on.

create_cnn blocks forever, doesn’t seem to return if you give it a resnet varient on Windows. Can anyone else confirm?

Someone mentioned it here as well.