Error loading data on Windows

I’m trying to run the notebooks on my Windows machine. I have tested the code on a paperspace machine and it works fine, I’m able to train my model even. However, on Windows, I can’t even seem to load the images properly. Here is my code

image_path = '.\custom_car_imgs'
data = (ImageItemList.from_folder(image_path)
        .transform(get_transforms(), size=64)

I get this error:

C:\Users\Gebrial\Anaconda3\lib\site-packages\fastai\ UserWarning: There seems to be something wrong with your dataset, can't access self.train_ds[i] for all i in [179, 288, 356, 354, 15, 171, 86, 355, 342, 115, 203, 442, 500, 303, 310, 234, 371, 93, 318, 307, 49, 196, 503, 91, 490, 159, 493, 157, 361, 414, 214, 384, 243, 215, 555, 50, 108, 464, 425, 317, 396, 9, 497, 98, 82, 373, 531, 395, 85, 418, 253, 55, 502, 485, 84, 450, 377, 204, 46, 435, 417, 68, 187, 216]
  warn(f"There seems to be something wrong with your dataset, can't access self.train_ds[i] for all i in {idx}")

Specifically, it seems to happen on the databunch() step.

When I run the code data.train_ds.x[404], it outputs the picture of a car and data.train_ds.y[404] outputs it’s class name, everytime. However, if I run data.train_ds[404] it outputs (Image (3, 64, 64), Category Honda CR-V) sometimes and sometimes it outputs an error:

RuntimeError                              Traceback (most recent call last)
<ipython-input-59-6bab94caef27> in <module>
----> 1 data.train_ds[404]

~\Anaconda3\lib\site-packages\fastai\ in __getitem__(self, idxs)
    524             else:                 x,y = self.item   ,0
    525             if self.tfms:
--> 526                 x = x.apply_tfms(self.tfms, **self.tfmargs)
    527             if hasattr(self, 'tfms_y') and self.tfm_y and self.item is None:
    528                 y = y.apply_tfms(self.tfms_y, **{**self.tfmargs_y, 'do_resolve':False})

~\Anaconda3\lib\site-packages\fastai\vision\ in apply_tfms(self, tfms, do_resolve, xtra, size, resize_method, mult, padding_mode, mode)
    111                 if resize_method in (ResizeMethod.CROP,ResizeMethod.PAD):
    112                     x = tfm(x, size=size, padding_mode=padding_mode)
--> 113             else: x = tfm(x)
    114         return x

~\Anaconda3\lib\site-packages\fastai\vision\ in __call__(self, x, *args, **kwargs)
    497     def __call__(self, x:Image, *args, **kwargs)->Image:
    498         "Randomly execute our tfm on `x`."
--> 499         return self.tfm(x, *args, **{**self.resolved, **kwargs}) if self.do_run else x
    501 def _resolve_tfms(tfms:TfmList):

~\Anaconda3\lib\site-packages\fastai\vision\ in __call__(self, p, is_random, *args, **kwargs)
    444     def __call__(self, *args:Any, p:float=1., is_random:bool=True, **kwargs:Any)->Image:
    445         "Calc now if `args` passed; else create a transform called prob `p` if `random`."
--> 446         if args: return self.calc(*args, **kwargs)
    447         else: return RandTransform(self, kwargs=kwargs, is_random=is_random, p=p)

~\Anaconda3\lib\site-packages\fastai\vision\ in calc(self, x, *args, **kwargs)
    449     def calc(self, x:Image, *args:Any, **kwargs:Any)->Image:
    450         "Apply to image `x`, wrapping it if necessary."
--> 451         if self._wrap: return getattr(x, self._wrap)(self.func, *args, **kwargs)
    452         else:          return self.func(x, *args, **kwargs)

~\Anaconda3\lib\site-packages\fastai\vision\ in coord(self, func, *args, **kwargs)
    165     def coord(self, func:CoordFunc, *args, **kwargs)->'Image':
    166         "Equivalent to `image.flow = func(image.flow, image.size)`."
--> 167         self.flow = func(self.flow, *args, **kwargs)
    168         return self

~\Anaconda3\lib\site-packages\fastai\vision\ in _symmetric_warp(c, magnitude, invert)
    236     m = listify(magnitude, 4)
    237     targ_pts = [[-1-m[3],-1-m[1]], [-1-m[2],1+m[1]], [1+m[3],-1-m[0]], [1+m[2],1+m[0]]]
--> 238     return _do_perspective_warp(c, targ_pts, invert)
    239 symmetric_warp = TfmCoord(_symmetric_warp)

~\Anaconda3\lib\site-packages\fastai\vision\ in _do_perspective_warp(c, targ_pts, invert)
    223     "Apply warp to `targ_pts` from `_orig_pts` to `c` `FlowField`."
    224     if invert: return _apply_perspective(c, _find_coeffs(targ_pts, _orig_pts))
--> 225     return _apply_perspective(c, _find_coeffs(_orig_pts, targ_pts))
    227 def _perspective_warp(c, magnitude:partial(uniform,size=8)=0, invert=False):

~\Anaconda3\lib\site-packages\fastai\vision\ in _find_coeffs(orig_pts, targ_pts)
    204     B = FloatTensor(orig_pts).view(8)
    205     #The 8 scalars we seek are solution of AX = B
--> 206     return torch.gesv(B,A)[0][:,0]
    208 def _apply_perspective(coords:FlowField, coeffs:Points)->FlowField:

RuntimeError: b should have at least 2 dimensions, but has 1 dimensions instead

Hi Sanjaya,

I had a similar issue and after much searching this post solved it for me. Might be relevant if you’re using Conda/Windows10.

1 Like

Thanks, I managed to fix this error, I should’ve updated this thread. I’m running into a problem now when I try to create_cnn with a resnet. It hangs and never finished loading, no error messages, nothing. But if I use a vgg model it loads fine.

Edit: I fixed it by updating fastai through pip. I had to download the updated torch version through the torch website though. I couldn’t update either through conda. I had to stop using conda altogether.

I was able to finish lesson one with fastai in a conda environment. I originally had create_cnn also hang with a resnet but upgrading fastai and if I remember correctly, downgrading torch fixed the solution. You may have to install torch using a different conda channel. I would recommend using conda because it makes installation and management of multiple environments very easy…

Thanks rastringer for the tip…this indeed solves the error when first running the ImageDataBunch command for first timers, I updated to 1.0.42…just leaving this here to help beginners find a quick solution…
conda install -c conda-forge -c pytorch -c fastai fastai=1.0.42