Hi,
I’m trying to simply show a batch of a dataset I have created. My code is as follows (I’m removing a lot of fluff for simplicity’s sake, but I am confident the code running before this is not related to the error, I have checked):
from fastai.vision.all import *
from pathlib import Path
def random_seed(seed_value):
import random
random.seed(seed_value) # Python
import numpy as np
np.random.seed(seed_value) # cpu vars
import torch
torch.manual_seed(seed_value) # cpu vars
if torch.cuda.is_available():
torch.cuda.manual_seed(seed_value)
torch.cuda.manual_seed_all(seed_value) # gpu vars
torch.backends.cudnn.deterministic = True #needed
torch.backends.cudnn.benchmark = False
path = Path.cwd() / "data"
seed = np.random.randint(0, 1000)
random_seed(seed)
data = ImageDataLoaders.from_folder(data_path.parent, train="train", valid="valid", bs=32, num_workers=0)
data.one_batch()
Inside my data folder three folders exist: train, valid and test, each with three sub-folders corresponding to the categories for a classifying CNN I want to build. A sample of the kind of image that can be found in these folders can be seen here:
This is, actually, the first image one_batch is trying to show. I am using fastai freshly cloned from the github repo and have been trying to debug what is happening on my own, but despite my best efforts, all I can get is:
Could not do one pass in your dataloader, there is something wrong in it
Traceback (most recent call last):
File "/home/grcarmenaty/paper-1/doe.py", line 206, in <module>
model_training()
File "/home/grcarmenaty/paper-1/doe.py", line 198, in model_training
doe.run(func, ["Accuracy", "Damage threshold", "Improvement threshold"], repetitions=25)
File "/home/grcarmenaty/paper-1/taguchi.py", line 241, in run
result = func(list(experimental_run.iloc[row, :]) + [repetition])
File "/home/grcarmenaty/paper-1/doe.py", line 138, in func
seed, epoch = train_model(path, model, model_name, repetition)
File "/home/grcarmenaty/paper-1/doe.py", line 27, in train_model
data.show_batch()
File "/home/grcarmenaty/fastai/fastai/data/core.py", line 100, in show_batch
if b is None: b = self.one_batch()
File "/home/grcarmenaty/fastai/fastai/data/load.py", line 148, in one_batch
with self.fake_l.no_multiproc(): res = first(self)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/basics.py", line 547, in first
return next(x, None)
File "/home/grcarmenaty/fastai/fastai/data/load.py", line 109, in __iter__
for b in _loaders[self.fake_l.num_workers==0](self.fake_l):
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
data = self._next_data()
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 561, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 34, in fetch
data = next(self.dataset_iter)
File "/home/grcarmenaty/fastai/fastai/data/load.py", line 118, in create_batches
yield from map(self.do_batch, self.chunkify(res))
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/basics.py", line 216, in chunked
res = list(itertools.islice(it, chunk_sz))
File "/home/grcarmenaty/fastai/fastai/data/load.py", line 133, in do_item
try: return self.after_item(self.create_item(s))
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/transform.py", line 200, in __call__
def __call__(self, o): return compose_tfms(o, tfms=self.fs, split_idx=self.split_idx)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/transform.py", line 150, in compose_tfms
x = f(x, **kwargs)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/transform.py", line 73, in __call__
def __call__(self, x, **kwargs): return self._call('encodes', x, **kwargs)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/transform.py", line 83, in _call
return self._do_call(getattr(self, fn), x, **kwargs)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/transform.py", line 90, in _do_call
res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/transform.py", line 90, in <genexpr>
res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/transform.py", line 89, in _do_call
return retain_type(f(x, **kwargs), x, ret)
File "/home/grcarmenaty/paper-1-env/lib/python3.8/site-packages/fastcore/dispatch.py", line 118, in __call__
return f(*args, **kwargs)
File "/home/grcarmenaty/fastai/fastai/vision/core.py", line 223, in encodes
def encodes(self, o:PILBase): return o._tensor_cls(image2tensor(o))
File "/home/grcarmenaty/fastai/fastai/vision/core.py", line 93, in image2tensor
res = tensor(img)
File "/home/grcarmenaty/fastai/fastai/torch_core.py", line 134, in tensor
else as_tensor(x, **kwargs) if hasattr(x, '__array__') or is_iter(x)
RuntimeError: Could not infer dtype of PILImage
I don’t know how to proceed further, I have checked everything I know to check. My most elaborate attempt was converting the PIL image to a numpy array, forcing it to have dtype np.uint8 and converting it back to PIL image just before fastai/fastai/torch_core.py line 134. Nothing has worked so far.
I must add this exact same code with the exact same input data works on Windows 10 in an environment with fastai 2.3.1 and python 3.8.10 created through conda, but not in my Ubuntu 20.04 with an environment created through python venv running fastai from the github repo and python 3.8.10 just the same. I have tried to do all this with fastai 2.3.1 in Ubuntu to no avail.
If someone can help I will be grateful, and I hope someone can benefit from this, I haven’t found any helpful information regarding this situation. If additional data is needed I remain at your disposal.