@jcreinhold, thanks so much! I’ve got my data loading now. One thing, though; for some reason my images don’t pair up correctly. Do lists in python load arbitrarily (i.e., should I explicitly sort them)? I notice you don’t have any sorting… Looking at them in Files (Ubuntu), they are alphanumerically sorted, as I would expect.
(I should clarify that I’m also using self.itemsB[i] in my get method.)
UPDATE: Yes, adding a couple of sort()s at the top of my __init__() does the trick. Strange though…
I’ve been going through the documentation and apparently, there is no test or example on how to use ImageDataBunch.create_from_ll. I have tried many things and I am unable to solve the following issue for a vision task:
my data is already split up in train and valid sets.
I have a list of dictionary with both the path and the target/label of the image
create_from_ll requires a LabelLists so I loaded both my torchvision.datasets.VisionDataset into train_set and val_set
il_train = vision.data.ImageList([path.joinpath(sample['path']) for sample in train_set.data])
ll_train = LabelList(x=il_train, y=ItemList([sample['target'] for sample in train_set.data]))
il_test = vision.data.ImageList([path.joinpath(sample['path']) for sample in val_set.data])
ll_test = LabelList(x=il_test, y=ItemList([sample['target'] for sample in val_set.data]))
data = vision.ImageDataBunch.create_from_ll(LabelLists(path, ll_train, ll_test),
bs=args.batch_size, ds_tfms=vision.get_transforms(),
size=224, num_workers=args.workers).normalize(vision.imagenet_stats)
Up until here, everything is working, but as soon as I try to create a learner with vision.cnn_learner passing the above data object, I get the following error:
File "/home/fg/miniconda3/lib/python3.7/site-packages/fastai/vision/learner.py", line 97, in cnn_learner
model = create_cnn_model(base_arch, data.c, cut, pretrained, lin_ftrs, ps=ps, custom_head=custom_head,
File "/home/fg/miniconda3/lib/python3.7/site-packages/fastai/basic_data.py", line 122, in __getattr__
def __getattr__(self,k:int)->Any: return getattr(self.train_dl, k)
File "/home/fg/miniconda3/lib/python3.7/site-packages/fastai/basic_data.py", line 38, in __getattr__
def __getattr__(self,k:str)->Any: return getattr(self.dl, k)
File "/home/fg/miniconda3/lib/python3.7/site-packages/fastai/basic_data.py", line 20, in DataLoader___getattr__
def DataLoader___getattr__(dl, k:str)->Any: return getattr(dl.dataset, k)
File "/home/fg/miniconda3/lib/python3.7/site-packages/fastai/data_block.py", line 640, in __getattr__
raise AttributeError(k)
AttributeError: c
I admit I used the create_from_ll out of desperation since I was not able to find other people using it, and thus I am unsure about whether I passed something wrong. Apparently, using this method, does not set the c attribute, but then I suppose this means I am not supposed to use this method?
Is there a better method to create the ImageDataBunch? Or is there a way to fix this?
So I managed to get around this using the from_dfmethod but I had to create the dataframe just for this purpose!
I guess it solves the core problem but being able to pass torch DataLoader to the constructor seemed appealing, but I always end up with the previous AttributeError.
Hello! I’ve taken a look at your CycleGAN code. I’m quite interested to implement CycleGAN with TIFF images as well. However, I want to check if the TIFF images are normalized to between -1 and 1? I noticed your train_loss is very high, so I am wondering if it is because the values in your TIFF images are not normalized? If not, do you know how I can normalize the values in the TIFF images?