Image Classifier with Multiple Data Loaders

Part 1 (2020)
1

I am currently working on building an image classifier that trains using data from two different dataloaders. I have a bunch of images in a dataframe with their labels all in a csv file and another set of images in labeled folders in my google drive. So, because the two images are stored in different settings, I use different datablocks to build dataloaders for each.

I’ve been trying to figure out how to get a working dataloader for the two - they both have the exact same item and batch transforms and yet I am getting errors. I’m not sure what I am missing or what the best solution might be - everything I’ve seen on the topic focuses on multi-modal problems which are a different level of complexity.

Here’s an excerpt of my current (failing) attempt -
dblock = DataBlock(blocks=(ImageBlock, MultiCategoryBlock),
splitter=splitter,
get_x=get_x,
get_y=get_y,
item_tfms=Resize(448, ResizeMethod.Pad, pad_mode=‘zeros’),
batch_tfms=aug_transforms(size=224, min_scale=0.75))
dls1 = dblock.dataloaders(df)

dblock2 = DataBlock(blocks = (ImageBlock, MultiCategoryBlock),
get_items=get_image_files,
splitter=RandomSplitter(seed=42),
get_y=Pipeline([using_attr(RegexLabeller(r’(.+)_\d+.jpg$’), ‘name’), lambda label: [label]]),
item_tfms=Resize(448, ResizeMethod.Pad, pad_mode=‘zeros’),
batch_tfms=aug_transforms(size=224, min_scale=0.75))
dls2 = dblock2.dataloaders(path2/“images”)

dls = DataLoaders(dls1, dls2)

learn = cnn_learner(dls, resnet50, metrics=partial(accuracy_multi, thresh=0.2))
learn.fine_tune(3, base_lr=3e-3, freeze_epochs=4)

Is there a thread or tutorial on this that I keep missing?

2

You need to go a step lower in the API.

Create two Datasets. Merge the two Datasets together. (Literally just dset_merged = dset1 + dset2 I believe). Make DataLoaders.

The Dataset level gathers all the information needed to build your data, and so from there everything is setup to where you just need your item and batch transforms.

2 Likes
3

Thank you for the fast response!

Is there any example of this anywhere? I still have a few questions. First, I ran into this - “TypeError: unsupported operand type(s) for +: ‘Datasets’ and ‘Datasets’” so I’m unclear on the correct operation for merging. Is that because I built my Datasets out of a DataBlock which isn’t really going a step lower in the API? If I do build datasets directly from the Datasets class, can that still work if one of my sets is a multi-category block of data?

4

I did say think, so wasn’t 100% sure that would work. IIRC (you can check and verify this), dset.train (dset is your generated Datasets) is of type L. If so, then you can do this:

d_train = dset1.train + dset2.train
d_valid = dset1.valid + dset2.valid

train_dl = TfmdDL(d_train, after_item=item_tfms (make sure to include ToTensor() in this list), after_batch=after_batch (make sure to include IntToFloatTensor() here), shuffle=True, drop_last=True)

Repeat the above for the valid but set shuffle to False and drop_last to False

L’s have a shortcut to append via the +

2 Likes
5

That doesn’t work either. It looks like the training and validation sets are also not of type L. I’m not sure what other steps to take other than building my datasets in straight PyTorch and seeing if I can merge them from there.