MIL question - variable length bags of images?

fastai
1

I’m working on a weakly-supervised learning project where I have bags of images and one or more labels for each bag (but not labels for each image).

I have developed a pytorch dataloader that loads several bags of images into a single 4D tensor, X, with sizes ( total # images, C, H, W). The dataloader also returns a 1D tensor, bagids, with size total # of images that contains an ID number for each bag.

My pytorch model expects a single input so call it like this: model( input ) where input is a tuple (X, bagids).

How can I get fastai to pass both X and bagids to the model? I’m not using the datablock API so I don’t know how to specify the number of inputs. I’ve looked at the Siamese and Image Sequences tutorials but in both of those the sequences (bags) have fixed length so it’s unnecessary to have an auxiliary tensor like bagids.

Any help or advice would be most appreciated.

2

This is a good question, just curious, what is your project on?

If you already have a PyTorch DataLoader already, you can pass it into fastai’s DataLoaders and then pass it into the Learner.

3

I guess you can set your dls.n_inp = 2 in your case.

Or you can use the midlevel API to build the dataloaders yourself. fastbook/11_midlevel_data.ipynb at master · fastai/fastbook · GitHub

Or you can just build your Dataloader in Pytorch and use fastai just for the training: Migrating PyTorch - Verbose | fastai

Hope it helps,

4

Thank you both for your suggestions. I’ll post again once I have something working.

Tanishq - each bag contains multiple ultrasound images from a single patient and we are trying to detect cancer within each bag.

1 Like