Classifier with multiple images as input and multiple labels as output

Part 1 (2019)
5

I’ve ported the original example to data_block api. I’ve tried to do that some times in the last weeks, trying to follow the evolution of library, but now I’ve to say that the actual version of fastai is very easy to customize.
The code that let you read a multi channel image is only:

def openMultiChannelImage(fpArr):
    '''
    Open multiple images and return a single multi channel image
    '''
    mat = None
    nChannels = len(fpArr)
    for i,fp in enumerate(fpArr):
        img = PIL.Image.open(fp)
        chan = pil2tensor(img, dtype='float').div_(255)
        if(mat is None):
            mat = torch.zeros((nChannels,chan.shape[1],chan.shape[2]))
        mat[i,:,:]=chan
    return Image(mat)
    
class MultiChannelImageItemList(ImageItemList):
    def open(self, fn):
        return openMultiChannelImage(fn)

So you can use the new “MultiChannelImageItemList” to create the multi channel image reader, both for the training and test set.

il = MultiChannelImageItemList.from_df(path=path, df=train_data_and_labels_df, cols=x_cols)
ils1 = il.random_split_by_pct()
ils2 = ils1.label_from_df(cols=y_cols) 
test_items = MultiChannelImageItemList.from_df(path=path, df=test_df, cols=x_cols) 
ils3 = ils2.add_test(items=test_items)
ils4 = ils3.transform(tfms, size=size)
data = ils4.databunch(bs=bs)

After that you’ve only to adapt your model to accept different input (ie: 4 channels instead of 3).

You can find a complete working example on:

15 Likes