Transforms when the target is a mask in the fastai library

Part 2 & Alumni (2018)
1

I know we will cover this later in the course with the Carvana data set, but I wanted to try a few things on the Data Science Bowl competition on Kaggle.

There is a weird thing happening when you create an ImageData object and your desired targets are masks for the image. Specifically my code is

tfms = tfms_from_model(resnet34, size, crop_type=CropType.NO, tfm_y=TfmType.PIXEL, aug_tfms=aug_tfms)
datasets = ImageData.get_ds(ImagesMasksDataset, (trn_img,trn_msk), (val_img,val_msk), tfms, path=PATH)
md = ImageData(PATH, datasets, batch_size, num_workers=0, classes=None)

The TfmType.PIXEL seems perfect for this since I want all the transformations done on the inputs to be done on the masks as well. And it works like a charm on all the data augmentation operations as well as the resizing of the images, but it also does two annoying things: it flips the channels (I can live with that) and it normalizes the image, like it does for the inputs.
The normalization changes all the values of the target mask which we don’t want (or if we do, I really don’t see why).

Looking at the carvana notebook I tried to figure out a way to turn it off but I missed it completely so if someone knows how to do it easily, please let me know. For now, the hackey thing I did to get going is adding a line

tfms = tfms_from_model(resnet34, size, crop_type=CropType.NO, tfm_y=TfmType.PIXEL, aug_tfms=aug_tfms)
list(tfms[0].tfms)[-2].tfm_y = TfmType.NO
datasets = ImageData.get_ds(ImagesMasksDataset, (trn_img,trn_msk), (val_img,val_msk), tfms, path=PATH)
md = ImageData(PATH, datasets, batch_size, num_workers=0, classes=None)

since I saw the Normalize Transform was the second to last in the list of transforms, and I chose an option where it wouldn’t do anything.

1 Like
2

Yeah I haven’t yet created a function wrapper for segmentation/classification. For now you’ll simply need to create the transform object yourself, rather than using tfms_from_model. (Or just replacing or editing a single transform from there, as you’ve done, is fine too).

1 Like
3

Got it, thanks!

4

This means fastai lib has not functions to define masks as targets?
Thanks

5

There has been some changes since then, and you should use tfm_y = TfmType.CLASS when your target is a mask.

2 Likes
6

Thanks!

7

upon doing so i consumes meomory like any thing take too long for iterations…
is it faced by every one… or m i missing some thing… ???