I was working on a pixel-level segmentation project with multiple classes using fastai. So in my example, there are 4 classes, and the information is encoded as integers 0 to 3.
I recently discovered when I resize my image in openCV, it would interpolate pixel values. So for example, my original mask may only has 0s and 3s, but after resizing I’m getting a small amount of 1s and 2s. While for an image this would be fine, this is a segmentation mask, and a different integer means a different class entirely, and I’m wondering if this is messing up my model.
Has anyone run into this issue? FYI, this resizing was done outside fastai using my own scripts, and now I’m wondering if the fastai resizing operations also introduce this artifact???
Obviously it was my fault (as usual). I had to do a little trick to be able to load RGB segmentation masks and convert them on the fly to grayscale. I wanted this feature because I wanted to be able to look at masks on disk “with the naked eye” and if you keep them in grayscale… well, you just see black. Anyway, I was doing this:
but it was not sufficient, Resize() in fastai/vision/augment.py was thinking those PILMasks were not true fastai.vision.core.PILMasks, so between the tuple resamples=(Image.BILINEAR, Image.NEAREST) it was applying Image.BILINEAR to my “false PILMasks”. I just added a cast to fastai.vision.core.PILMask at the end of my encodes() function and now it seems that Resize() behaves correctly.