Creating an Image Segmentation Dataset for Multiple Classes

(James) #1

I am using the fastai library to segment images with multiple classes on a personal dataset. Consequently, I have multiple masks per image. What is the best way to proceed? Do I consolidate the masks into a single image where the background is 0, and each subsequent class is assigned an integer (1, 2, 3 etc…)? Or do I extend the SegmentationDataset class to become more like ImageMultiDataset so that it can handle multiple masks? All the examples of the V1 library that I can find of segmentation are of single class segmentation datasets. Thank you in advance!

1 Like


The first solution is the best if the masks don’t overlap. There is an example with camvid of a segmentation problem with multiple classes.
If one pixel can belong to several classes though, you should have multiple channels in your mask.


(James) #3

Thank you for your prompt reply! For people who encounter a similar issue, the camvid lesson is linked below.

1 Like

(James) #4

I chose to focus on implementing a model that can segment one feature first before adding additional channels to handle overlapping features. When I call ImageBunch instance functions like show_batch() and normalize() on an ImageBunch object , I get an error along the lines of AttributeError: 'SegmentationDatasetOneFeature' object has no attribute 'normalize.' Why are the functions being called on the extended Dataset instead of the ImageBunch class?

class SegmentationDatasetOneFeature(SegmentationDataset):

def _get_y(self,i): 
    #checks if mask exists
    image_path = glob.glob(self.y[i] + "/abc.*")
    #if there is no abc, load an empty one 
    if len(image_path) == 0:
        tensor = torch.tensor((),
        tensor.new_zeros((sz, sz))
        return ImageSegment(Image(tensor))
        return open_mask(image_path[0])

data = DataBunch.create(train_ds=train_ds, valid_ds=valid_ds, test_ds=None,
bs=bs, num_workers=nw, tfms=tfms)

data = data.normalize(imagenet_stats)

AttributeError                            Traceback (most recent call last)
<ipython-input-30-f381a0614dd5> in <module>
----> 1 data = data.normalize(imagenet_stats)

~/anaconda3/envs/fastai-1.0/lib/python3.7/site-packages/fastai/ in __getattr__(self, k)
    114         return cls(*dls, path=path, device=device, tfms=tfms, collate_fn=collate_fn)
--> 116     def __getattr__(self,k:int)->Any: return getattr(self.train_dl, k)
    117     def holdout(self, is_test:bool=False)->DeviceDataLoader:
    118         "Returns correct holdout `Dataset` for test vs validation (`is_test`)."

~/anaconda3/envs/fastai-1.0/lib/python3.7/site-packages/fastai/ in __getattr__(self, k)
     50     def __len__(self)->int: return len(self.dl)
---> 51     def __getattr__(self,k:str)->Any: return getattr(self.dl, k)
     53     @property

~/anaconda3/envs/fastai-1.0/lib/python3.7/site-packages/fastai/ in DataLoader___getattr__(dl, k)
     34         super().__init__(len(classes))
---> 36 def DataLoader___getattr__(dl, k:str)->Any: return getattr(dl.dataset, k)
     37 DataLoader.__getattr__ = DataLoader___getattr__

AttributeError: 'SegmentationDatasetOneFeature' object has no attribute 'normalize'


That’s because DataBunch doesn’t have a normliaze function (so it’s trying to find it in its train_ds). You have to use an ImageDataBunch for this.

1 Like

(Hasib Zunair) #6

@Jamie which tool did you use to create the masks for the raw images?


(James) #7

Can you clarify what you mean by ‘raw images?’


(Hasib Zunair) #8

raw images are input features without any pre-processing


(hari rajeev) #9

In case you are looking for an annotation tool , try Labelme


(Antonio de Perio) #10

For a dataset that has multiple masks for a single image, is it also possible (and also is it recommended?) to create multiple training instances of that single image, paired with each image mask, rather than say combining all of the masks into one single mask where there are no overlaps? I guess this would increase training time? But would this improve or degrade accuracy?


(Antonio de Perio) #11

Also to add to the above, is there a way in fastai to combine these masks into one image in some sort of preprocessing step?