I am using the fastai library to segment images with multiple classes on a personal dataset. Consequently, I have multiple masks per image. What is the best way to proceed? Do I consolidate the masks into a single image where the background is 0, and each subsequent class is assigned an integer (1, 2, 3 etc…)? Or do I extend the SegmentationDataset class to become more like ImageMultiDataset so that it can handle multiple masks? All the examples of the V1 library that I can find of segmentation are of single class segmentation datasets. Thank you in advance!
The first solution is the best if the masks don’t overlap. There is an example with camvid of a segmentation problem with multiple classes.
If one pixel can belong to several classes though, you should have multiple channels in your mask.
Thank you for your prompt reply! For people who encounter a similar issue, the camvid lesson is linked below.
I chose to focus on implementing a model that can segment one feature first before adding additional channels to handle overlapping features. When I call ImageBunch instance functions like show_batch() and normalize() on an ImageBunch object , I get an error along the lines of
AttributeError: 'SegmentationDatasetOneFeature' object has no attribute 'normalize.' Why are the functions being called on the extended Dataset instead of the ImageBunch class?
def _get_y(self,i): #checks if mask exists image_path = glob.glob(self.y[i] + "/abc.*") #if there is no abc, load an empty one if len(image_path) == 0: tensor = torch.tensor((), dtype=torch.int) tensor.new_zeros((sz, sz)) return ImageSegment(Image(tensor)) else: return open_mask(image_path)
data = DataBunch.create(train_ds=train_ds, valid_ds=valid_ds, test_ds=None,
bs=bs, num_workers=nw, tfms=tfms)
data = data.normalize(imagenet_stats)
--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-30-f381a0614dd5> in <module> ----> 1 data = data.normalize(imagenet_stats) ~/anaconda3/envs/fastai-1.0/lib/python3.7/site-packages/fastai/basic_data.py in __getattr__(self, k) 114 return cls(*dls, path=path, device=device, tfms=tfms, collate_fn=collate_fn) 115 --> 116 def __getattr__(self,k:int)->Any: return getattr(self.train_dl, k) 117 def holdout(self, is_test:bool=False)->DeviceDataLoader: 118 "Returns correct holdout `Dataset` for test vs validation (`is_test`)." ~/anaconda3/envs/fastai-1.0/lib/python3.7/site-packages/fastai/basic_data.py in __getattr__(self, k) 49 50 def __len__(self)->int: return len(self.dl) ---> 51 def __getattr__(self,k:str)->Any: return getattr(self.dl, k) 52 53 @property ~/anaconda3/envs/fastai-1.0/lib/python3.7/site-packages/fastai/basic_data.py in DataLoader___getattr__(dl, k) 34 super().__init__(len(classes)) 35 ---> 36 def DataLoader___getattr__(dl, k:str)->Any: return getattr(dl.dataset, k) 37 DataLoader.__getattr__ = DataLoader___getattr__ 38 AttributeError: 'SegmentationDatasetOneFeature' object has no attribute 'normalize'
DataBunch doesn’t have a
normliaze function (so it’s trying to find it in its
train_ds). You have to use an
ImageDataBunch for this.
@Jamie which tool did you use to create the masks for the raw images?
Can you clarify what you mean by ‘raw images?’
raw images are input features without any pre-processing
In case you are looking for an annotation tool , try Labelme
For a dataset that has multiple masks for a single image, is it also possible (and also is it recommended?) to create multiple training instances of that single image, paired with each image mask, rather than say combining all of the masks into one single mask where there are no overlaps? I guess this would increase training time? But would this improve or degrade accuracy?
Also to add to the above, is there a way in fastai to combine these masks into one image in some sort of preprocessing step?