Multi class segmentation classes

[akadir (Abdulkadir Guner)]

My masks or only of unique values 0,1,2,3,4,5,6,7. So there need be 8 classes.
But why I am getting a 9th dimension in ‘class’ dimension of my get_preds results.
In other words get_preds[0] shape is : batchsize x 9 x h x w, where I have only 8 unique classes?
During training, my weighted loss function also complained if the weights is not 9 but 8 dimensional.
I am pretty sure, there are only 8 unique values in the masks, doublechecked several times.

And in relation to this, how to access to the classes if I have a dataloader from a datablock (imageblock, maskblock)?
Online examples are all about .classes method that works with databunch from SegmentationItemList and not a dataloadar object.

Thank you in advance.

[benkarr (Ben)]

I’d guess it’s a background/na label that is added by the loader automatically. What does learn.dls.vocab return?

[akadir (Abdulkadir Guner)]

@benkarr

learn.dls.vocab

…it returns an ‘attribute error’ for ‘vocab’

[akadir (Abdulkadir Guner)]

hard to grasp the need for that background NA (if exists here) if I only have 0,1,2,3,4,5,6,7, so 8 values in all the masks?

[akadir (Abdulkadir Guner)]

and attached how I did the simple mask values check…

[benkarr (Ben)]

Another wild guess, In your DataBlock you have MaskBlock(codes='codes.txt') could you load the codes into an array and pass that array to the MaskBlock? ‘codes.txt’ has 9 characters and I can’t see that the Block is able to load from paths.

Edit: The vocab should be at learn.dls.train.after_item.vocab

[akadir (Abdulkadir Guner)]

Thanks a lot:
DataBlock(blocks=(ImageBlock, MaskBlock(codes=np.loadtxt('codes.txt', dtype=str)))
solved.

You saved me one more time

[akadir (Abdulkadir Guner)]

I am also curiuos how you found that vocab attribute lying that deep?

[benkarr (Ben)]

I thought MaskBlock?? would be a good place to look at. It instanciates AddMaskCodes which sets a vocab attribute.

class AddMaskCodes(Transform):
    "Add the code metadata to a `TensorMask`"
    def __init__(self, codes=None):
        self.codes = codes
        if codes is not None: self.vocab,self.c = codes,len(codes)
    …

So we just have to find the place where the dataloader saves that transform.
Having a look at the attributes of learn.dls.train shows that AddMaskCodes is in a Pipeline at the after_item attribute. You can check which attributes an object has with learn.dls.train.__dict__ or vars(learn.dls.train).

learn.dls.train.__dict__

>> {'after_item': Pipeline: AddMaskCodes -> ToTensor,
 'before_batch': Pipeline: ,
 'after_batch': Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1} -> Normalize -- {'mean': tensor([[[[0.4850]],
 
          [[0.4560]],
          …
}

So by picking the first transform from the pipeline we get what we searched for: learn.dls.train.after_item[0].vocab.

learn.dls.train.after_item.vocab works because of… fastcore magic
I got that idea from making a symbolic search for vocab (Ctrl+t in VSCode)

The first thing that stood out to me was get_c since that is the function that derives the number of categories from your data/dataloaders to build the models classification layer and thus might also be a good place to look regarding your problem.

def get_c(dls):
    if getattr(dls, 'c', False): return dls.c
    if nested_attr(dls, 'train.after_item.c', False): return dls.train.after_item.c
    …

Since AddMaskCodes sets c as well as vocab you could expect that you can access the vocab in the same way as get_c accesses the count and it worked .

My response got quiet detailed but maybe someone finds that helpfull… basically you just guess and try a bunch of things and hope that it holds the information that you are searching for. The more you guess the more details you are going to pick up and your guesses become more educated

The code base might seem intimidating in the beginning, but it’s extremely worthwile to spend time in it since it speeds up your debuging process tremendously (that process took me ~1min.).

[akadir (Abdulkadir Guner)]

too good, thank you.
What I did was more like the ‘headless chicken’, where I make combinations of learner/dls/item/train/valid/c/classes/vocab including variations of ?, ?? and help(x) at the end and infront.
I am sharing this to show, ‘how not to do this’