Binary Segmentation: How to tell fastai to read images and masks as grayscale

Hi All,

fastai version: 2.3.1.

I want to tell fastai to read one channel grayscale images and segmentation masks. How this can be achieved?

My dls specification is as below:

binarySeg = DataBlock(blocks=(ImageBlock, MaskBlock(codes)),
get_items=get_image_files,
splitter=RandomSplitter(),
get_y=get_y,
item_tfms=[Resize(256)],
batch_tfms=[Normalize.from_stats(*imagenet_stats)])
dls = binarySeg.dataloaders(Path(‘dataset/barcodes’), bs=8)
dls.show

I will appreciate any guidance.
Many Thanks and
Kind Regards
Bilal

just use the PILImageBW class as a parameter to the blocks
blocks=(ImageBlock(PILImageBW), MaskBlock(codes))

Hi Narain,

Many thanks for the hint. While this will make the ImageBlock read black and white images but the output of the network that is specified by MaskBlock is still multi-channel. See the attached screenshot below:
image

How to fix this as well?

Many Thanks and
Kind Regards,
Bilal

If you want the output to have 1 channel then specify n_out=1 in the Unet Learner.

Hi Vishnu,

Thanks a lot for the input. After I specified n_out=1, lr_find and fit_one_cycle() is giving the following error:

Any idea of how to resolve this?

Many thanks and kind regards,
Bilal

When I take out the n_out=1 and keep using PILImageBW, everything works fine but the returned output is two-channel. However, a new issue is that I can’t export the model anymore. See the error below:

Got stuck now…

If you are using n_out=1 then you can change your loss function to BCEWithLogitsLossFlat. I am not sure why export is not working. You can try just saving the model weights, its just a PyTorch model.

Hi Vishnu,

Thanks for the reply. I will try changing the loss function.

For the export issue, I found its know problem with lambda functions and has been answered on the forum. I was able to resolve it. It has nothing to do with n_out or PILImageBW options.

Thanks again.

For n_out=1 and loss_func=BCEWithLogitsLossFlat given as below:

learn = unet_learner(dls, resnet34, metrics=[Dice], wd=1e-3, loss_func=BCEWithLogitsLossFlat, n_out=1)

The error has changed. Now I receive the following error:

You can try debugging to understand what is going wrong. Steps that I would use to debug

  1. Grab a batch of input and targets using dls.one_batch.
  2. Pass the input to learn.model.
  3. Grab output and pass to the loss function which you can access using learn.loss_fn
  4. Check where it is breaking and continue debugging.

Never did it before, but seems like a good plan to move forward and understand the causes of the issue.

I am on it. I will let you know my findings.

1 Like

Your issue here is BCELossLogits is a class, so you need to pass it in as such: loss_func = BCELossLogitsFlat()

To deal with your CUDA issues, follow what @bilalUWE said but make sure your data and model are on the CPU, so it gives you a better message

Hi @muellerzr,

Thanks for the hint. It made it to work successfully. This is how I initiated the learner:

learn = unet_learner(dls, resnet34, metrics=[Dice], wd=1e-3, loss_func=BCEWithLogitsLossFlat(), n_out=1)

But now the dice score has become 0. See below:

Overall, the model is learning to segment the barcodes.
Is this be the expected behaviour or I am doing something wrong?

Many Thanks and
Kind Regards,
Bilal

Looking at the code it appears that MaskBlock and TensorMask have no current functionality to convert to black and white. But it’s easy enough to copy the implementation of ImageBlock and PILImageBW to do it yourself like:

class PILMaskBW(PILMask): _open_args,_show_args = {'mode':'L'},{'alpha':0.5, 'cmap':'Greys'}
def MaskBlockBW(codes=None):
    return TransformBlock(type_tfms=PILMaskBW.create, item_tfms=AddMaskCodes(codes=codes), batch_tfms=IntToFloatTensor)

and then use it in your data block like:

binarySeg = DataBlock(blocks=(ImageBlock(PILImageBW), MaskBlockBW(codes)),
get_items=get_image_files,
splitter=RandomSplitter(),
get_y=get_y,
item_tfms=[Resize(256)],
batch_tfms=[Normalize.from_stats(*imagenet_stats)])

I tried this on CAMVID_TINY, and you do get the expected tensor shapes:

[ins] In [3]: x, y = dls.one_batch()

[ins] In [4]: x.shape
Out[4]: torch.Size([8, 1, 96, 128])

[ins] In [5]: y.shape
Out[5]: torch.Size([8, 96, 128])