Binary Segmentation: How to tell fastai to read images and masks as grayscale

fastai
1

Hi All,

fastai version: 2.3.1.

I want to tell fastai to read one channel grayscale images and segmentation masks. How this can be achieved?

My dls specification is as below:

binarySeg = DataBlock(blocks=(ImageBlock, MaskBlock(codes)),
get_items=get_image_files,
splitter=RandomSplitter(),
get_y=get_y,
item_tfms=[Resize(256)],
batch_tfms=[Normalize.from_stats(*imagenet_stats)])
dls = binarySeg.dataloaders(Path(‘dataset/barcodes’), bs=8)
dls.show

I will appreciate any guidance.
Many Thanks and
Kind Regards
Bilal

2

just use the PILImageBW class as a parameter to the blocks
blocks=(ImageBlock(PILImageBW), MaskBlock(codes))

3

Hi Narain,

Many thanks for the hint. While this will make the ImageBlock read black and white images but the output of the network that is specified by MaskBlock is still multi-channel. See the attached screenshot below:
image

How to fix this as well?

Many Thanks and
Kind Regards,
Bilal

4

If you want the output to have 1 channel then specify n_out=1 in the Unet Learner.

5

Hi Vishnu,

Thanks a lot for the input. After I specified n_out=1, lr_find and fit_one_cycle() is giving the following error:

image
image1946×718 96.2 KB

Any idea of how to resolve this?

Many thanks and kind regards,
Bilal

6

When I take out the n_out=1 and keep using PILImageBW, everything works fine but the returned output is two-channel. However, a new issue is that I can’t export the model anymore. See the error below:

image
image1662×734 100 KB

Got stuck now…

7

If you are using n_out=1 then you can change your loss function to BCEWithLogitsLossFlat. I am not sure why export is not working. You can try just saving the model weights, its just a PyTorch model.

8

Hi Vishnu,

Thanks for the reply. I will try changing the loss function.

For the export issue, I found its know problem with lambda functions and has been answered on the forum. I was able to resolve it. It has nothing to do with n_out or PILImageBW options.

Thanks again.

9

For n_out=1 and loss_func=BCEWithLogitsLossFlat given as below:

learn = unet_learner(dls, resnet34, metrics=[Dice], wd=1e-3, loss_func=BCEWithLogitsLossFlat, n_out=1)

The error has changed. Now I receive the following error:

image
image1516×778 83.1 KB

10

You can try debugging to understand what is going wrong. Steps that I would use to debug

  1. Grab a batch of input and targets using dls.one_batch.
  2. Pass the input to learn.model.
  3. Grab output and pass to the loss function which you can access using learn.loss_fn
  4. Check where it is breaking and continue debugging.
11

Never did it before, but seems like a good plan to move forward and understand the causes of the issue.

I am on it. I will let you know my findings.

1 Like
12

Your issue here is BCELossLogits is a class, so you need to pass it in as such: loss_func = BCELossLogitsFlat()

To deal with your CUDA issues, follow what @bilalUWE said but make sure your data and model are on the CPU, so it gives you a better message

13

Hi @muellerzr,

Thanks for the hint. It made it to work successfully. This is how I initiated the learner:

learn = unet_learner(dls, resnet34, metrics=[Dice], wd=1e-3, loss_func=BCEWithLogitsLossFlat(), n_out=1)

But now the dice score has become 0. See below:

image
image778×1394 106 KB

Overall, the model is learning to segment the barcodes.
Is this be the expected behaviour or I am doing something wrong?

Many Thanks and
Kind Regards,
Bilal

14

Looking at the code it appears that MaskBlock and TensorMask have no current functionality to convert to black and white. But it’s easy enough to copy the implementation of ImageBlock and PILImageBW to do it yourself like:

class PILMaskBW(PILMask): _open_args,_show_args = {'mode':'L'},{'alpha':0.5, 'cmap':'Greys'}
def MaskBlockBW(codes=None):
    return TransformBlock(type_tfms=PILMaskBW.create, item_tfms=AddMaskCodes(codes=codes), batch_tfms=IntToFloatTensor)

and then use it in your data block like:

binarySeg = DataBlock(blocks=(ImageBlock(PILImageBW), MaskBlockBW(codes)),
get_items=get_image_files,
splitter=RandomSplitter(),
get_y=get_y,
item_tfms=[Resize(256)],
batch_tfms=[Normalize.from_stats(*imagenet_stats)])

I tried this on CAMVID_TINY, and you do get the expected tensor shapes:

[ins] In [3]: x, y = dls.one_batch()

[ins] In [4]: x.shape
Out[4]: torch.Size([8, 1, 96, 128])

[ins] In [5]: y.shape
Out[5]: torch.Size([8, 96, 128])
1 Like
15

I’m having problems with my grayscale B&W RGB images and found this in a search. I added it to my code and am finally seeing train_loss and valid_loss values, they were 0 before and the model failed a prediction of any of the training images. I NEVER would have thought of this as a solution because I don’t see this level of explanation anywhere, in the API or the book. There’s a lot to this.

Just for grins, if you’re still paying any attention, where did you pick up this nugget of wisdom?

Thanks, Jeff