Can't do Segmentation: CUDA error: device-side assert triggered

[renato (Renato Hermoza)]

When defining the DataBunch, have you specified the classes?

[imanol (Imanol Pinto)]

I have found the solution. The problem is that my masks have the following values: [0, 255], and this is not supported by default by fast.ai. The problem is that the fast.ai function open_mask works with small mask pixel values like 0,1,2,3 by default. This function is called by the SegmentationLabelList class. To work with 255 values we should call the function open_mask with div=True: open_mask(fn, div=True). It divides the mask pixel values by 255.

In order to change the default behaviour of open_mask, I did the following:

class SegLabelListCustom(SegmentationLabelList):
    def open(self, fn): return open_mask(fn, div=True)

class SegItemListCustom(SegmentationItemList):
    _label_cls = SegLabelListCustom

codes = ['0','1']
src = (SegItemListCustom.from_folder(path_img)
       .random_split_by_pct(valid_pct=0.2, seed=33)
       .label_from_func(get_y_fn, classes=codes))

data = (src.transform(get_transforms(), size=size, tfm_y=True)
        .databunch(bs=bs)
        .normalize(imagenet_stats))

Hope this helps. Thank you for your replies!

[alex_zhang]

Hi, I used your codes, the error gone, but now by viewing data.show_batch, I found there is no mask. And if try" lr_find(learn);learn.recorder.plot()" after the learner created, the loss is 0, which confirm that the mask is gone…
How to fix that? Thanks

[alex_zhang]

I used “get_y_fn = lambda x: pathmask_train/f’{x.stem}_json.png’” before modification

[hasif]

Thanks @imanol, it works!

[ingbiodanielh (Danielh Carranza)]

I have the same issue, Did you solve it?

[alex_zhang]

Hey, guys! I’ve just partly fixed the problem(fix the problem using COCO dataset). Now, I can use lr_find(learn) and learn.recorder.plot() without error. Also see this link:Image Segmentation on COCO dataset - summary, questions and suggestions.

I think there are 2 problems in this error: 1)The value of the mask array should be {0,1}, not {0,255}or some other stuff, that’s what many other people said. The link above find ways to generate {0,1} mask. But how to fix it by not transforming data into COCO form, I haven’t tried; 2)The class list. Mine is a bynary segmentation project(to segment “row” ), so CATEGORY_NAMES=[0, ‘row’]. If CATEGORY_NAMES=[‘row’], I found the loss would always be 0, when using lr_find(learn) and learn.recorder.plot().
I think there still are something more to figure out, but for now, at leat I can use unet and lr_find(learn).

[alex_zhang]

I transformed data to COCO, and added 0 to class list(CATEGORY_NAMES=[0, ‘row’], instead[‘row’], e.g.), and it works.
I also updated fastai to 1.0.48, then the data.show_batch error disappeared, though I don’t know why.

[massaros (Simone Massaro)]

It works! thank you

[hotessy (Divyanshu Sharma)]

It works!
Should’t it be reported as a bug ?

[ptrampert (Patrick Trampert)]

Have a look here for those that still have problems:

Unet Segmentation Mask Converter to help against common errors / problems

Many seem to have problems with Segmentation and the used masks. Because of that I wrote some functions that can help anybody to overcome the problems. I did not test it thoroughly, only one RGB and one gray scaled image, where it worked as expected. You only have to provide the path, where the labels / masks are as well as a path to a folder where you want to save the converted masks. The new files will be named as the old ones. Converted masks contain only values from 0 to N, where N is the n…

[nithinnivi (Nithin Varghese)]

class SegLabelListCustom(SegmentationLabelList): def open(self, fn): return open_mask(fn, div=True) class SegItemListCustom(SegmentationItemList): _label_cls = SegLabelListCustom codes = [‘0’,‘1’] src = (SegItemListCustom.from_folder(path_img) .random_split_by_pct(valid_pct=0.2, seed=33) .label_from_func(get_y_fn, classes=codes)) data = (src.transform(get_transforms(), size=size, tfm_y=True) .databunch(bs=bs) .normalize(imagenet_stats))

Thanks so much, much needed one

[eljas1]

class SegLabelListCustom(SegmentationLabelList):
    def open(self, fn): return open_mask(fn, div=True)

class SegItemListCustom(SegmentationItemList):
    _label_cls = SegLabelListCustom

codes = ['0','1']
src = (SegItemListCustom.from_folder(path_img)
       .random_split_by_pct(valid_pct=0.2, seed=33)
       .label_from_func(get_y_fn, classes=codes))

data = (src.transform(get_transforms(), size=size, tfm_y=True)
        .databunch(bs=bs)
        .normalize(imagenet_stats))

Is it normal for this code to take quite a while to run? I’ve had it running for well over an hour with just 5000 128x128 images in the dataset. Hoping it will work but I’m not sure what’s going on. I only changed the paths.

[moeinh77 (moein hasani)]

t

Did u manage to fix the show batch problem ?

[alex_zhang]

Just update fastai, then show batch will be ok

[moeinh77 (moein hasani)]

I updated the fastai to 1.0.55 but still no mask is shown

[alex_zhang]

Sorry, I didn’t figure out why it worked, but it worked when I was using kaggle kernel

[eljas1]

I had the same problem. Turned out I had saved the modified images in the wrong format with floats

[MassimilianoG]

Hi, I’m trying to segmet some images where masks have data between [0,255]; after a lot of searches I tried your code that seems to solve my problem, but when I try do display data:

data.show_batch(2, figsize=(10,7))

I got this error:

BrokenPipeError: [Errno 32] Broken pipe

Any ideas? Thanks for your help.

(fastai version: 1.0.55)

[the_ccalderon (Cesar Calderon Muro)]

class SegLabelListCustom(SegmentationLabelList): def open(self, fn): return open_mask(fn, div=True) class SegItemListCustom(SegmentationItemList): _label_cls = SegLabelListCustom

Thanks @imanol this fixed my problem!