Unet_binary segmentation

(Daniel Grzenda) #48

This was such a helpful post. Thank you! I had to change the code a little to get it to work, because I think ImageItemList has been removed. Documenting it here for anyone else who finds this thread:

class SegLabelListCustom(SegmentationLabelList):
    def open(self, fn): return open_mask(fn, div=True)

class SegItemListCustom(SegmentationItemList):
    _label_cls = SegLabelListCustom
1 Like

(Miro) #49


I have RGB label-images with 6 labels.

codes = array([‘Impervious_surfaces’,‘Building’,‘Low_vegetation’,‘Tree’,‘Car’,‘Clutter_background’])

below is hoe they are encoded in RGB channel.

colors_LU ={‘Impervious_surfaces’: array([255, 255, 255]),
‘Building’: array([ 0, 0, 255]),
‘Low_vegetation’: array([0, 255, 255]),
‘Tree’: array([0, 255, 0]),
‘Car’: array([ 255, 255, 0]),
‘Clutter_background’: array([ 255, 0, 0])}

mask = open_mask(get_y_fn(img_f))
mask.show(figsize=(5,5), alpha =1)

I get…mask.data

tensor([[[255, 255, 255, …, 255, 255, 255],
[255, 255, 255, …, 255, 255, 255],
[255, 255, 255, …, 255, 255, 255],
[ 29, 29, 29, …, 255, 255, 255],
[ 29, 29, 29, …, 255, 255, 255],
[ 29, 29, 29, …, 255, 255, 255]]]))

So not for 0 to 5.
By examining the I figured that there are indeed 6 classes inside, which seem to be gray representations of RGB colors for each class.

{‘Impervious_surfaces’: 255,
‘Building’: 29,
‘Low_vegetation’: 178,
‘Tree’: 149,
‘Car’: 225,
‘Clutter_background’: 76}

I am using following to create the data source

src = (SegmentationItemList.from_folder(path)
.split_by_folder(train = ‘train’, valid = ‘valid’)
.label_from_func(get_y_fn, classes=codes))

data = (src.transform(get_transforms(), tfm_y=True, size = size)

This creates a problem where numbers in the mask are > number of classes, so I was getting that CUDA error.

I got around that by creating dummy classes (0-255), but then I start running out of memory, even with image size reduced to 200.

I do not know how to solve this problem. Dividing by 255 does not help me, since I have > 2 classes.

Any help would be much appreciated.


(Welton Rodrigo Torres Nascimento) #50


Not sure if I should start a new thread or post here. Please let me know.

I’m trying lesson 3 with road camera vehicle photos. The task here is creating a privacy mask hiding the inside of the vehicles.

I’m working in segmentating the windows. Right seems promising.

My doubt is: when an image contains no windows, should I present it to the network? With a full zeroed mask? Or should I just use photos where a window is presented?

As an aside, how accurate should I be when annotating data? There is a need to be pixel perfect?

Thank you.


(Patrick Trampert) #51

Transform your RGB masks into gray scale masks with values from 0 to 5 (for 6 classes). Use either Python or ImageJ. Using the transformed masks it should work.


(RobG) #52

Yes you should. Otherwise your model will see windows everywhere. But getting the percentage of null images right is hard to guess without experimentation.

Are these external road cameras? I think I would have approached it by segmenting non-window parts of vehicles to discover internal windows. If internal dash cams, I would detect the inside of the car like dashboards and passengers rather than the window.

Good luck!!


(Patrick Trampert) #53

I wrote some functions to help people having problems with labels in their masks. See here:

1 Like

(Miro) #54

thank you for great feedback. Images I am using are from Potsdam satellite image set.

All the pixels are labeled as something.

In case it may be of some use, this is the code I used to convert RGB lables.

files = get_image_files(path + ‘/original_RGB_lables’)
for file in files:
temp = Image.open(file)
temp = temp.convert(‘L’)
reshape = int(pixels.shape[0] ** .5)
pixels = pixels.reshape(reshape,reshape)
pixels = pixels.astype(int)
pixels = np.where(pixels==255,0,pixels)
pixels = np.where(pixels==225,4,pixels)
pixels = np.where(pixels==178,2,pixels)
pixels = np.where(pixels==149,3,pixels)
pixels = np.where(pixels==76,5,pixels)
pixels = np.where(pixels==29,1,pixels)
#for k in range(7):
# print(str(k) + ‘:\t’ + str(list(pixels.flatten()).count(k)))
array = np.array(pixels, dtype=np.uint8)
new_image = Image.fromarray(array)
#print (pixels)
new_image = new_image.convert(‘L’)
new_image.save(str(path) + ‘/monochrome_lables’ + ‘/’ + str(file.name))
#mask = open_mask(str(path) + ‘/monochrome_lables’ + ‘/’ + str(file.name))
#mask.show(figsize=(5,5), alpha =1)

after this I still got:
cuda error: device-side assert triggered. Apparently, this happens if number in the mask > num_classes.

After manually examining labelled images, it turned out that one of them (label 4_12) had values that were not in [0,5] range. I think something is wrong with this RGB labeled image. I removed it from the set and now it is training…

I’ll investigate latter what is going on with 4_12 image-label file, but in case someone plans to work with the same data set, pay attention to that.

One other thing I had to do is to label ‘background/clutter’ calls as void and exclude it from target matching in the accuracy function. Otherwise it was running out of memory.

codes = array([‘Impervious_surfaces’,‘Building’,‘Low_vegetation’,‘Tree’,‘Car’,‘Clutter_background’])

thanks again

1 Like

(hari rajeev) #55

thank you, post the RGB label conversion , you didn’t have to use “div=True” right ?


(Imran) #56

I have a unique problem

mask = open_mask(get_y_fn(img_f),div=False)

Gives me this


But, this gives me a blank images without annotation, for the same file

mask = open_mask(get_y_fn(img_f),div=True)


However, if I do this

mask = open_mask(get_y_fn(img_f),div=True)

I get the complete mask with the segmentation

Am I missing something? Please advise.


(Imran) #57

Found it, so PixelAnnotationTool creates mask by marking the pixel value of the masked area with the corresponding id value of the label being used. In my case, I used “road marking” with an id of 34, the mask area pixels were numbered as 34 and the other area with 0.
When i used div=true, it divided the entire mask by 255, making the mask area with also as 0.
Sorted this out by creating a custom open_image function where 34 was used as divisor instead of 255.


(Divyanshu Sharma) #58

In which method does the div=True argument go if I use the DataBlock API ?

I’m using v1.0.51



Nowhere directly now. You should subclass SegmentationLabelList and write the open function to fit your need, probably

def open(self, fn): return open_mask(fn, div=True)

(Divyanshu Sharma) #60

Oh, I thought the changes had been pushed.

I picked this version of the solution already from your answers in this thread. Thanks! :slight_smile:


(hari rajeev) #61

I have input images of size 4250 * 5500 , need to apply segmentation on it for a certain use case.
Do i need to bring down the resolution to say 960 * 720 ?. Or is it purely dependent on what will fit in to the GPU memory ?.
These are document images, what will be the best augmentation technique that can be applied on these images ?

1 Like

(RobG) #62

You can use large sizes if they fit in memory. I’ve worked with bs of 1 and 2.

Apart from bringing down resolution, another option is to use offset tiles. Practically, especially on inference, tiling has an advantage as Unet can struggle with edge accuracy. I just keep centre of tile predictions.

It all depends on your use case and I suspect size of objects in the image. Same goes for augmentations. Often you don’t need to be pixel perfect and downsizing is appropriate.



Agree, this is very helpful, got what I needed to work for Segmenting some cells. Did anyone try and deploy a serving of this? I can get it working locally on my Flask application but just including the above:

class SegLabelListCustom(SegmentationLabelList):
def open(self, fn): return open_mask(fn, div=True)

class SegItemListCustom(SegmentationItemList):
_label_cls = SegLabelListCustom

In my Flask app prior to loading the .pkl:

learn = load_learner(path, 'cell_export.pkl')

But when I try and deploy on Render (using Gunicorn), the same Flask app gives the following error:

  File "/opt/render/project/src/.venv/bin/gunicorn", line 10, in <module>
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/app/base.py", line 223, in run
    super(Application, self).run()
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/app/base.py", line 72, in run
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/arbiter.py", line 60, in __init__
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/arbiter.py", line 120, in setup
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/gunicorn/util.py", line 350, in import_app
  File "/opt/render/project/src/app.py", line 49, in <module>
    learn = load_learner(path, export_file_name)
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/fastai/basic_train.py", line 598, in load_learner
    state = torch.load(source, map_location='cpu') if defaults.device == torch.device('cpu') else torch.load(source)
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/torch/serialization.py", line 387, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)
  File "/opt/render/project/src/.venv/lib/python3.7/site-packages/torch/serialization.py", line 574, in _load
    result = unpickler.load()
AttributeError: Can't get attribute 'SegItemListCustom' on <module '__main__' from '/opt/render/project/src/.venv/bin/gunicorn'>

It seems like Gunicorn is not reading the custom classes. I have been searching StackOverflow but have not found an answer.


(hari rajeev) #64

has anybody tried comparing U-NET with Mask-RCNN ?. In fastai , why do we use U-NET for segmentation ?.

There is a paper which talks about an ensemble of U-NET and Mask-RCNN


(RobG) #65

Performance edge. Have a look at the kaggle data science bowl solutions to compare. Ensembles of unet/maskrcnn in this and other segmentation challenges (carvana, airbus, etc) haven’t been top performing. https://www.kaggle.com/c/data-science-bowl-2018/discussion

1 Like

(Mehran Ghandehari) #67

I am trying the following code:
class SegLabelListCustom(SegmentationLabelList):
def open(self, fn): return open_mask(fn, div=True)

class SegItemListCustom(SegmentationLabelList):
_label_cls = SegLabelListCustom

src = (SegItemListCustom.from_folder(path_img)
.label_from_func(get_y_fn, classes=codes))

and get the following error:

Traceback (most recent call last):
File “C:\Users\Mehran\Anaconda2\envs\test2\lib\site-packages\IPython\core\interactiveshell.py”, line 3296, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File “”, line 11, in
.label_from_func(get_y_fn, classes=codes))
File “C:\Users\Mehran\Anaconda2\envs\test2\lib\site-packages\fastai\data_block.py”, line 468, in _inner
File “C:\Users\Mehran\Anaconda2\envs\test2\lib\site-packages\fastai\data_block.py”, line 522, in process
for ds,n in zip(self.lists, [‘train’,‘valid’,‘test’]): ds.process(xp, yp, name=n)
File “C:\Users\Mehran\Anaconda2\envs\test2\lib\site-packages\fastai\data_block.py”, line 699, in process
File “C:\Users\Mehran\Anaconda2\envs\test2\lib\site-packages\fastai\data_block.py”, line 75, in process
for p in self.processor: p.process(self)
File “C:\Users\Mehran\Anaconda2\envs\test2\lib\site-packages\fastai\vision\data.py”, line 372, in process
def process(self, ds:ItemList): ds.classes,ds.c = self.classes,len(self.classes)
TypeError: object of type ‘NoneType’ has no len()


(rors) #68

Im also running in to some headaches here,

See attached screenshot. Now im wondering if it’s better to just try and convert into 0s and 1s?