Implementing Mask R-CNN

@jeremy I’ll help you with Tiramisu in PyTorch. I think it’s doable and will be more fun. Mask R-CNN is a great paper, but it is significantly more complex and we would only be able to tackle a very tiny part.

We can build on the PyTorch DenseNet model and the author’s Tiramisu implementation in Lasange.

@matthew Tiramisu is designed for semantic segmentation. Mask R-CNN is more extensible, but I haven’t been able to find an implementation of instance segmentation among its predecessors (Fast, Faster, RCNN)

1 Like

@brendan awesome! Are you free on the weekend? I am - so I’m hoping to keep hacking throughout the weekend. That might be enough time to do both properly…

1 Like

Sweet! I will be here for any questions and also can stop by 5 in person : )

Yep! I’ve been reading and Densenets look really promising. Should be easier/faster to train given the fewer parameters. I’ll start another wiki thread for Tiramisu later today so we can share notes.

Here is a network from this paper: Shape-aware Instance Segmentation

As you can see, it does Instance Segmentation and uses almost the same network:

DeepConvs + RPNs to pick the Region of Interest + Bounding box and lastly the Instance Segmentation.

The Mask R-CNN uses a much simpler and modular network to do the task. Modular as in, the ResNeXt can be swapped out and swapped in, meanwhile the layers of Instance Segmentation can be increases and reduced without making it effect the Object Detection + Bounding Box.

1 Like

Thought folks on this thread might enjoy reading this blog by Dhruv Parthasarathy about R-CNN, Fast-R-CNN, Faster-R-CNN and Mask-CNN

And here is my brief mind map after reading through the blog

13 Likes

Thanks @sravya8. That’s a very good summary!

I was Going through the Discussion above…

I was Wondering that do we have a implementation of the Mask R CNN in Keras …

No we ended up doing 100 layer tiramisu.

1 Like

What technique Should I use to Identify White Blood Cells(Purple color) in a Blood Smear using Deep Learning …I Have To show Bounding Boxes and Also Count the Numbers…

Is techniques Used in Lesson 7 Enough or Do I Need to Learning Some other technique…

Try using U-net segmentation.

Awesome! Did you guys manage to get this working?

Also, do you have any thoughts on applying this to a new dataset and roughly how much training data you’d need to annotate?

PLs Suggest

Any update on Mask R-CNN code?

I found AffordanceNet code can do multiclass instance segmentation, but they only used VGG16.

1 Like

Hey, did you implement the Mask R-CNN in PyTorch? Is there a good PyTorch implementation for it on github?

Hi, guys.
We are actually trying to use the following repository https://github.com/matterport/Mask_RCNN
What do you think about?
Any progress to implement it here?

I have been using that. I really like the notebooks and the examples using simple shapes as this allows you to experiment with small data. It definitely works. There is an existing port to pytorch but it is not working for training. Also they have not converted the notebooks and shapes; and have moved a lot of stuff around. I am still trying to understand all the different parts of maskRCNN but using both these projects to help me do that.

1 Like

I am working with the Matterport code for both a Kaggle competition and for work, and am having a lot of trouble with it. It does not integrate well with standard keras code, and I am having a difficult job implementing basic tasks like image augmentation. Does anybody have any advice or a more mature mask rcnn tool written in either pytorch or keras? I’m currently wrangling with a branch of the Matterport code that has implemented image augmentation, and working from there.

We’ll be doing something like this in the new part 2, except more like SSD and RetinaNet (i.e. 1-pass rather than 2-pass approach).

8 Likes

One of the core contributors of torchvision has said he’s working on faster/maskrcnn.

Right now, there isn’t an official roi align layer or non-max suppression in pytorch which means you have to either build one of the cuda extensions floating around or use a very slow naive version. Those problems + messy anchor box assignment code seem to be making most of the current pytorch attempts hard to work with. Also, I “think” roi align and nms are coming in the next version of torchvision.

Also also, I wonder why nobody ever mentions the Light-Head rcnn paper. https://arxiv.org/pdf/1711.07264v1.pdf. I’d be interested in throwing a mask-head on that and seeing how well it does at instance segmentation.

Edit: Also also also, @jeremy. I saw that you were looking for a 1-pass instance segmentation network on twitter. Did you ever find anything or are you just going to do object detection for part 2? I was actually trying (and failing) to hack a mask head onto retinanet as I saw that tweet. Perhaps you could point me in the right direction? The only thing I’ve been able to find is this (class agnostic) https://arxiv.org/abs/1711.07618 and the code isn’t available yet.

1 Like