Implementing Mask R-CNN

Deep Learning
43

Try using U-net segmentation.

44

Awesome! Did you guys manage to get this working?

Also, do you have any thoughts on applying this to a new dataset and roughly how much training data youā€™d need to annotate?

45

PLs Suggest

46

Any update on Mask R-CNN code?

I found AffordanceNet code can do multiclass instance segmentation, but they only used VGG16.

1 Like
47

Hey, did you implement the Mask R-CNN in PyTorch? Is there a good PyTorch implementation for it on github?

48

Hi, guys.
We are actually trying to use the following repository https://github.com/matterport/Mask_RCNN
What do you think about?
Any progress to implement it here?

49

I have been using that. I really like the notebooks and the examples using simple shapes as this allows you to experiment with small data. It definitely works. There is an existing port to pytorch but it is not working for training. Also they have not converted the notebooks and shapes; and have moved a lot of stuff around. I am still trying to understand all the different parts of maskRCNN but using both these projects to help me do that.

1 Like
50

I am working with the Matterport code for both a Kaggle competition and for work, and am having a lot of trouble with it. It does not integrate well with standard keras code, and I am having a difficult job implementing basic tasks like image augmentation. Does anybody have any advice or a more mature mask rcnn tool written in either pytorch or keras? Iā€™m currently wrangling with a branch of the Matterport code that has implemented image augmentation, and working from there.

51

Weā€™ll be doing something like this in the new part 2, except more like SSD and RetinaNet (i.e. 1-pass rather than 2-pass approach).

8 Likes
52

One of the core contributors of torchvision has said heā€™s working on faster/maskrcnn.

image
image.png814Ɨ146 25.8 KB

Right now, there isnā€™t an official roi align layer or non-max suppression in pytorch which means you have to either build one of the cuda extensions floating around or use a very slow naive version. Those problems + messy anchor box assignment code seem to be making most of the current pytorch attempts hard to work with. Also, I ā€œthinkā€ roi align and nms are coming in the next version of torchvision.

Also also, I wonder why nobody ever mentions the Light-Head rcnn paper. https://arxiv.org/pdf/1711.07264v1.pdf. Iā€™d be interested in throwing a mask-head on that and seeing how well it does at instance segmentation.

Edit: Also also also, @jeremy. I saw that you were looking for a 1-pass instance segmentation network on twitter. Did you ever find anything or are you just going to do object detection for part 2? I was actually trying (and failing) to hack a mask head onto retinanet as I saw that tweet. Perhaps you could point me in the right direction? The only thing Iā€™ve been able to find is this (class agnostic) https://arxiv.org/abs/1711.07618 and the code isnā€™t available yet.

1 Like
53

Hi,
I am to implement this model in production, and main constrain is inference time. Iā€™m going to start with https://github.com/facebookresearch/Detectron
Final goal is to run this model in TensorRT but at the moment it is not supporting custom layers.
Any suggestions about production performance is wellcome.

I have two more questions: what is ā€œBells and whistlesā€ from original article and what is Tiramisu?

54

Hi , is there an implementation of Mask R-CNN in fast ai ?

3 Likes
55

I have completed a version of maskrcnn for pytorch v1 and fastai v1. Based on matterport and multimodal but quite a lot of rewriting to simplify and make easier to understand. Have used the fastai training loop; learning rate finder; and other callbacks.

13 Likes
57

Kasianenko thank you for the link.

58

Hi, please be so kind to read patiently all the thread and use search before asking and most important - @ at mentioning people. @ mentioning is considered bad manners, if you can get an answer from anyone else in forum. Especially for admins like Jeremy.

So the last post before yours is the answer - fastai implementation. Implementing Mask R-CNN

In addition there is an it easy usage of pytorch: importing it from torchvision, follow this example https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html

2 Likes
59

Thank you so much for your help Kasianenko!
( I am sorry for my bad behaviour, wonā€™t happen again. )

1 Like
60

No worries, everyone has to start somewhere, I did the same mistake couple of years ago.

1 Like
62

I am trying to figure out what the best fastai-based solution is for working with images where the input annotations are bounding boxes (not segmentation masks). Things seem to be moving very fast; Fast R-CNN (Girshick 2015), Faster R-CNN (Ren et al. 2016), Feature Pyramid Networks (Lin et al 2017), Mask R-CNN (He et al. 2017), Mask scoring R-CNN (Huang et al. 2019), Detectron1 and now Detectron2 in PyTorch (Wu et al. 2019), etc.

Does anyone know what the state of the art would be in terms of models that are already implemented in either fastai v1 or fastai v2, for learning from bounding box annotations?

Thanks

2 Likes