We are looking to annotate images to segment, we have only 3 labels: class1, class2 and background.
( just to be clear, as there are many definitions of segmentation, each image pixel has only one label).
What I would like to do, is to Active Learning, so have the model in the loop, having this has (at least for me) two great advantages:
to help me find the hardest examples (which images I need to annotate next to improve my model performance.
Generate initial segmentation masks to “correct” an make the annotation process simpler. Instead of starting from a clean canvas.
Right now, I have 600 annotated images, and the fastai unet gives very good results. This initial images where annotated by another team, and have pretty low res. Also, they are provided as PNG files (with 0,122,255) values.
Questions:
How would you transform the output of fastai unet to make it compatible with annotation tools that expect COCO or JSON polygon regions.
What tool to use to do this, the simplest appears to be Label Studio, but is very basic and excruciatingly slow! CVAT is another tool we are testing.
Is this necessary or do you know a tool that can work directly with the Grayscale PNG image?
I want to be sure that the file format chosen to save annotation and the tools are up to the task.
So, after reasearching a lot. Testing many solution, open-source and paid ones. I have come to the conclusion that this is a topic that is in early stages.
So, I am building one.
What I want it to do:
Segment images, with polygon tools, brush tools.
Merge, cut, add, erase masks (this is not at all available on most of the tools I tested).
Example: you draw you polygon segmenting the object, but it has a hole that you want to substract from the mask, this is impossible to do on most tools. You would have to draw another polygon on top, with a different entity, and then perform some post-processing.
Have a Unet/Segment model in the loop.
Most of commercial tools propose this option at their most expensive tier. I want to be able to pass my Unet, or fastai one, and have state of the art fastai training, no some crappy training loop.
There are 2 options here, a model that propose and initial full image sementation, or a “magic” tool that help you segment parts of the image. The only tool I found that let me do this was: https://github.com/Britefury/django-labeller
It actually works pretty well, using as backend a pytorch DEXTR model that I was capable to train for my task.
I forked this repo, and removed all the django code to leave only the minimal flask part and put it here:
I am currently updating the flask code to revamp the interface, deporting the configs to .ini files and have to upgrade the training loop on the model, but it is looking promising. Any help woul dbe appreciated!
I am sure the guys from IceVision could give me a hand!