Segmentation and classification in the same pipeline

fastai fastai dev
1

I’m starting a new project with the idea to take an image, segment a specific area and classify what it’s on it. Let’s make an example:

Imagine I have images from open cardboard boxes with different sizes and shapes. I would like to classify the image taking into account what is inside the box but not outside it.

EDIT Actually what I have is a bunch of X-ray images that I would like to classify the lung part in different pathologies: control, pneumonia, stroke, etc

I know that I could manage to find a way to crop the image first but let’s say its impossible.
The proposed approach then would be first make a segmentation of the inner side of the box and the build a classifier on it.
I guess I could do it i a two step saving the image segmented image but I would like to know if there is any possibility to do it sequentially in the same training process.

Any ideas where I should start?

2

I think this Kaggle competition might give you some cool ideas :wink:

1 Like
3

Thanks for the feedback! Ok, I will clarify the post because I am not sure if Semantic segmentation would be the way to do this. I definetely give it a try but I am not sure if the image are different enough to make the classification part work. I will change the example to the actual problem:

I have a bunch of lung X-ray images and I would like to first segment the lung part, avoiding the bones and background and classify the lungs on different pathologies, let’s say control, pneumonia and stroke.

Is this suitable for semantic segmentation? Or should I go to some two step pipeline as I was proposing in the first post?

4

I have to say that I am not sure at all.
It would be a premiere for me too.

Do you have annotated bounding boxes (or actually segmented pixels) for those?

5

Yes, we have segmented masks for the lung

1 Like