Is it possible to label regions of an image?

I would like to apply multiple labels to an image by specifying x,y coordinates of bounding boxes. Is this possible? See the below image for an example.

box 1 coords: 400, 320
box 1 size: 200
box 1 label: ‘5 spokes’

box 2 coords: 800, 300
box 2 size: 200
box 2 label: ‘5 spokes’

I think you can use Labelme tool for this purpose, check out: https://github.com/wkentaro/labelme it will give you XML or a JSON file, which you can parse to get the coordinates of the boxes.

Thanks. But how can I feed its XML or JSON into fast.ai ImageDataBunch as the labels to use?

As I said before, you have to parse those files. For example you can use pandas to read a JSON file (pd.read_json(filename))and then save it as a csv. For your case you may have a lot of images, so you can read all those JSON files in a loop and append them to a pandas DataFrame and then save it as a csv file. Or you can use label_from_func method and write a function to do individual parsing given the name of the file. Refer docs.fast.ai for more information on how to do this. I recommend you go through this link (https://docs.fast.ai/vision.data.html) and find the type of data loader best suited for your problem. (Maybe you can use ObjectItemList because your problem seems to be an object detection problem).

1 Like

Great, I’ll take a look, thanks!

Maybe my suggestion could be a bit silly, but why not to use a segmentation approach?

1 Like

It’s not actually silly. But if there is no need for very precise pixel wise detection then I don’t think there is a point in complicating the problem. Also the labelme tool is generally used for labelling segmentation tasks only.:sweat_smile:

1 Like

Thanks @navidpanchi. My problem is a multi label classification one, sorry I didn’t make that clear in the first post. So in my original image, one bounding box might have the labels: [5 spokes, front] and the other might have: [5 spokes, back].

The example using lbl_bbox here is multi class, not multi label i.e. it would allow me to specify ‘5 spokes’ for a bounding box, but not necessarily ‘5 spokes’ and ‘front’. Do you know how I could get multi-label and multi-bounding box per image to work? I would like to be able to provide something like the below:

@sgugger hello, you seem to be very helpful on here when you have time, I was wondering if you could help with this issue please?