How to calculate new bbox and key points in random transforms

fastai
1

For example, when using a pipeline of random transforms on an image, how to calcualte the new coordinates of associated bounding box and key points? I think there are some parameters of the random transforms, e.g. affine matrix. I don’t know where are the parameters and how to apply them to the bboxes and key points before they change?

Thanks a lot!

2

You should use ImageBBox or ImagePoints, then the fastai library will do all the work for you :slight_smile:

3

Thanks @sgugger!

Yes, I tried ImageBBox and ImgaePoints.

Here is an example.
before transform. bbox displays right.
then

trans = get_transforms()[0]
bbox.apply_tfms(trans)
img.apply_tfms(trans).show(y=bbox)

I can see bbox is displayed in wrong place. Seems each time trans is used, it calls some random transfomation. There is something worong in my code. Is there any workable example for fastai 1.0? To train an object-detection net, the coordinates of bbox and key points after transformation are needed.

Thanks a lot!

4

You should use the example in the docs here for grabbing a DataBunch and here for applying transforms as a starter (scroll a bit to see the one with object detection). In general, you would do:

trans = get_transforms()[0]
img = img.apply_tfms(trans)
bbox = bbox.apply_tfms(trans, do_resolve=False) #Second arg to make sure you keep the same random params
img.show(y=bbox)
5

I’d appreate that! @sgugger

I tried your example and do_resolve=False is what I am looking for.

  • do_resolve : if False, the values of random parameters are kept from the last draw

One more question, if an image is rotated, why not the bbox is also rotated insteda of expanded, since the four corners can be recalculated.