# Part 2 Lesson 8 wiki

(Ramon) #297

Hi,

Iâm trying to extend the bounding boxes in lesson 8 by ârotated bounding boxesâ. Iâm doing this by passing in four coordinates top-right(x,y), bottom-right(x,y, bottom-left(x,y) and top-left(x,y).

By using the same code, only the first four values get passed through the different datasets and loaders:

tfms = tfms_from_model(f_model, sz, crop_type=CropType.NO, tfm_y=tfm_y, aug_tfms=augs)
md = ImageClassifierData.from_csv(PATH, JPEGS, BB_CSV, tfms=tfms, continuous=True, bs=4)
x,y=next(iter(md.aug_dl))
bbox = to_np(y[1])
print(bbox)
[194. 368. 217. 400. 0. 0. 0. 0.]

How come? Iâm trying to understand but I canât figure it out. Hints are very appreciated, thanks in advance!

(Michael) #298

Hey @ramon,

I was starting with lecture 8 today and I guess it only needs 2x x,y = 4 coordinates for the box to be sufficiently defined.

Do you use training data with rotated bounding boxes?
With the Pascal dataset I would guess that it will learn the âalignedâ bounding boxes.

Best regards
Michael

(Ramon) #299

Hi @MicPie, thanks for your reply. Yes I know the bounding boxes in the lecture only use two coordinates (top-left and bottom-right). But Iâm trying to extend this for a different dataset that is using rotated bounding boxes. Itâs defined by 8 values, the csv looks like ImageId, y1 x1 y2 x2 y3 x3 y4 x4

(Michael) #300

I am currently watching lesson 9 and there Jeremy talks about the transformations in the âBbox onlyâ section in the jupyter notebook. As you can see at the transformed/rotated pictures of the woman the bounding box is not rotated and is only resized (and still aligned vertically and horizontally).

I guess to rotate the bounding box you have to adapt the class âCoordTransformâ.
In addition, you have to use âtfm_y=TfmType.COORDâ to transform the coordinates of the bounding box too.

However, maybe the pixel transformation can be of use with TfmType.PIXEL (see class âTfmTypeâ, should be covered in later lessons on image segmentation).

(Ramon) #301

Hi @MicPie, Yes indeed, class "CoordTransformâ is the part of the code which is replacing the values by zeroâs. Thatâs what I was looking for, thanks for your help. Iâm trying to add a new TfmType to handle rotated boxes. Maybe in the end itâs better to use the PIXEL type, but in this way Iâm getting to understand the fast.ai frame more.

(Ramon) #302

Fixed it, if anyone is interested let me know

(Michael) #303

Nice work, looks very nice!

Do you also were able to train the neural network?
I would be interested if you use the coordinates of the four corners or maybe the two corners with a angle.

Best regards
Michael

(Ramon) #304

Hi @MicPie,

Not yet, but Ill continue next weekend. Ill keep you updated.
By the way, eventually Iâm trying to solve the Kaggle airbus challenge (https://www.kaggle.com/c/airbus-ship-detection), interested in teaming up?

Kind regards,
Ramon

(Thomas) #305

I am interested

(Ramon) #306

Hi @tcapelle,

Thanks for your response. Interested in the code and/or teaming up?

Regards,
Ramon

(David) #307

Would be interested in teaming up too!

(Ramon) #308

Great, I added you to the conversation, did you get that message?

(David) #309

âŚI did notâŚ

(Thomas) #310

teaming up!
I have been playing with a kernel to detect ship presence first.

#311

It seems there is no alias to data folder in the dl2 folder (contrary to dl1).

I just figured out how to create one on linux terminal to make the notebook work.
cd into dl2 folder, then :
âln -s ~/dataâ

#312

At the end of the pascal notebook, Jeremy unfreezes to layergroup -2. Then does lr find with the following result

From this graph, he chooses for the next iterations lr=5*10^-3

Can anyone explain to me wy he chooses such value ? (The graph doesnât look like the usual âdecrease then increaseâ graphâŚ)

Thanks a lot !

(Nikhil) #314

Hi I had the same question. that, will rotating the input image, will rotate bounding box co-ordinates.
Does `CoordTransform` achieve this?
Does `CoordTransform` increases accuracy?

(Luca) #315

Pay attention to which cells are actually run. He got me too! These commands are not supposed to be run in succession, just skip the second sched.plot. I believe that is used earlier to test which values would work better when unfreezing last two layers!
No need really for a second lr_find here, or you can try it with lr_find(lrs) instead of lrs/1000.

(Leonardo Lima) #316

Hello dude, I got the same error.

This fixed open_image(str(IMG_PATH/im0_d[FILE_NAME]))

âstr()â

Pd: Iâm using paperspace and just git pull

(Ramon) #317

CoordTransform takes to coordinates as input (top-left and bottom-right). On rotation (or other transformations), internally (in class CoordTransform):

1. it draws a rectangle (y = CoordTransform.make_square(y0, x)) - red in attached picture:
2. rotates this rectangle (y_tr = self.do_transform(y, True)) - green in attached picture
3. grap the new top-left and bottom-right (return to_bb(y_tr)) - blue in attached picture

So the return is a new bounding box (not rotated) thatâs larger than the original.