Object detection in fast.ai v1


Did you make Retinanet work?

I was successful with creating the data object using the datablocks api … but now I am strugelling with creating the model

model = RetinaNet(encoder, 6, -4)
ValueError Traceback (most recent call last)
1 encoder = create_body(tvm.resnet50(True), -2)
----> 2 model = RetinaNet(encoder, 6, -4)
3 model.eval()
4 x = torch.randn(2,3,256,256)
5 output = model(x)

<ipython-input-40-37554ab72d4b> in __init__(self, encoder, n_classes, final_bias, chs, n_anchors, flatten)
      6         self.n_classes,self.flatten = n_classes,flatten
      7         imsize = (256,256)
----> 8         sfs_szs,x,hooks = model_sizes(encoder, size=imsize)
      9         sfs_idxs = _get_sfs_idxs(sfs_szs)
     10         self.encoder = encoder

ValueError: too many values to unpack (expected 3)


I haven’t made it work yet. I’m still stuck with creating the ‘clas’ and ‘regr’ variables. Do you know how to create them?

I had the same error as you and replaced encoder:Model with encoder:nn.Module:

class RetinaNet(nn.Module):
“Implements RetinaNet from https://arxiv.org/abs/1708.02002
def init(self, encoder:Model, n_classes, final_bias=0., chs=256, n_anchors=9, flatten=True):

to this:

class RetinaNet(nn.Module):
“Implements RetinaNet from https://arxiv.org/abs/1708.02002
def init(self, encoder:nn.Module, n_classes, final_bias=0., chs=256, n_anchors=9, flatten=True):

Not sure if this is correct, though.

(Dien Hoa TRUONG) #9

@m000ritz, @hkristen Can you make Object Detection work with fast.ai v1 ? There are no more ObjectDetectDataset in fastai library . Are there any example of this in fast. ai v1, no matter Retina or the previous used technique SSD ?

Thank you


Hi @dhoa!

For the fastai coco subset I was able to create the data object with the datablocks api like this:

PATH = Path('.../coco_sample')
ANNOT_PATH = 'annotations'
images, lbl_bbox = get_annotations(PATH / ANNOT_PATH / 'train_sample.json')
img2bbox = dict(zip(images, lbl_bbox))
get_y_func = lambda o:img2bbox[o.name]
data = (ObjectItemList.from_folder(PATH / 'train_sample')
        #Where are the images?
        #How to split in train/valid? -> randomly with the default 20% in valid
        #How to find the labels? -> use get_y_func
        .transform(get_transforms(), tfm_y=True, padding_mode='zeros', do_crop=False, size=128,)
        #Data augmentation? -> Standard transforms with tfm_y=True
        .databunch(bs=16, collate_fn=bb_pad_collate))   
        #Finally we convert to a DataBunch and we use bb_pad_collate

Then I am stuck at the above described error. Ithink I`ll need more time or maybe help from @sgugger to resolve this :wink:

If you make any progress, please let us know!

(Michael Schuldes) #11

Since the creation of the notebook model_size changed its behaviour. You can now use:
sfs_szs = model_sizes(encoder, size=imsize)
hooks = hook_outputs(encoder)
instead. That should work for you.


Thanks @msandroid! I am one step further now, again running into an error that seems to come from changes in teh fastai source code…

For anybody working on this, you can use my GIST as a start …

Object Localization (COCO)
(Dien Hoa TRUONG) #13

Thanks @hkristen . So I think play with object detection in fast.ai now is quite difficult because it misses example and the library is still changing. I switch to read last year course with SSD and hope Jeremy will tell something about it in the last session of part 1 v3


No, it won’t be covered until part 2.

(Sebastien Derhy) #15

I think the conv2d_relu can be replaced by conv_layer. Or at least, that’s what I did, and it solved the issue.

I now have a notebook that runs until the learning part (included), but unfortunately, the loss function doesn’t go down when I’m trying to learn :frowning:

One more thing that I had to change in order to make the notebook run was to replace in the FocalLoss definition:

clas_loss = F.binary_cross_entropy_with_logits(clas_pred, encoded_tgt, weights, reduction=‘sum’)


weights_no_grad = weights.detach()
clas_loss = F.binary_cross_entropy_with_logits(clas_pred, encoded_tgt, weights_no_grad, reduction=‘sum’)

Otherwise, Pytorch was trying to compute the gradient of the weights and sending an error…

It’s frustrating, I feel like we’re close to making it work !!

(Aaron) #16

I found a working object detection algorithm y’all will find interesting.

Working notebook for Object Detection
(Rohit Singh) #17

@divyansh and I have gotten object detection to work with fastai v1. The dev notebook is at https://github.com/rohitgeo/singleshotdetector

This creates a Dynamic SSD based on the number of grid cells, zoom levels and aspect ratios for the anchor boxes.

We haven’t trained the network fully in the dev notebook… like unfreezing and fine-tuning, and using Focal Loss (which is supported) etc, but are getting good results already. Hope you find it useful!

See Dynamic SSD implementation for fastai v1 for more details.

(Francesco Gianferrari Pini) #18

So the code in fastai v1 for object detection will not be ready until part 2? I see already in the docs the yolov3 architecture… It is not yet 100% ready?


There is no YoloV3 in the fastai library and what’s in preparation is Retina Net, but no, it won’t be ready before part 2.

(Pierre Ouannes) #20

I think that what you saw is that darknet is available in the fastai CV model zoo. YOLOv3 uses darknet so that’s why YOLOv3 is mentioned.

(tester) #21

Seems like yolo is more famous. Just for curiosity, why would we pick retina net over yolov3 for lecture? Is retina net easier to teach or other reasons?

(Ben) #22

RetinaNet uses a pretrained ResNet backbone, while Yolov3 uses Darknet53 as a backbone. There is no pretrained Darknet53 model available in fastai, which you would need to implement Yolov3. This is why fastai will use RetinaNet.

FWIW, I tried to load the weights from the original Darknet53 implementation in C into the fastai implementation. I could never get it to quite work. I think the weights were loaded correctly, but since fastai uses a slightly different output layer than the original C implementation, I couldn’t validate it. I’ve since moved on to trying to train fastai Darknet53 on imagenet from scratch. I’ll share my results on the forum when I get the top-1 accuracy into the 70%+ range.

(James Adams) #23

Thanks for this, nice work!

(nick) #24

I have a problem. When I run the code to transfer the pre-training model: learn=create_cnn(…), the code will automatically download the model, but it will be interrupted due to time and network speed, so I download the pre-training model in advance. When you run this code again, it will be stuck here. How can I solve this problem? Please help everyone.

(nick) #25

(andreas) #26

Hi Sgugger,
I am trying to run the dev notebook 102a_coco and I receive the error objectdetectdataset not defined how do I fix this error?