Epochs to train head vs all layers

Part 1 (2020)
1

What is the optimum ratio of the number of epochs to train head vs all layers?

2

that completely depends on what you are trying to do and how you are approaching it :). can you provide more information?

3

I am just asking as a general rule of thumb. So I have this dataset which has 8000 training images for object detection.

4

If you are using a pretrained model with fastai I’d just go with learn.fine_tune() and train for 5,10, 20 epochs and just check the results.

Fine tune will train the head for one epoch and the whole model for the number epochs you pass to fine_tune().

1 Like
5

Is this an applicable rule of thumb for large datasets? As I said, I have 8000 training images for object detection.

6

in my opinion yes. you’ll get good results for any dataset just by using fine_tune.

Of course there’s room for improvement (model architecture / size, optimizer, activation layers, lr, lr schedule, etc …) but in my experience fastai with pretrained resnets I’ll get eg. 92% accuracy after a few minutes. Improving the result substantially (let’s say by 1-2%) took me always hours or days experimenting. for most of my projects it’s not worth it… I’ll just go with unfancy resent and fine_tune :D.

2 Likes
7

Ok. Thanks for the advice.

8

for object detection have a look at https://github.com/airctic/icevision . afaik a SOTA object detection library with fastai integration.

2 Likes
9

Thanks