Why use unfreeze() and fit_one_cycle()?

Part 1 (2019) Advanced (Part 1 v3)
1

Why did we use unfreeze() ? What does unfreeze(), actually do, how does it help in training?
Also what’s fit_one_cycle() and how is it different from the usual model.fit() ?

2

It is explained by jeremy in the lecture videos that when we don’t use unfreeze only the last few layers of the model are trained whereas when we unfreeze all the layers in the model are trained albeit with different learning rates.
fit_on_cycle is a special type of fit implemented from a paper called fit one cycle ( mostly) in which the neural net is trained with a variable learning rate such that it first increases with time and then decreases.

3

Thanks for explaining…

4

  1. Unfreeze()
    During transfer learning, each model can be considered to have a backbone/body + head.

    The training process happens as follows
    a. The backbone is intially kept fixed a.k.a. freeze the body (since through transfer learning, the backbone is reliably trained to a great extent) and the head is trained.
    b. Then, we unfreeze the body and train the body too, to some extent during a fine-tuning process.

    Hence when you say, unfreeze, you mean that the body is being unfrozen and being ready to be trained.

  2. fit_one_cycle -> Check out this great article by Sylvain.
    https://docs.fast.ai/callbacks.one_cycle.html

Hope that helps.