Lower accuracy when running fit_one_cycle again

This problem has already been discussed and answered here.

Basically, it is because the fit_one_cycle anneals the learning_rate towards 0.

29

So, when you retrain the model, the initial learning rate is higher than what previous training end up with, resulting in bigger steps that make your gradients go out from the minimum that previous training found, thus giving you and a higher loss value.

2 Likes