How to do Discriminative layer training with Cosine annealing with restarts in fastaiv1


I went via it seems that TrainingPhase class doesn’t support Discriminative layer training. What I want to do is with a model with 3 layer groups, I want to set a different max lr for each of the groups and do cosine annealing with several cycles. Is that something can be done out of the box or need to write a custom callbacks?


You can pass your lrs as np.arrays (or a tuple of those for start/end) to do discriminative learning rates in a TrainingPhase.