In lesson 2 at https://youtu.be/JNxcznsrRb8?t=5788, he simply chances cycle_mult to 2. cycle_mult makes the learning rate now decrease more slowly by a factor of 2 so why does the number of number of test runs double?
Look at training_phase.ipynb
in the dl2
folder. Inside there is a SGDR
part. It explains how it works and also allows you to plot what is happening to the learning rate as you change cycle_len
and cycle_mult
.