The learning rate finder explained in the video seems to differ from the way it is described in the paper referred. In the video, the learning rate is taken in the region where the slope is maximum and still converging. But in the paper, it is shown as varying between a maximum and minimum learning rate. I was wondering if both are the same or am I missing something?
Which paper are you speaking of? My understanding was that you start at a small learning rate then stop before the minima.
I felt in the video Jeremy was actually choosing a point that was between the inflection point (max slope) and the minimum. Does anyone know why?
1 Like