Learning rate finder: how do we know that the sample on which the "simulation" is done is representative of the rest of the dataset?

[zben]

Thanks!!
For those who want to have a look at how the loss evolves as a function of the learning rate for different batch sizes, i’ve found this valuable post from one of the fast.ai students :

Visualizing Learning rate vs Batch size

For small batch sizes, the loss moves around a lot indeed, and is more “stable” for larger batches.
What is interesting is that even for batch sizes for which the loss does not move around a lot, what we could infer as the learning rate (“a bit before the loss starts increasing”) is different from one batch size to another (see difference between plots BS=16,32 and 64).