Lr_find increases the learning rate exponentially every iteration and calculates the validation loss. However, calculating the validation loss for every single iteration, but that must need a lot of calculations.
It seems like that lr_find is using something like running mean in order to approximate it, but I’m not quite sure about it. How is lr_find possible to calculate the validation loss so fast?