I am working with the Protein challenge on Kaggle and I have a question
related to learning rates. There is a canonical image data set provided by Kaggle. Using standard fastai code, I estimate the learning rate as approximately 0.02.
But an intrepid person dug up another source for images, the so-called HPA data, and the learning rate, according to lr_find, needs to be 0.002, different by a factor 10. Both plots are clean and easy to interpret.
Running the training appears to give reasonable convergence for both.
Here is the question: both data sets have the same source for the images and they are of the same resolution and look similar. The only real difference that I see is that there are more that twice as many HPA images than canonical images. Can that difference in number alone give a substantially different learning rate, cut by a factor 10?