Fastai lessons often tell you what is the proper default for hyperparameters. It would be nice to have a web page that puts them together. Is there such a page?
Not that I’m aware but making a ‘best practices’ list from the lessons would be a great idea.
I’m actually going through all the lessons one last time to glean any missing details so I will start jotting things down and try to post it as a summary.
For starters = 3e-3 is Jeremy’s recommended default learning rate for frozen, and max_lr =slice(3e-3, 3e-4) for unfrozen.
anyway, good idea and I’ll try and make one if no one beats me to it.
2 Likes