Meet RAdam - imo the new state of the art AI optimizer

[Seb]

we can use this thread (or starting a new one is fine I guess)

Imagenette/ Imagewoof Leaderboards

I thought it’d be a good idea to start a thread dedicated to the Imagenette and Imagewoof leaderboards. I have some work in progress, and will publish if I get interesting results. But first, I tried to replicate the baseline results. And I am getting different results for Imagewoof, e.g. at 256 px, 5 epochs, I am getting a range of 57.4 -63% (ran it 4 times, average is 61.2%) compared to the 57.6% published. It seems that results vary a lot between runs (at least for 5 epochs).