I implemented the OpenAI paper “An Empirical Model of Large-Batch Training” for Fastai V2 (repo). This implementation is based on Batch size finder from OpenAI implemented using Fastai although I manage to simulate multiple GPUs as the original authors assume plus a couple of bug fixes. I hope that you find this code useful .
Finally, ¿do you think that this feature should be present in the standard Fastai V2 library? I’m willing to put a PR for that. As @muellerzr points out in his course, I find very useful for tabular data where we can use a huge batch size.
Hello Victor !
Great to see you taking it one step further !
I will have a look at this, and thanks for the help on this matter, I have to confess I haven’t found the time nor the courage to take another dive into this problematic but it’s great that you did !
This looks like a useful callback to add to the library. Make sure you adapt to the style of the rest of the library though: docstrings should be just one line and the rest of the documentation (for instance all the arguments doc) should be markdown prose in the cell after the definition of a function.