Batch size finder for Fastai V2

fastai fastai dev
1

Hi there!

I implemented the OpenAI paper “An Empirical Model of Large-Batch Training” for Fastai V2 (repo). This implementation is based on Batch size finder from OpenAI implemented using Fastai although I manage to simulate multiple GPUs as the original authors assume plus a couple of bug fixes. I hope that you find this code useful :slight_smile: .

Finally, ¿do you think that this feature should be present in the standard Fastai V2 library? I’m willing to put a PR for that. As @muellerzr points out in his course, I find very useful for tabular data where we can use a huge batch size.

12 Likes
2

Hello Victor !

Great to see you taking it one step further ! :smiley:

I will have a look at this, and thanks for the help on this matter, I have to confess I haven’t found the time nor the courage to take another dive into this problematic but it’s great that you did !

1 Like
3

This looks like a useful callback to add to the library. Make sure you adapt to the style of the rest of the library though: docstrings should be just one line and the rest of the documentation (for instance all the arguments doc) should be markdown prose in the cell after the definition of a function.

1 Like