How can you train your model on large batches when your GPU can’t hold more than a few samples?

Part 1 (2019) Advanced (Part 1 v3)
6

For fastai2, the docs are here:

https://dev.fast.ai/callback.training#GradientAccumulation

For fastaiv1 see this discussion: Accumulating Gradients

Though I highly recommend v2 :wink:

4 Likes