Is gradient accumulation implemented in fastai V2?

fastai fastai dev
1

Hi!

Is gradient accumulation implemented in fastai V2?

Cheers,

1 Like
2

Not yet no, but it’s a simple Callback to adapt.

2 Likes
3

thanks a lot @sgugger !

4

Mind if i ask, what is the use cases where you want your gradient to accumulate?

5

When you dont have much ram to have big batch sizes (such as dealing with high res images), gradient accumulation helps emulate bigger batches.

4 Likes