What is the v2 equivalent of "AccumulateScheduler"?

muellerzr · March 24, 2020, 1:32am

@wgpubs this was done in a very recent commit (like 12 hours ago).

Notebook: https://github.com/fastai/fastai2/blob/master/nbs/17_callback.training.ipynb

fastai/fastai2/blob/7845816a9b3d0a9b90a0a1dbb16e3c75219be3bd/fastai2/callback/training.py#L20


# Cell
class ShortEpochCallback(Callback):
    "Fit just `pct` of an epoch, then stop"
    def __init__(self,pct=0.01,short_valid=True): self.pct,self.short_valid = pct,short_valid
    def after_batch(self):
        if self.iter/self.n_iter < self.pct: return
        if self.training:    raise CancelTrainException
        if self.short_valid: raise CancelValidException


# Cell
class GradientAccumulation(Callback):
    "Accumulate gradients before updating weights"
    toward_end,run_before=True,MixedPrecision


    def __init__(self, n_acc=32): store_attr(self, 'n_acc')
    def begin_fit(self): self.count=0


    def after_backward(self):
        self.count += find_bs(self.learn.yb)
        if self.count < self.n_acc: raise CancelBatchException() #skip weight update
        else: self.count=0