I found there is a class in fastai codebase called SmoothenValue:
class SmoothenValue(): "Create a smooth moving average for a value (loss, etc) using `beta`." def __init__(self, beta:float): self.beta,self.n,self.mov_avg = beta,0,0 def add_value(self, val:float)->None: "Add `val` to calculate updated smoothed value." self.n += 1 self.mov_avg = self.beta * self.mov_avg + (1 - self.beta) * val self.smooth = self.mov_avg / (1 - self.beta ** self.n)
while I understand that
self.mov_avg is the (exponentially weighted) moving average, I am not clear what
self.smooth is actually doing…
Isn’t the moving average itself is already smooth? why is
self.smooth applied again to smooth it out?