I found there is a class in fastai codebase called SmoothenValue:
class SmoothenValue():
"Create a smooth moving average for a value (loss, etc) using `beta`."
def __init__(self, beta:float):
self.beta,self.n,self.mov_avg = beta,0,0
def add_value(self, val:float)->None:
"Add `val` to calculate updated smoothed value."
self.n += 1
self.mov_avg = self.beta * self.mov_avg + (1 - self.beta) * val
self.smooth = self.mov_avg / (1 - self.beta ** self.n)
while I understand that self.mov_avg
is the (exponentially weighted) moving average, I am not clear what self.smooth
is actually doing…
Isn’t the moving average itself is already smooth? why is self.smooth
applied again to smooth it out?