Interesting! Please note that the version plotted is already the smoothened version of the loss (so it’s already an exponential moving average of the loss)
2 Likes
Interesting! Please note that the version plotted is already the smoothened version of the loss (so it’s already an exponential moving average of the loss)