Lesson 4 - Non-beginner discussion

[ilovescience (Tanishq)]

@sgugger

The idea is that even if you have one wrongfully predicted sample, it’s good that it drags your loss up, and therefore gives a chance to your model to get more accurate.

What about if we have noisy/soft labels? Then is there an alternative to mean that could be used and places less weight in the case of outliers?

Of course, as mentioned, median wouldn’t work since it is not differentiable.