Lesson 2 In-Class Discussion ✅

[marcmuc (Marc P. Rostock)]

answered here

Why does "train-loss > val-loss" mean low learning rate?

From what I understand it’s the following. The model trains on the training set, and the training loss is calculated on that training set. So this loss is calculated on the same dataset the model was trained on and should thus be low. However, the model has never seen the validation set, so when the validation loss is calculated it is on new data for a model only trained on the training set. Hence the training loss ought to be lower than the validation set, because the model was fitted on the…