Let’s say I have split my dataset in three : 60% train, 20% val, 20% test.
An overfitted model has its validation error higher than its train error.
One of the usual “cure” in that case would be to add for example dropout.
Indeed that will usually raise the train error (calculated with droupout) and later, after a few iterations, diminish the validation error.
My question is general : to have a well fitted model, should I aim for a train error (calculated with droupout) nearly equal to the validation error ? Or a train error (calculated without droupout) nearly equal to the validation error ?
From what I have seen before, I would opt for answer 1. Then I can stop when training error (calc. with droupout) is for example 0.22 and validation error 0.23. The problem is : for my final well fitted model (that is, calculation made with no droupout), the error the validation and on the train is in that case very different (e.g. on dog breeds : 0.08 (train), 0.23 (val)).
Is it normal ? Is my model really a good fit ?
Thanks a lot for your help, “fasters” !