Suppose I train for several epochs and obtain a model with a low validation loss. Then, I train for a few more epochs and, due to overfitting, the validation loss increases. Finally, I save the model for later use.
Which model will be saved? The one with the lowest validation loss or the overfitted last model? If the latter, How can I make it save the lowest validation loss model?
Same question for predictions after training: which model will it use?
Sorry for the trivial question…I know this in Keras but haven’t figured it out yet in fastai.
learn.save(<MODEL_NAME>) save the weights of the current model inside the learn object. If you use learn.save() after overfitting then you will not have the one with low validation score that you trained before. You can access that model weights by saving the model after you got low validation score.
During prediction the model will be the one you have trained recently. You can access the low validation score model if you have saved that after training for the first time by using learn.load().