In fastai version 2.5.3, I’m trying to pass
SaveModelCallback twice to my learn.fit() function:
- To save the model with the best validation loss from the training cycle (’
- To save the model at the very end (i.e., to save the most overfit version of the model, ‘
However, when I run the code below, both
SaveModelCallbacks just save based on best validation (example output below).
learn.fit_one_cycle(100, lr_max = 0.003 , cbs=[ ActivationStats(with_hist=True), SaveModelCallback( monitor = 'mae', comp = np.less, fname = 'best', min_delta = 0.001), SaveModelCallback(fname='end', at_end=True) ] )
Better model found at epoch 0 with mae value: 21.10202980041504. Better model found at epoch 0 with valid_loss value: 805.0863037109375.
So the two questions (possibly related): 1. Why are both callbacks triggering even during the first epoch - shouldn’t the
at_end callback only trigger in the final iteration? 2. Why is
SaveModelCallback(fname='end', at_end=True) not triggering at the end? (I should add - I have run this to completion and have confirmed that the weights for the
end model are not different from those for the
at_end (whose meaning is not described in the documentation) doesn’t do what I expected it to do.