I was expecting the later to be 0.108513, the same score as reported against the validation set at the end of training.
Because during training we’re taking the average of mini-batches, which is a bit different to taking the average of the whole dataset.
1 Like
I take it that this is true for all “metrics”?
And, is there a particular reason it has to work this way and/or this is the way its being done?
Thanks.
Nope. Have a think about the arithmetic of what’s being done in each case, and see if you can tell us which metrics can be correctly averaged over mini-batches, and which can’t.
And if you want a further exercise, I can give you a tip as to how to fix exp_rmspe so it actually works correctly… 
I’m going to say that anything that does not include calculating a variance of some sort (e.g., accuracy, error_rate) will be correct while metrics like dice and exp_rmspe will not. For exp_rmspe I would say it is because of this line … pct_var = (targ - pred)/targ, the values will be different averaging this across minibatches vs. looking at the entire dataset in one go.
If I’m right, or even in the ballpark, I’d be glad to fix exp_rmspe. If not, you probably don’t want me touching it.
1 Like