I am having problems interpreting my training results. I use the default flattened MSE Loss to do regression on a dataset, and RMSE is used as ``metrics’’ when creating the Learner.

However, the problem is, the RMSE is not equal to the root of the valid loss (which is MSE). It is always smaller.

epoch

train_loss

valid_loss

root_mean_squared_error

time

0

0.223325

0.192066

0.437771

00:08

1

0.170728

0.126881

0.355616

00:09

The full code:

I am using FastAI 1.0.60.
Any idea why this happens?

I love these curiosity questions about an anomaly. Sometimes they lead to great discoveries. Sometimes “only” to better understanding.

First, I do not see this issue with fastai2. I am not using fastai1 currently.

If I had to risk a wild guess about fastai1:
valid_loss: the mean of all the errors squared for the whole validation set
root_mean_squared_error: the mean of the square roots of MSE across a smaller group, like batches.
And mean(sqrt()) != sqrt(mean())

You could find out exactly what is happening by putting a debugger on the metrics. Please let us know if you find the right answer!

I find that FastAI 1 calculates the valid loss and RMSE using the same method. It first calculates the score on mini-batches, then doing a weighted average to get the final loss/score, and the weight is the number of data in the mini-batch.