Using validation mean/std after normalization as an indicator for how good it is for evaluation?

Was thinking about how we normalize both the training and validation set via the mean/std of the training set and was wondering …

After normalization, can we infer how well our validation set reflects the data in the training set by how close its mean = 0 and its variance = 1?

If it is way off, it seems that would indicate that our training set does not reflect what is in the validation set and so our model is therefore likely to generalize poorly.

Does that intuition hold any weight?

Not really. See the Intro to Machine Learning course for ideas on how to test this properly.

2 Likes

Wow, I had completely missed that this course existed! Thank you