How to measure effect of added regularization?

If I have some initial value for some_param and I change it, how do I measure the effect of that change? The solutions that I think of include seeing the valid loss (or the test value) value or monitoring the rate of convergence. Something along those lines.

@jeremy can you help clarify this? I will be doing a detailed study on how to apply data augmentation. For example, should we go all out and apply all the transforms in the starting or apply the transforms as training progresses. So I need to measure the effect of adding those transforms. And I can think of only the above options. Are there some other things that can also help me measure the effects?

Sidenote, I will be doing an average of multiple runs (most probably 5) of the transforms because of the randomness involved.