Bah! You’re right, @Kagan! It doesn’t train!
Indeed I only tested the stats and assumed the rest would just work! Thank you for checking that my suggestion was wrong!
I tried to re-normalize the stats part again, so std+bias+std, but no, that doesn’t work either.
Re-running lsuv_module twice destroyes training too!
for m in mods: print(lsuv_module(m, xb), lsuv_module(m, xb))
Anybody has an idea why? It looks like it’s very important that the bias is not zero-centered!