MultiFit with 16fp - retrain LM?

Hi, I have been actively using MultiFit and it works actually quite well. Thanks for making your work available.

I would like to train at half-precision.
When I set in the architecture fp16 to True, everything runs but the accuracy numbers are very low.

Do I need to pre-train a new language model LM on fp16 or should it work with my current LM (paperversion), which is fp32?

Thanks for your response