Mixed precision training

Part 1 (2019) Advanced (Part 1 v3)
12

I was also playing with the mixed precision training and was not observing some NaN losses so far.

There seems to be also a callback to stop the training when the loss is getting NaN: https://github.com/fastai/fastai_docs/blob/master/dev_nb/new_callbacks.ipynb (However, I didn’t tested it so far.)

Here is the super explanation from Sylvain: Mixed precision training

PS: Maybe also for your interest: learn.TTA(is_test=True) not supporting half precision models?

1 Like