Torch error when predicting on exported/loaded model with fp16 precision

I posted a minimal reproduceable example here:

but the tl;dr is:

after retraining a cnn_learner, export()'ing it, and reloading it with load_learner(), when I then then run reloaded_model.get_preds(…), I get the following error:

“RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.cuda.HalfTensor) should be the same”

Any help would be much appreciated!

2 Likes

hm so if I comment out the .to_fp16() then the script above runs fine with no errors. And I think from this issue it looks like this is probably a bug? https://github.com/NVIDIA/apex/issues/168

…but maybe I’m just missing the correct way to export/reload a model trained with fp16 precision…

In case anyone else hits this problem, I figured out the solution and posted it on StackOverflow:

8 Likes

heyy, Thanks max for this simple solution, this really helped me lesson3-planet notebook

2 Likes