Tensor Cores and/or Half Precision support in FastAI/Pytorch?

Does anyone know if the current FastAI (2019) running on Pytorch v1 backend supports (i) tensor core acceleration in current generation GPUs like the NVIDIA RTX 2080 Ti and/or (ii) half-precision operations such that we can train on HD images without running into GPU RAM limitations?

Thanks!!!

fast AI v1 does support half precision models, you just need to call Learner.to_fp16
https://docs.fast.ai/basic_train.html#to_fp16

I am not 100% sure about the tensor cores, but I believe CUDA will just use the tensor cores when they are available for functions that is makes sense.

Beginning in CUDA 9 and cuDNN 7, the convolution operations are done using Tensor Cores whenever possible.

(https://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html)

cuDNN uses Tensor Cores to speed up both convolutions and recurrent neural networks (RNNs).

(https://devblogs.nvidia.com/programming-tensor-cores-cuda-9/)

(Very good reads btw!)

As long as your CUDA and cuDNN is recent enough (which should be the case if you installed according to fastai instructions) your tensor cores should be used since PyTorch uses cuDNN :wink:

1 Like