A guide to recovering from CUDA Out of Memory and other exceptions

(Stas Bekman) #1

This thread is to explain and help sort out the situations when an exception happens in a jupyter notebook and a user can’t do anything else without restarting the kernel and re-running the notebook from scratch. This usually happens when CUDA Out of Memory exception happens, but it can happen with any exception.

Please read the guide https://docs.fast.ai/troubleshoot.html#memory-leakage-on-exception and if you have any questions or difficulties with applying the information please ask the questions in this dedicated thread.

If you want to skip reading the guide, fastai-1.0.42 or higher has a built-in workaround just for the CUDA Out of Memory, so if you update your fastai install, chances are you’re already taken care of.

7 Likes

Lesson 3 In-Class Discussion ✅
Developer chat
Developer chat