I’m really new to FastAI and to this forum. I’ve just started reading the book through Jupyter pages and I am still at 01_intro. I continue to face the “CUDA out of memory” error when I execute the text_classifier_learner cell. I tried also with the suggestion to specify the batch size. I tried with 32, 16 and even 8 but, no matter what I do, I face the same error every time.
Is this a common issue? Are you aware of any problems from other users?
I am running the notebooks on my personal computer, which is quite new (bought last year). It has an NVidia PH-GTX1050TI-4G, CUDA 10.2, and Ubuntu 20.04 LTS. I am using Juypyter lab.
I did not have any issue with the previous experiments (those regarding image classification).
I thank in advance whoever will answer.
Best regards,
Michele
I ran into that even on sagemaker p2 instances, data loaders has something to do with it I fiddled around and it went away try restarting the Jupiter kernel once you change the batch size
@pineapple1 thanks for your suggestion. Unfortunately, I tried this already and it didn’t work. I tried also to monitor the GPU memory consumption through the nvtop command and I noticed that, obviously, I receive the error as soon as the memory occupation reaches 100% (which, for me, it is 4Gib).
I think there is something into the loader that leaks memory. It’s true that I don’t have a GPU with 8Gib RAM, but … I thought 4Gib should be enough, also because my GPU is quite new.
I made another trial of the same eperiment through Google Colab. In that case I didn’t have any issue. The training completed successfully. Is Colab so generous to provide free GPUs with more RAM than mine? This is the sole hypothesis that comes to my mind.
That’s completely normal. 4 gigabytes of GPU RAM is very, very tiny for deep learning (Colab provides up to 15 GB), and running even small models would be impractical. Receiving memory errors should be expected, especially for certain applications such as U-Nets or language models.
Is there a reason you’re not using Colab? It is free, integrates nicely with Google Drive (for larger datasets), and you don’t need to concern yourself with installing the latest GPU drivers or deep learning packages.
Thank you very much for the clarification @BobMcDear. I could not believe Google was so generous with GPUs and I could not find specifications.
One reason is pure desire of experimentation with what I have. I never did any natural language processing in the past and I was not aware that it is so demanding in terms of GPU. Thanks also for your clarification. I usually like to do as much as possible offline, but I also like Colab, so now, after your explanation, I know there is nothing wrong with fastai and the problem is simply my environment.
The main reason I tend to avoid using GPUs on Colab is that if you do that too often, then Google will interrupt your jobs. I experienced this in the past, with toy reinforcement learning jobs from the “Introduction to Deep Learning” MIT course (that uses TensorFlow). If you have a long-running training job, then Google at some point stops your kernel and refuses to provide you with a GPU. Then you need to wait (maybe some days) before being able to use one again. When you switch the runtime to use a GPU they actually warn you about this possibility.
Well, that said, I think I will continue as much as possible offline and I will switch to Colab when needed.
I understand your avoidance of Colab. When I had started deep learning, I was in a love-hate relationship with it: On one hand, it’s free and its GPUs are actually not bad. On the other, as you mentioned, it continuously disrupts your programs and isn’t a great choice for quick experiments because you have to upload your data every single time (you could move them to Drive, but that comes with its own issues), which is further augmented by its excruciatingly slow networking performance.
Nevertheless, it’s fine for many applications, and compared to other free services like Paperspace, it’s certainly better.