Runtimeerror cuda out of memory

Part 1 (2019)
1

I’m getting the following error on lesson3-1-planet.ipynb:

RuntimeError: CUDA out of memory. Tried to allocate 32.00 MiB (GPU 0; 7.43 GiB total capacity; 6.86 GiB already allocated; 28.94 MiB free; 18.26 MiB cached)

I’ve read similar threads and the docs guide on this issue and tried the following:

  • Set the batch size to 2
  • No other notebooks running
  • Restart kernel before running
  • confirm nothing is running on Nvidia-smi before starting

None of these work. I’m using Google Cloud Platform with n1-highmem-8 1 x NVIDIA Tesla P4. Does anyone have any ideas for how to fix this please?

3 Likes
2

First, I made sure that I do have memory available:

(base) jupyter@my-fastai-instance:~/course-v3$ top
top - 17:31:51 up  2:42,  1 user,  load average: 0.01, 0.57, 0.87
%Cpu(s):  0.0 us,  0.1 sy,  0.0 ni, 99.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 53595224 total, 45764032 free,  2758672 used,  5072520 buff/cache
KiB Swap:        0 total,        0 free,        0 used. 49129556 avail Mem

Next, I restarted the kernel in my Jupyter notebook and only ran the section I cared about. This error happened to me at the later stages (going from 128 to 256 images)