4GB of GPU RAM insufficient for IMDB text classification task?

Hi everyone,

I have trouble to get the IMBD text classification model to train within the first notebook (01_Intro.ipynb)

I followed the advice of reducing the batch size. However, I get a “CUDA out of memory” error even when I reduce the batch size to the minimum amount of 2:

from fastai.text.all import *

dls = TextDataLoaders.from_folder(untar_data(URLs.IMDB), valid='test', bs=2)
learn = text_classifier_learner(dls, AWD_LSTM, drop_mult=0.5, metrics=accuracy)
learn.fine_tune(4, 1e-2)

CUDA out of memory. Tried to allocate 92.00 MiB (GPU 0; 3.95 GiB total capacity; 3.18 GiB already allocated; 40.19 MiB free; 3.34 GiB reserved in total by PyTorch)

I have a GTX1050Ti with 4GB of VRAM.

According to nvtop the kernel of Jupyter notebooks uses 3975MiB of VRAM within the GPU when the training dies with the remaining 54MiB being used by the X server.

Is this amount of memory consumption to be expected or is there any workaround for this?

I understand that large models require lots of memory on the GPU but I am surprised that 4GB of VRAM are not enough for a simple text classification/sentiment analysis task.

Many thanks!
Christian

Hey Christian,

Unfortunately, older GPUs often aren’t compatible with newer versions of CUDA/software in addition to being slow and having very little VRAM. Standard GPUs have 11GB VRAM, and with less than 8 you will struggle. In addition to the input data (which is affected by the batch size), you also need to copy the model and calculate gradients on the GPU, and that alone can usually take more than 4GB. I recommend you try Google Colab or any of the other options mentioned in the documentation, they all provide you with a good GPU for free/cheap.

2 Likes

Dear orendar

Thanks a lot for the helpful reply. I had wondered whether the main issue might be the size of the model itself, not just the size of the training batch. It’s helpful to have that confirmed.

I will try the cloud-based options then.
Many thanks!
Christian

1 Like