OOM on GPU while fit_one_cycle

fastai
1

Hello,

I’m creating language_model_learner from the dataset which contains 18 millions short sentences (in csv it’s about 3GB).
When I start fit_one_cycle, after some time I have out of memory error on my GPU.

Is there possibility to train this model in parts which chunks? How can i do that?

My code:
bs = 32
data_lm = load_data(path, ‘data-lm.pkl’, bs=bs)
learn = language_model_learner(data_lm, AWD_LSTM, config=config, pretrained=False, drop_mult=0.3)
learn.fit_one_cycle(1, 0.01, moms=(0.8,0.7))

2

You can try lowering your batch size. How much GPU memory do you have available?

3

As @muellerzr suggested, you can reduce the batch size. You could also do the following
when you create the TextLMDatabunch,

  1. Reduce batch size
  2. Reduce bptt, since you mentioned the dataset consists of short sentences, you could reduce the bptt to 75 percentile text length in your dataset
  3. Reduce max_vocab size

data_lm = TextLMDataBunch.from_df(output_path,
train_df,
val_df,
bs=bs,
text_cols=‘body’,
max_vocab=60000,
min_freq=2,
bptt=bptt)

4

@muellerzr
I’ve got 16 GB memory on my GPU available.

@soorajviraat
I’m making my databunch with this command:
bs = 64
df = TextList.from_df(df, cols=‘xxx’, processor=SPProcessor.load(path_with_sp)).split_by_rand_pct(0.01).label_for_lm().databunch(bs=bs)

5

AFAIK, databunch is a base class for general use, it’s better to use TextLMDataBunch for LM which is derived from TextDataBunch which is then derived from DataBunch. You can read the code in the below link. https://github.com/fastai/fastai/blob/f75f58f68f94bbf841c2b70d911f46920992e3ba/fastai/text/data.py

Parameters such as
bptt is set inside TextLMDataBunch
max_vocab is set inside TextDataBunch
So you may or may not be able to set it just using base class data bunch

6

I understand, but I’d like to use sentencepiece with it. I’ve got my sentencepiece model trained already. So how can I load it with TextLMDataBunch?