Language_model_learner runtimeError

#1

hi, does anyone have an issue like this? Thank you very much indeed! :slight_smile:

I’m running in local python. The only change I made was to use pretrained_fnames rather than URLs. I downloaded the files from http://files.fast.ai/models/wt103_v1/.
I also updated the library to v28. The error is from running the below one line of code.
learn = language_model_learner(data_lm, pretrained_fnames=[pre_LM_path, pre_itos2_path], drop_mult=0.3)
#WindowsPath(‘data/models/wk103/lstm_wt103’), - LM_path
#WindowsPath(‘data/models/wk103/itos_wt103’)) - itos_path

RuntimeError Traceback (most recent call last)
in ()
----> 1 learn = language_model_learner(data_lm, pretrained_fnames=[pre_LM_path, pre_itos2_path], drop_mult=0.3)

C:\ProgramData\Anaconda3\lib\site-packages\fastai\text\learner.py in language_model_learner(data, bptt, emb_sz, nh, nl, pad_token, drop_mult, tie_weights, bias, qrnn, pretrained_model, pretrained_fnames, **kwargs)
115 fnames = [learn.path/learn.model_dir/f’{fn}.{ext}’ for fn,ext in zip(pretrained_fnames, [‘pth’, ‘pkl’])]
116 learn.load_pretrained(*fnames)
–> 117 learn.freeze()
118 return learn
119

C:\ProgramData\Anaconda3\lib\site-packages\fastai\basic_train.py in freeze(self)
181 “Freeze up to last layer.”
182 assert(len(self.layer_groups)>1)
–> 183 self.freeze_to(-1)
184
185 def unfreeze(self):

C:\ProgramData\Anaconda3\lib\site-packages\fastai\basic_train.py in freeze_to(self, n)
175 for g in self.layer_groups[:n]:
176 for l in g:
–> 177 if not self.train_bn or not isinstance(l, bn_types): requires_grad(l, False)
178 for g in self.layer_groups[n:]: requires_grad(g, True)
179

C:\ProgramData\Anaconda3\lib\site-packages\fastai\torch_core.py in requires_grad(m, b)
99 if not ps: return None
100 if b is None: return ps[0].requires_grad
–> 101 for p in ps: p.requires_grad=b
102
103 def trainable_params(m:nn.Module)->ParamList:

RuntimeError: you can only change requires_grad flags of leaf variables. If you want to use a computed variable in a subgraph that doesn’t require differentiation use var_no_grad = var.detach().

2 Likes

Loading pre-trained weights from a local file rather than from a URL
(fawaz almerbid) #2

I faced some errors when I tried to run the language model before and I solved it by reducing the batch size “bs” to 25…

so try to reduce the batch size to 25 and 20 and try… I think it will solve your problem.

0 Likes

(Tamlyn Rhodes) #3

I’m getting this error too. Tried dropping the batch size down to 10 but it doesn’t help.

0 Likes

(Tamlyn Rhodes) #4

I get the same error when running the unmodified IMDb notebook on Kaggle. :frowning:

UPDATE: got it working again, I think what did it was updating torch_nightly:

!pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cu90/torch_nightly.html
0 Likes

#5

Figured this out - my server basically lacked GPU memory and so restarting the kernel to free up that memory worked (and the error messages from pytorch_core weren’t reflecting the out of memory error)

Training the language model and then training the classifier on this salamander hardware
image
won’t cut it.

So I

  • restarted the notebook
  • loaded the fine tuned language model
  • all succeeding training steps for the classifier worked correctly
0 Likes