Language_model_learner and custom dataset: RunTime Error

I am just trying to reproduce the code found in the doc (here) where we want to fine-tune a pretrained language model.

I first created a language model with:

train_lm = (TextList.from_df(df=df[:4*split], cols=['sentences']).random_split_by_pct() .label_for_lm() .databunch())

which seems to work well sinceshow_batch. gives me:

idx text
0 xxbos xxmaj we have been to this place many times , and always have great food , wine , and service . xxbos xxmaj my friend xxunk her chicken and xxunk xxunk . xxbos i 'm xxunk up for my next visit . xxbos xxmaj xxunk fresh so brought hot hot hot to your table . xxbos xxmaj the place 's decor and hidden xxunk made for a good xxunk
1 xxunk … its quite xxunk - xxunk , and xxunk your xxunk to make this place look xxunk and modern . xxbos xxmaj price no more than a xxmaj xxunk xxunk but way better . xxbos i love it . xxbos a great place to xxunk up for some food and drinks … xxbos xxmaj the bagel was huge . xxbos xxmaj we ordered some beef and xxunk soup dishes

And then I defined a language model learner:

learn = language_model_learner(data_lm, pretrained_model=URLs.WT103_1)
learn.fit_one_cycle(2, 1e-2)'mini_train_lm')

However, I got the following error:

RuntimeError                              Traceback (most recent call last)
<ipython-input-617-28be5a339a08> in <module>
----> 1 learn = language_model_learner(train_lm, pretrained_model=URLs.WT103_1)
  2 learn.fit_one_cycle(2, 1e-2)
  4 learn.save_encoder('mini_train_encoder')

~\Anaconda3\envs\tensorflow\lib\site-packages\fastai\text\ in language_model_learner(data, bptt, emb_sz, nh, nl, pad_token, drop_mult, tie_weights, bias, qrnn, pretrained_model, pretrained_fnames, **kwargs)
136         fnames = [list(model_path.glob(f'*.{ext}'))[0] for ext in ['pth', 'pkl']]
137         learn.load_pretrained(*fnames)
--> 138         learn.freeze()
139     if pretrained_fnames is not None:
140         fnames = [learn.path/learn.model_dir/f'{fn}.{ext}' for fn,ext in zip(pretrained_fnames, ['pth', 'pkl'])]

~\Anaconda3\envs\tensorflow\lib\site-packages\fastai\ in freeze(self)
181         "Freeze up to last layer."
182         assert(len(self.layer_groups)>1)
--> 183         self.freeze_to(-1)
185     def unfreeze(self):

~\Anaconda3\envs\tensorflow\lib\site-packages\fastai\ in freeze_to(self, n)
175         for g in self.layer_groups[:n]:
176             for l in g:
--> 177                 if not self.train_bn or not isinstance(l, bn_types): requires_grad(l, False)
178         for g in self.layer_groups[n:]: requires_grad(g, True)

~\Anaconda3\envs\tensorflow\lib\site-packages\fastai\ in requires_grad(m, b)
102     if not ps: return None
103     if b is None: return ps[0].requires_grad
--> 104     for p in ps: p.requires_grad=b
106 def trainable_params(m:nn.Module)->ParamList:

RuntimeError: you can only change requires_grad flags of leaf variables. If you want to use a computed variable in a subgraph that doesn't require differentiation use var_no_grad = var.detach().

Any idea where it could come from?


Any guess?

Don’t forget to give the version of fastai you’re using, along with your vesion of pytorch. Ideally run fastai.show_install(1) and copy the result here.

Thanks for your answer. Here it is:

=== Software === 
python version : 3.6.7
fastai version : 1.0.30
torch version  : 0.4.1
torch cuda ver 
torch cuda is  : **Not available** 

=== Hardware === 
No GPUs available 

=== Environment === 
platform       : Windows-10-10.0.17134-SP0
conda env      : tensorflow
python         : C:\Users\XX\Anaconda3\envs\tensorflow\python.exe
sys.path       : 
no supported gpus found on this system

My environment is called tensorflow but Pytorch is indeed installed.

Ah, you don’t have pytorch v1, which is why you see this error.

Ah all right! Ok well I’m gonna work with the older version of fastai then. Thanks!!