Text_classifier_learner Everytime different shape and because of that I can't save learner

Part 2 (2019)
1

I have this code:

data = (TextList.from_csv('.', 'simple_test_data_text_included.csv')
         .random_split_by_pct()
         .label_from_df(cols=[1,2,3],label_cls=FloatList)
         .databunch(bs=8))

learn = text_classifier_learner(data, AWD_LSTM, pretrained=True, drop_mult=0.3)

learn.load('classifier')

learn.fit_one_cycle(1, 2e-2, moms=(0.8,0.7), wd=0.1)
learn.fit_one_cycle(1, 2e-2, moms=(0.8,0.7), wd=0.1)
learn.freeze_to(-2)
learn.fit_one_cycle(1, slice(1e-2/(2.6**4),1e-2), moms=(0.8,0.7), wd=0.1)
learn.freeze_to(-3)
learn.fit_one_cycle(1, slice(5e-3/(2.6**4),5e-3), moms=(0.8,0.7), wd=0.1)
learn.unfreeze()
learn.fit_one_cycle(2, slice(1e-3/(2.6**4),1e-3), moms=(0.8,0.7), wd=0.1)

learn.save('classifier')

First time the third row (learn.load('clas…) is commented. Code works normally and in the end I save the learner. Then I restart my kernel and run this code again but this time I load old learner. That doesn’t work and it print this error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-c852ac948b34> in <module>()
----> 1 learn.load('classifier')

~/Documents/Python/fastai/fastai/basic_train.py in load(self, name, device, strict, with_opt, purge)
    246         state = torch.load(self.path/self.model_dir/f'{name}.pth', map_location=device)
    247         if set(state.keys()) == {'model', 'opt'}:
--> 248             get_model(self.model).load_state_dict(state['model'], strict=strict)
    249             if ifnone(with_opt,True):
    250                 if not hasattr(self, 'opt'): self.create_opt(defaults.lr, self.wd)

~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
    767         if len(error_msgs) > 0:
    768             raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
--> 769                                self.__class__.__name__, "\n\t".join(error_msgs)))
    770 
    771     def _named_members(self, get_members_fn, prefix='', recurse=True):

RuntimeError: Error(s) in loading state_dict for SequentialRNN:
	size mismatch for 0.module.encoder.weight: copying a param with shape torch.Size([2020, 400]) from checkpoint, the shape in current model is torch.Size([1949, 400]).
	size mismatch for 0.module.encoder_dp.emb.weight: copying a param with shape torch.Size([2020, 400]) from checkpoint, the shape in current model is torch.Size([1949, 400]).

Is this error in Fastai code or am I using this incorrectly? It seems like that it every time change the shape and because of this it is not working.

simple_test_data_text_included.csv

text money travel sport
0 Debenhams chairman Sir Ian Cheshire is to step… 1 0 0
1 The FTSE 100 closed up 36.24 points at 6942.87… 1 0 0
2 Car giant Ford is set to axe thousands of jobs… 1 0 0

There is a news article and then category for it.

I’m using latest fastai which is 1.0.47.dev0.

2

That would be because you don’t specify a fixed vocab, so with the random split, a new one is created each time. I’d suggest saving your data once created and then load the save with load_data.

Also please post this kind of general fastai problems in the fastai users category as it can be helpful to others and this category is restricted for now.

4 Likes
3

I was running into this issue recently when I was loading the encoder from my language model to the classifier. I think it’s because of the way I was doing the splits of my data, where there is some I didn’t use at all for the LM (I was holding it out as the validation for the classifier).

In that case, when I was creating the data for my classifier it looked like the following, where I specify the vocal to be the same as that of the data used in my LM:

data_clas = TextDataBunch.from_df(path, df_trn_tagging, df_val_tagging, text_cols = 0, label_cols = 1, bs=48, vocab=data_lm.vocab)

1 Like
4

Thank you for giving answer. The sgugger’s answer fixed the problem but it was interesting to know another similar solution in case I some point face this problem.

5

Adding on to what people have already said.

I’m working with some custom text loading/processing that doesn’t pay well with the new save function. I’ve found the easiest solution is to use numpy to save the itos array in the vocab object after creating a language model. Then it’s easy to load the array and instantiate a vocab class and pass that to any dataloaders you make.

I’d also recommend you create a single defined validation/test set at the start, and add a column to the dataframe defining what rows go to what dataset. It makes a lot of things easier and more reproducible.