Hi all,
I am trying to continue training the language model generated by the lesson-3-imdb notebook on a different dataset. I would like to continue from the fine tuned model that Jeremy trained and not start from a new Ulmfit model. The dataset that I would like to continue training it on is the Yelp customer reviews dataset that comes with fastai as well.
Things seem to go well when I use the entire training dataset, and I believe that this working properly:
data_lmYelp = (TextList.from_csv(path=Path('yelp/yelp_review_polarity_csv/train/'), csv_name= 'train.csv', cols=1)
.split_by_rand_pct(0.1)
.label_for_lm()
.databunch(bs=bs))
data_lmYelp.save('data_lmYelp_newnb2.pkl')
data_lmYelp = load_data(path,'train/data_lmYelp_newnb2.pkl', bs)
data_lmYelp.show_batch()
learn = language_model_learner(data_lmYelp, AWD_LSTM, drop_mult=0.3)
learn.load('lm/models/fine_tuned')
and everything seems to work properly. That fine_tuned
is the fine_tuned.pth
file from the imdb notebook.
However, when I create a smaller version of the dataset, either by creating a new csv with 10,000 samples from the Yelp dataset or importing a subset of the dataset dataframe into the databunch, I get errors.
So the code is the same as above, I’m just importing a csv with only 10,000 samples instead of the entire training set (I can share the code of the from_df
if that matters but it’s the same error) and after the last line above I get:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-63-67473bdf2756> in <module>
1 learn = language_model_learner(data_lmYelp, AWD_LSTM, drop_mult=0.3)
----> 2 learn.load('lm/models/fine_tuned')
/usr/local/lib/python3.7/dist-packages/fastai/basic_train.py in load(self, file, device, strict, with_opt, purge, remove_module)
271 model_state = state['model']
272 if remove_module: model_state = remove_module_load(model_state)
--> 273 get_model(self.model).load_state_dict(model_state, strict=strict)
274 if ifnone(with_opt,True):
275 if not hasattr(self, 'opt'): self.create_opt(defaults.lr, self.wd)
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
828 if len(error_msgs) > 0:
829 raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
--> 830 self.__class__.__name__, "\n\t".join(error_msgs)))
831 return _IncompatibleKeys(missing_keys, unexpected_keys)
832
RuntimeError: Error(s) in loading state_dict for SequentialRNN:
size mismatch for 0.encoder.weight: copying a param with shape torch.Size([60000, 400]) from checkpoint, the shape in current model is torch.Size([12136, 400]).
size mismatch for 0.encoder_dp.emb.weight: copying a param with shape torch.Size([60000, 400]) from checkpoint, the shape in current model is torch.Size([12136, 400]).
size mismatch for 1.decoder.weight: copying a param with shape torch.Size([60000, 400]) from checkpoint, the shape in current model is torch.Size([12136, 400]).
size mismatch for 1.decoder.bias: copying a param with shape torch.Size([60000]) from checkpoint, the shape in current model is torch.Size([12136]).
Does anyone know what is going on?
Thanks!