@angelinayy, Nope but the function to convert from .h5 to .pth worked well for me.
1 Like
this worked for some of the senior batches as well
hi, thank you! the convert function worked for me too. did you run something like below to import the .pth and the itos103.pkl files?
I still have a keyerror: KeyError : ‘1.decoder.bias’
Did you have something like this? thanks!
learn = language_model_learner(data_lm, pretrained_fnames=[pre_LM_path, pre_itos2_path], drop_mult=0.3)
WindowsPath(‘data/models/wk103/fwd_wt103.pth’),
WindowsPath(‘data/models/wk103/itos_wt103.pkl’))
Hi,
I have the same problem as @angelinayy. In my case:
First i train on my own datasets LM:
data = TextLMDataBunch.from_csv(dir_name, file_name, text_cols=0)
learn = language_model_learner(data)
learn.fit(epochs=10, lr=0.01)
learn.save_encoder(‘lm_encoder’)
Second i want use this model as pretrained so:
new_data = TextLMDataBunch.from_csv(dir_name, file_name_2, text_cols=0)
learn = language_model_learner(data, pretrained_fnames=[‘path/to/lm_encoder’, ‘path/to/itos’])
and i have the same error:
File “…/fastai/text/learner.py”, line 20, in convert_weights
dec_bias, enc_wgts = wgts[‘1.decoder.bias’], wgts[‘0.encoder.weight’]
KeyError: ‘1.decoder.bias’
Any suggestions or solutions?
fastai==1.0.28
hello, I think this may work. I still have errors but not this one. we can connect on this, but you can try download these 2 files in the below link. thank you!
Yes, i use to save model this:
torch.save(learn.model.state_dict(), 'path/to/lm_encoder.pth')
and then i load the weights in this way:
learn = language_model_learner(data)
learn.load_pretrained(wgts_fname='path/to/lm_encoder.pth'), itos_fname='path/to/itos.pkl')
learn.freeze()
in this way torch.save() save all layers weigth . In original save_encoder() is this:
torch.save(self.model[0].state_dict(), 'path')
@kazutzu that code using v.07 or v.1 ?
and how to load the .h5 before you export to .pth ?
thanks for your time
same issue with KeyError: '1.decoder.bias'. any solution?
What worked for me but is not ideal is to set an empty tensor as the bias, so I modified the script provided by @zubair1.shah to add this line:
new_wgts['1.decoder.bias'] = torch.rand(60002)
It worked for me after that. If the 60002 size doesn’t work for you, try to find out the size of the decoder weight layer by doing old_wgts['1.decoder.weight'].shape and use that.
I am also trying to import an older model from v 0.70 in .h5 format to the new v 1.0 pth format. I am trying to do this for a CNN model. However, this function seems to be specific to RNNs - is there a way to do this for CNNs too?
Thanks in advance.
Hi.
I try it, but get the follow error:
Error(s) in loading state_dict for SequentialRNN:
Missing key(s) in state_dict: “0.rnns.0.weight_hh_l0_raw”, “0.rnns.0.module.weight_ih_l0”, “0.rnns.0.module.weight_hh_l0”, “0.rnns.0.module.bias_ih_l0”, “0.rnns.0.module.bias_hh_l0”, “0.rnns.1.weight_hh_l0_raw”, “0.rnns.1.module.weight_ih_l0”, “0.rnns.1.module.weight_hh_l0”, “0.rnns.1.module.bias_ih_l0”, “0.rnns.1.module.bias_hh_l0”, “0.rnns.2.weight_hh_l0_raw”, “0.rnns.2.module.weight_ih_l0”, “0.rnns.2.module.weight_hh_l0”, “0.rnns.2.module.bias_ih_l0”, “0.rnns.2.module.bias_hh_l0”.
Unexpected key(s) in state_dict: “0.rnns.0.linear.weight_raw”, “0.rnns.0.linear.module.weight”, “0.rnns.0.linear.module.bias”, “0.rnns.1.linear.weight_raw”, “0.rnns.1.linear.module.weight”, “0.rnns.1.linear.module.bias”, “0.rnns.2.linear.weight_raw”, “0.rnns.2.linear.module.weight”, “0.rnns.2.linear.module.bias”.