Great work! I’ve tried to reproduce the notebook but I haven’t found how to get the backbone model used. How could I get this model? Thanks so much!
Hey Daniel, I could share the weights with you so you can load them. I’ll try to upload them to Github tomorrow, if not possible I’ll upload them to GDrive.
Thanks for the useful comments in Github as well.
1 Like
Great! Thanks a lot.
Hi, thanks for sharing your model. The official fastai examples use .h5 format to save their models while yours uses .pt
When I try to torch.load() your language model, I get an AttributeError:
AttributeError: Can't get attribute '_rebuild_tensor_v2' on <module 'torch._utils' from '.../torch/_utils.py'>
This probably results from a version mismatch. Any ideas on how to overcome this (without having to retrain)? I’d really like to test out your pre-trained model.
i am also devote myself to the model’s apply to chinese
i am intrested in you try in chinese
i am also the problem this make me confused
you are lucky i am only 8G
so what your suggest for 8G RAM
Last I looked, it was the .h5 file name but not necessarily the the .h5 format - at least grepping for h5py, it only shows up in the inceptionv4.py.
This error happens when you use an old version of PyTorch. I usually use some form of master, but 0.4.1 should do. If you don’t want to change your main PyTorch version (you should, though!), you can load them with that and then save the model parameters with numpy or even h5py.
Best regards
Thomas
1 Like
when i rum pre_lm.py the errors are fllowing:
dir_path data/en_data; cuda_id 0; cl 12; bs 64; backwards False; lr 0.001; sampled True; pretrain_id
Traceback (most recent call last):
File “pretrain_lm.py”, line 53, in
if name == ‘main’: fire.Fire(train_lm)
File “/home/yhl/anaconda3/envs/fastai/lib/python3.6/site-packages/fire/core.py”, line 127, in Fire
component_trace = _Fire(component, args, context, name)
File “/home/yhl/anaconda3/envs/fastai/lib/python3.6/site-packages/fire/core.py”, line 366, in _Fire
component, remaining_args)
File “/home/yhl/anaconda3/envs/fastai/lib/python3.6/site-packages/fire/core.py”, line 542, in _CallCallable
result = fn(*varargs, **kwargs)
File “pretrain_lm.py”, line 42, in train_lm
learner,crit = get_learner(drops, 15000, sampled, md, em_sz, nh, nl, opt_fn, tprs)
File “/home/yhl/fastai/courses/dl2/imdb_scripts/sampled_sm.py”, line 85, in get_learner
m = to_gpu(get_language_model(md.n_tok, em_sz, nhid, nl, md.pad_idx, decode_train=False, dropouts=drops))
File “/home/yhl/fastai/courses/dl2/imdb_scripts/sampled_sm.py”, line 46, in get_language_model
rnn_enc = RNN_Encoder(n_tok, em_sz, n_hid=nhid, n_layers=nlayers, pad_token=pad_token,dropouti=dropouts[0], wdrop=dropouts[2], dropoute=dropouts[3], dropouth=dropouts[4])
TypeError: init() got an unexpected keyword argument ‘n_hid’
when i run the pretrain_lm.py
Traceback (most recent call last):
File “pretrain_lm.py”, line 53, in
if name == ‘main’: fire.Fire(train_lm)
File “/home/yhl/anaconda3/envs/fastai/lib/python3.6/site-packages/fire/core.py”, line 127, in Fire
component_trace = _Fire(component, args, context, name)
File “/home/yhl/anaconda3/envs/fastai/lib/python3.6/site-packages/fire/core.py”, line 366, in _Fire
component, remaining_args)
File “/home/yhl/anaconda3/envs/fastai/lib/python3.6/site-packages/fire/core.py”, line 542, in _CallCallable
result = fn(*varargs, **kwargs)
File “pretrain_lm.py”, line 42, in train_lm
learner,crit = get_learner(drops, 15000, sampled, md, em_sz, nh, nl, opt_fn, tprs)
File “/home/yhl/fastai/courses/dl2/imdb_scripts/sampled_sm.py”, line 85, in get_learner
m = to_gpu(get_language_model(md.n_tok, em_sz, nhid, nl, md.pad_idx, decode_train=False, dropouts=drops))
File “/home/yhl/fastai/courses/dl2/imdb_scripts/sampled_sm.py”, line 46, in get_language_model
rnn_enc = RNN_Encoder(n_tok, em_sz, n_hid=nhid, n_layers=nlayers, pad_token=pad_token,dropouti=dropouts[0], wdrop=dropouts[2], dropoute=dropouts[3], dropouth=dropouts[4])
TypeError: init() got an unexpected keyword argument ‘n_hid’
can you help me i am looking forward to you reply
You don’t seem to have the latest version of the library (the variable nhid was renamed n_hid in a PR a few months ago). Just do a git pull in the fastai folder.
Thank you for your prompt reply.It make me benfit a lot.
Thank you very much.
Wishes you to be happy daily, has a good mood!
I suggest doing the same in smaller chunks. I would increasingly double the number of chunks till you no longer get a memory error.
1 Like
Thank you for your kind suggest.
I found the question is result form the “np.save()”
So i split the tran.csv solved the question.
Thank you very much.
Wishes you to be happy daily, has a good mood!
1 Like
Hi Fabio,
What perplexity did you get? I’ve also implemented a Portuguese language with ULMFit.
Regarding your question about predicting the next word, I’m using sampling according to the softmax predicted distribution. At least, the texts get a little more “random”…
Cheers,
Monique
@sgugger do you have a pretrained French lm that can be I can use? Could not find one anywhere. Many thanks
1 Like
In just 10 epochs , your perplexity score is around 32 which is pretty good. Would you happen to know the state of the art for German LMs? I can’t seem to find any literature on this.
I haven’t saved one, sorry.