Before update 1.0.44, I used to save my vocab using the TextLMDataBunch.save method. It created a itos.pkl file that I could load to do classification. Since the last update, calling save on a TextLMDataBunch will generate a single file that contains the information needed to load the databunch but no itos.pkl . Is this intentional? And if it is, what is the best way to save my LM Vocab to load it with my classifier?
1 Like
That’s because save now works for every kind of DataBunch
. There are two ways to get your vocab for your classifier:
- manually save your vocab by pickling
data_lm.vocab.itos
- load your data for the LM with
load_data
then passdata_lm.vocab
I will add a save/load function for the Vocab
class to make this easier.
4 Likes
Thank you for your answer. Shall I submit a PR to add the save/load function for the Vocab class?
If you want to tackle this, go ahead yes!
super…its working
Hi !
Can you provide a link to how we can save our vocab thanks to your vocab ?
Thanks in advance !