Before update 1.0.44, I used to save my vocab using the TextLMDataBunch.save method. It created a itos.pkl file that I could load to do classification. Since the last update, calling save on a TextLMDataBunch will generate a single file that contains the information needed to load the databunch but no itos.pkl . Is this intentional? And if it is, what is the best way to save my LM Vocab to load it with my classifier?
That’s because save now works for every kind of
DataBunch. There are two ways to get your vocab for your classifier:
- manually save your vocab by pickling
- load your data for the LM with
I will add a save/load function for the
Vocab class to make this easier.
Thank you for your answer. Shall I submit a PR to add the save/load function for the Vocab class?
If you want to tackle this, go ahead yes!
Can you provide a link to how we can save our vocab thanks to your vocab ?
Thanks in advance !