Can you pass vocab file to TextBlock.from_df()

I am trying to train a language model from scratch.

I managed to train a tokenizer and save the tmp file with the model and vocab files to my google drive but I was logged out of colab and was unable to save the dataloaders.

To use pretrained tokenizer instead of retraining it, would I just do something like this?

TextBlock.from_folder(path, vocab='spm.vocab', is_lm=True, tok=None)

Instead of

TextBlock.from_folder(path, vocab=None, tok=SubWordTokenizer())


Did you try the above and it worked for you?

I think you have to add your spm.model file as an argument to SubWordTokenizer() and read your spm.vocab file into a list object and add that as an argument to .from_folder()