'Tokenizer' object has no attribute 'proc_all_mp'

(SH Ho) #1

Hi

I am just getting started with fast.ai by cloning the repo on Github this morning. I am aware of the fact that the notebooks under the course directory (part 2) do not work with fast.ai 1.0. I thus download the imdb.ipynb and moved it away from the fast.ai installation tree and tested it from there. It was able to get pass the first cell “from fastai.text import *”, so it appeared to be free to run on its own…

But when I got to

tok_trn, trn_labels = get_all(df_trn, 1)
tok_val, val_labels = get_all(df_val, 1)

I got the error msg: “‘Tokenizer’ object has no attribute ‘proc_all_mp’”. So it appears that the code can’t locate the additional class that fastai added on top of spacy. Any suggestion why that is the case? It appears that a similar thread was created in the forum last year though the it was supposed to be fixed… But I just downloaded and installed v1.0 today, and the issue resurfaced. Any thought? Has anyone encountered this problem?

Thank you for your help!
SH

3 Likes

(Vlad Getselevich) #2

I think the name changed in new library to Tokenizer().process_all(text)

0 Likes