I have a dataset where the inbuilt fastai tokenizer is getting quite slow whereas keras tokenizer does it in a matter of few minutes. Is there some trick to make the tokenization fast or should I continue to use keras?
I have a dataset where the inbuilt fastai tokenizer is getting quite slow whereas keras tokenizer does it in a matter of few minutes. Is there some trick to make the tokenization fast or should I continue to use keras?