I’ve been trying to create a simple custom n-gram tokenizer that can parse strings of non-Latin language.
I couldn’t find any resource for FastAI V2 about this.
I’ve been trying to create a simple custom n-gram tokenizer that can parse strings of non-Latin language.
I couldn’t find any resource for FastAI V2 about this.