Hi everyone,
I’ve been working on the ULMFIT approach and manage to train a language model in the Persian language.
The data That I’ve used was collected from various blog posts, news articles, and Wikipedia articles(~188k). Soon I’ll publish the dataset so that it will be available for further experiments.
Also, I finetuned it and got better results than parsbert for the same dataset.
for details, you can refer to these two blog posts:
Finally, I wanted to thank @florianl and @muellerzr for their insight. 