NLP in other languages (ancient books)-Week 4

[Moody (Sarada Lee)]

@bahman_apl you can download different languages from the Wiki by following the instruction here:

Language Model Zoo

To help you to get started, here is the procedure to download data from Wikipedia. Go to Wikimedia https://dumps.wikimedia.org/ Click on the “Database backup dumps” (WikiDumps) link. (It took me a while to figure out it is a link!) [image] There will be a long list inside the WikiDump. In this example, I pick ‘zh_yue’ for Cantonese (a subset of Chinese) and download it. (Warning: some of the file are very big) [image] Git Clone from WikiExtractor (https://github.com/attardi/wiki…

You may find the thread useful as well. Others trained a Persian model previously. Language Model Zoo 🦍