Language Model Zoo 🦍

[s.tsuruno (Shun Tsuruno)]

Hi everyone. I’ve applied ULMFiT to Japanese and started a thread. Let me know if you’re interested.

ULMFiT - Japanese

Continuing the discussion from Multilingual ULMFiT: I’ve been working on applying ULMFiT to Japanese language using fast.ai v1. The code and the pretrained models can be found here. Summary Pretrained a sentencepiece model for tokenization using Wikipedia dump Pretrained a language model using Wikipedia dump (100M tokens) Fine-tuned the language model and trained a classifier for the following datasets Aozora Bunko MedWeb Details of the classification tasks I could not find any famou…