Hi all, have any of you already pretrained the MultiFiT model on English Wikipedia data, using the method proposed in the paper by Julian Eisenschlos, Sebastian Ruder, Piotr Czapla, Marcin Kardas, Sylvain, and Jeremy of the fast.ai community?
(MultiFiT is an improved, more efficient version of ULMFiT, with subword tokenizaton, QRNNs, 1cycle policy, label smoothing, etc. It seems that pretrained models are only available in other languages in the official repo.)
If you have pretrained it and you could share the weights, it would be much appreciated. Thanks in advance!
I’m planning to do some fine-tuning experiments with it with different resource constraints, on English classification datasets, comparing against other models (e.g. ULMFiT, BERT). If noone has pretrained this model on English Wikipedia yet, I’ll try to do so, although I have limited hardware access currently.