ULMFiT pretraining + pretrained models

[muellerzr (Zachary Mueller)]

@danaludwig

I’ve always found it troubling that when BERT came out, nobody did a head-to-head comparison with ULMFiT

They actually have

BERT vs ULMFiT faceoff

Hi, when pytorch-transfomers was released, I wanted to try it out to see how well it performs and how convenient it is to use. So I got started, one thing led to another and ended up building this BERT vs ULMFiT sentence classfier (trained on IMDB dataset) web app. It is deployed on Google App engine. I have also added an option to provide feedback on reviews where a model does really well (or not) which we can then further analyze. Here’s the link: https://deployment-247905.appspot.com/. Giv…

https://towardsdatascience.com/battle-of-the-heavyweights-bert-vs-ulmfit-faceoff-91a582a7c42b

IIRC overall ULMFiT is still better in generalization scenarios with classification (we still use ULMFiT over BERT at work), though BERT is useful in certain situations