Imdb_scripts: Command line arguments for 95% ULMFIT performance?

(Sam Shleifer) #1

I am wondering how to invoke to achieve the 95% accuracy reported in the ULMFit paper.

in imdb_scripts/, it says setting cl=50 is optimal so I followed the imdb notebook to pretrain the language model,
then ran

 python imdb_scripts/ /home/paperspace/ 0 \
    --cl 50

(took 4h)
then ran

 python imdb_scripts/ /home/paperspace/ 0

and got only 94%.

Are there other command line args to pass the script (which is slightly different than the rest of the imdb notebook) to achieve 95% on the IMDB data?

Related post: ULMFiT train_clas training around 50% on IMDB


(Sam Shleifer) #2

This now works for forward, using the checked in notebook, but I can’t get equivalent performance for backwards. Has anybody tried?