I was expecting ulmfit to perform at least as good as plain bidirectional lstm but it didn’t happen on my dataset. To be specific, I was following the text classification tutorial using ULMFIT (https://www.analyticsvidhya.com/blog/2018/11/tutorial-text-classification-ulmfit-fastai-library/) but used on my own data with 1300 sentences and 3 classes (negative, neutral, positive). However, the accuracy I got is only 50% with ULMFIT.
Some obviously weird behaviors:
- The classification model doesn’t even perform well on the training data (accuracy 40%), even when I set fit_one_cycle running multiple rounds;
- The plain bidirectional LSTM got accuracy of 70%.