That sounds great! I’d love to collaborate on this. There are still few things to improve though: the export and predict functions, my code is still running really slow compared to the other model and I leave out a small portion of ULMFIT model (the SortishSampler). I will come back to this soon.
This is the other (faster) model implementation I was talking about: https://github.com/anhquan0412/fastai-tabular-text-demo/blob/master/mercari-tabular-text-version-2-all.ipynb
It does not require writing a new ItemLists and seems to be better in general. Maybe I will rewrite mine using this one.