Hi,
using the pretrained weights from fastai1 should be possible with fastai2. At some point there was a change of the hidden dimension size but that should also be fixable: Language_model_learner not working as before?
If you have access to a GPU pretraining the weitghts is not too hard. I recently released a GitHub Repo for ULMFiT training:
If you have any question let me know.
Florian