Pytorch implementation of ULMFit

I am curious if anyone has seen a purely Pytorch implementation of ULMFit, i.e. without going through fastai. I think it would be very educational, but was wondering if anyone has done it.
Thanks.

Check out the fastai source code of AWD-LSTM in PyTorch: https://github.com/fastai/fastai/blob/master/fastai/text/models/awdlstm.py#L81

Or the original PyTorch code of AWD-LSTM from Salesforce Research: https://github.com/salesforce/awd-lstm-lm/blob/master/model.py

And I highly recommend lesson 12 from course 2019 part 2 which shows how to code ULMFiT from scratch: https://course19.fast.ai/videos/?lesson=12

3 Likes