I’m looking for a simple code example that demonstrates how a pre-trained LM is updated on a new dataset. The version presented in the course uses the fastai library, but I’m interested in seeing a version in plain pytorch. Most examples out there use models such as BERT and GPT but I’m looking for something basic and barebones, like vanilla LSTM. If you’ve come across any such example, it would be immensely helpful to me.
you might check out the 12_nlp_dive.ipynb notebook in Jeremy and Sylvain’s fastbook. There is a code walk through showing how to create an LSTM model from scratch. The LSTM model is implemented in plain pytorch but the model training part is implemented using fastai version 2 (fastai2).
Thank you, Farid. I’m looking for an example that shows how a pre-trained LM is updated on a new dataset. The course notebooks provide examples via fastai APIs but I’m curious what the pure pytorch implementation would look like.
Sorry, I didn’t come across any example showing that.
Thank you, I’ll keep looking. If I end up creating my own, I’ll share the example here.
1 Like