Training Transformers model using TextLearner

I am trying to train a language model using the model microsoft/DialoGPT-medium from the transformers library from HuggingFace.

learn = TextLearner(dls, model, loss_func=CrossEntropyLossFlat(), cbs=[DropOutput], metrics=Perplexity())

I am unable to use the TextLearner Class to create my learner object.

AttributeError: ‘GPT2LMHeadModel’ object has no attribute ‘reset’.

Can someone pelase suggest how to solve this?

1 Like

You can check out this tutorial on fastai and transformers:
Tutorial - Transformers | fastai.

I also wrote a small library which allows to use most Hugginface models with fastai for text classification and sequence generation. I just open sourced it, maybe it helps you: GitHub - kbressem/fastai-transformers: use huggingface transformers with fastai

1 Like

I recommend blurr to use huggingface models with fastai GitHub - ohmeow/blurr: A library that integrates huggingface transformers with version 2 of the fastai framework

Florian

1 Like