I’m trying to familiarize myself with both the Transformers v4 library and the Fastai v2 library. I completed the OP’s tutorial and it seems to be for a text generation language model.
Would I be able to use fastai to train a different type of Transformer model, say, a Question-Answering model?
I fine tuned a GPT2 model with fastai. I can save the model as .pth file or export as .pkl file.
How can i import the model to huggingface with from_pretrained? - I need a .bin and .json file for the model to import from a directory.
You probably need to import the huggingface model (doesn’t have to be the pretrained weights) and then replace the weights in your imported huggingface model with the weights from your fastai model
Sounds good. Thanks!
What is the simplest way to replace the weight in the huggin face model?
I would LOVE it if it’s possible to take a transformer model that I fine-tuned using this library and the FastAI library and cyclical trainer and then output it into a format that allows the model to be imported using “.from_pretrained(model_dir)” or have it showcased on the huggingface website.
Hey! I was trying to access the tutorial at http://dev.fast.ai/tutorial.transformers but it doesn’t seem to open. Says “This site can’t be reached”. Any update on this or alternate links where I can find a tutorial to using the gpt-2 with fastai?
Replace dev with docs in the url. After fastai2’s release the docs were migrated
Thank you that worked!
Hi @sgugger,
have you prepared (or are you planning to prepare) a tutorial showing how to use transformers pretrained models for image classification? Thanks!
