I wasn’t able to find a list of legal parameters for ‘arch’ , or stated differently a list of available models in fastai, or list of available models in fastai.
I am especially interested in arch’s for
text_classifier_learner I’m doing a binary classification problem for where the inputs are 8000 integers.
You can find the list of available models here : https://fastai1.fast.ai/text.learner.html
The model used is given by
config. It can be:
Those are the same for the text_classifier_learner :
Here again, the backbone of the model is determined by
For the classifier, only the last layer is modified. The architecture remains the same.
Note that that is only applicable to fastai v1. In v2 the
TransformerXL are not a thing anymore that can be used by
text_learner. We just have an AWD-LSTM and QRNN
Ah yes, my bad. Thanks for the correction !
Thank you very much your your answers!
So I guess if I want to implement a transformer I will have to dive a a bit deeper down to the mid-levels and follow this guide https://docs.fast.ai/tutorial.transformers