text_classifier_learner is used for ULMFiT model, it use mean and max pooling output of LSTM.
if i donot use official text_classifier_learner, how to define custom classifier :
only use last output of lstm
add self-attention layer on LSTM for classifier
use CNN build on LSTM output
define own custom classifier loss function
or more generally , how to change text_classifier_learner to any pytorch model ?
is there any implemented code or tutorial or advices ?
thans very much
There is implemented code in the fastai library: you just have to define your custom head for the model like our LinearPoolingClassifier, then define your own function to get the model like this one. Then, you may have to create your own split for the model if the default text classifier one doesn’t work, but it should be all you need before adapting the code of
thanks a lot .
but I could not find the definition or pass of loss_function in these functions?
may i ask that where is the loss function definition part ?
The loss function is a flattened version of cross entropy (as
learn.loss_func will tell you). There is just the
RNNTrainer callback that gets in between, to return only the output (since the model spits more things).