Exporting a text_classifier_learner

Part 1 (2019) Advanced (Part 1 v3)
1

I have read some posts around the forums about exporting learners and loading them again, and have no trouble doing this. My code to do so is as follows:

learn_classifier.export(model_save_path / "export.pkl")
#Load the model again
learn_classifier = load_learner(model_save_path)

I am working with a text_classifier_learner and as such I first train a language_model_learner, train it on the data, then attach it as an encoder to the text_classifier_learner and finally train on the classification data.

My question is whether the export function for a text_classifier_learner also saves the encoder? Or when you reload it does it use some sort of default encoder.

Edit: Using version 1.0.52

2

I see no reason for it to not use the right weights for the encoder. The entire model is saved alongside the learner when you do the above.

3

The saved model size is only 82mb though…seems very small to have a saved encoder as well considering the AWD_LSTM download is about 1.8gb:

image
image.png1296×519 50.3 KB