I am building a text classification model.
When I first trained the model and had it make predictions on the validation set, I got an accuracy of 82.2%. However, upon restarting my jupyter notebook kernel and loading it, I get 69.6% accuracy instead.
I run the following to save the model:
learn.save('modelname')
(learn is a text classifier learner)
When I load the model, I run the following cell:
# Language model data
data_lm = TextLMDataBunch.from_folder(DATA_PATH)
# Classifier model data
data_clas = TextClasDataBunch.from_folder(DATA_PATH, vocab=data_lm.train_ds.vocab, bs=45)
learn = text_classifier_learner(data_clas, AWD_LSTM)
learn.load('modelname')
learn.load_encoder('encodername')
learn.model.eval()
Does anyone know what the problem is here? Is there something I’m doing wrong or is this a bug?