How to use saved model in evaluation for new dataset

Hi,

I’ve followed the new nlp tutorial here: https://docs.fast.ai/text.html to finetune a language model, and add a custom head for sentiment classification.

I then save the models to disk and reload them later in a new script.
I’ve seen a fews posts around the forum where people mention that you have to turn off training mode on the model, otherwise it will still apply regularisations such as dropout.

Should I turn on eval mode for the loaded model with:

learner.model.eval()
# or is it
learner.model.eval = True

Or is this applied when using predict by default?

Regards,
Tim

1 Like

Create another data bunch and reference Classifier error rate > 1? to get metrics.