Load data and model

Part 1 (2019)
1

I have developed a collab recommendation model, when the training is done I use the following code:

learn.export(file = Path(“mypath/model.pkl”))

I then load the model later using:

learn = load_learner(Path(’/mypath/’), ‘model.pkl’)

When I try to work with this model, for example use it to generate predictions for users, using the code developed in this other post: [Add New Feature Collab] Recommend Top N for Item/User

I get an error with this code learn.data.train_ds, I assume it is because, only the model and not the data is being loaded.

The code work fine when i train and test predictions in one go, but not when i load an previously trained model.

How can I solve this and save both the model and data so that the saved model can be load and ready for use.

2

When you run learn.export, it only saves the model weights and some additional info required for inference (activation function, train and val augmentations, etc).
It is meant to store awak models for inference specifically.

If you want to train further with the exported model, I’d suggest something like:

learn_exp = load_learner(...) # load exported model

learn_train = cnn_learner(...) # create a new `Learner` with the same setup as the previous one

# Both of these should work. The first is safer
learn_train.model.load_state_dict(learn_exp.model.state_dict())
learn_train.model = learn_exp.model

In the future, if you want to load models and continue saving, it’s better to save the model rather than export it.

learn1 = cnn_learner(...)
learn.fit(...)
learn.save('tmp_model')

### when resuming...

learn2 = cnn_learner(...)
learn.load('tmp_model')
3

I tried running the code you provided and I get the following Error:

No such file or directory: ‘models/tmp_model.pth’

4

Check learn.path and learn.model_dir, those are used to locate the model to load. Modify accordingly

5

learn.model_dir returns ‘models’

I am running this code on google colab, not sure if that affects it?

I don’t understand why I cant just save the model to a specific dir and then load it from that dir?

6

The model is loaded from learn.path/learn.model_dir/{fname}