Trouble using load_learner('model.pkl') in Jupyter Notebook

Hi all,

I’m trying to get my ML model onto Hugging Space (lesson 2). When I try to load my model from Kaggle into an app.ipynb (just like the video), I get this error attached below when I call load_learner. My model is very similar to the one in Lesson 1.

I’m very new to the Fastai course and would appreciate any help. Thanks!

It’s hard to guess where the error comes from just by the one line of code. Could you provide a link to your notebook, so we can have a closer look? It seems that there is something missing in the app.ipynb that the learner expected to be there, so you could double check that you ran all cells as in the lesson.

Here’s the code above it. I’m pretty sure I need to declare some functions but I’m not sure why or what.

Here’s the link to the code in model.pkl

I get a 404 error. You need to make the link public.

Sorry, I made it public now

Nothing to appologize for :slight_smile:

So I ran the notebook you linked, exported the model.pkl, created a new notebook, imported the model ,ran the code you provided in the screenshot and it loaded the model happily.
Could you double check, that the model.pkl is the one you expect? Just a guess but maybe there was a mixup and you loaded a different model.

1 Like

I just reran the code on kaggle, renamed the model so I wouldn’t mix anything up, and downloaded it. It’s still throwing the same error.

Ok, that would have been to easy :slight_smile:
Could you share the app notebook too, so I can reproduce the error for myself and have a closer look?

Was your re-run done on Kaggle or Huggingface?
It seems @benkarr’s success was running load_learner on Kaggle.
Lets get load_learner working for you on Kaggle before trying it of Huggingface.

You can work towards isolating your problem by:

  1. Copying your notebook
  2. After the cell-X… learn.export('model.pkl')
    add a new cell-Y… learn = load_learner('model.pkl')
    and see if rest of the notebook continues to wrok as before.
  3. Restart notebook and only execute that new cell-Y - presumably you’ll get an error.
  4. Now execute only the first cell in the notebook, then execute cell-Y.
  5. Now execute the second cell in the notebook, then execute cell-Y.
    …etc, until hopefully the error stops occuring.
1 Like

@benkarr uploaded the app notebook to hugging face

@bencoman hmm so load_learner works if I create the new cell-Y for learn = load_learner(‘model.pkl’), but if i restart the notebook, then cell y never works until i do learn.export

Ok, now I’m confused :sweat_smile:
Where do you execute the app.ipynb that produces the error? As @bencoman suggested, I thought it would be on kaggle.

But using different environments could explain the problem. As hinted before, the problem most likely is that the learner is expecting something that is not provided, this could also be that some library versions differ from saving to loading the model.

Is the error the same? The session storage is not persistent, so the model file would get deleted (at least with a Factory reset).

Ahh. I didn’t realise that. I’ve been using Paperspace for most of the course rather than Kaggle.

As @benkarr explains, thats because a restart deletes the file.
As a safety, add a check guard like…

from os.path import exists
if exists('model.pkl')
    print('Loading model...')
    learn = load_learner(‘model.pkl’)

However after hunting around, I found these thread that indicate there is a way to get persistent storage by doing a “Commit”.

I execute app.ipynb on my local jupyter notebook like what Jeremy does at 41:30 of Lesson 2. Should i create an app.ipynb on kaggle and then import the model there? Then download, and upload to hugging face? I just want to get it onto hugging face so I can use the model on a website when i call the hugging face API.

And yes the error is the same because the model gets deleted

At least that’s what I would do. If you get it to run on kaggle you can still investigate why it didn’t work locally but having anything working helps tremendously since you can compare step for step where it breaks as bencoman described above.

Just a note: using a notebook to generate the code is not mandatory. Huggingface builds the API from and Jeremy chose to use a notebook and nbdev to create it. You could just create that file manually and should be good to go (obviously its good practice to test it before).

Is this problem resolved? Can you please tell me how to resolve, I am encountering same issue while I trained model on my college server, I was trying to deploy model using my local machine which uses windows and I am encountering same issue