Binder: how to use larger models

not sure if this has been mentioned, but i made this repository which uses resnet50 in this binder app.


  • after you train your model and export it, upload it somewhere it can be downloaded. i chose dropbox. i also compressed it before i uploaded it but you dont need to. so the file i uploaded to dropbox was export.tar.gz (export.pkl compressed).

  • in your dropbox, on your export.tar.gz file, hit share, then copy link

  • this was my link: you need to change the dl=0 at the end to dl=1. this will download the file instead of going to the file in dropbox. so the link i used was:

  • in your app notebook (replace the url i used with your downloadable dropbox url):

path = Path()
# download_url and file_extract are functions from fastai2

this downloads the link to export.tar.gz in the current folder, extracts it (export.pkl), and deletes export.tar.gz.

  • so now you can do
    learn_inf = load_learner(path/'export.pkl', cpu=True)
    just as if you uploaded your model to github.

i also tried this loading a language model, but couldnt get it to work. the kernel crashes loading the learner. not sure why


Hi @pattyhendrix
I am trying the same thing. However, instead of downloading the model – have you tried reading it from memory as a bytestream ? I have some questions and comments using torch model zoo here

i havent tried that. could you do:

response = requests.get(model_url)

or is there a reason you have to use bytes?

but i think you should be able to use bytes because the torch.load() docs say it can take
a file-like object (has to implement read(), :methreadline, :methtell, and :methseek), or a string containing a file name.
which means bytes would work i think.

the UnpicklingError: invalid load key, '\xff' youre getting might be the bytes being encoded in a format it doesnt like so they dont get read or something like that? i also dont know much about pickle.

this link says torch.load: Uses pickle’s unpickling facilities to deserialize pickled object files to memory. which sounds similar to what bytes would do but im not totally sure