Help: Can't load a downloaded pkl file

Part 1 (2020)
#1

Hi Guys!

I trained my model for about 3 hours and I exported and downloaded the pkl file. But I can’t seem to load it into a learner.

I’ve downloaded the pkl file into kaggle and I used: learn = load_learner("../input/aptos-import/APTOS.pkl") to load it in.

But an error appears when I try to train it: /opt/conda/lib/python3.7/site-packages/fastprogress/fastprogress.py:74: UserWarning: Your generator is empty. warn("Your generator is empty.")

Thanks! :slight_smile:

#2

Can you show more code? Do you mean that you trained the model somewhere else and are trying to load it in a Kaggle Notebook now?

1 Like
#3

I trained the model somewhere else and I’ve exported and downloaded the pkl.

Now I’ve imported the pkl into the Kaggle notebook through datasets. I’m trying to load the pkl onto the learner but it says it’s empty.

My code is just the imports for FastAI (upgraded to v2), loading the pkl, then fit_one_cycle

Thanks for helping! :slight_smile:

#4

Found a related post which might give some more clue:

Have you tried first loading it inside a new, local notebook, to make sure that the problem really is because of Kaggle?

#5

Hello, I have a similar concern with .pkl files. I have a series of textural features saved as pkl files, and when I try to open them in pyCharm (see image attached) they are written in an unfamiliar syntax. How do I fix this to make it readable? The code should read as a series of digits.

image
image1506×916 95.6 KB

1 Like
#6

Hi @leah,

.pkl files are stored in binary format, so they will not be rendered as text in a text editor.

You could save files in a format which are understandable by the text editor and try.

For example, if you have a dictionary stored in your pickle file you could do something as follows

import pickle, json
your_pickle_file_path = "foo/bar.pkl"
with open(your_pickle_file_path, "rb") as f:
    data = pickle.load(f)

# Save this file as a json (If the underlying object is a json)
destination_path = "./destination_file.json"
if isinstance(data, dict):
   json.dump(data, open(destination_path, "w"))

# Save the file as a dataframe (If the underlying object is a dataframe)
import pandas as pd
destination_path = "./destination_file.csv"
if isinstance(data, pd.DataFrame):
   data.to_csv(destination_path)

# Save the file as a text file (if the underlying object is a string)
import pandas as pd
destination_path = "./destination_file.txt"
with open(destination_path, "w") as f:
    f.writelines(data)

It may not be the best idea to store model files which contain weights and biases in text format as you wouldn’t be able to get any actionable or insightful information by viewing them in a text file…

Hope this helps!

Thanks,
Vinayak.

2 Likes