Pkl and pth

Can someone help me understand the difference between the .pkl and the .pth files? I was looking at the repo for Render deployment and saw that one of the files has the message, “Download export.pkl in the current dir”, but when I open that file it instead says, “The .pth files should get downloaded here.”

I know the .pkl file has my full serialized model. So what’s the .pth file?

3 Likes

pkl stands for a “pickle” file which is a way of serializing objects in Python. Its contents can be almost anything; it just depends on what was serialized.

The pth file is your model’s weights (and optimizer state if saved with_opt).

3 Likes

Ah, great! So is there an advantage to using the pth file instead of the pkl file for my model deployment? For example, the pth file is more lightweight?

Depending on the model you may need both.

For example, for the pre-trained language models this is what gets downloaded behind the scenes:


A pkl file containing the vocabulary (integer -> string mapping) and a pth file containing the weights of the network.

On the other hand, for computer vision models, only the weights need to be downloaded:

image

8 Likes

Also, I somehow recall that
pkl is for deploying to production.

Am I correct?

Were you correct ? I have the same doubt.

is there a way to convert a pth to a pkl file? cant load a pth file using load_learner()

2 Likes

How do we convert from .pkl to .pth? I’ve tried using this but it just returns a string of integers:

with file.open('export.pkl', 'rb') as fid:
  print(pickle.loads(fid.read())) # => 102846209128121496
  # what now?