Error when trying to load tabular Pickle file- "'PosixPath' object has no attribute 'load'"

Part 1 (2020)
1

I’m following along with Chapter 9, and have reached the point where we’re persisting our instance of TabularPandas to a Pickle file:

to = TabularPandas(df, procs, cat, cont, y_names=dep_var, splits=splits)
save_pickle(path/'to.pkl',to)

This all works fine. The problem happens when I try to use the provided example code to read the Pickle file into memory:

to = (path/'to.pkl').load()

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-44-e1b539368431> in <module>
----> 1 to = (path/'to.pkl').load()

AttributeError: 'PosixPath' object has no attribute 'load'

I ran help(path/'to.pkl') to see if there is in fact a load() method on the PosixPath class, and I didn’t find one. I also checked the docs, and didn’t find one there either.

After a bit of research, I found this article, which talks about how to Pickle and un-Pickle scalars and objects in Python. This was educational, and I attempted to import the pickle library and run pickle.load(path/'to.pkl'), but got a new error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-46-178322946f3f> in <module>
----> 1 to = pickle.load(path/'to.pkl')

TypeError: file must have 'read' and 'readline' attributes

This is strange since I’ve verified that the path is in fact a file:

os.path.isfile(path/'to.pkl')
True

os.path.isdir(path/'to.pkl')
False

I feel like this chapter is unlikely to be wrong since it has been read and re-read many times by this point. I’m sure I’m missing something, I just don’t know what it is. Anybody see what I’m doing wrong?

2

In (path/'to.pkl').load() the load() method is provided by fastai and not by the python pathlib library.

The first error that you are getting is because of that. This usually occurs if you are trying to use the load method without importing fastai.

So, I must ask that are you trying to load the pickle file using the (path/'to.pkl').load() code in another notebook where fastai is not imported first?

3

Thanks @sapal6. Unless I missed something, I’m using all the import statements which were provided at the top of the Jupyter Notebook for Chapter 9:

!pip install -Uqq fastbook kaggle waterfallcharts treeinterpreter dtreeviz
import fastbook
fastbook.setup_book()
from fastbook import *
import kaggle
from kaggle import api
from pandas.api.types import is_string_dtype, is_numeric_dtype, is_categorical_dtype
from fastai.tabular.all import *
from sklearn.ensemble import RandomForestRegressor
from sklearn.tree import DecisionTreeRegressor
from dtreeviz.trees import *
from IPython.display import Image, display_svg, SVG

pd.options.display.max_rows = 20
pd.options.display.max_columns = 8
4

There is an equivalent load_pickle(path/'to.pkl') available (fastcore) as well. Can you try now?

or

using Pickle (basically the same load_pickle function):

f = open((path/'to.pkl'), 'rb')
pickle.load(f)
1 Like
5

Everything seems to be in place in the code you shared and the load() method should work.

You can tryout the options suggested by @thatgeeman . Since the file you had earlier saved is a pickle file, usin the regular open file and then pickle.load() should also work.

I will also try to run the chapter 9 code and see if I can reproduce the issue.

6

Thanks @thatgeeman and @sapal6, I did see load_pickle mentioned in the chapter, and that works fine in terms of loading the serialized object into memory. To clarify, my question is less around getting the pickle file to load (since load_pickle seems to do the trick), and more about why the line to = (path/'to.pkl').load() doesn’t seem work for me.

I should also note that just now I inserted the above .load() line into my copy of the original chapter 9 Jupyter notebook, just below the code snippet save_pickle(path/'to.pkl',to) and above the text snippet To read this back later, you would type:, and tried to run the entire notebook from start to finish with no other modifications to the chapter. I got the same error that I originally reported above:

Screen Shot 2021-06-03 at 6.39.09 AM
Screen Shot 2021-06-03 at 6.39.09 AM1710×844 94.5 KB

I thought about sharing a Voila link to the notebook in question, however I’ve been unable to get Voila itself setup properly on my side. I keep getting 404 links when I try to click the Voila link at the top of my notebook, but that’s a separate problem. So until I get that fixed, all I can do is ask whether anyone else has been able to get .load() to work successfully on their machine.

7

Hi, as far as I understand the line you refer to was fixed (see issue here) in the executable cells of the notebook. The comment To read this back later.. probably went unnoticed and remained in the main text. Maybe you can do a PR :slight_smile:

8

Thanks @thatgeeman . After reading that issue you linked to, Jeremy mentioned that any similar problems should be reported in a new Github issue. I filed one here, so hopefully that’ll do the trick.

Nice find, locating that original issue. Thanks for the help.

2 Likes
9

@toomanyrichies I think I misunderstood your original issue. I think the issue you raised on the branch ahould resolve the error.