Instantiation of DataBlock takes around 10 minutes. For this reason, I want to pickle it for later reuse:
path = untar_data(URLs.IMDB)
db = DataBlock(
blocks=(TextBlock.from_folder(path), CategoryBlock),
get_y = parent_label,
get_items=partial(get_text_files, folders=[‘train’, ‘test’]),
splitter=GrandparentSplitter(valid_name=‘test’)
)
with open('db_full.pickle', 'wb') as handle:
pickle.dump(db, handle)
pickle.dump
throws the following error.
AttributeError: Can't pickle local object 'GrandparentSplitter.<locals>._inner'
Any idea why I can’t save an instance?