Learner.export() problem: Can't pickle <function <self defined function> at 0x........>: it's not the same object as __main__.<self defined function>

Part 1 (2019)
1

I define my own transformation pad3d for databunch and create and train a learner using my own autoencoder models. But when we want to export the learner, I ran into an error:
PicklingError: Can’t pickle <function pad3d at 0x7fd07dee8d90>: it’s not the same object as main.pad3d

The traceback looks like this:
----> 2 learner.export(‘testexport.pkl’)

~/anaconda2/envs/my37/lib/python3.7/site-packages/fastai/basic_train.py in export(self, file, destroy)
240 state[‘data’] = self.data.valid_ds.get_state(**xtra)
241 state[‘cls’] = self.class
→ 242 try_save(state, self.path, file)
243 if destroy: self.destroy()
244

~/anaconda2/envs/my37/lib/python3.7/site-packages/fastai/torch_core.py in try_save(state, path, file)
404 def try_save(state:Dict, path:Path=None, file:PathLikeOrBinaryStream=None):
405 target = open(path/file, ‘wb’) if is_pathlike(file) else file
→ 406 try: torch.save(state, target)
407 except OSError as e:
408 raise Exception(f"{e}\n Can’t write {path/file}. Pass an absolute writable pathlib obj fname.")

~/anaconda2/envs/my37/lib/python3.7/site-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol)
222 >>> torch.save(x, buffer)
223 “”"
→ 224 return _with_file_like(f, “wb”, lambda f: _save(obj, f, pickle_module, pickle_protocol))
225
226

~/anaconda2/envs/my37/lib/python3.7/site-packages/torch/serialization.py in _with_file_like(f, mode, body)
147 f = open(f, mode)
148 try:
→ 149 return body(f)
150 finally:
151 if new_fd:

~/anaconda2/envs/my37/lib/python3.7/site-packages/torch/serialization.py in (f)
222 >>> torch.save(x, buffer)
223 “”"
→ 224 return _with_file_like(f, “wb”, lambda f: _save(obj, f, pickle_module, pickle_protocol))
225
226

~/anaconda2/envs/my37/lib/python3.7/site-packages/torch/serialization.py in _save(obj, f, pickle_module, pickle_protocol)
295 pickler = pickle_module.Pickler(f, protocol=pickle_protocol)
296 pickler.persistent_id = persistent_id
→ 297 pickler.dump(obj)
298
299 serialized_storage_keys = sorted(serialized_storages.keys())

PicklingError: Can’t pickle <function pad3d at 0x7fd07dee8d90>: it’s not the same object as main.pad3d

I have seen this two posts

In the latter post, I feel like it is because my code containing the customized function as follow:

@faiv.TfmPixel
@singledispatch
def pad3d(x, padding)->torch.Tensor:
# s = x.shape
npad = (padding[0],padding[0],padding[1],padding[1],padding[2],padding[2])
m = nn.ConstantPad3d(npad, 0)
r = m(x)
return r.contiguous()

tfms1 = pad3d(padding=(50,20,50))

I rerun the definition of the function before export, but it does work as well.
There it seems none of them will help me solve my problem.

Can any one help with this thanks very much.

1 Like
2

did you solve this problem? I have same problem:思维:

3

Hi Charming
Sorry, I didn’t I just use the learn.save and learn.load pair. Did you figure that out?