Lesson-3: Load Data Fails

Part 1 (2018)
1

Hi guys,
I’m working on the Lesson-3 IMDB. I have trained my model on AWS which is running on Ubuntu System. I have downloaded my trained models on the local system, the problem is I’m not able to load the data bunch by using the load_data(path,’ filename’). The error is cannot instantiate ‘PosixPath’ on your system. The problem is I used Path as on AWS which defaults to PosixPath and since the local system is Windows it is not able to load the model. Are there any dependency while saving the model.

The code is straight forward
from fastai.text import *
from fastai import *
#path ="/home/ubuntu/cleanClause/Clause" # while training on AWS
path = “C:/Users/abhis/Desktop/Ec2/cleanClause/Clause”
bs = 16
#data_lm = TextList.from_folder(path).split_by_rand_pct(0.0).label_for_lm().databunch(bs=bs) # while training on AWS
#data_lm.save(“vocabpath”)
data_lm = load_data(path,‘vocabpath’)

The stacktrace is
NotImplementedError Traceback (most recent call last)
in
----> 1 data_lm = load_data(path,‘vocabpath’)

F:\Anaconda\envs\fastai\lib\site-packages\fastai\basic_data.py in load_data(path, file, bs, val_bs, num_workers, dl_tfms, device, collate_fn, no_check, **kwargs)
276 source = Path(path)/file if is_pathlike(file) else file
277 print(source)
–> 278 ll = torch.load(source, map_location=‘cpu’) if defaults.device == torch.device(‘cpu’) else torch.load(source)
279 return ll.databunch(path=path, bs=bs, val_bs=val_bs, num_workers=num_workers, dl_tfms=dl_tfms, device=device,
280 collate_fn=collate_fn, no_check=no_check, **kwargs)

F:\Anaconda\envs\fastai\lib\site-packages\torch\serialization.py in load(f, map_location, pickle_module)
365 f = open(f, ‘rb’)
366 try:
–> 367 return _load(f, map_location, pickle_module)
368 finally:
369 if new_fd:

F:\Anaconda\envs\fastai\lib\site-packages\torch\serialization.py in _load(f, map_location, pickle_module)
536 unpickler = pickle_module.Unpickler(f)
537 unpickler.persistent_load = persistent_load
–> 538 result = unpickler.load()
539
540 deserialized_storage_keys = pickle_module.load(f)

F:\Anaconda\envs\fastai\lib\pathlib.py in new(cls, *args, **kwargs)
995 if not self._flavour.is_supported:
996 raise NotImplementedError(“cannot instantiate %r on your system”
–> 997 % (cls.name,))
998 self._init()
999 return self

NotImplementedError: cannot instantiate ‘PosixPath’ on your system

3 Likes
2

me too, have you fixed that error??

3

Me too. On windows not sure how to fix this?

4

@jeremy Hi Jeremy, can you please provide a possible workaround for this?
The path in Linux/Unix system(here, I am using Colab) defaults to PosixPath type, which is not implemented in Windows. Thus if I load my model in Windows, I get the following error : NotImplementedError: cannot instantiate ‘PosixPath’ on your system

I tried changing the type of Path using
path = PureWindowsPath(path)
However, the path is still saved as a PosixPath.

5

The same error happened for me on windows, have you found the solution?

6

These steps solved the issue for me:
1- go to the F:\Anaconda\envs\fastai\lib\ and open pathlib.py
2- go to line 998 and change the function new in the class Path(PurePath) to :

 def __new__(cls, *args, **kwargs):
        if cls is Path:
            cls = WindowsPath
        self = cls._from_parts(args, init=False)
        self._init()
        return self

3- save the file and re-run you code again

3 Likes
7

I too had the same issue. Below is how this can be resolved.

F:\Anaconda\envs\fastai\lib\site-packages\torch\serialization.py in load(f, map_location, pickle_module)
365 f = open(f, ‘rb’)

please change this to as below

f = f.open(‘rb’)

reference : https://github.com/pytorch/pytorch/issues/16607

8

Works like a charm, even for fastai v2, thank you!!

9

Thanks a lot…