Course NLP Notebook nn-vietnamese data_load cannot find the saved databunch

(A Philip) #1

Hi

After creating the databunch, the data batch is visible, but on using the data.save option, as in the notebook, no new files or folders are formed in the path folder. when trying to load this data, I am getting this error.

Steps to reproduce:

data = (TextList.from_folder(dest)
        .split_by_rand_pct(0.1, seed=42)
        .label_for_lm()           
        .databunch(bs=bs, num_workers=1))

data.save('vi_databunch')
data = load_data(path, 'vi_databunch', bs=bs)

Error:

FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/.fastai/data/viwiki/vi_databunch'
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-28-5627808bffd6> in <module>
----> 1 data = load_data(path, 'vi_databunch', bs=bs)

~/anaconda3/lib/python3.7/site-packages/fastai/basic_data.py in load_data(path, file, bs, val_bs, num_workers, dl_tfms, device, collate_fn, no_check, **kwargs)
    279     source = Path(path)/file if is_pathlike(file) else file
    280     distrib_barrier()
--> 281     ll = torch.load(source, map_location='cpu') if defaults.device == torch.device('cpu') else torch.load(source)
    282     return ll.databunch(path=path, bs=bs, val_bs=val_bs, num_workers=num_workers, dl_tfms=dl_tfms, device=device,
    283                         collate_fn=collate_fn, no_check=no_check, **kwargs)

~/anaconda3/lib/python3.7/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    420     elif (sys.version_info[0] == 3 and isinstance(f, pathlib.Path)):
    421         new_fd = True
--> 422         f = f.open('rb')
    423     try:
    424         if sys.version_info >= (3, 0) and 'encoding' not in pickle_load_args.keys():

~/anaconda3/lib/python3.7/pathlib.py in open(self, mode, buffering, encoding, errors, newline)
   1201             self._raise_closed()
   1202         return io.open(self, mode, buffering, encoding, errors, newline,
-> 1203                        opener=self._opener)
   1204 
   1205     def read_bytes(self):

~/anaconda3/lib/python3.7/pathlib.py in _opener(self, name, flags, mode)
   1056     def _opener(self, name, flags, mode=0o666):
   1057         # A stub for the opener argument to built-in open()
-> 1058         return self._accessor.open(self, flags, mode)
   1059 
   1060     def _raw_open(self, flags, mode=0o777):

FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/.fastai/data/viwiki/vi_databunch'

Environment:

python        : 3.7.6
fastai        : 1.0.60
fastprogress  : 0.2.2
torch         : 1.3.1
nvidia driver : 435.21
torch cuda    : 10.0.130 / is available
torch cudnn   : 7603 / is enabled

I am a complete beginner to python and programming, so if this is a misplaced comma type error please excuse.

Regards

Philip

0 Likes

#2

Look at your data.path, since it’s where the file is saved. It looks like it’s not the same as path.

0 Likes