Hi, I am trying to run the code for lesson 4 on my own dataset. My dataset is a csv file, so I am not very sure how I can convert it to the same format as the IMDB data? I created a dataframe using pandas for my csv file but I am not sure how to incorporate it into this code:
FILES = dict(train=TRN_PATH, validation=VAL_PATH, test=VAL_PATH)
md = LanguageModelData.from_text_files(PATH, TEXT, **FILES, bs=bs, bptt=bptt, min_freq=10)
This is what I’m currently doing:
FILES = dict(train=df_trn[‘text’], validation=df_val[‘text’], test=df_val[‘text’])
md = LanguageModelData.from_text_files(PATH, TEXT, **FILES, bs=bs, bptt=bptt, min_freq=10)
But I am receiving an error message saying:
TypeError Traceback (most recent call last)
in
1 FILES = dict(train=df_trn[‘text’], validation=df_val[‘text’], test=df_val[‘text’])
----> 2 md = LanguageModelData.from_text_files(PATH, TEXT, **FILES, bs=bs, bptt=bptt, min_freq=10)
C:\Windows\System32\fastai\courses\dl1\fastai\nlp.py in from_text_files(cls, path, field, train, validation, test, bs, bptt, **kwargs)
310 “”"
311 trn_ds, val_ds, test_ds = ConcatTextDataset.splits(
–> 312 path, text_field=field, train=train, validation=validation, test=test)
313 return cls(path, field, trn_ds, val_ds, test_ds, bs, bptt, **kwargs)
314
C:\ProgramData\Anaconda3\envs\fastai\lib\site-packages\torchtext\data\dataset.py in splits(cls, path, root, train, validation, test, **kwargs)
74 path = cls.download(root)
75 train_data = None if train is None else cls(
—> 76 os.path.join(path, train), **kwargs)
77 val_data = None if validation is None else cls(
78 os.path.join(path, validation), **kwargs)
C:\ProgramData\Anaconda3\envs\fastai\lib\ntpath.py in join(path, *paths)
113 return result_drive + result_path
114 except (TypeError, AttributeError, BytesWarning):
–> 115 genericpath._check_arg_types(‘join’, path, *paths)
116 raise
117
C:\ProgramData\Anaconda3\envs\fastai\lib\genericpath.py in _check_arg_types(funcname, *args)
147 else:
148 raise TypeError(’%s() argument must be str or bytes, not %r’ %
–> 149 (funcname, s.class.name)) from None
150 if hasstr and hasbytes:
151 raise TypeError(“Can’t mix strings and bytes in path components”) from None
TypeError: join() argument must be str or bytes, not ‘Series’
I would appreciate any help, thank you very much!