Lesson 10 No such file or directory: '/content/train/train_tok/counter.pkl'

Part 1 (2020)
1

dls_lm = DataBlock(
blocks = TextBlock.from_folder(hindi_txt_path,is_lm=True),
get_items = get_text_files(hindi_txt_path),
splitter = RandomSplitter(0.1)
).dataloaders(hindi_txt_path,path = hindi_txt_path ,bs = 128,
seq_len = 80 )

I have 2 questions

  1. here in this “hindi_txt_path” I have 130,000 .txt files (around 300 Mb) data but for testing and check everything working I need only 500 files so how to achieve with the above code
    I try with get_text_files(hindi_txt_path)[:500] but it did’t work

  2. and after running the above code 2nd time ("because I just deliberately stops code execution for change first time ") it gives this error

FileNotFoundError: [Errno 2] No such file or directory: ‘/content/train/train_tok/counter.pkl’

I am running this code in colab
Thanks for any help

2

Sorry 2nd problem solve with this

import shutil
shutil.rmtree(’/content/train/train_tok/’)

3

after 15 mins of dl_lm gives
TypeError: ‘L’ object is not callable

4

TypeError: ‘L’ object is not callable this error comes when u use
get_items=get_text_files(hindi_txt_path)
but if u use like this then there is no such error
get_hindi = partial(get_text_file)
get_items=get_hindi