Lesson 4 Advanced Discussion ✅

Part 1 (2019) Advanced (Part 1 v3)
106

You can set the batch size as follows (it’s in the updated course-v3)

bs =24
data_lm = TextLMDataBunch.load(path, 'tmp_lm', bs=bs)

You can specify the settings of max_vocab and the language as follows.

txt_proc = [
    TokenizeProcessor(tokenizer=Tokenizer(lang='nl') ),
    NumericalizeProcessor(min_freq=1, max_vocab=10000 )
]

data_lm = (TextList.from_df(df, cols='text', processor=txt_proc)           
            .random_split_by_pct(0.1)
            .label_for_lm()           
            .databunch())
3 Likes