Where is LanguageModelData?

(Javier Rodriguez Zaurin) #1

Maybe I am bit late to the party and this is solved somewhere, but, LanguageModelData is not a available in v1. I am wondering what is the equivalent to this in v1:

trn_dl = LanguageModelLoader(np.concatenate(trn_lm), bs, bptt)
val_dl = LanguageModelLoader(np.concatenate(val_lm), bs, bptt)
md = LanguageModelData(PATH, 1, vs, trn_dl, val_dl, bs=bs, bptt=bptt)




What are train_lm and val_lm? In my case, when I had two pandas dataframes, I concatenated them into one dataframe, adding a fake column is_valid, and used datablock API to create a databunch.