Hi, I have a question about this file pretrain_lm.py, if run it with backwards=True
, is the line 36 correct?
trn_dl = LanguageModelLoader(trn_lm, bs, bptt)
or it should be as follows:
trn_dl = LanguageModelLoader(trn_lm, bs, bptt, backwards=True)
Btw, I have converted ids backward using codes in this post
@sebastianruderr