LanguageModelData bptt and backwards

Part 2 & Alumni (2018)
1

I am looking at the init function of LanguageModelData and wanted to check with some people that are smarter than me on this.

Here is the whole piece I’m looking at in fastai.text:

class LanguageModelData():
    def __init__(self, path, pad_idx, nt, trn_dl, val_dl, test_dl=None, bptt=70, backwards=False, **kwargs):
        self.path,self.pad_idx,self.nt = path,pad_idx,nt
        self.trn_dl,self.val_dl,self.test_dl = trn_dl,val_dl,test_dl

    def get_model(self, opt_fn, emb_sz, n_hid, n_layers, **kwargs):
        m = get_language_model(self.nt, emb_sz, n_hid, n_layers, self.pad_idx, **kwargs)
        model = LanguageModel(to_gpu(m))
        return RNN_Learner(self, model, opt_fn=opt_fn)

So when I initialize a new md, I pass in bptt and backwards=False, but that is not really doing anything is it? I’m under the impression that this should possibly have another line that says

self.bptt,self.backwards= bptt,backwards

But as far as I can tell, this isn’t actually being used anywhere. So my question is, what am I missing here?

2

As far as I know, you construct trn_dl and val_dl with LanguageModelLoader, which will handle bptt

Something like:

trn_dl = LanguageModelLoader(trn_lm, bs, bptt, backwards=False)
val_dl = LanguageModelLoader(val_lm, bs, bptt, backwards=False)
md = LanguageModelData(PATH, 1, vs, trn_dl, val_dl)

But you’re right, looks like LanguageModelData is not making use of those params!

1 Like
3

Good call, that is where I am actually using those two parameters. Thanks for pointing that out.

4

I think it’s an heritage from the old class defined in nlp.py, but yeah, those params aren’t used there anymore.

3 Likes
5

Added the smallest pull request ever to clean that up: https://github.com/fastai/fastai/pull/367

1 Like