Using QRNN in Language Models

You are not the first on to be confused by this! Rather than defining the config as a dict, you make a copy of the default and then tweak and pass to the language_model_learner constructor. This way you are sure to have all the params that are needed to train. I put that into your notebook, can you see it. It should look like this:

config = awd_lstm_lm_config.copy()
config['qrnn'] = True

learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.5,config=awd_lstm_lm_config,pretrained=False)
learn.fit_one_cycle(1, 1e-2)

Then proceed as usual. Let me know if you have any trouble with it.

3 Likes