How to predict next word with fastai.text?

Part 2 & Alumni (2018)
1

I have trained a language model on a custom data set, did the same things Jeremy did in his IMDb notebook but I can’t figure out how to use the language model to predict the next word without torchtext. To be clear, I want to get the equivalent of this code without using torchtext:

m=learner.model
ss=""". So, it wasn't quite was I was expecting, but I really liked it anyway! The best"""
s = [spacy_tok(ss)]
t=TEXT.numericalize(s)
' '.join(s[0])

# Set batch size to 1
m[0].bs=1
# Turn off dropout
m.eval()
# Reset hidden state
m.reset()
# Get predictions from model
res,*_ = m(t)
# Put the batch size back to what it was
m[0].bs=bs

print(ss,"\n")
for i in range(50):
     n=res[-1].topk(2)[1]
     n = n[1] if n.data[0]==0 else n[0]
     print(TEXT.vocab.itos[n.data[0]], end=' ')
     res,*_ = m(n[0].unsqueeze(0))
print('...')

And, one more question. How to make the language model not repeat itself every N words (like in Jeremy’s example i was very impressed . i was very impressed with the acting . i was very impressed with the acting .? I tried to increase bptt to 140, it somewhat helped, but not very significantly.

2

I’m not sure how you would predict without torchtext. You need to feed a torch Variable to the model to make a prediction, use torch functions to grab the top predictions, and use torchtext to work with word to integer mapping.

As for the repeats I’ve found that using torch.topk for predictions results in loops. You can get more variable predictions from torch. multinomial. If you want an example I did a quick notebook on topk vs multinomial for RNNs.

1 Like