Tricks for using language models to generate text

In lesson four @jeremy said although language models are not the focus of this course, there are tricks to use language models to generate text, in this thread let’s share them.


Main one I was thinking of was beam search.


I was looking into text generation models. I’ve tried to implement beam search while working on a char LSTM problem for joke generation. The beam search part is taking loads of time to generate outputs however, I guess it is because I’m running predictions on a model still on the GPU.

I’ve put this here since this is about text generation specifically and I’m not using the latest fastai codebase yet. Hoping to polish this up more!