Here’s the ref to Jeremy’s note (no further elaboration): Tricks for using language models to generate text - #2 by jeremy
I believe you get the same output for fixed input (ex. translation). In language generation you can always change seed or starting text (ex. use a random word from vocab). Beam search should be better than ‘next token prediction’.
I don’t know how to implement beam search or GANs for language generation in fastai but I am interested in exploring it when available. I tried poetry generation earlier (using language model learner prediction - predict after xxbos or xxfld), results were not that impressive.