Repeating text during generation isn’t unexpected, especially, as you say, when it’s common in the input. You’ll need to come up with some different way to generate the next word, rather than always picking the highest probability prediction. E.g. if you’ve picked that word recently, pick the 2nd highest instead.