Other text generation models

Part 1 (2020) Non-beginner
1

Hi guys,

Can we actually use other types of models for text generation? We learned on the NLP section that ULMFit and vanilla RNN’s, LSTMs and GRUs can do that just fine. But what about Elmo, BERT and other kinds of models?

2

Yes, there are different techniques for text generation other than RNN [1].

Few things to clarify:

  • ELMo is a word embedding system and not a model. It can be used with LSTM model.
  • BERT is a language model based Transformers model, which are different than RNN models.

[1] The survey: Text generation models in deep learning

2 Likes