I am just venturing into NLP and have a seq2seq dataset on which I would like to apply things as I learn. On this page (https://docs.fast.ai/text.models.html), why isn’t AWD LSTM listed as decoder? Isn’t it a good idea to use it both as an encoder and as a decoder?
Thank you for your inputs!
I guess the page you refer to is more focused on language modeling and text classification, where you don’t really need an RNN as a decoder since you don’t generate an output sequence.
For seq2seq problems, I highly recommend notebooks 7, 7b and 8 from the fastai NLP course: https://github.com/fastai/course-nlp
In notebook 7, they use a 2-layer GRU with standard nn.Dropout both as encoder and decoder, but I don’t see any reason why you couldn’t use an AWD-LSTM instead. Might be worth trying out 