Using an AWD LSTM as a seq2seq decoder

I am just venturing into NLP and have a seq2seq dataset on which I would like to apply things as I learn. On this page (, why isn’t AWD LSTM listed as decoder? Isn’t it a good idea to use it both as an encoder and as a decoder?

Thank you for your inputs!

I guess the page you refer to is more focused on language modeling and text classification, where you don’t really need an RNN as a decoder since you don’t generate an output sequence.

For seq2seq problems, I highly recommend notebooks 7, 7b and 8 from the fastai NLP course:

In notebook 7, they use a 2-layer GRU with standard nn.Dropout both as encoder and decoder, but I don’t see any reason why you couldn’t use an AWD-LSTM instead. Might be worth trying out :slight_smile: