I am experimenting with some Seq2Seq models using pytorch, such as End to End Speech Recognition and Neural Machine Translation. I was trying to figure out a way to leverage ULMFIT models for Seq2Seq tasks and it seems that this paper has a good approach (Cold Fusion):
However, once I am still a begginer, I am scratching my head on how to plug a fixed ULMFIT logit layer to my SeqSeq Pytorch models.
Anyone else experimenting with this? Any ideas?