Using Cold Fusion with ULMFIT on Seq2Seq tasks

Deep Learning
1

Hello all.

I am experimenting with some Seq2Seq models using pytorch, such as End to End Speech Recognition and Neural Machine Translation. I was trying to figure out a way to leverage ULMFIT models for Seq2Seq tasks and it seems that this paper has a good approach (Cold Fusion):

However, once I am still a begginer, I am scratching my head on how to plug a fixed ULMFIT logit layer to my SeqSeq Pytorch models.

Anyone else experimenting with this? Any ideas?

Best Regards,
Fabio.

1 Like