Extracting word embeddings from ULMFiT

[knesgood (Kyle Nesgood)]

This thread might be of interest:

ULMFiT to Generate Embeddings

Hey, You can I think. I’ll have to go through the paper again but Ulmfit was designed to function differently. Basically its a language model trained to be a classifier. So you train a generic language model on your own text from which you want to do classification but you train it like a language model. And you use the last state and the mean of the previous hidden states (you might want to go through lesson 12 of the new deep learning course for this) and then pass it to a classifier. You’re…