Hi
I was wondering how to get contextual word representations like ELMO from ULMFIT . I have the fine tuned language model and need to get the word embeddings to be able to use somewhere . Any code should be helpful
Thank you
Hi
I was wondering how to get contextual word representations like ELMO from ULMFIT . I have the fine tuned language model and need to get the word embeddings to be able to use somewhere . Any code should be helpful
Thank you
Iām curious about this as well ā did you ever figure it out?
Hi
Zache
You can use the final layer of the LSTM encoder . The embedding size is 400
load_model(encoder, './models/lm1_enc.h5')
encoder.reset()
encoder.eval()
encoder.cuda(0)
model = BiLSTMModel(embedding_dim=EMBEDDING_DIM, hidden_dim=HIDDEN_DIM, label_size=3,\
use_gpu=True, batch_size=32)
model.cuda()
for i,batch in enumerate(data):
raw_outputs,outputs = encoder.forward(Variable(batch[0]))
outputs is a tensor sentencelen* batch_size*embeddingdimension
you have to use this ahead for your BILSTM model
Hope this helps
Thank you!!