Extract feature vector/latent factors from Embedding layer in Pytorch

Deep Learning
2

@jonmunm

  1. You can simply do list(model.children()) (or list(learn.model.children() if you have directly defined a learner) and see the index of the embedding vector, and tap the .parameters() attribute of that layer.

  2. Well, the answer is not definitive. I would expect the RF model to work somewhat accurately if i were to directly take out the embedding vector from a trained NN. But to make it work well, you’d need to train it in the RF as well. Thats because the parameters of the embedding vectors themselves dont make any sense, only the combination of various layers of parameters makes sense, if you understand what I’m trying to say.

  3. Please elaborate on this a little. I think i dont understand your point. Sorry for that.

  4. You cant use the test data in that case. First of all, the model has no mechanism to take data as inputs, that are not compatible with the architecture. Then, there are no weights to handle extra features. You’d either have to remove the extra features from the test/valid set, or access training data that has those extra features as well

Cheers!

1 Like