How to extract embedding vectors after m.fit()


(Mike) #1

I have fit a model to my data and I have 1 categorical variable that I created embedding vectors for (from Lesson 4 Rossman ipynb). How do I now extract those embedding vectors for each level of the factor variable?


(Sam Lloyd) #2

learn.model.embs gets a list of your embeddings. In your case you do:
emb = to_np(learn.model.embs[0].weight.data)
This shall be a number of embeddings x size of embedding sized matrix. It shall be ordered by the categorical index that was passed to it


(Mike) #3

Thank you! p.s. how did you know that? Is there documentation on the attributes of the learner?


(Sam Lloyd) #4

Just looking at the source code:

  • StructuredLearner, and from that
  • MixedInputModel
  • To looking up how to get the embedding matrix from a nn.Embedding objects

(Karl) #5

This is generally how you get the weights for any pytorch model. The only fastai bits are having to use learn.model to get the actual pytorch model out of the learner object and to_np which just converts the weights from a torch tensor to a numpy array.