I’ve got the wikitext103 rnn model that looks like this:
SequentialRNN(
(0): RNN_Encoder(
(encoder): Embedding(6408, 400, padding_idx=1)
(encoder_with_dropout): EmbeddingDropout(
(embed): Embedding(6408, 400, padding_idx=1)
)
(rnns): ModuleList(
(0): WeightDrop(
(module): LSTM(400, 1150)
)
(1): WeightDrop(
(module): LSTM(1150, 1150)
)
(2): WeightDrop(
(module): LSTM(1150, 400)
)
)
(dropouti): LockedDropout()
(dropouths): ModuleList(
(0): LockedDropout()
(1): LockedDropout()
(2): LockedDropout()
)
)
(1): LinearDecoder(
(decoder): Linear(in_features=400, out_features=6408, bias=False)
(dropout): LockedDropout()
)
)
My goal is to put in a sequence and get a feature vector to do e.g. cos similarity between sequences. Here’s code I found on this forum to accomplish this:
h = m[0](V(T([tokensAsNumbers))) #convert array of tokensAsNumbers to tensor then variable with grad=false. Run that through the rnn encoder on m[0], m is a pytorch model. m[1] has decoder.
feature_vec = to_numpy(h[0][2][0][2])
What’s the thinking behind h[0][2][0][2]? I could see how the 2s could correspond to the 2s in the repr string above, but not the 0s. What’s the logic here? Is there a better way to get feature vectors like this from pytorch models?