What would be the best way to get activations of penultimate dense layer of RNNLearner.classifier, i.e. the one I highlighted in bold below, after the model is trained:
(0): BatchNorm1d(1200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): Linear(in_features=1200, out_features=50, bias=True)
(4): BatchNorm1d(50, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(6): Linear(in_features=50, out_features=1, bias=True)
I used to pass the inputs directly to the trained model one by one, but it looks like there should be some easier and more efficient way to get the activations of certain layer.
My previous code of getting the 1200-tensor activations btw:
rnn_encoder = learner.model;
out, hidden = rnn_encoder_output
h = hidden[-1]
# h is with size 1*seq_len*400
max_pool = F.adaptive_max_pool1d(h.permute(0, 2, 1), (1,)).view(-1)
avg_pool = F.adaptive_avg_pool1d(h.permute(0, 2, 1), (1,)).view(-1)
cat = torch.cat([h[-1], max_pool, avg_pool]) # tensor of size (1200,)
encoded = np.array([concat_pooling(rnn_encoder(V(T([encoding_np[i]])))) for i in range(len(encoding_np))])
If i understand correctly, you want the weights of a certain layer after training?
You can get the weights with
list(learner.model.parameters()) and then choose the corresponding layer from the model. The order is the same as the model layers.
In your case, it should be
Or do you want the output from an input up to that highlighted layer?
If so, see:
I suspect it has to do with
register_forward_hook in lesson7-CAM, cause I just came across this lecture 2 days ago. You should try searching this forum for
I saw something posted from @krishnavishalv
def hook(module, input, output):
res50_model = models.resnet50(pretrained=True)
out = res50_model(res)
out = res50_model(res1)
You should use the
ActivationStats callback or one of the other callbacks in the fastai hooks module.
Hello @jeremy I am still not able to do it. Can you provide an example to do it?
Refer to Hooks in fastai documentation -
Also, wrote a sample code. See if it works for you.
layers_ = children(learn.model).layers
act_means = [ for _ in layers_]
act_stds = [ for _ in layers_]
outputs = [ for _ in layers_]
def append_stats(i, mod, inp, outp):
for i,m in enumerate(layers_): m.register_forward_hook(partial(append_stats, i))
Link doesn’t work anymore, unfortunately
docs.fast.ainow refers to v2.
Note that " Important : This documentation covers fastai v2, which is a from-scratch rewrite of fastai. The v1 documentation has moved to fastai1.fast.ai."
Is there an update on v2?