I’ve a question regarding usage of ActivationStats class.
My goal is to be able to record all of the stats of activated layers, ie VGG16 has
6 activated layers, while training the model, from scratch.
As far as I understand it, is that we can obtain stats for activations, mean and std.
How this actually works and what is returned?
At the moment I am using it like this, inside batch iteration.
stats = ActivationStats.hook(net, nn.ReLU, inputs, outputs)
means += stats
stds += stats
But above is returning only one pair of values (mean, std) per batch. Is this average of all activations, or this is the last activation in the network architecture?
Anyone here could help me with this issue, please?