ActivationStats how to get stats of all activations

fastai
1

Hi all,

I’ve a question regarding usage of ActivationStats class.
My goal is to be able to record all of the stats of activated layers, ie VGG16 has
6 activated layers, while training the model, from scratch.
As far as I understand it, is that we can obtain stats for activations, mean and std.

How this actually works and what is returned?

At the moment I am using it like this, inside batch iteration.
stats = ActivationStats.hook(net, nn.ReLU, inputs, outputs)
means += stats[0]
stds += stats[1]
But above is returning only one pair of values (mean, std) per batch. Is this average of all activations, or this is the last activation in the network architecture?

Anyone here could help me with this issue, please?
Marcin

2

You should use it as a Callback and not just for the hook function:

cb = ActivationStats(learn, list_of_layers_you_want)
learn.callbacks.append(cb)

or something like that.

1 Like
3

Thanks for quick response.
Would you have something like full example?

I don’t understand how to implement it fully.
Thanks a lot.