Parameter for BatchNorm1d is twice the number of features

Part 1 (2018)
1

self.nf = num_features(layers)*2
ni=self.nf
nn.BatchNorm1d(num_features=ni)

in ConvnetBuilder class of fastai self.nf is evaluated to 512*2 = 1024. self.nf is then passed as num_features to nn.BatchNorm1d.

How can BatchNorm1d accept 1024 features when the layer prior to that (8th layer of resnet34) outputs 512 features.

1 Like
2

I guess it’s due to the AdaptiveConcatPool2d.

Kind regards
Michael

1 Like
3 
self.ap = nn.AdaptiveAvgPool2d(sz)
self.mp = nn.AdaptiveMaxPool2d(sz)
def forward(self, x): return torch.cat([self.mp(x), self.ap(x)], 1)

Ah ok, I see that the above code will result in output becoming twice the input because there is column wise concatenation of features.

Thanks for pointing me to the docs link.