Hi!
I am finishing my first round on Part2 course and I am quite surprised by the batch norm issue that is discussed around 1:02h. It seems that, when freeze, the batchnorm layers should not be updated. I more or less understand the logic behind but, in practice, I am not sure if this is actually what default (X)ResNets are doing in fastai library.
Could anybody tell me if I should add:
def set_grad(m, b):
if isinstance(m, (nn.Linear,nn.BatchNorm2d)): return
if hasattr(m, 'weight'):
for p in m.parameters(): p.requires_grad_(b)
apply_mod(learn.model, partial(set_grad, b=False))
to my code if I want to unfreeze?
Thanks!
P.S.: I see that @xnet did a similar question before but nobody answers