I am finishing my first round on Part2 course and I am quite surprised by the batch norm issue that is discussed around 1:02h. It seems that, when freeze, the batchnorm layers should not be updated. I more or less understand the logic behind but, in practice, I am not sure if this is actually what default (X)ResNets are doing in fastai library.
Could anybody tell me if I should add:
def set_grad(m, b): if isinstance(m, (nn.Linear,nn.BatchNorm2d)): return if hasattr(m, 'weight'): for p in m.parameters(): p.requires_grad_(b) apply_mod(learn.model, partial(set_grad, b=False))
to my code if I want to unfreeze?