pretrained=True but all layergroups unfrozen?

Part 1 (2019)
1

Hello together,

regarding transfer learning in fastai v1 I have a question:
is a model with:

  • normalize(imagenet_stats), pretrained=True, and all layers unfrozen (learn.unfreeze())
    and a model with:
  • normalize(), pretrained=False, and all layers unfrozen (learn.unfreeze())
    the same?
    My results suggest that there is a difference, but I don’t know where there should be a difference since transfer learning is just about using weights from another model…

Best regards

2

Normalizing with imagenet_stats will normalize the input pixels using the mean and standard deviation from the imagenet image catalog. If imagenet_stats is not passed into normalize() then it will use the images present to normalize the input pixels, so I would not expect two models to be equivalent that use two different sets of normalizing stats.

1 Like
3

Thanks for your reply!
The question is more about the pretrained/unfrozen. When I choose to use pretrained=True but re-train ALL layers, in my understanding, it should be the same as when using a not pretrained model where I also train all layers (or in fastAI all layergroups)