Why do we need to unfreeze the learner everytime before retarining even if learn.fit_one_cycle() works fine without learn.unfreeze()

Part 1 (2019)
10

To answer this question, we only need to enter the source code.
---------------- First, when we create a cnn model using cnn_learner -----------------------------

def cnn_learner(data:DataBunch, base_arch:Callable, cut:Union[int,Callable]=None, pretrained:bool=True,
                lin_ftrs:Optional[Collection[int]]=None, ps:Floats=0.5, custom_head:Optional[nn.Module]=None,
                split_on:Optional[SplitFuncOrIdxList]=None, bn_final:bool=False, init=nn.init.kaiming_normal_,
                concat_pool:bool=True, **kwargs:Any)->Learner:
    "Build convnet style learner."
    meta = cnn_config(base_arch)
    model = create_cnn_model(base_arch, data.c, cut, pretrained, lin_ftrs, ps=ps, custom_head=custom_head,
        split_on=split_on, bn_final=bn_final, concat_pool=concat_pool)
    learn = Learner(data, model, **kwargs)
    learn.split(split_on or meta['split'])
"----------------watch out!  our's 'pre-trian model' has been 'freeze' ----------------------------"
    if pretrained: learn.freeze()     # there
    if init: apply_init(model[1], nn.init.kaiming_normal_)
    return learn

-----------------------------------now , we move to learn.freeze()----------------------------------

    def freeze(self)->None:
        "Freeze up to last layer group."
        assert(len(self.layer_groups)>1)
" there , we can see only trian the last layer group(such as fully-connect layers)"
        self.freeze_to(-1)  
        self.create_opt(defaults.lr)

So, if you don’t run learn.unfreeze() before learn.fit_one_cycle(1) just train fully connect layers, the convolutional framework has been pre-trained and freeze , everything works fine.

This is equivalent to using the pre-trained convolution network backbone as a feature extractor,only training the fully connected layer learning classification.

4 Likes