I am trying split my layers to be able to freeze some of them.
I want to pass in a pre-trained model called questions
as a sub-module of my main model. I want to freeze questions
and fine tune it later. Here is how I do it:
class ParentChildModel(Module):
def __init__(self, questions):
self.questions = questions
self.results = TabularModel(results_emb_szs, len(results_cont_names), 10, [200, 50], ps=[0.01, .1], embed_p=0.04, bn_final=False)
self.head = nn.Sequential(*[LinBnDrop(60, 1, p=0., bn=False), SigmoidRange(*[-.1, 1.1])])
...
def split_layers(m):
return L(m.questions, m.results, m.head).map(params)
learnQuestions = load_learner('models/NextQuestionPrediction_export.pkl')
learner = Learner(dls, ParentChildModel(learnQuestions.model), loss_func=combined_loss, splitter=split_layers)
learner.freeze_to(1)
Freezing to 1 here should only freeze m.questions right?
Thanks,