Can we use the result of a hook in the forward of a module?
For example I have a Module where I pass in a pre-trained model named questions. questions is a TabularModel, but I want the result of a Linear layer inside of it to do more computations with it:
class ParentChildModel(Module): def __init__(self, questions): self.results = TabularModel(results_emb_szs, len(results_cont_names), 10, [200, 50], ps=[0.01, .1], embed_p=0.04, bn_final=False) self.questions = questions self.head = nn.Sequential(*[LinBnDrop(20, 1, p=0.1), SigmoidRange(*[-.1, 1.1])]) def forward(self, data, children): [... some code here...] # self.questions.layers[-3] is a Linear layer with hook_output(self.questions.layers[-3], detach=False) as h: result = self.questions(cat, cont) mid = h.stored
Then I use mid variable in some other calculations later. I am not getting really good results with that so I was wondering if this is valid and if the gradients will flow well.