Can we use the result of a hook in the forward of a module?
For example I have a Module where I pass in a pre-trained model named questions. questions is a TabularModel, but I want the result of a Linear layer inside of it to do more computations with it:
class ParentChildModel(Module):
def __init__(self, questions):
self.results = TabularModel(results_emb_szs, len(results_cont_names), 10, [200, 50], ps=[0.01, .1], embed_p=0.04, bn_final=False)
self.questions = questions
self.head = nn.Sequential(*[LinBnDrop(20, 1, p=0.1), SigmoidRange(*[-.1, 1.1])])
def forward(self, data, children):
[... some code here...]
# self.questions.layers[-3][2] is a Linear layer
with hook_output(self.questions.layers[-3][2], detach=False) as h:
result = self.questions(cat, cont)
mid = h.stored
Then I use mid variable in some other calculations later. I am not getting really good results with that so I was wondering if this is valid and if the gradients will flow well.