Unable to compute intermediate layer's output to get embeddings

Part 1 (2020)
4

We have our answer! :smiley:

Here’s what I did, presume this is your fastai tabular model for the Adult Sample dataset. I want to grab the intermediate layer output of 100, when our FC layers are [200,100]. I do the following:

with hook_outputs(learn.model.layers[:-1]) as hooks:
    _ = learn.model(cat,cont)

I can then access these intermediate layer activations by doing:
hooks.stored[-1]. This will be of shape [64,100] in this particular example :slight_smile:

For another example, if we want to get the raw embeddings to use with RF, we do:

with hook_outputs(learn.model.embeds) as hooks:
    _ = learn.model(cat,cont)
embeds = torch.cat(list(hooks.stored), 1)

Hope this helps @tsm_tau and @johannesstutz

8 Likes