Individual layers in AWD_LSTM

Part 2 (2019)
1

I want to print the output of each of the layers in AWD_LSTM, the encoder+classifier architecture. I.e. When I pass a text as input, I want to see the tensors after individual layers. And play with them, like you do with BERT to extract word embeddings. How do I get hold of individual layers in fastai v1?

2

You should register/use a pytorch hook similar to:

  • Grad-Cam: with hook_output(m[0]) as hook_a:
  • CUda-cnn-hooks
    with Hooks(model, append_stats) as hooks: run.fit(1, learn)
1 Like