Solved: Help with implementing Grad-CAM at inference time

As a followup, here is the updated version:

Unfortunately I am stuck again. In the second gist I try to get grad-CAMs for all 11 classes…

However, the CAMs I get look wrong. I guess something is up with the way I store backdrop results?