Not sure if it’s to do with normalisation, but if it is, I think you can normalize using Normalize.from_stats(*imagenet_stats)
Also, I wonder if your separate calling of the models onto your images are causing the params to change because they are still trainable? Try doing it in eval() mode. My post here shows what I did, though it’s in the process of me trying to hook in GradCAM.
Let us know how you get on. Thanks.
Yijin