Introducing Convolutional Layer with a Twist

I’m putting together a notebook that hopefully explains Convolutional Layer with a Twist for everyone. For those who have never heard about it, I recently used it on ResNet for the Imagenette/Imagewoof challenge, and it appears to be working pretty well (with some surprises). As it can replace any 3x3 convolutional layer in your model, I’d very much like to see it applied to other kinds of CNN models (detection, GANs, etc.) and datasets.

I’m trying to use fastai’s fastpages to “publish” it, and it’s still a draft. For one thing, I couldn’t find instruction for writing LaTeX which it promised to support. [Update: I was dumb… simply using $ signs in markdown works.] (Jupyter notebook, LaTeX, and a public comment section is a killer combination, from my view.)

I’d also like to ask for suggestion of a notebook that has code that takes an image, turns it to a PyTorch tensor, passes through a neural net, and “plots” the outcome (or a feature map) as an image. That would help me a lot. Thanks in advance!

5 Likes

You mean like class activation maps?

Thanks, I should try that at some point. But for this it’s probably too big a hammer.

It seems that kornia is great library that’s equivalent to OpenCV but using PyTorch. I think I can find an example or two that could get me started.

Why is it “too big a hammer”? It’s already in fastai v1, and I just linked code for doing it in fastai v2.

Looks like it’s showing what part of the original image is “activated” at a certain feature map, while what I’m looking for is just showing the feature map, of a model that is simply one Conv layer (not a model at all). I’ll take a look.

It’s too big a hammer because I’m new to fastai. Nothing against it.

Updated the notebook. I simply used OpenCV’s imread, and use matplotlib to plot the before and after.

The math is still to be written. But if you don’t care about that, you can already play with it (with your own image).