.normalize(imagenet_stats) will silently add channels

(Erez ) #1

After a lot of debugging, I realized that calling .normalize(imagenet_stats) on a 1-channel image dataset will silently turn the images into 3-channel. I understand that the imagenet_stats are 3-channel, but I seems like throwing an error would be a better user experience.

Would an error like that make sense in this case? - I’m happy to try to add it and file a PR.

0 Likes

#2

I disagree. ImageNet has 3-channel images so if you are normalizing with those, you a kind of implying that you want 3-channel images. If not, you should normalize with custom channel stats.

Also, most pretrained models take in 3-channel images, so it is helpful to convert them directly to 3-channel images. Otherwise, you will have to adjust the first layer of your pretrained model.

0 Likes

(Erez ) #3

I agree with almost everything you wrote, but not your conclusion. Normalizing a 1-channel image with 3-channel stats shouldn’t just work, you should be warned or an error should be thrown. If you are normalizing a 1-channel image with imagenet-stats, you are probably making an error.

Anyway, I’ll leave it to the group - thanks for the reply!

2 Likes

#4

I agree, you should know when you are doing something wrong and without an error (or at least a warning) people may not even realize their image is converted to 3-channel.

0 Likes

#5

Problem is that a channel dimension of 1 is compatible with a channel dimension of 3 in the normalizing stats because that’s a broadcasting rule. So it’s normal you don’t get an error + get automatic conversion to 3 channels.

Maybe at the time you call normalize, we could try to load one batch and return a warning if the stats have 3 channels but the images only one.

1 Like

#6

I also read the image normalization provides almost no improvements so you can avoid it. What is the benefit of using normalization if you use BatchNorms?

0 Likes

#7

Calling normalize() without specifying any parameters also seems to add extra channels silently. Is this expected behavior? I have a Tensor of shape (1, 24, 24) and calling normalize() converted it to (4, 24, 24). Unfortunately I spent the whole day trying to debug where I’d gone wrong with my code. :sweat_smile:

0 Likes