Calculating New stats

Part 1 (2019) Advanced (Part 1 v3)
1

Hi all

  1. Can we always use the default stats ,that are the ones calculated for ImageNet stats ?
  2. If i were to calculate the new stats to be used for normalization …
    How would i do for RGB image…
    Any pointers
    say I have set of 1000 Traning Images, and i want to calculate the new set of stats ,as the images are different from the ones used in Imagnet…
2

  1. If you are fine-tuning a pretrained model trained on imagenet, you should use the default stats. This seems like most of the situations, so I kind of mindlessly use the resnets in fastai library

  2. If you are training everything from scratch, then you may need to calculate your own stats i think. try this if you wish to calculate your own stats

4 Likes
3

Thanku
Should we always use default stats or we should re calculate

4

I think for most situations we are using the Resnets in fastai, so i don’t think we need to calculate the stats ourselves.
I’ll just add something from DL1 lesson 7

stats —When we use pre-trained models, you can call tfms_from_model which creates the necessary transforms to convert our data set into a normalized dataset based on the means and standard deviations of each channel in the original model that was trained in. Since we are training a model from scratch (Note: in lesson 7 we are training using Cifar10, which consists of small-sized images), we need to tell it the mean and standard deviation of our data to normalize it. Make sure you can calculate the mean and the standard deviation for each channel.

Edit: this answer below by @bluesky314 is way better

5

If using pretrained: You must use ImageNet stats cause the model has learnt corelations specific to that normalization

If using your own: Write a function to load the images in a np array . If there are too many then you may pick a good enough sample. Then call np.mean(data,axis=channel axis). and np.std similarly. Channel axis will most likely be 3 as your np array will be [image number, h,w,3] for RGB

1 Like
6

Taking what @wyquek has said, annd the script can be simplied as following:

data_dir = "xxxxx" # your image directory
TRAIN = Path(data_dir)
images = (plt.imread(str(i)) for i in TRAIN.iterdir()) # generator comprehension
images = np.stack(images)  # this takes time 
means = np.mean(images, axis=(0, 1, 2))
stds = np.std(images, axis=(0, 1, 2))
1 Like