How to access Normalization stats when NOT normalized from imagenet_stats and not used DataBunch?

Part 1 (2020)
1

I have trained a model with Normalize() passed in as a batch_tfms during forming the DataBlock.

Now, how do I access the stats with which the data was normalized? If I had used Normalize.from_stats(*imagenet_stats), then I would have access to the global variable imagenet_stats or could have simply Googled it.

But as I did not use it, how do I find with which mean and std my images were normalized?

Something called batch_stats() existed in fastai v1, which you could call on an ImageDataBunch.

How do I access the stats with which my data was normalized?

My goal is to use the same stats to normalize the images during inference time before passing it to the model for inference.

And, what really happens, when you pass NO mean or std to the Normalize() function? Passing a mean or std is not required, after all.

2

Assuming you have passed the normalizer as a batch transform to your DataBlock, you could do:

batch_tfms = datablock.batch_tfms
# Go through all the transforms
for tfm in batch_tfms:
    # Find the normalizer
    if isinstance(tfm, Normalize):
        # The mean and std are accesible through the .mean and .std attributes
        print(tfm.mean, tfm.std)

By the way, for inference, if you’re using a Learner that has a normalizer as a transform, no need to manually normalize the input; fastai takes care of it for you.

EDIT: The following is true only when one doesn’t run the setup function. Otherwise, please see the answer below by idraja.

To answer your second question, you will get an error when you don’t pass in the mean or standard deviation because you can’t do arithmetic with tensors and None.

Cheers!

3 Likes
3

And, what really happens, when you pass NO mean or std to the Normalize() function? Passing a mean or std is not required, after all.

You can see the code in data/transforms.py

If either std or mean is none, it gets one batch from the data loader, and uses its statistics for the std and mean.

    def setups(self, dl:DataLoader):
        if self.mean is None or self.std is None:
            x,*_ = dl.one_batch()
            self.mean,self.std = x.mean(self.axes, keepdim=True),x.std(self.axes, keepdim=True)+1e-7
2 Likes