Exporting normalization stats

AdrianT · January 6, 2019, 7:14am

Is there a simple way to export just the normalization stats from the leaner? Something like imagenet_stats thats already in the library. My goal is to train a model using a subset of the data before loading in the entire set for fine tuning.

rohit_gr · January 6, 2019, 11:50am

I think learn.data.stats gives normalization stats. And its already calculated on a batch.

github.com

fastai/fastai/blob/03974cd47470063665d64b5a164390c52fe7dc3c/fastai/vision/data.py#L172




def batch_stats(self, funcs:Collection[Callable]=None)->Tensor:
    "Grab a batch of data and call reduction function `func` per channel"
    funcs = ifnone(funcs, [torch.mean,torch.std])
    x = self.one_batch(ds_type=DatasetType.Valid, denorm=False)[0].cpu()
    return [func(channel_view(x), 1) for func in funcs]


def normalize(self, stats:Collection[Tensor]=None, do_x:bool=True, do_y:bool=False)->None:
    "Add normalize transform using `stats` (defaults to `DataBunch.batch_stats`)"
    if getattr(self,'norm',False): raise Exception('Can not call normalize twice')
    if stats is None: self.stats = self.batch_stats()
    else:             self.stats = stats
    self.norm,self.denorm = normalize_funcs(*self.stats, do_x=do_x, do_y=do_y)
    self.add_tfm(self.norm)
    return self


def download_image(url,dest, timeout=4):
try: r = download_url(url, dest, overwrite=True, show_progress=False, timeout=timeout)
except Exception as e: print(f"Error {url} {e}")


def _download_image_inner(dest, url, i, timeout=4):

AdrianT · January 6, 2019, 5:05pm

If I change the data, the normalization stats will be calculated on that batch. However, I want to normalize it based on the data I trained the model with, so that I can use a different dataset to fine tune the model.

I can’t seem to get that bit to work, learn.data shows that the data is loaded onto the leaner:

but both learn.data.stats and learn.data.batch_stats() raises an attribute error.

I’m working with tabular data which does not seem to have that. Do you think it would work if I just copied that part over?