Apply transform only to input not target

etremblay · July 29, 2020, 2:33am

Let’s say I want to do a denoising autoencoder in fastai v2, I want to add random noise to my input, but I don’t want it applied to my target. I thought the transforms would be the best place to add that:

class AddNoiseTransform(Transform):
    "Add noise to image"
    order = 11
    def __init__(self, noise_factor=0.3): store_attr(self, 'noise_factor')
    def encodes(self, o:TensorImage): return o + (self.noise_factor * torch.randn(*o.shape).to(o.device))

mnist = DataBlock(blocks=(ImageBlock(cls=PILImageBW), ImageBlock(cls=PILImageBW)), 
                 get_items=get_image_files,
                 splitter=RandomSplitter(),
                 batch_tfms=[AddNoiseTransform])

But then when I do show_batch() on my DataSets, I see the transform applied to both input and target:

I could create a subclass of TensorImage to represent my target so that my transform is not applied to it… But there must be a better way? How can I apply a transform only to the input, not the target?

muellerzr · July 29, 2020, 2:34am

Give it a split_idx property of 0. (There are some examples in the vision augmentation file I believe with this property). As you can imagine, 0 is train only, 1 is validation only, none is both

Edit: right here is one example:

github.com

fastai/fastai2/blob/master/fastai2/vision/augment.py#L22


from .core import *
from .data import *


# Cell
from torch import stack, zeros_like as t0, ones_like as t1
from torch.distributions.bernoulli import Bernoulli


# Cell
class RandTransform(Transform):
    "A transform that before_call its state at each `__call__`"
    do,nm,supports,split_idx = True,None,[],0
    def __init__(self, p=1., nm=None, before_call=None, **kwargs):
        super().__init__(**kwargs)
        self.p,self.before_call = p,ifnone(before_call,self.before_call)


    def before_call(self, b, split_idx):
        "before_call the state for input `b`"
        self.do = self.p==1. or random.random() < self.p


    def __call__(self, b, split_idx=None, **kwargs):
        self.before_call(b, split_idx=split_idx)

etremblay · July 29, 2020, 2:51am

Thanks for your quick reply, but what you are referring to is train vs validation. I want the transform to apply only to my x and not my y.

etremblay · July 29, 2020, 3:05am

Found a solution but it feel a bit dirty… Basically since I want the transform to only apply to my X and not my Y, I created a new class so that the type dispatch for my transform only apply it to the X:

class PILImageBWNoised(PILImageBW): pass
class TensorImageBWNoised(TensorImageBW): pass
PILImageBWNoised._tensor_cls = TensorImageBWNoised

class AddNoiseTransform(Transform):
    "Add noise to image"
    order = 11
    def __init__(self, noise_factor=0.3): store_attr(self, 'noise_factor')
    def encodes(self, o:TensorImageBWNoised): return o + (self.noise_factor * torch.randn(*o.shape).to(o.device))

mnist = DataBlock(blocks=(ImageBlock(cls=PILImageBWNoised), ImageBlock(cls=PILImageBW)), 
                 get_items=get_image_files,
                 splitter=RandomSplitter(),
                 batch_tfms=[AddNoiseTransform])

This yield what I want, a X with noise added to it and a clean target:

utkb · July 29, 2020, 12:06pm

Hi,

I think you will need to use fastai-v2’s “mid-level API” for this, which will give you more flexibility over the higher-level DataBlock API. Have a look at this chapter in fastbook, specifically the part about Datasets that allows specification of x_tfms and y_tfms.

But I guess if you already have something that works, can just stick with that?

Yijin