Tutorial on adding new data augs (Gaussian Noise)?

Part 1 (2020)
1

Hi there, are there any good guides/tutorial on how to add new data augs that is compatible with the fastai data aug RandTransform class? I need to add some noise to my images, and I’ve only come across pytorch implementations but not fastai. Thanks.

Tried the following but it’s erroring out in other parts of the fastai aug pipeline:

class AddNoise(RandTransform):
    
    def __init__(self, mean=0., std=1.):
        self.std = std
        self.mean = mean
        
    def encodes(self, x:(Image.Image,TensorBBox,TensorPoint)):
        return x + torch.randn(x.size()) * self.std + self.mean
2

+1 for this request. And not only for extensions of RandTransform. Would be nice to have tutorials explaining how to create custom fastai transforms in general.

1 Like
3

I just some small changes and your transform works :).

  1. call super().init() to initialize the RandTransform correctly
  2. Changed the type of X to TensorImage
class AddNoise(RandTransform):
    
    def __init__(self, mean=0., std=1., **kwargs):
        self.std = std
        self.mean = mean
        super().__init__(**kwargs)
        
    def encodes(self, x:TensorImage):
        return x + torch.randn(x.size()) * self.std + self.mean

Test:

path = untar_data(URLs.MNIST)

class AddNoise(RandTransform):
    
    def __init__(self, mean=0., std=1., **kwargs):
        self.std = std
        self.mean = mean
        super().__init__(**kwargs)
        
    def encodes(self, x:TensorImage):
        return x + torch.randn(x.size()) * self.std + self.mean


db = DataBlock(blocks=(ImageBlock(cls=PILImage), CategoryBlock), 
                  get_items=get_image_files, 
                  batch_tfms=[AddNoise(mean=0., std=100.)],
                  get_y=parent_label)

dls = db.dataloaders(path)

dls.show_batch()

Bildschirmfoto 2020-10-30 um 17.02.34
Bildschirmfoto 2020-10-30 um 17.02.341390×772 135 KB

To apply the transform only with a 50 % chance do:

batch_tfms=[AddNoise(mean=0., std=100., p=0.5)],

Florian

4 Likes
4

In your work, did you realize there is something different about implementation of Transform vs RandTransform? I’ve been working on color jitter

class AddJitter(RandTransform):
    
    def __init__(self, color_jitter=1, **kwargs):
        super().__init__(**kwargs)
        self._color_jitter = color_jitter
            
    def jitter(self, x):

        cjitter = kornia.augmentation.ColorJitter(self._color_jitter, self._color_jitter, self._color_jitter, self._color_jitter)
        return cjitter(x)
    
    def encodes(self, x:TensorImage):
        
        return self.jitter(x)

And this is the non-random version

class AddFixedJitter(Transform):
    
    def __init__(self, color_jitter=1, **kwargs):
        super().__init__(**kwargs)
        self._color_jitter = color_jitter
        
    def jitter(self, x):
        
        cjitter = kornia.augmentation.ColorJitter(self._color_jitter, self._color_jitter, self._color_jitter, self._color_jitter)
        return cjitter(x)
    
    def encodes(self, x:TensorImage):
        
        return self.jitter(x)

As you see the functions above are identical. But I get way different results when I use AddJitter(p=1), which should be the same as AddFixedJitter. I think it’s something on the validation transforms being applied or not applied… but I can’t figure out the exactly what’s going on.

1 Like
5

That’s exactly it. RandTransforms aren’t applied on the validation set. They have a hard coded split_idx of 0 IIRC