This is a powerful abstraction for image data augmentation that generalizes MixUp, Cutout, CutMix, RICAP, and more, and allows annealing of the amount of augmentation during training (to support curriculum learning) - Jeremy on Twitter
I’ve been thinking lately about the new types of image data transformation (random erasing, cutout, mixup, ricap, cutmix, etc.) and have come up with a single transform that can achieve many types of similar transformations. I call it Blend, as I think that is what most of this transformations have in common. They all blend pixels in a certain area with something else.
Many of the ideas I’ve integrated in it come from the original cutout, mixup, ricap and cutmix, as well as the great CutMix > Mixup (?) and Progressive Sprinkles (cutout variation) - my new data augmentation & 98% on NIH Malaria dataset threads.
I’d like to all participants in those threads, and specially @LessW2020 for creating those threads and sharing lots of creative ideas in them, @rwightman for his ‘frosted sprinkles’ idea, and Jeremy for bringing up ricap (I wasn’t aware of it).
I wanted to learn more about fastai and callbacks (although I know they will be changed soon) and have created a notebook to explain how to use this transform.
Also, related to this I have created a transform scheduler that allows to modify any parameters of the transform during training. It’s the TfmScheduler. This also stems from an idea @LessW2020 shared in the threads above. I’ve always been intrigued by this and wanted to learn how to code it.
I have run a few tests (you will see that there are endless possibilities) and have seen some promising results. But I don’t have the hardware to really test this in depth as it takes a considerable amount of time in my case.
I have added it to the fastai_extensions repo that I built a few days ago (repo)
I hope some of you may find this useful. In any case it’s been a great learning experience.
PS. I don’t have a CS background (I have a pharmacy degree), so apologies if the code is not at the right level.
Edit: I’ve added above a great definition of what this code is made by Jeremy on Twitter.