Hi All,
Does anybody know how one could “replace” an item in the batch with a different item as an augmentation?
I am working in the remote-sensing domain, where it is important to ensure that the model is robust against seasonal changes.
One simple augmentation technique is to consider the exact same location over multiple timesteps as augmented views of the first image.
Say, we have to pictures of a bridge: One in winter and one in summer. Now, I would only sample from all images in winter, but would like to randomly “augment” the winter to the summer “view”.
From a usability perspective, it would be nice, if it would be a “normal” Transformer
, just so one can quickly add/remove the augmentation where the other ones are used. This transformation should be at the very beginning of the pipeline, as all the other transformations would be “overwritten” by the replacement operation.
But how would you actually implement the transformation?
Or would you go a different route, to reduce unnecessary data loading?
Would you try to infer the index of the current item with val2idx
or from the dataloader?
Or get the hash of the current item and then “look-up” files from different time steps?
This would require the transformer to also include the function to do the type-transform
.
I am not sure if I am missing something and if there is a “nicer” way to do it.
I would appreciate any input.
EDIT: A different approach would be to subclass from TensorImage, or monkey-patch, some metadata information into the tensor about the current item, like the name/path for example.