Unsupervised data augmentation (UDA) from Google...better than mixup

[jamesrequa]

@JoshVarty

Hmmm, there’s no code available for this yet is there?

Not that I am aware of no. I personally found the MixMatch paper (also shared here Good readings 2019 - #54 by jamesrequa) to be much more interesting, also involves using an unlabeled dataset but is a lot more straightforward to implement (plus it uses mixup ). For UDA paper, I actually found the TSA technique the most interesting part.

@JoshVarty

Does anyone know how to select training examples from the labelled and unlabelled samples? Do we shuffle the datasets together, but keep track of labelled vs. unlabelled? Or can we train exclusively on labelled examples at the start of training and then process the unlabelled training examples afterward?

I think it makes sense to have two separate data loaders one for the labeled dataset and one from unlabeled so you can grab a batch from each separately so that the losses for each could be also processed separately/calculated differently and then combine the batch randomly. I am not sure yet on the specifics of how UDA should be done but at least this is the case with MixMatch.