Domain randomization for better data augmentation?

Part 1 (2018)
1

OpenAI published a really groundbreaking result early this year, showing how to, in a VR simulation environment, generate thousands of variations in the background surrounding an object, in order to help train a computer vision model supporting a robot arm. The technique was called domain randomization.

Here’s the paper: Domain Randomization for Transferring Deep Neural Networks from
Simulation to the Real World.

I was wondering about doing this to extend data augmentation in images, in a way that’s similar to what we’ve learned here over the past few weeks. Here’s how I think it would work:

  1. Obtain a bounding box for the object you’d like to recognize. This could happen in the final step of e.g. the dog breeds challenge dataset. This would mean generating a new file for each image in e.g. the test set that was correctly classified, with a bounding box around the dog.
  2. Cut out the background of the bounding box i.e. what’s not in the bounding box, and save that background.
  3. Generate images with random noise, or with data generated from shuffling all other cutouts generated from step 2.
  4. Save images. This means you have a lot more training data.
  5. Do another step of data augmentation.

Has anyone tried this? I don’t have time to attempt this in the next few days (traveling), but it seems like an interesting enough approach to warrant some attention.

3 Likes
2

I have not, but if you gonna do this and looking for some company and know how to do bounding boxing, I am glad to join in this research :).

1 Like
3

Sure, go ahead! :muscle: The method can probably even go as far as a bounding shape.

4

interesting idea! as for the image data augmentation, i’m currently using the CycleGAN approach - so far i’m cautiously optimistic :slight_smile:

2 Likes
5

Otavio Good has done this for video, as shown in the autonomous car section of this video https://www.youtube.com/watch?v=Oqm9vsf_hvU . (It’s the same video we saw a clip from earlier in the course).

1 Like
6

Very cool! I should watch this video. Thanks!

7

So… CycleGAN + background swapping? How would that work?

8

i’m using CycleGAN in a somewhat different domain - my app is an OCR-ish type so it’ more like fonts, background textures - but in the process i became so impressed with the results that started thinking about applying it to other problems. Like in this paper: Optimizing Realism of Synthetic Agricultural Images using CycleGAN that might be more along the lines of your idea (including the object segmentation etc).

1 Like