Creating synthetic training data for computer vision in Unity


the Unity game engine offers a new package for creating synthetic training data in the area of computer vision. It creates the images, as well as the annotations (bounding boxes, segmentations). Here is a video on an example I created for training a segmentation model with fastai:



This is super cool!

1 Like

This is awesome, thanks for the pointer! I’ve been waiting for something like this to come along.

Also, time to buy some Unity stock :slight_smile:

1 Like

We are doing the same thing.

Nice! Nvidia created a library using Unreal engine done time back. Very popular paper.
And I recently saw a paper where the authors created a pipeline in Blender for generating synthetic data.

I’m building a Blender pipeline myself to render synthetic data of transparent objects for an ongoing project. My prev project, ClearGrasp, also used similar synthetic data and we found it to work very well. It generalized on the real world test set quite nicely.

What are you all using synthetic data for?

1 Like