Hi Everyone,
I hope this is the right place to share this. Lately, I have been interested in the potential applications of combining deep learning with real-time development platforms like Unity and Unreal Engine
I decided to start with Unity as they actually have a cross platform inference library called Barracuda.
I like the idea of mapping a user’s movements to a virtual character in real-time, so I started with a pose estimation model. I made a tutorial to help document what I learned and recently updated it.
Barracuda PoseNet Tutorial
GitHub: GitHub - cj-mills/Barracuda-PoseNet-Tutorial: This tutorial series provides step-by-step instructions for how to perform human pose estimation in Unity with the Barracuda inference library.
Tutorial - Part 1: Barracuda PoseNet Tutorial 2nd Edition Pt. 1 | Christian Mills
I also recently worked out how to make plugins for Unity that leverage Intel’s OpenVINO toolkit to improve inference speed. The performance gains over Unity’s Barracuda library are significant (approx. 5-6x faster in my experience).
I made a tutorial for how to make a plugin for a modified fast-neural-style model to demonstrate the performance gains.
OpenVINO Plugin for Unity
GitHub: GitHub - cj-mills/Unity-OpenVINO-Plugin: This tutorial series covers how to create a plugin that leverages the OpenVINO™ Toolkit for the Unity game engine.
Tutorial - Part 1: OpenVINO Plugin for Unity Tutorial Pt.1 | Christian Mills
Related Links:
End-to-End In-Game Style Transfer Tutorial: GitHub - cj-mills/End-to-End-In-Game-Style-Transfer-Tutorial: This tutorial series covers how to train your own style transfer model with PyTorch and implement it in Unity using the Barracuda library.
Targeted In-Game Style Transfer Tutorial: GitHub - cj-mills/End-to-End-In-Game-Style-Transfer-Tutorial-Intel at targeted-style-transfer
Anyways, I just wanted to contribute back to fast.ai community in some way and hope that theses tutorials can be of use.
If someone does happen to read them, constructive feedback would be especially appreciated. I know my writing has room for improvement.
Also, if there are specific requests for tutorials in this area, I would be happy to look into them.