Transfer learning on drones: From simulation to real world

The current issue (138) of the newsletter reports a very interesting use case of transfer learning. Without much ado, here is the essence of the story:

"They train the (simulated) drones using Proximal Policy Optimization (PPO) with a cost function designed to maximize stability of the drone platforms. They sanity-check the trained policies by running them in a different simulator (in this case, Gazebo using the RotorS package) and observing how well they generalize. “This sim-to-sim transfer helps us verify the physics of our own simulator and the performance of policies in a more realistic environment,” they write.

They also validate their system on three real quadcopters, built around the ‘Crazyflie 2.0’ platform. “We build heavier quadrotors by buying standard parts (e.g., frames, motors) and using the Crazyflie’s main board as a flight controller,” they explain…"

What’s your thoughts about the potential of converging reinforcement learning, simulation and engineering?


A few years ago for a hackathon I made a Unity simulation that modeled drone-based search and rescue. It used IBM’s image classification API to search for an overturned boat in realistically-simulated ocean. (The API did not perform well at all). There’s a lot of cool possibilities for simulated environments.

Check out some of the stuff at SpatialOS, basically a cloud computing framework for running huge agent-based models.