Saw this phenomenal 2 minute papers video on a recent publication by the DeepMind team on relational reasoning with images and text [1].
The paper describes composing an input deep-net that produces feature-maps describing a set of objects together with a functional unit called a relation network that learns relations between those objects. The resulting network is able to learn complex relations between a variety of objects, outperforming even humans sometimes!
Now lets see if I can make my little driving robot navigate through a door, to the right of the second plant on the far end of my living room 
[1] Santoro, Adam, et al. “A simple neural network module for relational reasoning.” arXiv preprint arXiv:1706.01427 (2017).