Just announced by OpenAi and Google:
What have neural networks learned that allows them to classify images so well?
Activation Atlases provide a new way to peer into convolutional vision networks, giving a global, hierarchical, and human-interpretable overview of concepts within the hidden layers of a network. We think of activation atlases as revealing a machine-learned alphabet for images — an array of simple, atomic concepts that are combined and recombined to form much more complex visual ideas. [text from google AI blog]