In the first lecture Jeremy mentions, in his list of misconceptions about Deep Learning, the idea that Neural Networks are Black Boxes. This resonated with me deeply, particularly because I am currently pursuing a business opportunity, where the client is leery of using Deep Learning “because it is difficult to know why a Neural Net made a particular decision, or which data features where most important in making that decision.”
In doing a quick bit or research on the topic, I came across this excellent article outlining the “Information Bottleneck Theory” of Neural Network decision making. I also came across the paper " Opening the black box of neural nets:case studies in stop/top discrimination" out of the Physics Dept. at Harvard.
The paper is a long one, but the aim of the approach seems to be to make “contour maps of [a] network’s classification function.” In light of all this, I was just wondering if anyone had any insights as where Jeremy was headed with this line of thought, as well as knowledge about features of the new library that might help one better understand a model’s decision making process.