Same here - the black-box view has always bothered me, mainly because I like understanding how things work, and secondarily because it seems to encourage a “magical thinking” mindset. (On the other hand, viewing deep learning as a general-purpose tool makes it easier to think of potential applications for it.)
@r2d2 came up with a PCA approach for inspecting the final features of a model, similar to the convolution visualizations that Jeremy showed in the lecture: https://forums.fast.ai/t/share-your-work-here/27676/26 . I have been trying it out on my data, and it’s very cool!
On the information bottleneck theory: I wonder how it applies to the transfer-learning type of training which we’re doing in lesson 1? The Imagenet pretrained model must have reached the “compression phase” of training (stage D in the last figure in the article), since general features of real-world photographs have been compressed and can generalize to many different image datasets. But when we do the training on a specific dataset, does it run through all the stages again (fitting, then the phase change to compression), perhaps just in the final layers and on a smaller scale? What about when we unfreeze the rest of the layers? Lots to think about!