Hello all! This is the continuation of topic Good Readings 2021, for the year 2022. It is for sharing deep learning papers that are important, interesting, or otherwise worth reading.
Comments and discussion are welcome.
1 Like
Automatic feature extraction is an amazing thing. But here’s an “artisanal” approach that required less training data, worked too, and generalizes to different domains. The article also includes a brief history of deep learning for vision.
The Pastry A.I. That Learned to Fight Cancer
(The article was published in 2021, but some of us take a while to get around to it.)
In this course we use GPUs and abstract mathematical computations to propagate forward and to adjust weights backwards. Yet the only example we have of true intelligence, the brain, is built on actual physical processes. And it uses a tiny fraction of the energy consumed by warehouses full of GPUs and TPUs.
Researchers are experimenting with non-linear physical systems that can learn, using memristers, light, and even sound. The big issue of course is how do the “weights” get adjusted in such systems, i.e., what serves for back propagation.
Here are a couple of articles that report what is happening. The first is conversational, the second technical.
How to Make the Universe Think for Us
Deep physical neural networks trained with backpropagation
Why I think this research is important:
-
Computational silicon will hit its limits due to fabrication and energy consumption long before the field of machine learning/AI does. Even now, only megacorporations can afford to train massive models like DALL-E and Imagen.
-
Machine learning is partially modelled after the brain. Yet we don’t even yet know how learning works there! There is lots more to discover from biological and physical systems.
-
Huge counts of weights and massively parallel computation will get us to the next level. At least that is obvious to me and it seems to be the way developments are going.
-
Inconceivable ethical issues are on the horizon. They will be far more complex and consequential than data biases, surveillance, and disinformation.
Enjoy and beware,
