Hi @Pomo glad you like the article! I recently listened to Lex Fridman’s interview with Ilya Sutskever and it made me think about how some of the visual abilities we take for granted can be transferred to neural networks. I don’t think there is a universal way of this transfer yet since we don’t know how our brain works. But there definitely can be ways to tackle specific small problems.

The universal approximation theorem tells us that neural networks can approximate any function. Take this spiral data as an example, we could give it some prior, say, we formulate it as a regression instead of a classification and let it fit the spirals. An even stronger prior is to manually tell it to fit a spiral-like function in polar coordinates. Of course, that defeats the purpose of letting it learn the form by itself. But you see, it’s hard to draw the line how much of prior knowledge we can give it.

As for the question of how we intuitively do this, I think what we do is just a kind of regression in our mind, so it’s not too different from pre-programming with a prior. We are more sensitive to simple functions that exist in our world. e.g. spirals in nature like a cyclone, spiral shells, galaxies, water swirl, or just in a math class. So I think given a certain task formulation and a certain number of examples, artificial NNs are capable of solving these specific problems.