In lesson 4, the concept of embeddings is presented for categorical data. Jeremy states the values for the embedding columns are randomly initialized. It is never explained how the embeddings are subsequently modified from that point.
In an effort to learn on my own, I found Rachel’s article where she mentions as an aside (in parentheses!) that the values of the categorical variables are learned as the network is trained. That is a major concept, not intuitive, and strangely (to me at least) never addressed.
My understanding currently is that the weights of the neural net are trained based on the data presented. The embeddings are the data. How and where does the system update the actual data values in addition to weights? I have trouble even understanding how that could work.
Could anyone explain or direct me to a resource that concisely explains this in great detail?