Hi guys,
First of all, the fast.ai community is awesome. I learned a lot here!
I have been writing a blog entry to understand backpropagation mathematically for myself starting from very simple examples - https://treeinrandomforest.github.io/deep-learning/2018/10/30/backpropagation.html.
Some of the notation seems daunting and excessive but a lot of the length is because I tried to be extra detailed. If this is interesting to you, please let me know and I would really appreciate your feedback. In a way, this seems over-complicated because implementing backprop in code is so easy but I was pretending to be someone in the 1970s trying to solve this problem by looking at simple architectures first.
I still need to finish section IV here and then there’ll be a follow up post about actually implementing backprop from scratch.
Thank you,
Sanjay