Based on the visualisations of gradient descent in Lecture 3 (and Lecture 2 of version 3), I have been investigating gradient descent and producing a bunch of visuals of my own. Some of the visuals are funky and emotive and almost lifelike!
- Part 1. Gradient descent with simple models on simple data
- Part 2. Gradient descent with neural networks models on simple data
- Part 3. Stochastic gradient descent.
Though (I think) I understand the theory of gradient descent and of vanilla neural networks, it is evident that the whole is greater than the sum of its parts. I under-estimated what I could have learnt by experimenting.