Thanks to @fmussari for sharing the colab for searching youtube! I found the following by using the colab.
Hi Mike, here is Jeremy answering the question on Affine function in which he talked about:
- What is an Affine function (linear function)? What does it do? (matrix multiplication and sum up)
- What does it mean by adding one Affine function on top of another? (still another linear function)
- What does it do by putting a non-linear activation (ReLU) in between Affine functions? (essentially building a neuralnet)
More to explore?
There are some good papers or posts we should read on ReLU or non-linear function in general to just have a one-step further understanding of the magic of non-linear function to neuralnet?