RNN Backprop Derivations - Blog Post

Dear all, after developing some intuition about RNNs from lectures 6 & 7, I took it upon myself to derive all the maths required to backprop through an RNN. I have written a detailed blogpost describing those calculations, and I have also written a Notebook building an RNN (without use of Pytorch or Tensorflow), using the equations I derived.

I hope this is useful for fellow Part 1 learners. Will be great to get your feedback or comments. Thanks

Blog Post - Build an RNN from scratch (with derivations!)

Accompanying Jupyter Notebook

1 Like

Nice blog post (especially because I am currently going through lecture 6 and 7)! :smiley:
However, I will need some time to go through your post and understand it. :wink:

I guess you (unintentionally) changed something in your GitHub folder, as I get a 404 error when trying to access the accompanying Jupyter notebook?

Thank you & best regards

I found the file in another folder in your repo: https://github.com/talwarabhimanyu/Learning-by-Coding/blob/master/Deep%20Learning/RNN%20from%20Scratch/RNN%20from%20Scratch.ipynb
Really nice work!

Thanks for your encouraging comments Michael :slight_smile:

Yes you are right - I had reorganized folders of my Git repository and that messed up hyperlinks in the blog post. Thanks for the catch - I’ve fixed those now!

Hi all,
Following up on my blog post on Creating an RNN from Scratch in Python (without using Pytorch, Tensorflow), I put out a new post on LSTMs in which:
(1) I derive the maths required for backprop through an LSTM,
(2) Create an accompanying Jupyter Notebook which builds an LSTM from scratch.

Here’s the link to my post. Hope y’all find it useful, and I welcome any feedback!