The Deep Learning text book by Goodfellow, Bengio and Courville mentions an interesting trick that uses complex numbers to calculate a numerical estimate of a gradient. I made a short notebook to explain it: https://github.com/science4performance/DataScience/blob/master/Derivative%20using%20complex%20trick.ipynb
2 Likes