Micrograd/Tinygrad small Autograd engines(like pytorch) to lean how it all works

Deep Learning
1

Both these project do some of the basic operations of Pytorch but are very small and very readable. They are based off numpy so it easier to understand.
I found them very useful to learn how all the basic operations for forward and backward passes are working. You can also see that that tinygrad is going to be hardware agnostic by using PyOpenCL as a GPU backend.
micrograd
tinygrad

1 Like