A Recipe for Training Neural Networks

Hi everyone!

Andrej Karpathy, the guy that first created the now famous cs231n class at Stanford and is now Director of AI at Tesla, just published a blog post about how to train Neural Networks that I find very very interesting and so I thought I’d share it here. Enjoy!

Also while I’m talking about Tesla, they had an “Autonomy Day” a few days ago which was basically talking about how they design every aspect of their AutoPilot hardware and software. Very interesting also.

15 Likes

Good read, thanks for posting!

Welcome to the real world! Thanks for sharing!

Thanks. It’s a great read, written in accessible language. Karpathy stresses the importance of visualising what is happening during training, which reminded me of Lesson 10.

1 Like

Thanks for sharing.

suffering is a perfectly natural part of getting a neural network to work well, but it can be mitigated by being thorough, defensive, paranoid, and obsessed with visualizations of basically every possible thing.

:laughing: :rofl: :grinning:


I enjoyed the “Neural Network Essentials” part from Karpathy’s Tesla “Autonomy Day” presentation.

Many practical advice.

4 Likes

Finally had some time to watch it. This was def. well worth a watch.

1 Like