Resources Master Thread

[ecdrid]

@jeremy

Source - my github repository…

Machine Learning & Deep Learning

Hope it helps…Feel free to add more …

Loose path:

0. Fast.ai( Complete them before anything Though knowledge of the commonly used frameworks (Pytorch, Keras, Tensorflow etc…) would be a huge advantage…)
1. Math
2. Programming
3. Machine Learning concepts
4. Specializations

Fast.ai(Ground zero)

1. Deep Learning Part 1: Practical Deep Learning for Coders

2. What we cover in the course

3. Deep Learning Part 2: Cutting Edge Deep Learning for Coders

4. Computational Linear Algebra: Online textbook and Videos

5. A Unique Path to Deep Learning Expertise—our teaching approach

6. Pytorch Tutorial

7. github link

The best point is the 5th one…

Math

Understanding Math is pivotal. You can never be a good Machine Learning Scientist
by skipping the Math.

1. Probability & Statistics
   Basic Probability and Stats will be helpful in understanding ML algorithms like Naive Bayes.

2. Statistics 101 - Udacity
   Taught by the founder of GoogleX it’s full of exercises in Python so you won’t get bored.

3. MIT 18.06 Linear Algebra
   Prof. Strang is terrific! Not only he’ll make you fall in love in Linear Algebra but you’ll learn
   important concepts like SVD and matrix algebra. You might wanna grab this PDF
   as well. Be sure to also solve the exam question papers from here: link

4. MIT Single Variable Calculus
   This is my personal favorite book, use it for SVC + MVC link
   Amazing course but it gets quite tedious in the middle, you might wanna skim some geometry, but the key is
   to understand how optimization works. Be sure to solve questions from here: link

5. MIT Multi Variable Calculus
   Understanding vector calculus is necessary for algorithms like SVM, you might wanna skim some parts
   which are purely theoretical. Be sure to solve questions from here: link

6. (Optional) Stanford Convex Optimization
   WARNING: Do this course only if you’re very good at math. Convex Optimization will teach you numerous
   functions used in Machine Learning. But this course is extremely heavy on Math!

Introduction to Programming & Algorithms

• Python - Any one, both courses are equally good

  1. Complete Python BootCamp
  2. Complete Python Masterclass

• Algorithms

  Since you’ll be coding a lot of algorithms yourself basic understanding is necessary

  1. Basic Algorithms and Complexity Theory

  In case you want to go deeper

  1. Algorithms Stanford I
  2. Algotithms Stanford II

Introduction to Machine Learning

1. Machine Learning by Andrew Ng
   A must do course, best course of Introduction to Machine Learning so far, light on Math and focuses more on concepts.

Complete one out of two:

1. Machine Learning A-Z
   Introductory course on ML focusing on not only Python but also R, one of the best sellers on Udemy.

2. Introduction to Machine Learning - Udacity
   Sebastian Thrun does an awesome job explaining various approaches in ML. It gets a little boring in the middle
   but overall it’s very good.

Applied Machine Learning

Two quick courses on applying the theory you learnt. They’re short so I recommend doing both of them.

1. Python for Data Science and Machine Learning Bootcamp

2. Machine Learning with Python - Hands On!

Specializations

• Deep Learning

  1. Neural Networks by Geofrrey Hinton
     This guy is the creator of backpropagation algorithm! Warning: very heavy on Math.

  2. MIT Introduction to Deep Learning

  3. Deep Learning A-Z

  4. Creative Applications of Deep Learning with TensorFlow

  5. Must read book on Deep Learning: Free HTML book

• Big Data & Large Scale Machine Learning

  1. Introduction to Big Data
  2. Spark and Python for Big Data with PySpark
  3. Distributed Machine Learning with Apache Spark

• Natural Language Processing

  1. Introduction to Natural Language Processing UMichigan
  2. Natural Language Processing by Stanford

• Self Driving Car

  1. MIT Deep Learning for Self Driving Cars
  2. Self Driving Car Nano Degree

• Scientific Computing

  1. Scientific Computing
  2. High Performance Scientific Computing

Bonus Material

General Neural Network References:

Books/Guides on Deep/Machine Learning: (all excellent)

1. http://neuralnetworksanddeeplearning.com

2. http://machinelearningmastery.com

3. https://www.deeplearningbook.org

Hacker’s Guide to Neural Nets by karpathy(My Favourite)

Tutorials/Videos:

1. Youtube Playlist on “Deep Learning”, t from Oxford U. by Nando de Freitas

2. Andrew Ng’s online course on ML at Stanford comes highly recommended

3. Stanford Tutorial:

Concepts in NN/Deep Learning:

1. [Backpropagation (i.e. the chain rule)](neuralnetworksanddeeplearning.org book), chapter 2

2. Chris Olah on backprop

3. Karpathy on backprop

Recurrent Neural Networks (RNN) (which mostly feature LSTM nowadays):

1. Karpathy post

2. Karpathy talk

3. Excellent annotated Char-NN in Keras tutorial

4. Andrew Trask post/tutorial

5. Denny Britz post

6. Class notes/tutorial (long!)

7. CS class notes (short)

8. Excellent post by Ross Goodwin RNNs

9. Great List of references;

10.in TensorFlow

11. Theano tutorial

12. Batch Normalization to the hidden-to-hidden transitions of our RNNs

Traditional RNNs suffer from vanishing/exploding gradient. Hence LSTM & others…

Long Short-Term Memory (LSTM):

1. Tutorial

2. Chris Olah post

3. Zach Lipton post, “Demystifying LSTM” (with Tutorial theano code)

4. Demo: Lightweight Theano-LSTM

5. Massive 33-page review article by Lipton et al

6. LSTM tutorial in Tensorflow

7. Stateful LSTM in Keras for time-series prediction

8. Much-need Docs on stateful LSTM in Keras

9. Tensorflow sequence prediction