Hello everyone!
This semester, I’m a TA in my college for the deep learning course and I was supposed to gather some high-quality resources/tutorials for people in the course at all levels of experience. I decided that if things have to be fast and practical, then the resources need to:
- Be more code-focused, less theory-based. You should be able to directly jumpy into practical exercises for whatever topic you choose to learn.
- Be possibly jupyter notebook driven.
- Avoid video when possible because having to watch a video brings both experts and beginners at the same rate of learning. If I’ve been coding for a while, I’d want to accelerate through the resources. If I haven’t, I’d want to read them at my own pace. Having a video to watch makes this difficult.
- Not necessarily sacrifice depth in favour of approachability.
- Be limited in number, possibly ≤ 2 resources for each topic. Having too many resources sometimes makes it very hard for people to choose the best ones for them and get started.
With these goals in mind, I sent out a mail to all the students registered in the course recently and I thought it’d be good if I put the same here.
Note that you do not need to (and shouldn’t) learn everything beforehand to start practicing deep learning. Ideally, you should learn the tools you need only when you need them. The reasoning here is that the amount of topics/tools that comprise effective practice of deep learning is very high and is ever-growing, and trying to learn everything beforehand will only slow you down before you do any actual significant work.
Hope this helps!
Here’s the list:
Linux, shell tools and scripting, command line environment, git (version control), regular expressions, CLI editors, data wrangling, etc:
Most people who do significant work on their computers end up having to learn all of this in a slow, ineffective and hacky way throughout their coding jounrey. Thankfully, some folks at MIT realize this and have designed a small course (replete with detailed notes if you do not want to watch the lectures) with practical components for all these topics and more:
Starting from the basics: Python, Jupyter, NumPy, Pandas, etc.
If you need to brush up your python skills, or have never coded before, this course will get you up to speed very quickly while still covering all the important parts of the language with respect to data science (exercises included): https://www.kaggle.com/learn/python
Introduction to Jupyter Notebooks : https://realpython.com/jupyter-notebook-introduction/#getting-up-and-running-with-jupyter-notebook
NumPy refresher:
Introductory: http://cs231n.github.io/python-numpy-tutorial/#numpy
Comprehensive (recommended): https://nbviewer.jupyter.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-2-Numpy.ipynb
Pandas, Matplotlib, Seaborn:
This is a great series of notebooks to get an ML-focused practical introduction to pandas, data vizualization and some exploratory data analysis:
- https://mlcourse.ai/articles/topic1-exploratory-data-analysis-with-pandas/
- https://mlcourse.ai/articles/topic2-visual-data-analysis-in-python/
- https://mlcourse.ai/articles/topic2-part2-seaborn-plotly/
Alternatives (in depth):
Pandas: https://www.kaggle.com/learn/pandas
Data Vizualization using Matplotlib, Seaborn, Plotly, etc: https://www.kaggle.com/learn/data-visualization
Matplotlib fundamentals in depth: https://matplotlib.org/matplotblog/posts/an-inquiry-into-matplotlib-figures/ ( I authored this so I had to put it here! 
)
A single resource to cover most of what you will ever need for scientific programming in python: http://scipy-lectures.org
The above series covers Python, NumPy, Matplotlib and SciPy at both introductory and advanced levels.
Deep Learning Theory
A course on deep learning concepts for beginners and intermediates alike:
The course gives a fast-paced, yet very comprehensive coverage of the most important and practical aspects of deep learning. The lectures cover a huge amount of content within a single hour. For example, by the end of the 4th lecture, you’ll already have understood the theory behind neural networks, activation functions, loss functions, backward propagation, regularization, adaptive learning rates, weight initialization, RNNs, LSTMs, gated LSTMs, BPTT, CNNs, image captioning, segmentation, autoencoders, VAEs, GANs, CycleGANs, etc.
Also, the lab exercises of the above course serve as a good introduction to Tensorflow 2.0
The Math you need for reading and understanding Deep Learning research papers:
Matrix Calculus Refresher: https://explained.ai/matrix-calculus/index.html
And a full (free) book to help you review all of the math required for deep learning: https://mml-book.com
Tensorflow 2.0
[Need to add more Resources here]
[I’m not familiar with Tensorflow, perhaps someone else can contribute here]
PyTorch:
If you prefer video : https://deeplizard.com/learn/playlist/PLZbbT5o_s2xrfNyHZsM6ufI0iZENK9xgG
If you prefer text :
An all-in-one beginner friendly introduction by Jeremy: https://pytorch.org/tutorials/beginner/nn_tutorial.html
Extras:
A Neural Network Playground to experiment and build intuition with different datasets, architectures, activation functions, and regularization: https://playground.tensorflow.org
Interactive tutorial on weight initialization: https://www.deeplearning.ai/ai-notes/initialization/
Interactive tutorial on different optimizers: https://www.deeplearning.ai/ai-notes/optimization/
Articles on advanced deep learning topics: https://distill.pub (not necessarily beginner friendly)
Small, practical courses on several data science tools: https://www.kaggle.com/learn/overview
