Fast and Practical resources/tutorials for all levels of coding experience

Hello everyone!
This semester, I’m a TA in my college for the deep learning course and I was supposed to gather some high-quality resources/tutorials for people in the course at all levels of experience. I decided that if things have to be fast and practical, then the resources need to:

  1. Be more code-focused, less theory-based. You should be able to directly jumpy into practical exercises for whatever topic you choose to learn.
  2. Be possibly jupyter notebook driven.
  3. Avoid video when possible because having to watch a video brings both experts and beginners at the same rate of learning. If I’ve been coding for a while, I’d want to accelerate through the resources. If I haven’t, I’d want to read them at my own pace. Having a video to watch makes this difficult.
  4. Not necessarily sacrifice depth in favour of approachability.
  5. Be limited in number, possibly ≤ 2 resources for each topic. Having too many resources sometimes makes it very hard for people to choose the best ones for them and get started.

With these goals in mind, I sent out a mail to all the students registered in the course recently and I thought it’d be good if I put the same here.

Note that you do not need to (and shouldn’t) learn everything beforehand to start practicing deep learning. Ideally, you should learn the tools you need only when you need them. The reasoning here is that the amount of topics/tools that comprise effective practice of deep learning is very high and is ever-growing, and trying to learn everything beforehand will only slow you down before you do any actual significant work.

Hope this helps!

Here’s the list:


Linux, shell tools and scripting, command line environment, git (version control), regular expressions, CLI editors, data wrangling, etc:

Most people who do significant work on their computers end up having to learn all of this in a slow, ineffective and hacky way throughout their coding jounrey. Thankfully, some folks at MIT realize this and have designed a small course (replete with detailed notes if you do not want to watch the lectures) with practical components for all these topics and more:

https://missing.csail.mit.edu

Starting from the basics: Python, Jupyter, NumPy, Pandas, etc.

If you need to brush up your python skills, or have never coded before, this course will get you up to speed very quickly while still covering all the important parts of the language with respect to data science (exercises included): https://www.kaggle.com/learn/python

Introduction to Jupyter Notebooks : https://realpython.com/jupyter-notebook-introduction/#getting-up-and-running-with-jupyter-notebook

NumPy refresher:

Introductory: http://cs231n.github.io/python-numpy-tutorial/#numpy

Comprehensive (recommended): https://nbviewer.jupyter.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-2-Numpy.ipynb

Pandas, Matplotlib, Seaborn:

This is a great series of notebooks to get an ML-focused practical introduction to pandas, data vizualization and some exploratory data analysis:

  1. https://mlcourse.ai/articles/topic1-exploratory-data-analysis-with-pandas/
  2. https://mlcourse.ai/articles/topic2-visual-data-analysis-in-python/
  3. https://mlcourse.ai/articles/topic2-part2-seaborn-plotly/

Alternatives (in depth):

Pandas: https://www.kaggle.com/learn/pandas

Data Vizualization using Matplotlib, Seaborn, Plotly, etc: https://www.kaggle.com/learn/data-visualization

Matplotlib fundamentals in depth: https://matplotlib.org/matplotblog/posts/an-inquiry-into-matplotlib-figures/ ( I authored this so I had to put it here! :sweat_smile:)

A single resource to cover most of what you will ever need for scientific programming in python: http://scipy-lectures.org
The above series covers Python, NumPy, Matplotlib and SciPy at both introductory and advanced levels.


Deep Learning Theory

A course on deep learning concepts for beginners and intermediates alike:

The course gives a fast-paced, yet very comprehensive coverage of the most important and practical aspects of deep learning. The lectures cover a huge amount of content within a single hour. For example, by the end of the 4th lecture, you’ll already have understood the theory behind neural networks, activation functions, loss functions, backward propagation, regularization, adaptive learning rates, weight initialization, RNNs, LSTMs, gated LSTMs, BPTT, CNNs, image captioning, segmentation, autoencoders, VAEs, GANs, CycleGANs, etc.

Also, the lab exercises of the above course serve as a good introduction to Tensorflow 2.0

The Math you need for reading and understanding Deep Learning research papers:

Matrix Calculus Refresher: https://explained.ai/matrix-calculus/index.html

And a full (free) book to help you review all of the math required for deep learning: https://mml-book.com

Tensorflow 2.0

[Need to add more Resources here]

[I’m not familiar with Tensorflow, perhaps someone else can contribute here]

PyTorch:

If you prefer video : https://deeplizard.com/learn/playlist/PLZbbT5o_s2xrfNyHZsM6ufI0iZENK9xgG

If you prefer text :

An all-in-one beginner friendly introduction by Jeremy: https://pytorch.org/tutorials/beginner/nn_tutorial.html

Extras:

A Neural Network Playground to experiment and build intuition with different datasets, architectures, activation functions, and regularization: https://playground.tensorflow.org

Interactive tutorial on weight initialization: https://www.deeplearning.ai/ai-notes/initialization/

Interactive tutorial on different optimizers: https://www.deeplearning.ai/ai-notes/optimization/

Articles on advanced deep learning topics: https://distill.pub (not necessarily beginner friendly)

Small, practical courses on several data science tools: https://www.kaggle.com/learn/overview

16 Likes

@arora_aman @init_27 @muellerzr @barnacl
Improvements / Additions?

This is a beautiful thread. You’ve got it all covered I think :slight_smile:

1 Like

I think so too. Thanks, Akash!

1 Like

Looks good, i have to check out a few myself. :slight_smile:
May be some OOP?

This is one of my favourite resources. Similar to this Jeremy has a SGD notebook i think that is also great :slight_smile: All the best TAing :slight_smile:

1 Like

This is a wonderful idea, I love it :slight_smile:

1 Like

What sort of OOP would be relevant here? Can you provide some links/examples ?

++ This. Yes I have to keep this in mind when people asking for suggestions and hence intentionally decided not to give lot of resources. Max 2 which are easy enough to get started and build momentum from there.

3 Likes