Congratulation to all my fellow students who received the invitation to Part 2 of the course! All of you have been wonderful members of our community and I cannot wait to take on the exciting journey of Part 2 with you all.
As we know all, in Part 2 we will we diving deep into both the realm of deep learning and the fast.ai library itself. To really suck out all the marrow out of it, and there will be a lot, we need to be better prepared. Here is a quote of Jeremy from Part 1’s course website:
We teach how to train PyTorch models using the fastai library. These two pieces of software are deeply connected—you can’t become really proficient at using fastai if you don’t know PyTorch well, too.
It is clear that we need to be more comfortable and familiar with PyTorch to do well in Part 2.
There have been many wonderful posts about PyTorch out there in the forum. This post is intended to be a synthesis and update. Here are a few that I can find:
- Getting Comfortable with Pytorch & Projects
- PyTorch resources?
- Plain PyTorch implementation of fast.ai notebooks
Below is a list of resources organized into different themes and topics. It is intended to be short, highly relevant to the to value quality over quantity. (When the list becomes as long as Awesome PyTorch List, it really loses its value as there too many options.) It is also intended to focus on PyTorch framework itself, rather than deep learning concepts, which Part 1 has covered well.
Note that this is a forum wiki thread, so you all can edit this post to add/change/organize info to help make it better! To edit, click on the little edit icon at the bottom of this post. Here’s a pic of what to look for:
What to read
Tutorials
- Jeremy’s amazing What is
torch.nnreally ?- “A quick journey: from neural net “from scratch”, to fully utilizing
torch.nn,torch.optim,Dataset, andDataLoader” – Jeremy - @SHAR1 , the author of Getting Comfortable with Pytorch & Projects post, finds it way better than the official introductory PyTorch tutorials.
- This one will get you familiar with the core of PyTorch.
- “A quick journey: from neural net “from scratch”, to fully utilizing
- Other official PyTorch tutorials
- PyTorch: Zero to GANs - Tutorial series going from basics to advanced topics
- Development Notebooks written when developing the library and course materials.
- NLP Tutorial for Deep Learning Researchers: From basics to advanced concepts in PyTorch
Video-Tutorials/Courses
Udacity Course by facebook ai: Intro to DL with pytorch.
Internals
- PyTorch under the Hood
- CAUTION: This tutorial goes very deep into the lower-level design of PyTorch.
- Selective excursion into PyTorch internals not as comprehensive as Christian’s talk, focuses on how functions are exposed to PyTorch
What to do
Contribute to fastai
Since the fastai library itself is written with PyTorch, you could always learn loads about PyTorch by diving into the library’s source codes written heavily with PyTorch.
- You could write tests for it. It is a great way to learn because to write tests you must first understand what a piece of code does exactly.
- You don’t need to have any previous experience. Here is a very detailed guide on how to get started.
- Here is Jeremy’s tip on how to get started:
-
Documentation improvements never get old. Similar to writing tests, it will force you to truly understand the library.
-
The wonderful How to contribute to fastai general guide by @PierreO