Diving Deep into PyTorch

(George Zhang) #1

Congratulation to all my fellow students who received the invitation to Part 2 of the course! All of you have been wonderful members of our community and I cannot wait to take on the exciting journey of Part 2 with you all.

As we know all, in Part 2 we will we diving deep into both the realm of deep learning and the fast.ai library itself. To really suck out all the marrow out of it, and there will be a lot, we need to be better prepared. Here is a quote of Jeremy from Part 1’s course website:

We teach how to train PyTorch models using the fastai library. These two pieces of software are deeply connected—you can’t become really proficient at using fastai if you don’t know PyTorch well, too.

It is clear that we need to be more comfortable and familiar with PyTorch to do well in Part 2.

There have been many wonderful posts about PyTorch out there in the forum. This post is intended to be a synthesis and update. Here are a few that I can find:

Below is a list of resources organized into different themes and topics. It is intended to be short, highly relevant to the to value quality over quantity. (When the list becomes as long as Awesome PyTorch List, it really loses its value as there too many options.) It is also intended to focus on PyTorch framework itself, rather than deep learning concepts, which Part 1 has covered well.


Note that this is a forum wiki thread, so you all can edit this post to add/change/organize info to help make it better! To edit, click on the little edit icon at the bottom of this post. Here’s a pic of what to look for:
discourse_edit_btn


What to read

Tutorials

Video-Tutorials/Courses

Udacity Course by facebook ai: Intro to DL with pytorch.

Internals

What to do

Contribute to fastai

Since the fastai library itself is written with PyTorch, you could always learn loads about PyTorch by diving into the library’s source codes written heavily with PyTorch.

  • You could write tests for it. It is a great way to learn because to write tests you must first understand what a piece of code does exactly.
    • You don’t need to have any previous experience. Here is a very detailed guide on how to get started.
    • Here is Jeremy’s tip on how to get started:
43 Likes

Summarizing Part 2 Category
(George Zhang) #2

Would you mind make this a wiki, @jeremy?

0 Likes

(ecdrid) #3

Also the raw notebooks of fastai in their GitHub repo are very helpful.

1 Like

(Vijay Narayanan Parakimeethal) #4

@aakashns runs a PyTorch workshop titled “PyTorch: Zero to GANs” and they are available as blog posts on Medium. You can get the first part of the series here. They will also be available as youtube videos in the future, I believe. He had also done some videos on the fast-ai dev note books which can be accessed here.

I have also found the series by deep learning wizard on PyTorch to be good. The site can be accessed from here. I will add more resources as I go through them.

7 Likes

#5

I did it (at least I think I did, not used to the Admin super-powers yet).

5 Likes

(George Zhang) #6

Thank you so much! Hurray!

0 Likes

(George Zhang) #7

I believe that Sylvain just wikified this post. Please add it on the top post wherever you find it appropriate. I am sure that these will be helpful to other students.

0 Likes

(George Zhang) #8

Hey man, glad to see you here. Do you mean the dev notebooks? If I guessed correctly what you mean, they are actually added to the post.

1 Like

(Aakash N S) #9

I’ve been working a series of tutorials called PyTorch: Zero to GANs, inspired from the FastAI development notebooks. The first 3 tutorials are live at the moment:

  1. PyTorch Basics: Tensors & Gradients
  2. Linear Regression and Gradient Descent
  3. Image Classification using Logistic Regression
  4. more coming soon (CNNs, RNNs, GANs etc.)

With some help from the Bangalore study group, we are also planning to conduct workshops and put the recordings online. I hope this will be useful resource for learning PyTorch in depth.

13 Likes

(George Zhang) #10

Thank you @aakashns! I am sure that these would be wonderful references for many people.

1 Like

(Jeff Hale) #11

If you are familiar with Numpy, you might find this one-to-one comparison of Numpy functions to PyTorch functions helpful.

4 Likes

(Stas Bekman) #12

This is an excellent tutorial, @aakashns! Thank you!

Except it has a very convoluted setup (which as I pointed out in the comment on medium doesn’t work on linux w/o fixing the conda env file). Suggestions to make it more accessible to more people:

  • it could just say conda install pytorch torchvision -c pytorch
  • if you do continue with the setup you have now you first need to tell the user to quit the currently activated conda env, conda activate base otherwise what a mess it creates!
  • github repo would have been a lot easier to keep up with updates and being able to submit corrections.
4 Likes

(Stas Bekman) #13

This one is well designed too: http://deeplizard.com/learn/video/v5cngxo4mIg
But it has been sitting there for a few months now without continuation, incomplete… but things are explained quite well.

4 Likes

(Axel Straminsky) #14

I found this tutorial useful: https://www.youtube.com/watch?v=_H3aw6wkCv0

0 Likes

(Michael) #15

With our local learning community we are currently going through part 1 and look deeper into PyTorch. For that we set up a repo which could be of interest for others:


Currently, we just started with lesson 1 and we will expand it with information and notebooks as we move along. So far you will find there a lot of very good PyTorch resources to get going.

We will also update it to include the great material posted in this thread!
Thank you all for the contributions!

7 Likes

(George Zhang) #16

This is wonderful! I am sure that it will be super helpful to many people. Could you please find a place to put it in the top post?

0 Likes

#18

The deeplizard PyTorch series is really good. I can’t recommend it enough. It’s short, but the guy is an amazing teacher.

1 Like

(Joseph Catanzarite) #19

Sounds great – could you please provide the URL?

0 Likes

(Marc P. Rostock) #20

Somehow this one is still missing in this thread (it‘s mentioned in others here on the forum though), a free udacity course by the makers of pytorch (facebook ai) on dl with pytorch, I found this to be pretty good! Includes sections you can probably skip depending on your level and non core sections like an interview with Soumith, but I can definitely recommend it.

4 Likes

Study group in SF / Berkeley
(Neeraj Agrawal) #21

I do agree it’s a very nice course.

0 Likes