Part 1 notebooks in PyTorch

Part 1 (2017)
1

Hi,

First of all big kudos to Rachel and Jeremy for your amazing work! I really can’t thank you enough! I’m super excited to see what the buzz is all about for Part 2 :smiley:

I’ve just released PyTorch implementations for notebooks 1, 4, 5 and 6. If you have been waiting for an excuse to learn PyTorch, this is it. :wink: I tried my best to follow the original spirit of the notebooks where possible. Lesson 1 was a bit tricky to port but the rest are almost identical feature-wise.

I believe Part 2 of this course focuses on PyTorch, so this might be a nice intro for those who haven’t done any PyTorch before but have completed Part 1.

I have personally enjoyed working with PyTorch over the last couple of months. I love its flexibility, easier debugging and the community around it. While it is significantly harder to use than Keras I really think it is worth the trouble.

If you are new to PyTorch, I recommend reading the official 60-minute crash course and Brendan’s cheatsheet.
For more advanced architectures, the official examples repo would be a good starting point.

Let me know if you find these notebooks useful and if you would like to help, PRs are welcome for lessons 2 and 3.

Rodrigo

28 Likes
2

These are the must-have resources that helped me conquer Pytorch in a week:

  1. PyTorch for Deep Learning Researchers
  2. PyTorch Official Tutorials - best to go through all of them
  3. PyTorch Docs - not as nice as Keras, but still solid
  4. PyTorch Forums - if you have a question, likely it’s been answered
  5. PyTorch Official Examples
  6. PyTorch Source Code - if all else fails, this stuff is open source after all . . .
  7. Torchvision Source Code
14 Likes
3

The community in PyTorch forums is extremely active. They take the time to reply to most questions, something I rarely see in open source. I highly recommend joining it.

I would add to your list:

  1. Torchtext Source Code Not as complete as Torchvision, but has some common NLP datasets
  2. Torchsample High-Level Training, Data Augmentation, and Utilities for Pytorch
4

PyTorch looks great. I am just starting on the course. Would you recommend I work with PyTorch as I go… or should I circle back to PyTorch after completing the course? My skill level: advanced python developer not data scientist.

5

If you haven’t done any Machine Learning before, I would recommend you use the original Keras notebooks.

You will be learning many new topics & concepts in every lesson, so trying to learn a new framework at the same time might be too much. Once you have mastered the concepts, learning new frameworks is something you could do very quickly.
You will also be much more likely to find answers to common questions by following the standard path.

In my case, this is why I used PyTorch instead. In the end, it is entirely up to you :slight_smile:

I hope this helps.

2 Likes
6

Ok, thanks for your guidance!

1 Like
7

Thanks for sharing - this is great! :slight_smile:

1 Like
8

I finally got the time to implement Lesson 7 in PyTorch. It currently requires a fork of torchsample that includes many bug fixes for version 0.1.3 and introduces a loss_weights feature in ModuleTrainer. I am hoping this requirement will go away in the near future. Keep an eye on torchsample’s issue tracker to see if this is no longer the case.

Almost everything works as well as in the original Keras implementation. The one thing I haven’t been able to properly reproduce is getting accurate bounding boxes when using multiple outputs (classifier + regressor). Other than that, this lesson was particularly heavy on GPU, RAM and disk requirements. I recommend using a p2.xlarge AWS instance with at least 80GB SSD and 60B of RAM.

The notebook for Lesson 7 in PyTorch is available on Github.

Any questions or issues, happy to help.

1 Like
9

Thank you for sharing your PyTorch version! I forked your version of torchsample, and tried to run lesson 1 (on my MacBook’s CPU), however I encountered an error when fitting the parameters of the model. Do you have any suggestions to fix it?

Here is the detail;

trainer = getTrainer()
trainer.fit_loader(loader=train_loader, val_loader=val_loader, num_epoch=3)
# This gets a validation accuracy of 98.9 when using the whole dataset

AttributeError Traceback (most recent call last)
in ()
1 trainer = getTrainer()
----> 2 trainer.fit_loader(loader=train_loader, val_loader=val_loader, num_epoch=3)
3 # This gets a validation accuracy of 98.9 when using the whole dataset

~/dogscats/pytorch/torchsample/modules/module_trainer.py in fit_loader(self, loader, val_loader, num_epoch, cuda_device, verbose)
345 tmp_callbacks.append(ConstraintCallback(self.constraint_container))
346 if self._has_metrics:
–> 347 self.metric_container.set_helper(fit_helper)
348 tmp_callbacks.append(MetricCallback(self.metric_container))
349

AttributeError: ‘ModuleTrainer’ object has no attribute ‘metric_container’

10

Hi Yohei, glad your are finding them useful. Regarding your issue, you may be using an old version of torchsample. Try upgrading it to the latest release (which is a bit buggy), or use my fork while my fixes get merged upstream.

Let me know if that helps.

1 Like
11

Hi Rodrigo, thank you for your comment, and sorry for my late reply. Actually, I’m using “losses” branch in your folk, but got this error. I also tried on GPU server, but faced the same issue. That’s weird… Anyway, I’ll try to investigate further and wait for the updates which your fixed are merged.

1 Like
12

These notebooks are awesome. Thanks for sharing!

I expanded my original cheatsheet into a more comprehensive starter kit targeted towards Kaggle competitions. Right now it includes sample code for Dogs/Cats, Planet Amazon Rainforest, and the new Carvana Image Segmentation challenge, but the code itself can be used for any deep learning experiment.

3 Likes
13

Hi ysaz, you should indeed be using the losses branch. I have updated the link on my previous comment to reflect this.

Have you figured out what your problem was? If not, what is the stack trace that you are getting when using the losses branch?

Also, how are you installing torchsample? I usually do the following:

cd torchsample
pip install -e .
14

Hi Rodrigo, Thank you for the comments. I don’t want to bother you further, but here are what I’ve done.

  • 11 days ago: simply downloaded, unzipped, and put your torchsample ‘losses’ branch in the same directory as lesson1-pytorch.ipynb. Then, I’ve got ‘metric_container’ error in fit_loader.

  • today: install torchsample with pip install git+ https://github.com/recastrodiaz/torchsample.git (under anaconda/lib/~/site-package), and replaced all the torchsample-v0.1.3 files with your ‘losses’ branch files. However, the same ‘metric_container’ error occurs. The detail is as follows.

      trainer = getTrainer()
  trainer.fit_loader(loader=train_loader, val_loader=val_loader, num_epoch=3)

    AttributeError Traceback (most recent call last)
    in ()
    1 trainer = getTrainer()
    ----> 2 trainer.fit_loader(loader=train_loader, val_loader=val_loader, num_epoch=3)
    3 # This gets a validation accuracy of 98.9 when using the whole dataset

    ~/anaconda/lib/python3.6/site-packages/torchsample/modules/module_trainer.py in fit_loader(self, loader, val_loader, num_epoch, cuda_device, verbose)
    345 tmp_callbacks.append(ConstraintCallback(self.constraint_container))
    346 if self._has_metrics:
    –> 347 self.metric_container.set_helper(fit_helper)
    348 tmp_callbacks.append(MetricCallback(self.metric_container))
    349

    AttributeError: ‘ModuleTrainer’ object has no attribute ‘metric_container’

15

From your stack trace, it looks like you are using the original zip file you downloaded and not the one from the losses branch. I am not super familiar with anaconda, but if pip works with it, you could try ‘installing’ my losses branch with:

cd torchsample # the one that has the losses branch
pip install -e .

To verify that the install command worked, you can check the contents of the ~/anaconda/lib/python3.6/site-packages/torchsample/modules/module_trainer.py. If it matches the content of the same file in the losses branch, then you are good to go! :slight_smile:

Any issues I’d be happy to give you a hand!

16

This is great work! Thanks for sharing it and making the code MIT! I have had a look at the first 2 files and I am impressed by the time you have committed to them! I really love the ROC curves at the bottom.

I hope you don’t mind me asking, but I am curious as to whether you plan to apply your ML/DL knowledge into commercial products? Or is it purely for research purposes?

Cheers,

Rodrigo

1 Like
17

Since you ask, my dream scenario is to start a company that requires regularly reading and implementing the latest research to make people happier/healthier/safer/more educated/financially better off. A tall feat I admit, haha. :slight_smile:

Currently I’m working on a few small projects, exploring some consulting gigs, and doing Kaggle. I’m about to start work on the TSA passenger screening challenge, but only casually.

How about you?

1 Like
18

Hi Rodrigo,

Thanks for doing this. Before I take the plunge, are these notebooks compatible with Python 2 or do they assume Python 3?

19

Hi Patrick, I didn’t test for Python 2 compatibility, so If I were you I would go for Python 3 and save yourself some trouble :slight_smile:

20

That sounds exactly like the same (vague?) description of my Machine Learning company :smiley: I suppose many people in the field have the same dream? :stuck_out_tongue:

I’m admin of https://picmatix.com which is based on the Style Transfer algo from Gatys et al. and have worked with some other cool computer vision projects too. One of those projects involved developing a fast style transfer PyTorch network, which I will be making open source sometime this year…

I’ve also done some Kaggle competitions, but all of them after the deadline. I sometimes wish the day had 36 hours so I could do everything I want, haha :smile:

Are you doing consulting work in computer vision too?