First of all big kudos to Rachel and Jeremy for your amazing work! I really can’t thank you enough! I’m super excited to see what the buzz is all about for Part 2
I’ve just released PyTorch implementations for notebooks 1, 4, 5 and 6. If you have been waiting for an excuse to learn PyTorch, this is it. I tried my best to follow the original spirit of the notebooks where possible. Lesson 1 was a bit tricky to port but the rest are almost identical feature-wise.
I believe Part 2 of this course focuses on PyTorch, so this might be a nice intro for those who haven’t done any PyTorch before but have completed Part 1.
I have personally enjoyed working with PyTorch over the last couple of months. I love its flexibility, easier debugging and the community around it. While it is significantly harder to use than Keras I really think it is worth the trouble.
The community in PyTorch forums is extremely active. They take the time to reply to most questions, something I rarely see in open source. I highly recommend joining it.
I would add to your list:
Torchtext Source Code Not as complete as Torchvision, but has some common NLP datasets
Torchsample High-Level Training, Data Augmentation, and Utilities for Pytorch
PyTorch looks great. I am just starting on the course. Would you recommend I work with PyTorch as I go… or should I circle back to PyTorch after completing the course? My skill level: advanced python developer not data scientist.
If you haven’t done any Machine Learning before, I would recommend you use the original Keras notebooks.
You will be learning many new topics & concepts in every lesson, so trying to learn a new framework at the same time might be too much. Once you have mastered the concepts, learning new frameworks is something you could do very quickly.
You will also be much more likely to find answers to common questions by following the standard path.
I finally got the time to implement Lesson 7 in PyTorch. It currently requires a fork of torchsample that includes many bug fixes for version 0.1.3 and introduces a loss_weights feature in ModuleTrainer. I am hoping this requirement will go away in the near future. Keep an eye on torchsample’s issue tracker to see if this is no longer the case.
Almost everything works as well as in the original Keras implementation. The one thing I haven’t been able to properly reproduce is getting accurate bounding boxes when using multiple outputs (classifier + regressor). Other than that, this lesson was particularly heavy on GPU, RAM and disk requirements. I recommend using a p2.xlarge AWS instance with at least 80GB SSD and 60B of RAM.
Thank you for sharing your PyTorch version! I forked your version of torchsample, and tried to run lesson 1 (on my MacBook’s CPU), however I encountered an error when fitting the parameters of the model. Do you have any suggestions to fix it?
Here is the detail;
trainer = getTrainer()
trainer.fit_loader(loader=train_loader, val_loader=val_loader, num_epoch=3)
# This gets a validation accuracy of 98.9 when using the whole dataset
AttributeError Traceback (most recent call last)
in ()
1 trainer = getTrainer()
----> 2 trainer.fit_loader(loader=train_loader, val_loader=val_loader, num_epoch=3)
3 # This gets a validation accuracy of 98.9 when using the whole dataset
~/dogscats/pytorch/torchsample/modules/module_trainer.py in fit_loader(self, loader, val_loader, num_epoch, cuda_device, verbose)
345 tmp_callbacks.append(ConstraintCallback(self.constraint_container))
346 if self._has_metrics:
–> 347 self.metric_container.set_helper(fit_helper)
348 tmp_callbacks.append(MetricCallback(self.metric_container))
349
AttributeError: ‘ModuleTrainer’ object has no attribute ‘metric_container’
Hi Yohei, glad your are finding them useful. Regarding your issue, you may be using an old version of torchsample. Try upgrading it to the latest release (which is a bit buggy), or use my fork while my fixes get merged upstream.
Hi Rodrigo, thank you for your comment, and sorry for my late reply. Actually, I’m using “losses” branch in your folk, but got this error. I also tried on GPU server, but faced the same issue. That’s weird… Anyway, I’ll try to investigate further and wait for the updates which your fixed are merged.
I expanded my original cheatsheet into a more comprehensive starter kit targeted towards Kaggle competitions. Right now it includes sample code for Dogs/Cats, Planet Amazon Rainforest, and the new Carvana Image Segmentation challenge, but the code itself can be used for any deep learning experiment.
Hi Rodrigo, Thank you for the comments. I don’t want to bother you further, but here are what I’ve done.
11 days ago: simply downloaded, unzipped, and put your torchsample ‘losses’ branch in the same directory as lesson1-pytorch.ipynb. Then, I’ve got ‘metric_container’ error in fit_loader.
today: install torchsample with pip install git+ https://github.com/recastrodiaz/torchsample.git (under anaconda/lib/~/site-package), and replaced all the torchsample-v0.1.3 files with your ‘losses’ branch files. However, the same ‘metric_container’ error occurs. The detail is as follows.
AttributeError Traceback (most recent call last)
in ()
1 trainer = getTrainer()
----> 2 trainer.fit_loader(loader=train_loader, val_loader=val_loader, num_epoch=3)
3 # This gets a validation accuracy of 98.9 when using the whole dataset
~/anaconda/lib/python3.6/site-packages/torchsample/modules/module_trainer.py in fit_loader(self, loader, val_loader, num_epoch, cuda_device, verbose)
345 tmp_callbacks.append(ConstraintCallback(self.constraint_container))
346 if self._has_metrics:
–> 347 self.metric_container.set_helper(fit_helper)
348 tmp_callbacks.append(MetricCallback(self.metric_container))
349
AttributeError: ‘ModuleTrainer’ object has no attribute ‘metric_container’
From your stack trace, it looks like you are using the original zip file you downloaded and not the one from the losses branch. I am not super familiar with anaconda, but if pip works with it, you could try ‘installing’ my losses branch with:
cd torchsample # the one that has the losses branch
pip install -e .
To verify that the install command worked, you can check the contents of the ~/anaconda/lib/python3.6/site-packages/torchsample/modules/module_trainer.py. If it matches the content of the same file in the losses branch, then you are good to go!
This is great work! Thanks for sharing it and making the code MIT! I have had a look at the first 2 files and I am impressed by the time you have committed to them! I really love the ROC curves at the bottom.
I hope you don’t mind me asking, but I am curious as to whether you plan to apply your ML/DL knowledge into commercial products? Or is it purely for research purposes?
Since you ask, my dream scenario is to start a company that requires regularly reading and implementing the latest research to make people happier/healthier/safer/more educated/financially better off. A tall feat I admit, haha.
Currently I’m working on a few small projects, exploring some consulting gigs, and doing Kaggle. I’m about to start work on the TSA passenger screening challenge, but only casually.
That sounds exactly like the same (vague?) description of my Machine Learning company I suppose many people in the field have the same dream?
I’m admin of https://picmatix.com which is based on the Style Transfer algo from Gatys et al. and have worked with some other cool computer vision projects too. One of those projects involved developing a fast style transfer PyTorch network, which I will be making open source sometime this year…
I’ve also done some Kaggle competitions, but all of them after the deadline. I sometimes wish the day had 36 hours so I could do everything I want, haha
Are you doing consulting work in computer vision too?