Using PyTorch 0.4 and resultant slowdown

Using PyTorch 0.4 and resultant slowdown

Here’s how I managed to start using PyTorch 0.4 with the fastai lessons. Disclaimer: I’m a Ubuntu beginner, and do not claim this is the ideal method.

First in a Terminal, run…

source activate fastai
git pull
conda env update

These commands insure that fastai is up to date. Next…


Within the Navigator GUI, find the list of conda environments. Clone fastai to create a new environment, PyTorch04. Quit.
Back in Terminal…

source activate PyTorch04
conda update PyTorch
jupyter notebook

You should now be running the lesson notebooks with PyTorch 0.4.

The good news is that some warnings disappear. However, I expected the latest PyTorch to show some speed improvements. But a quick test with Lesson1 finds that it is slower at every step than PyTorch 0.3.1.

The most extreme place is at, 3, cycle_len=1) in the image transformations section. With GTX 1070 and PyTorch 0.3.1, this function runs at 84 s/it. With PyTorch 0.4.1, it runs at 175 s/it, half the speed.

Google finds several reports about PyTorch being slower for certain operations. I am not sure if the above observed slowdown has the same cause.

I would greatly appreciate it if anyone can explain what is going on (and correct any mistakes I’ve made). For now, reverting to PyTorch 0.3.1 for the fastai lessons.


To try to answer your question I would like to request a little more information about your current system:
1-Are you using Windows or Linux ?
2-What cuda or cuda(s) version(s) you have concurrently on your machine ? Because anaconda can use a total different cuda apart from the installed on your system.
3-What driver version and build do you have installed for cuda ?