I was testing out the new show_install command and I noticed something weird in my output:
It looks like I have a different cuda version for torch and nvcc. The other thing I noticed is that the torch.cuda.is_available() returns False for me. I’m wondering if I need to upgrade my cuda version or I guess if anybody else has seen this where they have torch cuda 9.2 and nvcc cuda 9.1 and if that is an issue.
show_install(0)
platform : Linux-4.15.0-36-generic-x86_64-with-debian-stretch-sid
distro : Ubuntu 16.04 Xenial Xerus
python : 3.6.6
fastai : 1.0.6.dev0
torch : 1.0.0.dev20181007
nvidia dr. : 390.67
torch cuda : Not available
torch cuda : 9.2.148
nvcc cuda : 9.1.85
torch gpus
I am going to remove the output of nvcc in show_install, since as system’s cuda version doesn’t have to match torch’s cuda version anymore - it’s just confusing.
The problem is the nvidia driver, for some reason torch can’t detect it. Most people had success by reinstalling it, in particular making sure that you don’t have more than one driver installed. So usually doing a full uninstall, and then installing the latest solid version that works for your card. The link you included goes into more details.
You don’t even need to have normal CUDA installed on your system unless you need it for other software (i.e. you only need pytorch w/ its pre-packaged cuda library)
So would that actually be impacting being able to use my GPU? I hadn’t even noticed this was an issue. I guess I haven’t done a ton of training since I installed v1 though either.
You shouldn’t be able to use torch w/ GPU if you get torch.cuda.is_available() == False. You can always run:
watch -n 1 nvidia-smi
and see whether the gpu is used when you run some notebook.
If it works despite it reporting not available, then something gets mixed up in your environment. For example you’re running the reporting script from a different environment than your jupyter (e.g. a shell w/o matching env activated or none activated at all).
which means that reporting conda env should be helpful too in show_install.
That makes sense. I think that show_install is going to be super useful. I’m probably going to be putting that in my notebooks just as a verification. One thought, if you wanted to have a more detailed dump of their environment, you could do a conda list and output all the packages they have installed.
I am not sure we want conda list by default, since its output is huge. Let’s see if it becomes needed then we will add it via an optional flag, like we do now with show_nvidia_smi option.
But I added python’s location and sys.path, I think those will be very useful when users report the nb can’t import fastai.
Give the updated version a try after git pull, also to check that I didn’t leave any holes in the updated code. Thanks.
This is good, I committed yet another iteration to show how many GPUs nvidia sees and how many if any torch sees, otherwise ‘no gpus’ is a misleading statement.
So your edit report is from the fixed system, correct? got gpu? great!
``` are there is to help users to properly format the code when they paste it in the forums, as some paste without formatting it. In theory I could indent the output 4 spaces, but if the previous text uses nested lists I think it might not format properly, and if some long lines hardwrap in copy-n-paste it’ll mess up the markdown too. So it feels like a safe simple approach. Of course, you don’t have to paste the formatting as suggested and re-format it anyway you want. But I’m open to suggestions if you think not using them is better for the general case.
edit: oh sorry, I didn’t see the coloring as it was hidden by the scroll bar, I can see what you mean now.
I see what you mean. so yes, ``` it is:
=== Software ===
python version : 3.6.6
fastai version : 1.0.6.dev0
torch version : 1.0.0.dev20181013
nvidia driver : 396.44
torch cuda ver : 9.2.148
torch cuda is : available
=== Hardware ===
nvidia gpus : 1
torch available : 1
- gpu0 : 8119MB | GeForce GTX 1070 Ti
=== Environment ===
platform : Linux-4.15.0-36-generic-x86_64-with-debian-buster-sid
distro : Ubuntu 18.04 Bionic Beaver
conda env : pytorch-dev
python : /home/stas/anaconda3/envs/pytorch-dev/bin/python
sys.path :
/home/stas/anaconda3/envs/pytorch-dev/lib/python3.6/site-packages/_pdbpp_path_hack
/home/stas/anaconda3/envs/pytorch-dev/lib/python36.zip
/home/stas/anaconda3/envs/pytorch-dev/lib/python3.6
/home/stas/anaconda3/envs/pytorch-dev/lib/python3.6/lib-dynload
/home/stas/.local/lib/python3.6/site-packages
/home/stas/anaconda3/envs/pytorch-dev/lib/python3.6/site-packages
/mnt/disc1/fast.ai-1/br/fastai/master
/home/stas/anaconda3/envs/pytorch-dev/lib/python3.6/site-packages/IPython/extensions
Mon Oct 15 22:07:16 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.44 Driver Version: 396.44 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 107... Off | 00000000:02:00.0 Off | N/A |
| 0% 37C P8 6W / 180W | 495MiB / 8119MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 29285 C ...s/anaconda3/envs/pytorch-dev/bin/python 485MiB |
+-----------------------------------------------------------------------------+
I like having the formatting. I just would maybe add ```text to the starting one since it isn’t really code and that formats it better, but just having that information in general is awesome.
yes, sorry, I only saw what you meant when I pasted it myself I added ```text. Thank you for a great suggestion, @KevinB. Let’s hope people will copy-n-paste it too
I started a new document https://docs-dev.fast.ai/gpu.html to collect gpu-related tips, so if you have other suggestions please send them my way. Thanks.
I added ```text. Thank you for a great suggestion,