Fastai v1 install issues thread

(Devon Kaberna) #280

Hello,

What seems to be best practice (if there even is one) for upgrading to a new version of a Nvidia driver, or a new Pytorch version? I’ve read the documentation on https://docs.fast.ai/troubleshoot.html, but couldn’t find guidance. What I’m asking is, do you wait a while, so as to make sure it’s stable, or do you upgrade as soon as a new version is released?

0 Likes

(Stas Bekman) #281

Usually, you want to upgrade things like nvidia drivers, when the new release includes:

  • fixes to something that was broken that you needed to work
  • new features you actually need
  • speed improvements that are important to you

and when you upgrade hardware and then you may have to get a newer driver.

In all other circumstances, if you have a satisfactory setup - save your time and sanity and don’t update, especially since often updates break other things.

Another time I update everything is when I revamp everything anyway, e.g. new Ubuntu version once in a few years.

Of course, YMMV.

1 Like

(Malcolm McLean) #282

Hi helpers. I have been getting this conda error for about a week whenever trying to update the fastai conda environment. Can anyone help?

Some clues…

  • The conda command below used to work.

  • The error might have started after conda asked me to update conda. I did this by dutifully copy/pasting the command given, without understanding what it did.

  • My setup is local, with Ubuntu 16.04 LTS, GTX 1070.

  • conda list(s) currently the following versions. They work together. I don’t know which are needed, but I prefer to use Cuda92, or whichever is the most recent stable version.

    pytorch 1.0.0 py3.6_cuda9.0.176_cudnn7.4.1_1 pytorch
    pytorch-nightly 1.0.0.dev20181126 py3.6_cuda9.2.148_cudnn7.4.1_0 [cuda92] pytorch
    python 3.6.7 h0371630_0
    fastai 1.0.42 1 fastai
    fastprogress 0.1.18 py_0 fastai
    torchvision 0.2.1 py36_1 pytorch
    torchvision-nightly 0.2.1 py_0 fastai
    cuda92 1.0 0 pytorch

  • I am a Linux/conda igoramus whose primary goal is to get some work done and not break anything. So please use small words.:slightly_smiling_face:

    (fastaiv3) malcolm@PC-GPU:~/course-v3$ conda install -c pytorch -c fastai fastai
    Collecting package metadata: done
    Solving environment: done

    Package Plan

    environment location: /home/malcolm/anaconda3/envs/fastaiv3
    
    added / updated specs:
      - fastai
    

    The following packages will be downloaded:

      package                    |            build
      ---------------------------|-----------------
      torchvision-nightly-0.2.1  |             py_0          39 KB  fastai
      ------------------------------------------------------------
                                             Total:          39 KB
    

    The following NEW packages will be INSTALLED:

    bleach             pkgs/main/linux-64::bleach-3.1.0-py37_0
    cudatoolkit        pkgs/main/linux-64::cudatoolkit-10.0.130-0
    pip                pkgs/main/linux-64::pip-19.0.1-py37_0
    soupsieve          pkgs/main/linux-64::soupsieve-1.7.1-py37_0
    urllib3            pkgs/main/linux-64::urllib3-1.24.1-py37_0
    

    The following packages will be UPDATED:

    beautifulsoup4                               4.6.3-py36_0 --> 4.7.1-py37_1
    cryptography                         2.4.1-py36h1ba5d50_0 --> 2.4.2-py37h1ba5d50_0
    decorator                                    4.3.0-py36_0 --> 4.3.2-py37_0
    entrypoints                                  0.2.3-py36_2 --> 0.3-py37_0
    idna                                           2.7-py36_0 --> 2.8-py37_0
    ipython                              7.0.1-py36h39e3cac_0 --> 7.2.0-py37h39e3cac_0
    jedi                                        0.13.1-py36_0 --> 0.13.2-py37_0
    jupyter_client                               5.2.3-py36_0 --> 5.2.4-py37_0
    markupsafe                             1.0-py36h14c3975_1 --> 1.1.0-py37h7b6447c_0
    msgpack-python                       0.5.6-py36h6bb024c_1 --> 0.6.1-py37hfd86e86_1
    notebook                                     5.7.0-py36_0 --> 5.7.4-py37_0
    packaging                                     18.0-py36_0 --> 19.0-py37_0
    pandas                              0.23.4-py36h04863e7_0 --> 0.24.1-py37he6710b0_0
    parso                                        0.3.1-py36_0 --> 0.3.2-py37_0
    pillow                               5.3.0-py36h34e0f95_0 --> 5.4.1-py37h34e0f95_0
    prometheus_client                            0.4.2-py36_0 --> 0.5.0-py37_0
    prompt_toolkit     pkgs/main/linux-64::prompt_toolkit-2.~ --> pkgs/main/noarch::prompt_toolkit-2.0.8-py_0
    pygments                                     2.2.0-py36_0 --> 2.3.1-py37_0
    pyopenssl                                   18.0.0-py36_0 --> 19.0.0-py37_0
    pyparsing                                    2.2.2-py36_0 --> 2.3.1-py37_0
    pyqt                         pkgs/free::pyqt-5.6.0-py36_2 --> pkgs/main::pyqt-5.6.0-py37h22d08a2_6
    python                                   3.6.7-h0371630_0 --> 3.7.1-h0371630_7
    python-dateutil                              2.7.3-py36_0 --> 2.7.5-py37_0
    pytorch              1.0.0-py3.6_cuda9.0.176_cudnn7.4.1_1 --> 1.0.1-py3.7_cuda10.0.130_cudnn7.4.2_0
    pytz                                        2018.5-py36_0 --> 2018.9-py37_0
    requests                                    2.19.1-py36_0 --> 2.21.0-py37_0
    setuptools                                  40.4.3-py36_0 --> 40.7.3-py37_0
    six                                         1.11.0-py36_1 --> 1.12.0-py37_0
    thinc              pkgs/main::thinc-6.12.0-py36h4989274_0 --> fastai::thinc-6.12.1-py37h637b7d7_1000
    torchvision        pytorch/linux-64::torchvision-0.2.1-p~ --> pytorch/noarch::torchvision-0.2.1-py_2
    tqdm               pkgs/main/linux-64::tqdm-4.26.0-py36h~ --> pkgs/main/noarch::tqdm-4.29.1-py_0
    wheel                                       0.32.2-py36_0 --> 0.32.3-py37_0
    

    The following packages will be DOWNGRADED:

    asn1crypto                                  0.24.0-py36_0 --> 0.24.0-py37_0
    backcall                                     0.1.0-py36_0 --> 0.1.0-py37_0
    bottleneck                           1.2.1-py36h035aef0_1 --> 1.2.1-py37h035aef0_1
    certifi                                 2018.11.29-py36_0 --> 2018.11.29-py37_0
    cffi                                1.11.5-py36he75722e_1 --> 1.11.5-py37he75722e_1
    chardet                                      3.0.4-py36_1 --> 3.0.4-py37_1
    cycler                                      0.10.0-py36_0 --> 0.10.0-py37_0
    cymem                                2.0.2-py36hfd86e86_0 --> 2.0.2-py37hfd86e86_0
    cytoolz                            0.9.0.1-py36h14c3975_1 --> 0.9.0.1-py37h14c3975_1
    dill                                       0.2.8.2-py36_0 --> 0.2.8.2-py37_0
    ipykernel                            5.1.0-py36h39e3cac_0 --> 5.1.0-py37h39e3cac_0
    ipython_genutils                             0.2.0-py36_0 --> 0.2.0-py37_0
    jinja2                                        2.10-py36_0 --> 2.10-py37_0
    jsonschema                                   2.6.0-py36_0 --> 2.6.0-py37_0
    jupyter_core                                 4.4.0-py36_0 --> 4.4.0-py37_0
    kiwisolver                           1.0.1-py36hf484d3e_0 --> 1.0.1-py37hf484d3e_0
    matplotlib                           2.2.2-py36hb69df0a_2 --> 2.2.2-py37hb69df0a_2
    mistune                              0.8.4-py36h7b6447c_0 --> 0.8.4-py37h7b6447c_0
    mkl_fft                              1.0.6-py36h7dd41cf_0 --> 1.0.6-py37h7dd41cf_0
    mkl_random                           1.0.1-py36h4414c95_1 --> 1.0.1-py37h4414c95_1
    msgpack-numpy                              0.4.3.2-py36_0 --> 0.4.3.2-py37_0
    murmurhash                           1.0.1-py36he6710b0_0 --> 1.0.1-py37he6710b0_0
    nb_conda                                     2.2.1-py36_0 --> 2.2.1-py37_0
    nb_conda_kernels                             2.2.0-py36_0 --> 2.2.0-py37_0
    nbconvert                                    5.3.1-py36_0 --> 5.3.1-py37_0
    nbformat                                     4.4.0-py36_0 --> 4.4.0-py37_0
    ninja                                1.8.2-py36h6bb024c_1 --> 1.8.2-py37h6bb024c_1
    numexpr                              2.6.8-py36hd89afb7_0 --> 2.6.8-py37hd89afb7_0
    numpy                               1.15.4-py36h1d66e8a_0 --> 1.15.4-py37h1d66e8a_0
    numpy-base                          1.15.4-py36h81de0dd_0 --> 1.15.4-py37h81de0dd_0
    olefile                                       0.46-py36_0 --> 0.46-py37_0
    pandocfilters                                1.4.2-py36_1 --> 1.4.2-py37_1
    pexpect                                      4.6.0-py36_0 --> 4.6.0-py37_0
    pickleshare                                  0.7.5-py36_0 --> 0.7.5-py37_0
    plac                                         0.9.6-py36_0 --> 0.9.6-py37_0
    preshed                              2.0.1-py36he6710b0_0 --> 2.0.1-py37he6710b0_0
    ptyprocess                                   0.6.0-py36_0 --> 0.6.0-py37_0
    pycparser                                     2.19-py36_0 --> 2.19-py37_0
    pysocks                                      1.6.8-py36_0 --> 1.6.8-py37_0
    pyyaml                                3.13-py36h14c3975_0 --> 3.13-py37h14c3975_0
    pyzmq                               17.1.2-py36h14c3975_0 --> 17.1.2-py37h14c3975_0
    regex                        2018.01.10-py36h14c3975_1000 --> 2018.01.10-py37h14c3975_1000
    scipy                                1.1.0-py36hfa4b5c9_1 --> 1.1.0-py37hfa4b5c9_1
    send2trash                                   1.5.0-py36_0 --> 1.5.0-py37_0
    sip                                 4.19.8-py36hf484d3e_0 --> 4.18.1-py37hf484d3e_2
    spacy                            2.0.18-py36hf484d3e_1000 --> 2.0.18-py37hf484d3e_1000
    terminado                                    0.8.1-py36_1 --> 0.8.1-py37_1
    testpath                                     0.4.2-py36_0 --> 0.4.2-py37_0
    toolz                                        0.9.0-py36_0 --> 0.9.0-py37_0
    tornado                              5.1.1-py36h7b6447c_0 --> 5.1.1-py37h7b6447c_0
    traitlets                                    4.3.2-py36_0 --> 4.3.2-py37_0
    typing                                       3.6.4-py36_0 --> 3.6.4-py37_0
    ujson                                 1.35-py36h14c3975_0 --> 1.35-py37h14c3975_0
    wcwidth                                      0.1.7-py36_0 --> 0.1.7-py37_0
    webencodings                                 0.5.1-py36_1 --> 0.5.1-py37_1
    wrapt                              1.10.11-py36h14c3975_2 --> 1.10.11-py37h14c3975_2
    
    
      Proceed ([y]/n)? y
    
    
      Downloading and Extracting Packages
      torchvision-nightly- | 39 KB     |                                                                            |   0% 
    
      CondaHTTPError: HTTP 404 NOT FOUND for url <https://conda.anaconda.org/fastai/noarch/torchvision-nightly-0.2.1-py_0.tar.bz2>
      Elapsed: 00:00.385509
      CF-RAY: 4a501457fcca2a61-SEA
    
      An HTTP error occurred when trying to retrieve this URL.
      HTTP errors are often intermittent, and a simple retry will get you on your way.
    

Thanks so much for sorting this out!

P.S. Sorry for the bad formatting. Is there a way to tell the forum software not to alter a text block in any way? I tried with Preformatted Text.

0 Likes

(Stas Bekman) #283

What is the error? I see that it’s trying to move you from py36 to py37 - you must be referring to that - otherwise I see no error.

and why are you having pytorch* nightly packages and not stable ones?

In any case, it has all been changed since yesterday, since the released pytorch-1.0.1 changed how they install the cuda builds, https://github.com/pytorch/pytorch/releases/tag/v1.0.1
so if you try now it’ll be installing the cudatoolkit=10.0 by default, and you can ask for cudatoolkit=9.0 if your driver is old and doesn’t support CUDA 10. It doesn’t matter if you have CUDA installed system-wide and if you do which is its version - it packs its own binaries.

So in your case, uninstall first and then re-install:

conda uninstall fastai pytorch pytorch-nightly torchvision-nightly cuda92
conda install -c pytorch -c fastai fastai

And even better start with a fresh conda environment with just:

conda install -c pytorch -c fastai fastai

I think your problem mainly has to do with you having a conda environment with packages pre-pytorch-1.0.0 release, that’s why it’s messed up.

0 Likes

(Malcolm McLean) #284

Thanks, the first method worked perfectly after updating the nvidia driver to v415.27.

1 Like

(Stas Bekman) split this topic #285

A post was merged into an existing topic: Misc issues

0 Likes

(Battle500) #286

Hello,

I have a question. Do you know whether the fastai 1.0.42 version has changed the CUDA drivers or anything else relative to NVIDIA? I upgraded to v. 1.0.42 from 1.038 and it can’t import all libraries. It give me the following error:

NVMLError_LibraryNotFound: NVML Shared Library Not Found

Any idea why? ( I am using windows 10 with a conda environment)

0 Likes

(Stas Bekman) #287

fastai did not, but pytorch did a few days ago, see: https://github.com/pytorch/pytorch/releases

to debug load pytorch directly, without going through fastai, and report any issues with it at https://github.com/pytorch/pytorch/

Remember that fastai is a front-end for pytorch, so any driver issues belong to pytorch.

0 Likes

(Battle500) #288

Thanks, first of all.

Yes, I noticed that Pytorch went from version 1.0.0 to 1.0.1, but as I upgraded fastai to v. 1.0.42 it shows me the above error. So, considering I upgraded both packages at once, I was not sure if it an issue I have with fastai, or torch :).

0 Likes

(Stas Bekman) #289

Pytorch-1.0.1 switched to a different method of installing cuda, please refer to the release notes. It now by default installs cudatoolkit==10.0, you probably need to install cudatoolkit==9.0

To test my suggestion that it has nothing to do with fastai you can alternatively downgrade to pytorch==1.0.0 and I’m pretty sure your problem will go away. And if it is, then try what I said in the previous paragraph.

0 Likes

(Battle500) #290

Great, it totally makes sense.

I’ll try both suggestions.

Thank you so much :wink:

1 Like

(Brad) #291

Following the instructions on the Installation page, I was getting an error installing the Jupyter notebook dependencies:

PackagesNotFoundError: The following packages are not available from current channels: jupyter_contrib_nbextensions

This error was fixed by enabling conda-forge with the following command:

conda config --add channels conda-forge

EDIT: Dear future people: See next post for why you shouldn’t do this

1 Like

(Stas Bekman) #292

Thank you for the heads up, @yeldarb.

The correct instruction is:

conda install -c conda-forge jupyter_contrib_nbextensions

do not add the channel like you did, since now you will be getting everything from the conda-forge channel which may break some fastai dependencies, since it currently relies on the main anaconda channel.

I fixed the doc.

1 Like

(Stas Bekman) split this topic #294

A post was merged into an existing topic: Misc issues

0 Likes

(Stas Bekman) split this topic #295

A post was merged into an existing topic: Misc issues

0 Likes

#296

I made a thread before, but I think it’s clear I should have posted here.

I have been using google colab without issue, until needing to run widgets. I don’t have a GPU on my laptop but see no reason I can’t clean data on it.

However, attempting to install/run on my Ubuntu 18.04 laptop, is not working. I am sure I am missing something basic but I have struggled for some hours now, so here I am.

I have installed fastai with conda.

conda install -c pytorch -c fastai fastai

And my binaries seem to be correct?

(base) geoff@cs-macbook:~/src/neuralnet_stuff/local$ which jupyter
/home/geoff/anaconda2/bin/jupyter

(base) geoff@cs-macbook:~/src/neuralnet_stuff/local$ which python
/home/geoff/anaconda2/bin/python

(base) geoff@cs-macbook:~/src/neuralnet_stuff/local$ which pip
/home/geoff/anaconda2/bin/pip

When I run a notebook

(fastai) geoff@cs-macbook:~/src/neuralnet_stuff/local$ jupyter notebook

And try and import fastai, I get the error:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-3749d44be698> in <module>
      1 from fastai import *
----> 2 from fastai.vision import *
      3 from fastai.metrics import error_rate

ModuleNotFoundError: No module named 'fastai.vision'

If I clone the fastai repo and put it in the same directory, and try and import fastai by path it happens like this:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-3-d20a49ecd0c9> in <module>
      1 from fastai.fastai import *
----> 2 from fastai.fastai.vision import *
      3 from fastai.fastai.metrics import error_rate

~/src/neuralnet_stuff/local/fastai/fastai/vision/__init__.py in <module>
----> 1 from .. import basics
      2 from ..basics import *
      3 from .learner import *
      4 from .image import *
      5 from .data import *

~/src/neuralnet_stuff/local/fastai/fastai/basics.py in <module>
----> 1 from .basic_train import *
      2 from .callback import *
      3 from .core import *
      4 from .basic_data import *
      5 from .data_block import *

~/src/neuralnet_stuff/local/fastai/fastai/basic_train.py in <module>
      1 "Provides basic training and validation with `Learner`"
----> 2 from .torch_core import *
      3 from .basic_data import *
      4 from .callback import *
      5 from .data_block import *

~/src/neuralnet_stuff/local/fastai/fastai/torch_core.py in <module>
      1 "Utility functions to help deal with tensors"
----> 2 from .imports.torch import *
      3 from .core import *
      4 
      5 AffineMatrix = Tensor

~/src/neuralnet_stuff/local/fastai/fastai/imports/__init__.py in <module>
----> 1 from .core import *
      2 from .torch import *

~/src/neuralnet_stuff/local/fastai/fastai/imports/core.py in <module>
     11 from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
     12 from copy import copy, deepcopy
---> 13 from dataclasses import dataclass, field, InitVar
     14 from enum import Enum, IntEnum
     15 from functools import partial, reduce

ModuleNotFoundError: No module named 'dataclasses'

I am stuck, any help would be appreciated. The computer’s “name” is “C’s Macbook” but it’s not a mac in software or hardware.

0 Likes

(Stas Bekman) #297

Please refer to https://docs.fast.ai/support.html to see what information we need to help out.

And I guess you haven’t read:
https://docs.fast.ai/troubleshoot.html#modulenotfounderror-no-module-named-fastaivision

1 Like

(Alankar) #298

I was installing the fast ai library (both the 0.7 and the recent v1) in my windows laptop in anaconda. After creating environments for both the versions I got the same error when i tried to run any notebook from the old course and the new course. The error was:

from torch._C import *

ImportError: DLL load failed: The specified module could not be found.
0 Likes

(Stas Bekman) #299

That’s a pytorch/windows problem, not fastai’s - please look at the various solutions at various places on google and lots here at fastai forums.

0 Likes

#300

I had not read that, thank you @stas. I still was not able to get it working. I opted to completely remove conda, and start over from scratch. Now when I start a jupyter notebook the kernel fails to even load because it is trying to access something from my old installation which no longer exists.

The kernel error I get when starting my notebook:

Traceback (most recent call last):
...snip...
FileNotFoundError: [Errno 2] No such file or directory: '/home/geoff/anaconda2/envs/fastai/bin/python': '/home/geoff/anaconda2/envs/fastai/bin/python'

where ~/anaconda2/envs/fastai/ is my old, now deleted, no longer existent conda environment.

(fastai-3.6) geoff@cs-macbook:~/src/fastai$ which python
/home/geoff/anaconda3/envs/fastai-3.6/bin/python
(fastai-3.6) geoff@cs-macbook:~/src/fastai$ which jupyter
/home/geoff/anaconda3/envs/fastai-3.6/bin/jupyter
=== Software === 
python       : 3.6.8
fastai       : 1.0.46.dev0
fastprogress : 0.1.19
torch        : 1.0.1.post2
torch cuda   : 10.0.130 / is **Not available** 

=== Hardware === 
No GPUs available 

=== Environment === 
platform     : Linux-4.18.0-15-generic-x86_64-with-debian-buster-sid
distro       : Pop!_OS 18.04 bionic
conda env    : fastai-3.6
python       : /home/geoff/anaconda3/envs/fastai-3.6/bin/python
sys.path     : 
/home/geoff/anaconda3/envs/fastai-3.6/lib/python36.zip
/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6
/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/lib-dynload
/home/geoff/.local/lib/python3.6/site-packages
/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages
no supported gpus found on this system
0 Likes