Fastai v1 install issues thread

(Stas Bekman) #263

I was able to reproduce this problem following your instructions. Mine was installed via conda and had no such problems.

So as you identified pip installs incompatible versions of some of the numpy functionality and bottleneck, conda’s versions of these packages don’t have that problem. The problem happens during pandas’ import.

There is a third workaround:

pip install git+https://github.com/kwgoodman/bottleneck

I filed a bug report: https://github.com/kwgoodman/bottleneck/issues/204

It looks like a new long overdue release is being tested: https://github.com/kwgoodman/bottleneck/issues/191, so perhaps let’s wait a little bit and perhaps it’ll get resolved automatically with the new release.

If, however, it gets delayed and/or new numpy is still not released, we will pin pip to bottleneck=1.2.0 for the next fastai release.

As you can see from the bug report, I managed to reduce the problem to:

python -c "import pandas"
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
1 Like

(Devon Kaberna) #264

Hi everyone,

Forgive the noob question here, but I am purposely working under 1.0.36.post1. I am using bash-git-prompt. When I am in the fastai folder, I am able to confirm I am using version 1.0.36.post1. When create a new folder for Kaggle competitions within the fastai folder, and then cd into the Kaggle folder, I check the version and now see that I am using 1.0.40.

All of this is done within a virtual environment. Any ideas on how I can make sure I am still under 1.0.36.post1? Apologies if I am supposed to be asking this question under a different thread than this one.

0 Likes

Developer chat
(Stas Bekman) #265

You probably have fastai installed in two places (and perhaps you have a symlink that gets the wrong version).

See this https://docs.fast.ai/troubleshoot.html

and certainly follow the instructions here: https://docs.fast.ai/support.html

Use locate fastai/data_block.py if you’re on unix to find all the instances and then you will see if you have it installed twice. You probably have some symlinks pointing to a second install or something similar. Can’t tell without a proper report.

p.s. if you’re working with the git checkout, there was originally also a git cross-over in the 1.0.36 branch (with master HEAD) that has been fixed since then, so if you’re using the 1.0.36 branch, you need to update it.

1 Like

(Stas Bekman) split this topic #266

A post was merged into an existing topic: Misc issues

0 Likes

(Stas Bekman) pinned #267
0 Likes

(Stas Bekman) split this topic #268

2 posts were merged into an existing topic: Misc issues

0 Likes

(Stas Bekman) split this topic #269

4 posts were merged into an existing topic: Performance Improvement Through Faster Software Components

0 Likes

(Masaki Kozuki) #270

Weird dependency error regarding to installation of dataclasses.

No module named 'dataclasses'

This is how I installed fastai in Docker

# Python Anaconda default.
RUN wget -q https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh -O ~/anaconda.sh && \
    /bin/bash ~/anaconda.sh -b -p /opt/conda && \
    rm ~/anaconda.sh
# Install PyTorch V1.
ENV PATH /opt/conda/bin:$PATH
ARG PYTHON_VERSION
RUN conda install -y python=$PYTHON_VERSION && \
    conda install -c conda-forge imbalanced-learn && \
    conda install -y -c conda-forge feather-format && \
    conda install -y -c conda-forge jupyterlab && \
    conda install -y -c conda-forge jupyter_contrib_nbextensions && \
    jupyter contrib nbextension install --system
RUN conda install -y -c pytorch -c fastai fastai==1.0.42
RUN pip install --no-cache-dir -U pip && \
    pip install --no-cache-dir -U dpkt scapy protobuf

Could anyone help me?

0 Likes

#271

I am having trouble upgrading to the latest version of fastai on Windows 10 using conda. I haven’t been able to update since 1.0.38. Running conda update fastai returns “all packages already installed”, even though there have been several new versions.

The only thing I can see is that at version 1.0.39, the build changed from py_1 to 1. Could this be causing the issue? I tried removing fastai and reinstalling, but it just installed 1.0.38 again. Specifying the latest version explicitly doesn’t work either.

Here’s the output of conda search fastai. The new versions are there, but it won’t install.
Name Version Build Channel
fastai 1.0.37 py_1 fastai
fastai 1.0.38 py_1 fastai
fastai 1.0.39 1 fastai
fastai 1.0.40 1 fastai
fastai 1.0.41 1 fastai
fastai 1.0.42 1 fastai

1 Like

(Stas Bekman) #272

Conda is tricky that way - it tells you nothing about the conflicts and then quits, please see: https://docs.fast.ai/troubleshoot.html#cant-install-the-latest-fastai-conda-package

0 Likes

(Stas Bekman) #273

fastai doesn’t install dataclasses for py37, but it’s needed for py36. Most likely what happened is that you installed fastai with py37, but then some other package downgraded python to py36 and you’re running fastai with py36, while you installed it with py37?

It’d help if you were to follow the guideliness for support: https://docs.fast.ai/support.html
You’re not saying when you encountered the error. Please put yourself in the shoes of the person that has no idea what you did and then you will know what information to share :wink:

we currently don’t use conda-forge and all package dependencies are tested against the anaconda channel, so it’s possible that something is off in your case, since you’re on the fringe. But again I have no way of telling.

and of course, you can just install dataclasses

0 Likes

(Masaki Kozuki) #274

sorry I missed to clarify the python version. I installed python 3.6 after the conda installation (So what you wrote is not true for my case). Then I install fastai via conda -c pytorch -c fastai fastai.

The error of No module named 'dataclasses' occurred after I ran the cell of from fastai.tabular import *.

I created the environment on nvidia-docker. There, I used anaconda to follow the instruction on the github repo as possible.

1 Like

(Stas Bekman) #275

Thank you for providing these details, @crcrpar. Indeed there was a bug in the conda package setup for py36, should be resolved in fastai-1.0.43 when that is released. Until then, please add conda install dataclasses to your docker build script.

1 Like

(Masaki Kozuki) #276

Thank you.

I’m glad to hear that because I thought the conda would automatically install dataclasses if python version was 3.6 IIRC.

ps. To get familiar with, I decided to migrate to python 3.7.

1 Like

(Stas Bekman) #277

yeah, conda is not very flexible in this situation. pip allows defining a dynamic package dependency which gets sorted out during install. conda only during package build and then it’s set in stone. So the variant is only doable if py36 and py37 packages are built. Since fastai is noarch, this variant unfortunately doesn’t work. But luckily no harm in reinstalling dataclasses for py37, so all is good.

1 Like

#278

Hi,
I get the following error:
DistributionNotFound: The ‘fastprogress>=0.1.18’ distribution was not found and is required by the application

when running
from fastai.vision import *
or other import operations in an Jupyter Notebook.

It runs fine when running it just in python.

I installed it through pip
fastai 1.0.42
fastprogress 0.1.18

Python version
3.6.3 (default, Sep 2 2018, 00:38:05)

I am running Ubuntu 16.04.

I should also note that this might be related to my pyenv setup, which I am using. However, I have not had any problems with the pyenv-jupyter combination so far.

Best regards

0 Likes

#279

Okay, problem seems to be my pyenv enviroment. Problem is solved (not quite, but the problem is on my side).

0 Likes

(Devon Kaberna) #280

Hello,

What seems to be best practice (if there even is one) for upgrading to a new version of a Nvidia driver, or a new Pytorch version? I’ve read the documentation on https://docs.fast.ai/troubleshoot.html, but couldn’t find guidance. What I’m asking is, do you wait a while, so as to make sure it’s stable, or do you upgrade as soon as a new version is released?

0 Likes

(Stas Bekman) #281

Usually, you want to upgrade things like nvidia drivers, when the new release includes:

  • fixes to something that was broken that you needed to work
  • new features you actually need
  • speed improvements that are important to you

and when you upgrade hardware and then you may have to get a newer driver.

In all other circumstances, if you have a satisfactory setup - save your time and sanity and don’t update, especially since often updates break other things.

Another time I update everything is when I revamp everything anyway, e.g. new Ubuntu version once in a few years.

Of course, YMMV.

1 Like

(Malcolm McLean) #282

Hi helpers. I have been getting this conda error for about a week whenever trying to update the fastai conda environment. Can anyone help?

Some clues…

  • The conda command below used to work.

  • The error might have started after conda asked me to update conda. I did this by dutifully copy/pasting the command given, without understanding what it did.

  • My setup is local, with Ubuntu 16.04 LTS, GTX 1070.

  • conda list(s) currently the following versions. They work together. I don’t know which are needed, but I prefer to use Cuda92, or whichever is the most recent stable version.

    pytorch 1.0.0 py3.6_cuda9.0.176_cudnn7.4.1_1 pytorch
    pytorch-nightly 1.0.0.dev20181126 py3.6_cuda9.2.148_cudnn7.4.1_0 [cuda92] pytorch
    python 3.6.7 h0371630_0
    fastai 1.0.42 1 fastai
    fastprogress 0.1.18 py_0 fastai
    torchvision 0.2.1 py36_1 pytorch
    torchvision-nightly 0.2.1 py_0 fastai
    cuda92 1.0 0 pytorch

  • I am a Linux/conda igoramus whose primary goal is to get some work done and not break anything. So please use small words.:slightly_smiling_face:

    (fastaiv3) malcolm@PC-GPU:~/course-v3$ conda install -c pytorch -c fastai fastai
    Collecting package metadata: done
    Solving environment: done

    Package Plan

    environment location: /home/malcolm/anaconda3/envs/fastaiv3
    
    added / updated specs:
      - fastai
    

    The following packages will be downloaded:

      package                    |            build
      ---------------------------|-----------------
      torchvision-nightly-0.2.1  |             py_0          39 KB  fastai
      ------------------------------------------------------------
                                             Total:          39 KB
    

    The following NEW packages will be INSTALLED:

    bleach             pkgs/main/linux-64::bleach-3.1.0-py37_0
    cudatoolkit        pkgs/main/linux-64::cudatoolkit-10.0.130-0
    pip                pkgs/main/linux-64::pip-19.0.1-py37_0
    soupsieve          pkgs/main/linux-64::soupsieve-1.7.1-py37_0
    urllib3            pkgs/main/linux-64::urllib3-1.24.1-py37_0
    

    The following packages will be UPDATED:

    beautifulsoup4                               4.6.3-py36_0 --> 4.7.1-py37_1
    cryptography                         2.4.1-py36h1ba5d50_0 --> 2.4.2-py37h1ba5d50_0
    decorator                                    4.3.0-py36_0 --> 4.3.2-py37_0
    entrypoints                                  0.2.3-py36_2 --> 0.3-py37_0
    idna                                           2.7-py36_0 --> 2.8-py37_0
    ipython                              7.0.1-py36h39e3cac_0 --> 7.2.0-py37h39e3cac_0
    jedi                                        0.13.1-py36_0 --> 0.13.2-py37_0
    jupyter_client                               5.2.3-py36_0 --> 5.2.4-py37_0
    markupsafe                             1.0-py36h14c3975_1 --> 1.1.0-py37h7b6447c_0
    msgpack-python                       0.5.6-py36h6bb024c_1 --> 0.6.1-py37hfd86e86_1
    notebook                                     5.7.0-py36_0 --> 5.7.4-py37_0
    packaging                                     18.0-py36_0 --> 19.0-py37_0
    pandas                              0.23.4-py36h04863e7_0 --> 0.24.1-py37he6710b0_0
    parso                                        0.3.1-py36_0 --> 0.3.2-py37_0
    pillow                               5.3.0-py36h34e0f95_0 --> 5.4.1-py37h34e0f95_0
    prometheus_client                            0.4.2-py36_0 --> 0.5.0-py37_0
    prompt_toolkit     pkgs/main/linux-64::prompt_toolkit-2.~ --> pkgs/main/noarch::prompt_toolkit-2.0.8-py_0
    pygments                                     2.2.0-py36_0 --> 2.3.1-py37_0
    pyopenssl                                   18.0.0-py36_0 --> 19.0.0-py37_0
    pyparsing                                    2.2.2-py36_0 --> 2.3.1-py37_0
    pyqt                         pkgs/free::pyqt-5.6.0-py36_2 --> pkgs/main::pyqt-5.6.0-py37h22d08a2_6
    python                                   3.6.7-h0371630_0 --> 3.7.1-h0371630_7
    python-dateutil                              2.7.3-py36_0 --> 2.7.5-py37_0
    pytorch              1.0.0-py3.6_cuda9.0.176_cudnn7.4.1_1 --> 1.0.1-py3.7_cuda10.0.130_cudnn7.4.2_0
    pytz                                        2018.5-py36_0 --> 2018.9-py37_0
    requests                                    2.19.1-py36_0 --> 2.21.0-py37_0
    setuptools                                  40.4.3-py36_0 --> 40.7.3-py37_0
    six                                         1.11.0-py36_1 --> 1.12.0-py37_0
    thinc              pkgs/main::thinc-6.12.0-py36h4989274_0 --> fastai::thinc-6.12.1-py37h637b7d7_1000
    torchvision        pytorch/linux-64::torchvision-0.2.1-p~ --> pytorch/noarch::torchvision-0.2.1-py_2
    tqdm               pkgs/main/linux-64::tqdm-4.26.0-py36h~ --> pkgs/main/noarch::tqdm-4.29.1-py_0
    wheel                                       0.32.2-py36_0 --> 0.32.3-py37_0
    

    The following packages will be DOWNGRADED:

    asn1crypto                                  0.24.0-py36_0 --> 0.24.0-py37_0
    backcall                                     0.1.0-py36_0 --> 0.1.0-py37_0
    bottleneck                           1.2.1-py36h035aef0_1 --> 1.2.1-py37h035aef0_1
    certifi                                 2018.11.29-py36_0 --> 2018.11.29-py37_0
    cffi                                1.11.5-py36he75722e_1 --> 1.11.5-py37he75722e_1
    chardet                                      3.0.4-py36_1 --> 3.0.4-py37_1
    cycler                                      0.10.0-py36_0 --> 0.10.0-py37_0
    cymem                                2.0.2-py36hfd86e86_0 --> 2.0.2-py37hfd86e86_0
    cytoolz                            0.9.0.1-py36h14c3975_1 --> 0.9.0.1-py37h14c3975_1
    dill                                       0.2.8.2-py36_0 --> 0.2.8.2-py37_0
    ipykernel                            5.1.0-py36h39e3cac_0 --> 5.1.0-py37h39e3cac_0
    ipython_genutils                             0.2.0-py36_0 --> 0.2.0-py37_0
    jinja2                                        2.10-py36_0 --> 2.10-py37_0
    jsonschema                                   2.6.0-py36_0 --> 2.6.0-py37_0
    jupyter_core                                 4.4.0-py36_0 --> 4.4.0-py37_0
    kiwisolver                           1.0.1-py36hf484d3e_0 --> 1.0.1-py37hf484d3e_0
    matplotlib                           2.2.2-py36hb69df0a_2 --> 2.2.2-py37hb69df0a_2
    mistune                              0.8.4-py36h7b6447c_0 --> 0.8.4-py37h7b6447c_0
    mkl_fft                              1.0.6-py36h7dd41cf_0 --> 1.0.6-py37h7dd41cf_0
    mkl_random                           1.0.1-py36h4414c95_1 --> 1.0.1-py37h4414c95_1
    msgpack-numpy                              0.4.3.2-py36_0 --> 0.4.3.2-py37_0
    murmurhash                           1.0.1-py36he6710b0_0 --> 1.0.1-py37he6710b0_0
    nb_conda                                     2.2.1-py36_0 --> 2.2.1-py37_0
    nb_conda_kernels                             2.2.0-py36_0 --> 2.2.0-py37_0
    nbconvert                                    5.3.1-py36_0 --> 5.3.1-py37_0
    nbformat                                     4.4.0-py36_0 --> 4.4.0-py37_0
    ninja                                1.8.2-py36h6bb024c_1 --> 1.8.2-py37h6bb024c_1
    numexpr                              2.6.8-py36hd89afb7_0 --> 2.6.8-py37hd89afb7_0
    numpy                               1.15.4-py36h1d66e8a_0 --> 1.15.4-py37h1d66e8a_0
    numpy-base                          1.15.4-py36h81de0dd_0 --> 1.15.4-py37h81de0dd_0
    olefile                                       0.46-py36_0 --> 0.46-py37_0
    pandocfilters                                1.4.2-py36_1 --> 1.4.2-py37_1
    pexpect                                      4.6.0-py36_0 --> 4.6.0-py37_0
    pickleshare                                  0.7.5-py36_0 --> 0.7.5-py37_0
    plac                                         0.9.6-py36_0 --> 0.9.6-py37_0
    preshed                              2.0.1-py36he6710b0_0 --> 2.0.1-py37he6710b0_0
    ptyprocess                                   0.6.0-py36_0 --> 0.6.0-py37_0
    pycparser                                     2.19-py36_0 --> 2.19-py37_0
    pysocks                                      1.6.8-py36_0 --> 1.6.8-py37_0
    pyyaml                                3.13-py36h14c3975_0 --> 3.13-py37h14c3975_0
    pyzmq                               17.1.2-py36h14c3975_0 --> 17.1.2-py37h14c3975_0
    regex                        2018.01.10-py36h14c3975_1000 --> 2018.01.10-py37h14c3975_1000
    scipy                                1.1.0-py36hfa4b5c9_1 --> 1.1.0-py37hfa4b5c9_1
    send2trash                                   1.5.0-py36_0 --> 1.5.0-py37_0
    sip                                 4.19.8-py36hf484d3e_0 --> 4.18.1-py37hf484d3e_2
    spacy                            2.0.18-py36hf484d3e_1000 --> 2.0.18-py37hf484d3e_1000
    terminado                                    0.8.1-py36_1 --> 0.8.1-py37_1
    testpath                                     0.4.2-py36_0 --> 0.4.2-py37_0
    toolz                                        0.9.0-py36_0 --> 0.9.0-py37_0
    tornado                              5.1.1-py36h7b6447c_0 --> 5.1.1-py37h7b6447c_0
    traitlets                                    4.3.2-py36_0 --> 4.3.2-py37_0
    typing                                       3.6.4-py36_0 --> 3.6.4-py37_0
    ujson                                 1.35-py36h14c3975_0 --> 1.35-py37h14c3975_0
    wcwidth                                      0.1.7-py36_0 --> 0.1.7-py37_0
    webencodings                                 0.5.1-py36_1 --> 0.5.1-py37_1
    wrapt                              1.10.11-py36h14c3975_2 --> 1.10.11-py37h14c3975_2
    
    
      Proceed ([y]/n)? y
    
    
      Downloading and Extracting Packages
      torchvision-nightly- | 39 KB     |                                                                            |   0% 
    
      CondaHTTPError: HTTP 404 NOT FOUND for url <https://conda.anaconda.org/fastai/noarch/torchvision-nightly-0.2.1-py_0.tar.bz2>
      Elapsed: 00:00.385509
      CF-RAY: 4a501457fcca2a61-SEA
    
      An HTTP error occurred when trying to retrieve this URL.
      HTTP errors are often intermittent, and a simple retry will get you on your way.
    

Thanks so much for sorting this out!

P.S. Sorry for the bad formatting. Is there a way to tell the forum software not to alter a text block in any way? I tried with Preformatted Text.

0 Likes