2 posts were merged into an existing topic: Misc issues
4 posts were merged into an existing topic: Performance Improvement Through Faster Software Components
Weird dependency error regarding to installation of dataclasses.
No module named 'dataclasses'
This is how I installed fastai in Docker
# Python Anaconda default.
RUN wget -q https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh -O ~/anaconda.sh && \
/bin/bash ~/anaconda.sh -b -p /opt/conda && \
rm ~/anaconda.sh
# Install PyTorch V1.
ENV PATH /opt/conda/bin:$PATH
ARG PYTHON_VERSION
RUN conda install -y python=$PYTHON_VERSION && \
conda install -c conda-forge imbalanced-learn && \
conda install -y -c conda-forge feather-format && \
conda install -y -c conda-forge jupyterlab && \
conda install -y -c conda-forge jupyter_contrib_nbextensions && \
jupyter contrib nbextension install --system
RUN conda install -y -c pytorch -c fastai fastai==1.0.42
RUN pip install --no-cache-dir -U pip && \
pip install --no-cache-dir -U dpkt scapy protobuf
Could anyone help me?
I am having trouble upgrading to the latest version of fastai on Windows 10 using conda. I havenât been able to update since 1.0.38. Running conda update fastai returns âall packages already installedâ, even though there have been several new versions.
The only thing I can see is that at version 1.0.39, the build changed from py_1 to 1. Could this be causing the issue? I tried removing fastai and reinstalling, but it just installed 1.0.38 again. Specifying the latest version explicitly doesnât work either.
Hereâs the output of conda search fastai. The new versions are there, but it wonât install.
Name Version Build Channel
fastai 1.0.37 py_1 fastai
fastai 1.0.38 py_1 fastai
fastai 1.0.39 1 fastai
fastai 1.0.40 1 fastai
fastai 1.0.41 1 fastai
fastai 1.0.42 1 fastai
1 Like
Conda is tricky that way - it tells you nothing about the conflicts and then quits, please see: https://docs.fast.ai/troubleshoot.html#cant-install-the-latest-fastai-conda-package
fastai doesnât install dataclasses for py37, but itâs needed for py36. Most likely what happened is that you installed fastai with py37, but then some other package downgraded python to py36 and youâre running fastai with py36, while you installed it with py37?
Itâd help if you were to follow the guideliness for support: https://docs.fast.ai/support.html
Youâre not saying when you encountered the error. Please put yourself in the shoes of the person that has no idea what you did and then you will know what information to share 
we currently donât use conda-forge and all package dependencies are tested against the anaconda channel, so itâs possible that something is off in your case, since youâre on the fringe. But again I have no way of telling.
and of course, you can just install dataclasses
sorry I missed to clarify the python version. I installed python 3.6 after the conda installation (So what you wrote is not true for my case). Then I install fastai via conda -c pytorch -c fastai fastai.
The error of No module named 'dataclasses' occurred after I ran the cell of from fastai.tabular import *.
I created the environment on nvidia-docker. There, I used anaconda to follow the instruction on the github repo as possible.
1 Like
Thank you for providing these details, @crcrpar. Indeed there was a bug in the conda package setup for py36, should be resolved in fastai-1.0.43 when that is released. Until then, please add conda install dataclasses to your docker build script.
1 Like
Thank you.
Iâm glad to hear that because I thought the conda would automatically install dataclasses if python version was 3.6 IIRC.
ps. To get familiar with, I decided to migrate to python 3.7.
1 Like
yeah, conda is not very flexible in this situation. pip allows defining a dynamic package dependency which gets sorted out during install. conda only during package build and then itâs set in stone. So the variant is only doable if py36 and py37 packages are built. Since fastai is noarch, this variant unfortunately doesnât work. But luckily no harm in reinstalling dataclasses for py37, so all is good.
1 Like
Hi,
I get the following error:
DistributionNotFound: The âfastprogress>=0.1.18â distribution was not found and is required by the application
when running
from fastai.vision import *
or other import operations in an Jupyter Notebook.
It runs fine when running it just in python.
I installed it through pip
fastai 1.0.42
fastprogress 0.1.18
Python version
3.6.3 (default, Sep 2 2018, 00:38:05)
I am running Ubuntu 16.04.
I should also note that this might be related to my pyenv setup, which I am using. However, I have not had any problems with the pyenv-jupyter combination so far.
Best regards
Okay, problem seems to be my pyenv enviroment. Problem is solved (not quite, but the problem is on my side).
Hello,
What seems to be best practice (if there even is one) for upgrading to a new version of a Nvidia driver, or a new Pytorch version? Iâve read the documentation on https://docs.fast.ai/troubleshoot.html, but couldnât find guidance. What Iâm asking is, do you wait a while, so as to make sure itâs stable, or do you upgrade as soon as a new version is released?
Usually, you want to upgrade things like nvidia drivers, when the new release includes:
- fixes to something that was broken that you needed to work
- new features you actually need
- speed improvements that are important to you
and when you upgrade hardware and then you may have to get a newer driver.
In all other circumstances, if you have a satisfactory setup - save your time and sanity and donât update, especially since often updates break other things.
Another time I update everything is when I revamp everything anyway, e.g. new Ubuntu version once in a few years.
Of course, YMMV.
1 Like
Hi helpers. I have been getting this conda error for about a week whenever trying to update the fastai conda environment. Can anyone help?
Some cluesâŚ
-
The conda command below used to work.
-
The error might have started after conda asked me to update conda. I did this by dutifully copy/pasting the command given, without understanding what it did.
-
My setup is local, with Ubuntu 16.04 LTS, GTX 1070.
-
conda list(s) currently the following versions. They work together. I donât know which are needed, but I prefer to use Cuda92, or whichever is the most recent stable version.
pytorch 1.0.0 py3.6_cuda9.0.176_cudnn7.4.1_1 pytorch
pytorch-nightly 1.0.0.dev20181126 py3.6_cuda9.2.148_cudnn7.4.1_0 [cuda92] pytorch
python 3.6.7 h0371630_0
fastai 1.0.42 1 fastai
fastprogress 0.1.18 py_0 fastai
torchvision 0.2.1 py36_1 pytorch
torchvision-nightly 0.2.1 py_0 fastai
cuda92 1.0 0 pytorch -
I am a Linux/conda igoramus whose primary goal is to get some work done and not break anything. So please use small words.
(fastaiv3) malcolm@PC-GPU:~/course-v3$ conda install -c pytorch -c fastai fastai
Collecting package metadata: done
Solving environment: donePackage Plan
environment location: /home/malcolm/anaconda3/envs/fastaiv3 added / updated specs: - fastaiThe following packages will be downloaded:
package | build ---------------------------|----------------- torchvision-nightly-0.2.1 | py_0 39 KB fastai ------------------------------------------------------------ Total: 39 KBThe following NEW packages will be INSTALLED:
bleach pkgs/main/linux-64::bleach-3.1.0-py37_0 cudatoolkit pkgs/main/linux-64::cudatoolkit-10.0.130-0 pip pkgs/main/linux-64::pip-19.0.1-py37_0 soupsieve pkgs/main/linux-64::soupsieve-1.7.1-py37_0 urllib3 pkgs/main/linux-64::urllib3-1.24.1-py37_0The following packages will be UPDATED:
beautifulsoup4 4.6.3-py36_0 --> 4.7.1-py37_1 cryptography 2.4.1-py36h1ba5d50_0 --> 2.4.2-py37h1ba5d50_0 decorator 4.3.0-py36_0 --> 4.3.2-py37_0 entrypoints 0.2.3-py36_2 --> 0.3-py37_0 idna 2.7-py36_0 --> 2.8-py37_0 ipython 7.0.1-py36h39e3cac_0 --> 7.2.0-py37h39e3cac_0 jedi 0.13.1-py36_0 --> 0.13.2-py37_0 jupyter_client 5.2.3-py36_0 --> 5.2.4-py37_0 markupsafe 1.0-py36h14c3975_1 --> 1.1.0-py37h7b6447c_0 msgpack-python 0.5.6-py36h6bb024c_1 --> 0.6.1-py37hfd86e86_1 notebook 5.7.0-py36_0 --> 5.7.4-py37_0 packaging 18.0-py36_0 --> 19.0-py37_0 pandas 0.23.4-py36h04863e7_0 --> 0.24.1-py37he6710b0_0 parso 0.3.1-py36_0 --> 0.3.2-py37_0 pillow 5.3.0-py36h34e0f95_0 --> 5.4.1-py37h34e0f95_0 prometheus_client 0.4.2-py36_0 --> 0.5.0-py37_0 prompt_toolkit pkgs/main/linux-64::prompt_toolkit-2.~ --> pkgs/main/noarch::prompt_toolkit-2.0.8-py_0 pygments 2.2.0-py36_0 --> 2.3.1-py37_0 pyopenssl 18.0.0-py36_0 --> 19.0.0-py37_0 pyparsing 2.2.2-py36_0 --> 2.3.1-py37_0 pyqt pkgs/free::pyqt-5.6.0-py36_2 --> pkgs/main::pyqt-5.6.0-py37h22d08a2_6 python 3.6.7-h0371630_0 --> 3.7.1-h0371630_7 python-dateutil 2.7.3-py36_0 --> 2.7.5-py37_0 pytorch 1.0.0-py3.6_cuda9.0.176_cudnn7.4.1_1 --> 1.0.1-py3.7_cuda10.0.130_cudnn7.4.2_0 pytz 2018.5-py36_0 --> 2018.9-py37_0 requests 2.19.1-py36_0 --> 2.21.0-py37_0 setuptools 40.4.3-py36_0 --> 40.7.3-py37_0 six 1.11.0-py36_1 --> 1.12.0-py37_0 thinc pkgs/main::thinc-6.12.0-py36h4989274_0 --> fastai::thinc-6.12.1-py37h637b7d7_1000 torchvision pytorch/linux-64::torchvision-0.2.1-p~ --> pytorch/noarch::torchvision-0.2.1-py_2 tqdm pkgs/main/linux-64::tqdm-4.26.0-py36h~ --> pkgs/main/noarch::tqdm-4.29.1-py_0 wheel 0.32.2-py36_0 --> 0.32.3-py37_0The following packages will be DOWNGRADED:
asn1crypto 0.24.0-py36_0 --> 0.24.0-py37_0 backcall 0.1.0-py36_0 --> 0.1.0-py37_0 bottleneck 1.2.1-py36h035aef0_1 --> 1.2.1-py37h035aef0_1 certifi 2018.11.29-py36_0 --> 2018.11.29-py37_0 cffi 1.11.5-py36he75722e_1 --> 1.11.5-py37he75722e_1 chardet 3.0.4-py36_1 --> 3.0.4-py37_1 cycler 0.10.0-py36_0 --> 0.10.0-py37_0 cymem 2.0.2-py36hfd86e86_0 --> 2.0.2-py37hfd86e86_0 cytoolz 0.9.0.1-py36h14c3975_1 --> 0.9.0.1-py37h14c3975_1 dill 0.2.8.2-py36_0 --> 0.2.8.2-py37_0 ipykernel 5.1.0-py36h39e3cac_0 --> 5.1.0-py37h39e3cac_0 ipython_genutils 0.2.0-py36_0 --> 0.2.0-py37_0 jinja2 2.10-py36_0 --> 2.10-py37_0 jsonschema 2.6.0-py36_0 --> 2.6.0-py37_0 jupyter_core 4.4.0-py36_0 --> 4.4.0-py37_0 kiwisolver 1.0.1-py36hf484d3e_0 --> 1.0.1-py37hf484d3e_0 matplotlib 2.2.2-py36hb69df0a_2 --> 2.2.2-py37hb69df0a_2 mistune 0.8.4-py36h7b6447c_0 --> 0.8.4-py37h7b6447c_0 mkl_fft 1.0.6-py36h7dd41cf_0 --> 1.0.6-py37h7dd41cf_0 mkl_random 1.0.1-py36h4414c95_1 --> 1.0.1-py37h4414c95_1 msgpack-numpy 0.4.3.2-py36_0 --> 0.4.3.2-py37_0 murmurhash 1.0.1-py36he6710b0_0 --> 1.0.1-py37he6710b0_0 nb_conda 2.2.1-py36_0 --> 2.2.1-py37_0 nb_conda_kernels 2.2.0-py36_0 --> 2.2.0-py37_0 nbconvert 5.3.1-py36_0 --> 5.3.1-py37_0 nbformat 4.4.0-py36_0 --> 4.4.0-py37_0 ninja 1.8.2-py36h6bb024c_1 --> 1.8.2-py37h6bb024c_1 numexpr 2.6.8-py36hd89afb7_0 --> 2.6.8-py37hd89afb7_0 numpy 1.15.4-py36h1d66e8a_0 --> 1.15.4-py37h1d66e8a_0 numpy-base 1.15.4-py36h81de0dd_0 --> 1.15.4-py37h81de0dd_0 olefile 0.46-py36_0 --> 0.46-py37_0 pandocfilters 1.4.2-py36_1 --> 1.4.2-py37_1 pexpect 4.6.0-py36_0 --> 4.6.0-py37_0 pickleshare 0.7.5-py36_0 --> 0.7.5-py37_0 plac 0.9.6-py36_0 --> 0.9.6-py37_0 preshed 2.0.1-py36he6710b0_0 --> 2.0.1-py37he6710b0_0 ptyprocess 0.6.0-py36_0 --> 0.6.0-py37_0 pycparser 2.19-py36_0 --> 2.19-py37_0 pysocks 1.6.8-py36_0 --> 1.6.8-py37_0 pyyaml 3.13-py36h14c3975_0 --> 3.13-py37h14c3975_0 pyzmq 17.1.2-py36h14c3975_0 --> 17.1.2-py37h14c3975_0 regex 2018.01.10-py36h14c3975_1000 --> 2018.01.10-py37h14c3975_1000 scipy 1.1.0-py36hfa4b5c9_1 --> 1.1.0-py37hfa4b5c9_1 send2trash 1.5.0-py36_0 --> 1.5.0-py37_0 sip 4.19.8-py36hf484d3e_0 --> 4.18.1-py37hf484d3e_2 spacy 2.0.18-py36hf484d3e_1000 --> 2.0.18-py37hf484d3e_1000 terminado 0.8.1-py36_1 --> 0.8.1-py37_1 testpath 0.4.2-py36_0 --> 0.4.2-py37_0 toolz 0.9.0-py36_0 --> 0.9.0-py37_0 tornado 5.1.1-py36h7b6447c_0 --> 5.1.1-py37h7b6447c_0 traitlets 4.3.2-py36_0 --> 4.3.2-py37_0 typing 3.6.4-py36_0 --> 3.6.4-py37_0 ujson 1.35-py36h14c3975_0 --> 1.35-py37h14c3975_0 wcwidth 0.1.7-py36_0 --> 0.1.7-py37_0 webencodings 0.5.1-py36_1 --> 0.5.1-py37_1 wrapt 1.10.11-py36h14c3975_2 --> 1.10.11-py37h14c3975_2 Proceed ([y]/n)? y Downloading and Extracting Packages torchvision-nightly- | 39 KB | | 0% CondaHTTPError: HTTP 404 NOT FOUND for url <https://conda.anaconda.org/fastai/noarch/torchvision-nightly-0.2.1-py_0.tar.bz2> Elapsed: 00:00.385509 CF-RAY: 4a501457fcca2a61-SEA An HTTP error occurred when trying to retrieve this URL. HTTP errors are often intermittent, and a simple retry will get you on your way.
Thanks so much for sorting this out!
P.S. Sorry for the bad formatting. Is there a way to tell the forum software not to alter a text block in any way? I tried with Preformatted Text.
What is the error? I see that itâs trying to move you from py36 to py37 - you must be referring to that - otherwise I see no error.
and why are you having pytorch* nightly packages and not stable ones?
In any case, it has all been changed since yesterday, since the released pytorch-1.0.1 changed how they install the cuda builds, https://github.com/pytorch/pytorch/releases/tag/v1.0.1
so if you try now itâll be installing the cudatoolkit=10.0 by default, and you can ask for cudatoolkit=9.0 if your driver is old and doesnât support CUDA 10. It doesnât matter if you have CUDA installed system-wide and if you do which is its version - it packs its own binaries.
So in your case, uninstall first and then re-install:
conda uninstall fastai pytorch pytorch-nightly torchvision-nightly cuda92
conda install -c pytorch -c fastai fastai
And even better start with a fresh conda environment with just:
conda install -c pytorch -c fastai fastai
I think your problem mainly has to do with you having a conda environment with packages pre-pytorch-1.0.0 release, thatâs why itâs messed up.
Thanks, the first method worked perfectly after updating the nvidia driver to v415.27.
1 Like
A post was merged into an existing topic: Misc issues
Hello,
I have a question. Do you know whether the fastai 1.0.42 version has changed the CUDA drivers or anything else relative to NVIDIA? I upgraded to v. 1.0.42 from 1.038 and it canât import all libraries. It give me the following error:
NVMLError_LibraryNotFound: NVML Shared Library Not Found
Any idea why? ( I am using windows 10 with a conda environment)
fastai did not, but pytorch did a few days ago, see: https://github.com/pytorch/pytorch/releases
to debug load pytorch directly, without going through fastai, and report any issues with it at https://github.com/pytorch/pytorch/
Remember that fastai is a front-end for pytorch, so any driver issues belong to pytorch.