Fastai v1 install issues thread

(Stas Bekman) #289

Pytorch-1.0.1 switched to a different method of installing cuda, please refer to the release notes. It now by default installs cudatoolkit==10.0, you probably need to install cudatoolkit==9.0

To test my suggestion that it has nothing to do with fastai you can alternatively downgrade to pytorch==1.0.0 and I’m pretty sure your problem will go away. And if it is, then try what I said in the previous paragraph.

0 Likes

(Battle500) #290

Great, it totally makes sense.

I’ll try both suggestions.

Thank you so much :wink:

1 Like

(Brad) #291

Following the instructions on the Installation page, I was getting an error installing the Jupyter notebook dependencies:

PackagesNotFoundError: The following packages are not available from current channels: jupyter_contrib_nbextensions

This error was fixed by enabling conda-forge with the following command:

conda config --add channels conda-forge

EDIT: Dear future people: See next post for why you shouldn’t do this

1 Like

(Stas Bekman) #292

Thank you for the heads up, @yeldarb.

The correct instruction is:

conda install -c conda-forge jupyter_contrib_nbextensions

do not add the channel like you did, since now you will be getting everything from the conda-forge channel which may break some fastai dependencies, since it currently relies on the main anaconda channel.

I fixed the doc.

1 Like

(Stas Bekman) split this topic #294

A post was merged into an existing topic: Misc issues

0 Likes

(Stas Bekman) split this topic #295

A post was merged into an existing topic: Misc issues

0 Likes

#296

I made a thread before, but I think it’s clear I should have posted here.

I have been using google colab without issue, until needing to run widgets. I don’t have a GPU on my laptop but see no reason I can’t clean data on it.

However, attempting to install/run on my Ubuntu 18.04 laptop, is not working. I am sure I am missing something basic but I have struggled for some hours now, so here I am.

I have installed fastai with conda.

conda install -c pytorch -c fastai fastai

And my binaries seem to be correct?

(base) geoff@cs-macbook:~/src/neuralnet_stuff/local$ which jupyter
/home/geoff/anaconda2/bin/jupyter

(base) geoff@cs-macbook:~/src/neuralnet_stuff/local$ which python
/home/geoff/anaconda2/bin/python

(base) geoff@cs-macbook:~/src/neuralnet_stuff/local$ which pip
/home/geoff/anaconda2/bin/pip

When I run a notebook

(fastai) geoff@cs-macbook:~/src/neuralnet_stuff/local$ jupyter notebook

And try and import fastai, I get the error:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-3749d44be698> in <module>
      1 from fastai import *
----> 2 from fastai.vision import *
      3 from fastai.metrics import error_rate

ModuleNotFoundError: No module named 'fastai.vision'

If I clone the fastai repo and put it in the same directory, and try and import fastai by path it happens like this:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-3-d20a49ecd0c9> in <module>
      1 from fastai.fastai import *
----> 2 from fastai.fastai.vision import *
      3 from fastai.fastai.metrics import error_rate

~/src/neuralnet_stuff/local/fastai/fastai/vision/__init__.py in <module>
----> 1 from .. import basics
      2 from ..basics import *
      3 from .learner import *
      4 from .image import *
      5 from .data import *

~/src/neuralnet_stuff/local/fastai/fastai/basics.py in <module>
----> 1 from .basic_train import *
      2 from .callback import *
      3 from .core import *
      4 from .basic_data import *
      5 from .data_block import *

~/src/neuralnet_stuff/local/fastai/fastai/basic_train.py in <module>
      1 "Provides basic training and validation with `Learner`"
----> 2 from .torch_core import *
      3 from .basic_data import *
      4 from .callback import *
      5 from .data_block import *

~/src/neuralnet_stuff/local/fastai/fastai/torch_core.py in <module>
      1 "Utility functions to help deal with tensors"
----> 2 from .imports.torch import *
      3 from .core import *
      4 
      5 AffineMatrix = Tensor

~/src/neuralnet_stuff/local/fastai/fastai/imports/__init__.py in <module>
----> 1 from .core import *
      2 from .torch import *

~/src/neuralnet_stuff/local/fastai/fastai/imports/core.py in <module>
     11 from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
     12 from copy import copy, deepcopy
---> 13 from dataclasses import dataclass, field, InitVar
     14 from enum import Enum, IntEnum
     15 from functools import partial, reduce

ModuleNotFoundError: No module named 'dataclasses'

I am stuck, any help would be appreciated. The computer’s “name” is “C’s Macbook” but it’s not a mac in software or hardware.

0 Likes

(Stas Bekman) #297

Please refer to https://docs.fast.ai/support.html to see what information we need to help out.

And I guess you haven’t read:
https://docs.fast.ai/troubleshoot.html#modulenotfounderror-no-module-named-fastaivision

1 Like

(Alankar) #298

I was installing the fast ai library (both the 0.7 and the recent v1) in my windows laptop in anaconda. After creating environments for both the versions I got the same error when i tried to run any notebook from the old course and the new course. The error was:

from torch._C import *

ImportError: DLL load failed: The specified module could not be found.
0 Likes

(Stas Bekman) #299

That’s a pytorch/windows problem, not fastai’s - please look at the various solutions at various places on google and lots here at fastai forums.

0 Likes

#300

I had not read that, thank you @stas. I still was not able to get it working. I opted to completely remove conda, and start over from scratch. Now when I start a jupyter notebook the kernel fails to even load because it is trying to access something from my old installation which no longer exists.

The kernel error I get when starting my notebook:

Traceback (most recent call last):
...snip...
FileNotFoundError: [Errno 2] No such file or directory: '/home/geoff/anaconda2/envs/fastai/bin/python': '/home/geoff/anaconda2/envs/fastai/bin/python'

where ~/anaconda2/envs/fastai/ is my old, now deleted, no longer existent conda environment.

(fastai-3.6) geoff@cs-macbook:~/src/fastai$ which python
/home/geoff/anaconda3/envs/fastai-3.6/bin/python
(fastai-3.6) geoff@cs-macbook:~/src/fastai$ which jupyter
/home/geoff/anaconda3/envs/fastai-3.6/bin/jupyter
=== Software === 
python       : 3.6.8
fastai       : 1.0.46.dev0
fastprogress : 0.1.19
torch        : 1.0.1.post2
torch cuda   : 10.0.130 / is **Not available** 

=== Hardware === 
No GPUs available 

=== Environment === 
platform     : Linux-4.18.0-15-generic-x86_64-with-debian-buster-sid
distro       : Pop!_OS 18.04 bionic
conda env    : fastai-3.6
python       : /home/geoff/anaconda3/envs/fastai-3.6/bin/python
sys.path     : 
/home/geoff/anaconda3/envs/fastai-3.6/lib/python36.zip
/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6
/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/lib-dynload
/home/geoff/.local/lib/python3.6/site-packages
/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages
no supported gpus found on this system
0 Likes

(Stas Bekman) #301

clearly you have some left-overs from the old environment getting in the way.

perhaps you have some aliases or symlinks that point to the old dirs? use grep(1) and see if you have /home/geoff/anaconda2 string somewhere in your .bashrc or whatever you use?

perhaps try to start jupyter with an explicit path /home/geoff/anaconda3/envs/fastai-3.6/bin/jupyter

You didn’t show the traceback so it’s impossible to see what may have caused it.

0 Likes

#302

I am sorry the whole traceback is

Traceback (most recent call last):
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/web.py", line 1592, in _execute
    result = yield result
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/sessions/handlers.py", line 73, in post
    type=mtype))
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 79, in create_session
    kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
    yielded = self.gen.throw(*exc_info)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 92, in start_kernel_for_session
    self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 326, in wrapper
    yielded = next(result)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/kernels/kernelmanager.py", line 160, in start_kernel
    super(MappingKernelManager, self).start_kernel(**kwargs)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel
    km.start_kernel(**kwargs)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/manager.py", line 259, in start_kernel
    **kw)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/manager.py", line 204, in _launch_kernel
    return launch_kernel(kernel_cmd, **kw)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/launcher.py", line 138, in launch_kernel
    proc = Popen(cmd, **kwargs)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/geoff/anaconda2/envs/fastai/bin/python': '/home/geoff/anaconda2/envs/fastai/bin/python'

I have that grep running, it’s not finding anything but logs at this point, but it’snot in my bash profile or bashrc

0 Likes

#303
(fastai-3.6) geoff@cs-macbook:~$ /home/geoff/anaconda3/envs/fastai-3.6/bin/jupyter notebook
[I 15:02:50.589 NotebookApp] Serving notebooks from local directory: /home/geoff
[I 15:02:50.589 NotebookApp] The Jupyter Notebook is running at:
[I 15:02:50.589 NotebookApp] http://localhost:8888/?token=6eb4c2b78fe5fd8ecb8cf8743955cc5800661604846c95d0
[I 15:02:50.589 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 15:02:50.595 NotebookApp] 
    
    To access the notebook, open this file in a browser:
        file:///run/user/1000/jupyter/nbserver-22970-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=6eb4c2b78fe5fd8ecb8cf8743955cc5800661604846c95d0
[23028:23028:0220/150251.004029:ERROR:sandbox_linux.cc(364)] InitializeSandbox() called with multiple threads in process gpu-process.
Opening in existing browser session.
[I 15:02:55.756 NotebookApp] Creating new notebook in 
[E 15:02:57.100 NotebookApp] Failed to run command:
    ['/home/geoff/anaconda2/envs/fastai/bin/python', '-m', 'ipykernel_launcher', '-f', '/run/user/1000/jupyter/kernel-f099c8cf-0cd9-47d2-8806-2d72e2f10fde.json']
        PATH='/home/geoff/anaconda3/envs/fastai-3.6/bin:/home/geoff/anaconda3/envs/fastai-3.6/bin:/home/geoff/anaconda3/condabin:/home/geoff/bin:/home/geoff/.local/bin:/home/geoff/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/geoff/go/bin:/home/geoff/.vimpkg/bin'
        with kwargs:
    {'stdin': -1, 'stdout': None, 'stderr': None, 'cwd': '/home/geoff', 'start_new_session': True}
    
[E 15:02:57.102 NotebookApp] Uncaught exception POST /api/sessions (127.0.0.1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='POST', uri='/api/sessions', version='HTTP/1.1', remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/web.py", line 1592, in _execute
        result = yield result
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/sessions/handlers.py", line 73, in post
        type=mtype))
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 79, in create_session
        kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1141, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 92, in start_kernel_for_session
        self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 1133, in run
        value = future.result()
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/tornado/gen.py", line 326, in wrapper
        yielded = next(result)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/notebook/services/kernels/kernelmanager.py", line 160, in start_kernel
        super(MappingKernelManager, self).start_kernel(**kwargs)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel
        km.start_kernel(**kwargs)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/manager.py", line 259, in start_kernel
        **kw)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/manager.py", line 204, in _launch_kernel
        return launch_kernel(kernel_cmd, **kw)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/site-packages/jupyter_client/launcher.py", line 138, in launch_kernel
        proc = Popen(cmd, **kwargs)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/subprocess.py", line 729, in __init__
        restore_signals, start_new_session)
      File "/home/geoff/anaconda3/envs/fastai-3.6/lib/python3.6/subprocess.py", line 1364, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: '/home/geoff/anaconda2/envs/fastai/bin/python': '/home/geoff/anaconda2/envs/fastai/bin/python'
[W 15:02:57.109 NotebookApp] Unhandled error
[E 15:02:57.111 NotebookApp] {
      "Host": "localhost:8888",
      "Connection": "keep-alive",
      "Content-Length": "92",
      "Origin": "http://localhost:8888",
      "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36",
      "Content-Type": "application/json",
      "Accept": "application/json, text/javascript, */*; q=0.01",
      "X-Requested-With": "XMLHttpRequest",
      "X-Xsrftoken": "2|7673f82d|a6c2965d3cbb549d184b68e240cd7023|1550589817",
      "Dnt": "1",
      "Referer": "http://localhost:8888/notebooks/Untitled6.ipynb?kernel_name=python3",
      "Accept-Encoding": "gzip, deflate, br",
      "Accept-Language": "en-US,en;q=0.9,es;q=0.8,es-419;q=0.7",
      "Cookie": "_xsrf=2|7673f82d|a6c2965d3cbb549d184b68e240cd7023|1550589817; username-localhost-8889=\"2|1:0|10:1550627070|23:username-localhost-8889|44:MDM2ZTY3YTQyZDkxNDVmMDk1YjYzZDQ4N2YwMDJjZDE=|429e39b8122fc44594d8fdb82e1ef98139ff9ebfc27a7a2a7ebd823b17ae3a84\"; username-localhost-8888=\"2|1:0|10:1550692972|23:username-localhost-8888|44:ZjhhYzU1NjdmMjU2NGM3ZDg1NTE0NTBmMjBlMjJjMjE=|9ca5d022a7a1d95c2020180fa4395a678bbe2605ce3b83be48b5ff9944690b94\""
    }
[E 15:02:57.111 NotebookApp] 500 POST /api/sessions (127.0.0.1) 68.61ms referer=http://localhost:8888/notebooks/Untitled6.ipynb?kernel_name=python3
0 Likes

(Stas Bekman) #304

thanks for the backtrace.

hmm, could it be your cached PATH values in bash - try reset with:

hash -r

it’s now a jupyter issue (not fastai). There are many posts about it, with a bunch of proposed solutions:

https://www.google.com/search?q=jupyter+notebook+“No+such+file+or+directory”+“launch_kernel”

all seem to indicate some outdated environment.

This suggests that you have a stale:
~/.local/share/jupyter/kernels/python3
that you should delete.
but if no change check other values under ~/.local/share/jupyter/ - back it up and delete and see if it resolves it.

there is also ~/.jupyter to check.

1 Like

#305

Thank you, deeply. Deleting that folder worked.

1 Like

#306

Incredibly, I still am having issues. I was able to from fastai import * cleanly, and figured I must be back to baseline.

But just from fastai.vision import * results in

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-7-8be277ac0761> in <module>
      1 from fastai import *
----> 2 from fastai.vision import *

ModuleNotFoundError: No module named 'fastai.vision'

What follows are my attempts to spit out the current environment with the recommended script.

(fastai-3.6) geoff@cs-macbook:~/src/fastai$ git pull
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 7 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), done.
From https://github.com/fastai/fastai
   eb0c8061..3a445cb5  master     -> origin/master
Updating eb0c8061..3a445cb5
Fast-forward
 fastai/gen_doc/doctest.py | 22 +++++++++++++++++++++-
 tests/conftest.py         |  7 ++++++-
 2 files changed, 27 insertions(+), 2 deletions(-)
(fastai-3.6) geoff@cs-macbook:~/src/fastai$ python -c 'import fastai.utils.collect_env; fastai.utils.collect_env.show_install(1)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/geoff/src/fastai/fastai/utils/collect_env.py", line 2, in <module>
    from ..imports.torch import *
  File "/home/geoff/src/fastai/fastai/imports/__init__.py", line 1, in <module>
    from .core import *
  File "/home/geoff/src/fastai/fastai/imports/core.py", line 1, in <module>
    import csv, gc, gzip, os, pickle, shutil, sys, warnings, yaml
ModuleNotFoundError: No module named 'yaml'
(fastai-3.6) geoff@cs-macbook:~/src/fastai$ pip install yaml
Collecting yaml
  Could not find a version that satisfies the requirement yaml (from versions: )
No matching distribution found for yaml
(fastai-3.6) geoff@cs-macbook:~/src/fastai$ conda install yaml
Collecting package metadata: done
Solving environment: done

# All requested packages already installed.

(fastai-3.6) geoff@cs-macbook:~/src/fastai$ 

0 Likes

(Stas Bekman) #307

Perhaps you need to show how you installed fastai.

You shouldn’t need to install yaml by hand. The deps should get installed themselves. Go back to the troubleshooting doc and check your paths. You probably have more than one environment and you mix them up.

And often starting with a totally fresh conda environment is the easiest way to sort this out. It’s just 3 commands and you should be done.

If you’re trying to use the dev version, remember to install fastai as explained for dev in the README.md

1 Like

#308

I was installing everything via the docs (at least a faithful attempt). Obviously I had left the system in a dirty state previously. I believe I had to install jupyter into the conda environment to finally fix that last error. Thanks again for your hand holding and help.

1 Like

(Kristian Rother) #309

Hi,

I did some quick tests and there must have been a change in how the libraries are loaded or something going from fast.ai 1.0.41->1.0.42. Everything works fine with both pytorch 1.0.0 and pytorch 1.0.1 in fastai 1.0.41 but after upgrading to fastai .42 and above the “NVML Shared Library Not Found” error appears (when running from fastai.text import *). When I downgrade to fastai .41 everything works fine again so I’m assuming it must be a fast.ai change that broke things.

Not a huge issue, I’m working with .41 for now but it would be nice to have load_data() etc. from the most recent version :slight_smile:

I’m currently training a Language Model so the computer will be blocked for some days but I can re-break things and provide a full report later if desired.

tl;dr: I think it might actually be a fast.ai change that breaks things on Windows and not a pytorch/windows issue since I’m fairly sure my pytorch/cuda is set up correctly (and works nicely with fastai <1.0.42 both for a forced pytorch 1.0.0 and the current version)

(Windows 10)

0 Likes

Nvml.dll loading issue in nvidia-ml-py3-7.352.0-py_0