Developer chat


(Kaspar Lund) #122

Look exactly like the messages i got on Windows 10 / WSL. I abononned it:(


(Stas Bekman) #123

I was able to reproduce your problem.

You need to:

conda install jupyter

will update the docs/deps


(Marco Ribeiro) #124

That made the notebook get the right path.

The jupyterlab package was installed in the conda root environment. After I install the package in the fastai_v1 env the path changed to the right one.

Even after this install, the kernel is still restarting. The –debug flag did not helped to find what is causing that.
Do you know other way to debug this?


(Stas Bekman) #125

update: I made a new release: fastai-1.0.3 which includes this dependency.

If you could kindly re-run the complete install from scratch and the newly made (again) environment and report results that would be very helpful:

conda create -y  --name fastai_v3
conda activate fastai_v3
conda install -y -c pytorch -c fastai fastai pytorch-nightly cuda92
conda uninstall -y fastai
pip install -e .

(Stas Bekman) #126

Can you try the complete rebuild and test it in jupyter notebook environment first? I have just tested it to work as I instructed above.

I haven’t yet tried jupyterlab, perhaps someone else did… I’d imagine you need some other packages installed?


(Marco Ribeiro) #127

The kernel is still restarting every time I import fastai.

I got this error on pip install -e .

https://pastebin.com/MbnnHFpm

That is the restarting log:

[I 03:04:09.207 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
WARNING:root:kernel ec8fe490-05d7-4dbc-8803-ef553727ab4c restarted
[D 03:04:09.210 NotebookApp] Starting kernel: ['/home/marco/anaconda3/envs/fastai_v3/bin/python', '-m', 'ipykernel_launcher', '-f', '/run/user/1000/jupyter/kernel-ec8fe490-05d7-4dbc-8803-ef553727ab4c.json']
[D 03:04:09.223 NotebookApp] Connecting to: tcp://127.0.0.1:39013
[D 03:04:09.774 NotebookApp] activity on ec8fe490-05d7-4dbc-8803-ef553727ab4c: status
[D 03:04:09.825 NotebookApp] activity on ec8fe490-05d7-4dbc-8803-ef553727ab4c: status
[D 03:04:09.829 NotebookApp] activity on ec8fe490-05d7-4dbc-8803-ef553727ab4c: status
[D 03:04:10.045 NotebookApp] 304 GET /static/base/images/favicon-notebook.ico (127.0.0.1) 3.57ms
[D 03:04:12.226 NotebookApp] KernelRestarter: restart apparently succeeded
[D 03:04:12.478 NotebookApp] activity on ec8fe490-05d7-4dbc-8803-ef553727ab4c: status
[D 03:04:12.485 NotebookApp] activity on ec8fe490-05d7-4dbc-8803-ef553727ab4c: execute_input
[D 03:04:13.659 NotebookApp] 304 GET /static/base/images/favicon-busy-1.ico (127.0.0.1) 1.52ms

(Stas Bekman) #128
$ pipdeptree --reverse --packages regex
Warning!!! Possibly conflicting dependencies found:
* tensorflow==1.9.0
 - setuptools [required: <=39.1.0, installed: 40.2.0]
* jupyter==1.0.0
 - ipywidgets [required: Any, installed: ?]
* fastai==1.0.3
 - fastprogress [required: >=0.1.9, installed: 0.1.8]
------------------------------------------------------------------------
regex==2017.4.5
  - spacy==2.0.12 [requires: regex==2017.4.5]
    - en-core-web-sm==2.0.0 [requires: spacy>=2.0.0a18]
    - fastai==1.0.3 [requires: spacy]
    - fr-core-news-sm==2.0.0 [requires: spacy>=2.0.0a19]

spacy needs regex.

Install:

$ pip install pipdeptree

what do you get for?

$ pipdeptree --reverse --packages regex

You did follow the exact steps from here, correct?

Both space and regex should have been installed by conda then. why would pip try to install them again?

Also it suggests ipywidgets aren’t installed, but it’s there.

I think pipdeptree is glitchy.


(Marco Ribeiro) #129
Warning!!! Possibly conflicting dependencies found:
* Twisted==18.7.0
 - PyHamcrest [required: >=1.9.0, installed: ?]
* spacy==2.0.12
 - regex [required: ==2017.4.5, installed: 2018.7.11]
* mkl-random==1.0.1
 - cython [required: Any, installed: ?]
------------------------------------------------------------------------
regex==2018.7.11
  - spacy==2.0.12 [requires: regex==2017.4.5]
    - fastai==1.0.3 [requires: spacy]

conda list:

https://pastebin.com/ywiWrn7x


(Stas Bekman) #130

Please try the two suggestions here

But try the second first. And then the first.

Let me know which if any worked.


(Marco Ribeiro) #131

Both failed,

https://pastebin.com/91t22y03


(Stas Bekman) #132

Thank you,

I figured it out, you are on python 3.7, and there is no wheel for regex==2017.4.5 for python 3.7. only the newer 2018 versions have python 3.7 build.

While I only see 2018 wheels here https://pypi.org/project/regex/#files. You can see all of them here: https://pypi.org/simple/regex/

So if you are not attached to python3.7, use 3.6 and it will just work, since there is a wheel for that.

 conda create -y  python=3.6 --name fastai_xxx

If however you want python 3.7, you will need to figure the build from source. Probably some dev packages missing.

Let us know whether this resolved it?

Otherwise somebody who knows spacy will need to step in to figure out its dependency on regex==2017.4.5 - why can’t we have 2018 ed?.

FYI, my system builds regex==2017.4.5 wtih python 3.7 just fine:

pip install spacy
Building wheels for collected packages: regex
  Running setup.py bdist_wheel for regex ... done
  Stored in directory: /home/stas/.cache/pip/wheels/75/07/38/3c16b529d50cb4e0cd3dbc7b75cece8a09c132692c74450b01
Successfully built regex

Your compiler env is incomplete, try this

If that wasn’t it, It’s hard for me to search for the exact solution since your gcc error output is not in english, but explore google results with: “/usr/lib/gcc/” “include-fixed/limits.h” “no such” and probably your linux distro. and install whatever build/make tools you’re missing.


(Marco Ribeiro) #133

The g++ package was already installed.

I got the same errors with python 3.6:

https://pastebin.com/7L5u0SvW

I will try again tomorrow. Thanks for your help.


(Stas Bekman) #134

wrt 3.6, you’re correct, I didn’t pay close attention. All the wheels are built for the windows platform - none for linux.

So basically unless you have a healthy build environment on linux you won’t be able to build regex not on py3.6 nor py3.7, and the regex year edition doesn’t matter.

Bottom line: using pip install setup - if a user needs spacy, she needs to be able to build regex from source (via pip install)

But since conda has the binary packages, we just need to figure out why after installing them via conda install you got this issue triggered by pip install -e ..

So I would start from the beginning and carefully try to see where things go astray. Perhaps you can post the complete output of that run somewhere on paste.

Unfortunately, I will be mostly out tomorrow, so unless someone else will have an insight beforehand, I will be most likely have time to dig more into it on Wed.


(Arka Sadhu) #135

@stas sorry for the delayed report. I am using miniconda if that helps. I created a new environment (python 3.7).I am not sure why it needed conda install conda but that is the command I used. I am guessing for some reason the new channels were not updated, doing conda install conda did the trick and updated the channels.


(Marco Ribeiro) #136

Trying to discover what is causing the kernel to restart in my conda environment, I executed each one of these lines in the python console:

from fastai import *
from fastai.vision import *
from fastai.collab import *
from fastai.tabular import *
from fastai.text import *
from fastai.docs import *

Every one of them stopped in the following lines:

# code object from '/home/marco/anaconda3/envs/fastai/lib/python3.6/site-packages/fastai/vision/__pycache__/transform.cpython-36.pyc'
import 'fastai.vision.transform' # <_frozen_importlib_external.SourceFileLoader object at 0x7fd88eb5f2b0>

Is the ‘fastai.vision.transform’ been imported with collab, tabular, text or docs too?


(Marco Ribeiro) #137

To maintain the conda packages and not install the pip ones I did:

pip install -e . --no-dependencies

The spacy package is working fine and I can import fastai from the repository folder (until the kernel restarts, as described above).


(Matthew Honnibal) #138

Currently building a v2.0.13.dev3 of spaCy with the updated regex version. The exact pin was unfortunate, but regex doesn’t semver, making it hard to give a range :(. The dev version should be uploaded within the next half hour or so (CI can take some time).

Once it’s up, you should be able to set your version pin to spacy==2.0.13.dev3 to verify that it all works. I can then publish 2.0.13 properly, so you can set your pin to spacy>=2.0.13,<2.1.0

Edit: Getting test failures with regex==2018.08.29; some of the tests are simply hanging. I guess I can try some earlier versions, but it makes me nervous about rushing out a version on this. I’m worried performance could be much worse on some inputs, for some languages.


(Stas Bekman) #139

Please note a minor change in dev install steps

In addition to having all pip dependencies managed in a single file (setup.py),

To do an editable install, do:

pip install -e .[dev]

It’s almost the same as:

pip install -e .

but the former will also install extra dependencies needed only by developers. Of course, the latter works as before…

Down the road we can define a whole bunch of sub-project dependencies (e.g. just NLP-related deps) and install them with:

pip install -e .[nlp]

(Willismar Medeiros) #140

May be this ?

Wooowww sorry now I get it you guys are trying to build a environment for python 3.7… Sorry for that

conda update regex -c conda-forge

Package Plan

environment location: /home/ubuntu/miniconda3/envs/fastai

added / updated specs:
- regex

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
semver-2.8.1               |             py_1           7 KB  conda-forge
sputnik-0.9.3              |           py36_0          45 KB  conda-forge
thinc-5.0.8                |           py36_0         1.3 MB
spacy-0.101.0              |           py36_0         5.7 MB
preshed-0.46.4             |           py36_0         223 KB  conda-forge
regex-2018.08.29           |   py36h470a237_0         686 KB  conda-forge
murmurhash-0.26.4          |           py36_0          37 KB  conda-forge
------------------------------------------------------------
                                       Total:         8.0 MB

The following NEW packages will be INSTALLED:

semver:          2.8.1-py_1            conda-forge
sputnik:         0.9.3-py36_0          conda-forge

The following packages will be UPDATED:

ca-certificates: 2018.03.07-0                      --> 2018.8.24-ha4d7672_0      conda-forge
certifi:         2018.8.24-py36_1                  --> 2018.8.24-py36_1001       conda-forge
openssl:         1.0.2p-h14c3975_0                 --> 1.0.2p-h470a237_0         conda-forge
regex:           2017.11.09-py36_0     conda-forge --> 2018.08.29-py36h470a237_0 conda-forge

The following packages will be DOWNGRADED:

murmurhash:      0.28.0-py36hfc679d8_0 conda-forge --> 0.26.4-py36_0             conda-forge
preshed:         1.0.1-py36hfc679d8_0  conda-forge --> 0.46.4-py36_0             conda-forge
spacy:           2.0.12-py36hf8a1672_0 conda-forge --> 0.101.0-py36_0                       
thinc:           6.10.3-py36hf8a1672_3 conda-forge --> 5.0.8-py36_0

(Stas Bekman) #141

I started a new thread for install issues - please use that for any future reports. Thank you.

@elmarculino, thank you for your patience. Will you kindly post the updated state of the issues you’re experiencing to the thread I mentioned above. I’m also not sure whether you have this issue in jupyter notebook or lab? does it work ok in the notebook? can you run other non-fastai code? Thank you.

Also see 1, 2 - perhaps any of these is your culprit? but continue in the other thread with your details - since it’ll probably help others in the same boat down the river. thanks.