Conda env update does not work


(Jinu Daniel) #1

Hi,
I followed the installation guideline by first creating an Ubuntu 16.04 VM on Google Cloud Platform, cloned fast ai, downloaded anaconda distribution but when I do a conda env update on fast ai folder I am getting the below error.

    Collecting fiona (from geopandas>=0.3.0->plotnine->-r /home/jinudaniel74/fastai/condaenv.zghddok7.requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/e3/bf/029958f4e3811ce7017fb5805d5203e8bde6c1816b902964acb2dec67863/Fiona-1.7.13-cp36-cp36m-manylinux1_x86_64.whl
Collecting pyproj (from geopandas>=0.3.0->plotnine->-r /home/jinudaniel74/fastai/condaenv.zghddok7.requirements.txt (line 9))
  Using cached https://files.pythonhosted.org/packages/29/72/5c1888c4948a0c7b736d10e0f0f69966e7c0874a660222ed0a2c2c6daa9f/pyproj-1.9.5.1.tar.gz
    Complete output from command python setup.py egg_info:
    using bundled proj4..
    Traceback (most recent call last):
      File "/home/jinudaniel74/anaconda3/envs/fastai/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
        extra_postargs)
      File "/home/jinudaniel74/anaconda3/envs/fastai/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/home/jinudaniel74/anaconda3/envs/fastai/lib/python3.6/distutils/spawn.py", line 36, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/home/jinudaniel74/anaconda3/envs/fastai/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
        % (cmd, exit_status))
    distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-asji0eqv/pyproj/setup.py", line 72, in <module>
        objects = cc.compile(['nad2bin.c', 'src/pj_malloc.c'])
      File "/home/jinudaniel74/anaconda3/envs/fastai/lib/python3.6/distutils/ccompiler.py", line 574, in compile
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/home/jinudaniel74/anaconda3/envs/fastai/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-asji0eqv/pyproj/

CondaValueError: pip returned an error

Not sure how to resolve this issue. Please help.


Conda env update fails
(Willismar Medeiros) #2

You need to update your pip package and try it again , also install Gcc

If you are using Ubuntu on the cloud you can do this:

1-this will install a set of tools to build packages, because not all downloaded pip packages cames in whl format, some of them comes in zip or tar (as source codes to be built).

sudo apt install -y build-essential

2-this update from pip 10.xx to pip 18.xx. Activate your conta environment first then:

pip install -U pip -y

then again:

conda env update


(David Henderson) #3

Installing build-essential worked for me. I think this is an upstream bug that should be hopefully fixed soon. Don’t forget that you might need to install jupyter as well afterwards as Jeremy has commented those packages out to get around the problem.


(Jed Bartlet) #4

This turned out to be an error with the dependency β€˜testpath’, which needs to be downgraded. Refer to the following SO answer: https://stackoverflow.com/a/52598942/2437479


(Jinu Daniel) #5

Thank you. Yes this did work for me. Installing Build essentials and upgrading pip did the trick. Also I had an issue with testpath so as mentioned I modified the environment.yml to be testpath<0.4


(Willismar Medeiros) #7

Man I cannot approve you give me access to your credentials…
I prefer help you identify the problem and learn with it. I believe people have complicated lives and cannot be available… I cannot give my problems to other people so solve it for me … And I not intend too…

So I prefer to make you LEARN how to deal with it.
Send me prints of the terminal of what is going on. Send me the error so I Can see and let’s figure out together.


(Nafiseh Salmaniniyasar) #8

Thanks,
pip is 18.1
(fastai) paperspace@ps1r995v4:~/fastai$ pip install -U pip
Requirement already up-to-date: pip in /home/paperspace/anaconda3/lib/python3.6/site-packages (18.1)

Here is the error I get after running coda env update

Exception:
Traceback (most recent call last):
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2862, in _dep_map
return self.__dep_map
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2669, in getattr
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py”, line 93, in init
req = REQUIREMENT.parseString(requirement_string)
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py”, line 1632, in parseString
raise exc
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py”, line 1622, in parseString
loc, tokens = self._parse( instring, 0 )
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py”, line 1379, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py”, line 3395, in parseImpl
loc, exprtokens = e._parse( instring, loc, doActions )
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py”, line 1383, in _parseNoCache
loc,tokens = self.parseImpl( instring, preloc, doActions )
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pyparsing.py”, line 3183, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2942, in init
super(Requirement, self).init(requirement_string)
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py”, line 97, in init
requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at β€œβ€™; extra '”

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_internal/basecommand.py”, line 228, in main
status = self.run(options, args)
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_internal/commands/install.py”, line 291, in run
resolver.resolve(requirement_set)
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_internal/resolve.py”, line 103, in resolve
self._resolve_one(requirement_set, req)
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_internal/resolve.py”, line 307, in _resolve_one
set(req_to_install.extras) - set(dist.extras)
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2819, in extras
return [dep for dep in self._dep_map if dep]
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2864, in _dep_map
self.__dep_map = self._compute_dependencies()
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2874, in _compute_dependencies
reqs.extend(parse_requirements(req))
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2935, in parse_requirements
yield Requirement(line)
File β€œ/home/paperspace/anaconda3/envs/fastai/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py”, line 2944, in init
raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at β€œβ€™; extra '”

CondaValueError: pip returned an error.


(Willismar Medeiros) #9

I saw other poeple talking about this today …
Please go to your base environment
conda activate base

and try to update pip there as well then get back to fastai and try to update the environment again… let me know if it worked


(Willismar Medeiros) #10

Try also update setuptools

conda update setuptools


(Won Seob Seo) #12

Thanks for suggestions! this didn’t solve my problem on Google cloud platform ubuntu 18.04 yet. but wanted to note you that -y option on pip install might be not available.


(Won Seob Seo) #13

Feels like this is quite complicated task to make conda env update work. Is there a work around maybe to downgrade fastai version or downgrading some dependencies ?


(Willismar Medeiros) #14

Hi

There is not complicated at all. From last week to this week this thing appear to get stated to happen in mass for many people. When things like that happens is because some package has been upgraded and others dependent libraries may not yet.

When conda or the author of some software not make limitations about the version of some package new versions may leak on update then this kind of problem happens.

I can suggest you a bunch of stuff todo but also.

I stopped to work with conda many months ago, because I like to have more control of my packages myself.

But that is my business. So tell me about your system, what can you say to me before I try to medicate it. ?


(Won Seob Seo) #15

Hello all ! For me this commit that is just applied to github repo fixed the problem. Try git pull again and run conda env update. For me now it works.


(Willismar Medeiros) #16

Just what I was telling about ! hhhhhehehhehee


(Won Seob Seo) #17

Nice !! thank you so much for contributions and helping ! @willismar


(Willismar Medeiros) #18

yes let’s spread the word ! :smiley:


(Won Seob Seo) #19

With the current version of fast ai repo clone, from the clean ubuntu with only anaconda installed, running conda env update kinda works but has an error in the end. I wonder if this is something that I can ignore. The relevant output in the terminal says

Collecting pyproj (from geopandas>=0.3.0->plotnine->-r /home/pghoou/fastai/condaenv.lbjraq97.requirements.txt (line 9))
  Downloading https://files.pythonhosted.org/packages/29/72/5c1888c4948a0c7b736d10e0f0f69966e7c0874a660222ed0a2c2c6daa9f/pyproj-1.9.5.1.tar.gz (4.4MB)
    74% |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ        | 3.3MB 33.4MB/s eta 0:00:01debug2: channel 0: window 999362 sent adjust 49214
    100% |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 4.4MB 9.4MB/s
    Complete output from command python setup.py egg_info:
    using bundled proj4..
    Traceback (most recent call last):
      File "/home/pghoou/anaconda3/envs/fastai/lib/python3.6/distutils/unixccompiler.py", line 118, in _compile
        extra_postargs)
      File "/home/pghoou/anaconda3/envs/fastai/lib/python3.6/distutils/ccompiler.py", line 909, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/home/pghoou/anaconda3/envs/fastai/lib/python3.6/distutils/spawn.py", line 36, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/home/pghoou/anaconda3/envs/fastai/lib/python3.6/distutils/spawn.py", line 159, in _spawn_posix
        % (cmd, exit_status))
    distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-h5bju6oj/pyproj/setup.py", line 72, in <module>
        objects = cc.compile(['nad2bin.c', 'src/pj_malloc.c'])
      File "/home/pghoou/anaconda3/envs/fastai/lib/python3.6/distutils/ccompiler.py", line 574, in compile
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/home/pghoou/anaconda3/envs/fastai/lib/python3.6/distutils/unixccompiler.py", line 120, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-h5bju6oj/pyproj/

CondaValueError: pip returned an error

Does anyone have the same problem ?

The last commit the current version of github has is c17bc19

Btw, the jupyter notebook can be opened despite the error message. So it might be ok to ignore but might be good to fix since getting error is confusing and misleading even if everything might be ok.


(Divyojyoti Sinha) #20

Hi @wontheone1, you were able to import fastai library successfully? Do we need to activate fastai library as earlier? I am facing the below error while trying to activate it -


(Won Seob Seo) #21

activate fastai looks not familiar to me (maybe because i am noob)

I always run

conda activate fastai

source activate fastai


(Stas Bekman) #22

fyi testpath 0.4.2 was released on conda, so the deps were updated to reflect that.