Howto: installation on Windows


(Andrea de Luca) #61

Use the absolute path. Eg, if you cloned the repository in C:, move into dl1, then do:

mklink /d fastai C:\fastai\fastai


#62

I tried to install Link Shell Extension. The installation puts HardlinkShellExt.dll, LSEConfig.exe, Ro
ckallDLL.dll, symlink.exe and uninst-HardlinkShellExt_X64.exe into Program Files/LinkShellExtension, but it there is no HardlinkShellExt_X64.exe which seems to be the program. I installed the prerequisite redistribution code for both 64 and 32 bit before installing.


#64

The problem was that the Python code was not being installed in the fastai/fastai directory. I started everything over from scratch and this time it was successful. dl1/lesson1.ipynb now works fine (EVGA 1080Ti). Thanks for your help.


#65

Twice today I’ve come across interesting packages I’d like to experiment with fastai. (but not at risk of breaking fastai functionality)
First pandas-profiling, then tpot. Both are available at conda, but both want to downgrade a whole heap of dependencies. Can anyone here advise how to deal with that?


#66

Just create/clone a new conda environment from the fastai one


#67

funny you should suggest that… I got an error trying to clone…

which I’ve now fixed and submitted a conda pull request. (my first, so blaring trumpet here loudly, hoping/presuming it gets accepted)


(Aditya) #68

tpot didn’t require a lot of package changes as many will be installed with Anaconda by default…


#69

Hi, thanks Jeremy for creating this guide for us Windows users. I’m on a windows 10 desktop and managed to get it working.

I have a problem when trying to install the spacy english model. My anaconda 3 is in the home directory.

(fastai) C:\WINDOWS\system32>pip install spacy && python -m spacy download en
Requirement already satisfied: spacy in c:\users\user\anaconda3\envs\fastai\lib\site-packages
Requirement already satisfied: numpy>=1.7 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: murmurhash<0.29,>=0.28 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: cymem<1.32,>=1.30 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: preshed<2.0.0,>=1.0.0 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: thinc<6.11.0,>=6.10.1 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: six in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: pathlib in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: ujson>=1.35 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: dill<0.3,>=0.2 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: regex==2017.4.5 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: ftfy<5.0.0,>=4.4.2 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: msgpack-python in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: msgpack-numpy==0.4.1 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from spacy)
Requirement already satisfied: wrapt in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from thinc<6.11.0,>=6.10.1->spacy)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from thinc<6.11.0,>=6.10.1->spacy)
Requirement already satisfied: cytoolz<0.9,>=0.8 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from thinc<6.11.0,>=6.10.1->spacy)
Requirement already satisfied: termcolor in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from thinc<6.11.0,>=6.10.1->spacy)
Requirement already satisfied: pyreadline>=1.7.1 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from dill<0.3,>=0.2->spacy)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: idna<2.7,>=2.5 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: html5lib in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from ftfy<5.0.0,>=4.4.2->spacy)
Requirement already satisfied: wcwidth in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from ftfy<5.0.0,>=4.4.2->spacy)
Requirement already satisfied: toolz>=0.8.0 in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from cytoolz<0.9,>=0.8->thinc<6.11.0,>=6.10.1->spacy)
Requirement already satisfied: webencodings in c:\users\user\anaconda3\envs\fastai\lib\site-packages (from html5lib->ftfy<5.0.0,>=4.4.2->spacy)
Traceback (most recent call last):
File “C:\Users\user\Anaconda3\envs\fastai\lib\runpy.py”, line 183, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File “C:\Users\user\Anaconda3\envs\fastai\lib\runpy.py”, line 142, in get_module_details
return get_module_details(pkg_main_name, error)
File “C:\Users\user\Anaconda3\envs\fastai\lib\runpy.py”, line 109, in get_module_details
import(pkg_name)
File "C:\Users\user\Anaconda3\envs\fastai\lib\site-packages\spacy_init
.py", line 4, in
from .cli.info import info as cli_info
File "C:\Users\user\Anaconda3\envs\fastai\lib\site-packages\spacy\cli_init
.py", line 1, in
from .download import download
File “C:\Users\user\Anaconda3\envs\fastai\lib\site-packages\spacy\cli\download.py”, line 10, in
from .link import link
File “C:\Users\user\Anaconda3\envs\fastai\lib\site-packages\spacy\cli\link.py”, line 7, in
from …compat import symlink_to, path2str
File “C:\Users\user\Anaconda3\envs\fastai\lib\site-packages\spacy\compat.py”, line 11, in
from thinc.neural.util import copy_array
File "C:\Users\user\Anaconda3\envs\fastai\lib\site-packages\thinc_init
.py", line 1, in
from .about import *
ModuleNotFoundError: No module named ‘thinc.about’

It appears I have everything I need to install spacy, but it can’t seem to find a module I should have? Do I need to do a manual install? I alrready tried an env update.


#70

can’t say I understood that. when i ran conda install under fastai it wanted to downgrade heaps (~20). but when i cloned that and installed, it seemed to do very little. pandas-profiling did similar.

once pandas-profiling is installed, the code is basic

# can use pandas profiling to get an overview of the data
import pandas_profiling
#profile = pandas_profiling.ProfileReport(df_all)
profile = pandas_profiling.ProfileReport(df_all[:2000]) #run faster on subset of data
profile.to_file(outputfile="./tmp/myprofilereport.html")
profile

(Aditya) #71

Will try it out today…


(nok) #72


Somehow there is a missing tab in the lastest master branch. Easy fix, just add a tab into nasnet.py


(QuantScientist) #73

You can verify your CUDA installation on Windows using either:
PyTorch:

OR:
PyCuda: https://github.com/QuantScientist/Deep-Learning-Boot-Camp/blob/master/day02-PyTORCH-and-PyCUDA/PyCUDA/01%20PyCUDA%20verify%20CUDA%208.0.ipynb


(Andrea de Luca) #74

Don’t clone. Create a new env, then install fastai ex novo (maybe changing the name in environment.yml).


#75

already done it after fixing Conda Windows bug. why do you suggest the other way?


#76

Help please. Wndows install of xgboost is giving me grief.

I’d installed tpot, without hassles. I went to run some code and its using xgboost, which wasn’t installed.

anaconda search -t conda xgboost
only shows 1 windows version, but it uses py27, so i get
UnsatisfiableError: The following specifications were found to be in conflict:

  • defaults/win-64::jupyter_client==5.2.2=py36_0
  • xgboost

what next?
I’d prefer to use conda rather than trying to compile or use the py36 version found at
https://www.lfd.uci.edu/~gohlke/pythonlibs/#xgboost

For anyone searching for a previous problem, conda install xgboost fails on Windows at the moment, as does conda install -c conda-forge install xgboost. (see above to locate distributions)

thanks in advance


(Aditya) #77

@Brad_S

Use the wheel file to install xgboost…

Here’s what I always use to install anything…

https://www.lfd.uci.edu/~gohlke/pythonlibs/#xgboost

Download the corresponding .whl file according to your machine and then just navigate to your default download location and do a
pip install *.whl(if using some environment then activate that and then do)

And we are good to go

** Be sure to download the corresponding version**

Then this link is from the docs…

http://xgboost.readthedocs.io/en/latest/parameter.html


#78

except conda is not managing dependencies.
I can’t recall having had to use pip for anything in my environment


(Aditya) #79

@Brad_S
It’s not a problem…
Whether we do a condo install *.whl(haven’t tried)

In my case I had Anaconda setup and did pip install *.whl in the Anaconda terminal…

This works out of box…

Regarding the dependencies, pip will take care of that…

Everything might keep working if you use pip to install vs conda. However, Conda cannot manage dependencies that pip has installed - it cannot upgrade them, or remove them. More importantly, conda will install a package even if its already been installed with pip!


#80

I have a problem running the import in lesson 1…

# This file contains all the main external libs we'll use

from fastai.imports import *


#81

I have a problem running import in lesson 1…

I use Windows 7. I use Jupyter Notebook started through Anaconda Navigator.

We have this cell:

# This file contains all the main external libs we'll use
from fastai.imports import *

It produces the following output:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-7-c51a52575a56> in <module>()
      1 # This file contains all the main external libs we'll use
----> 2 from fastai.imports import *

~\Documents\Fast AI\fastai\courses\dl1\fastai\imports.py in <module>()
      3 import random, pandas as pd, pickle, sys, itertools, string, sys, re, datetime, time, shutil
      4 import seaborn as sns, matplotlib
----> 5 import IPython, graphviz, sklearn_pandas, sklearn, warnings
      6 from abc import abstractmethod
      7 from glob import glob, iglob

ModuleNotFoundError: No module named 'sklearn_pandas'

I’ve tried to install sklearn_pandas with conda:

(fastai) C:\Users\PJvG\Documents\Fast AI\fastai>conda install sklearn_pandas
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current cha
nnels:

  - sklearn_pandas

Current channels:

  - https://repo.continuum.io/pkgs/main/win-64
  - https://repo.continuum.io/pkgs/main/noarch
  - https://repo.continuum.io/pkgs/free/win-64
  - https://repo.continuum.io/pkgs/free/noarch
  - https://repo.continuum.io/pkgs/r/win-64
  - https://repo.continuum.io/pkgs/r/noarch
  - https://repo.continuum.io/pkgs/pro/win-64
  - https://repo.continuum.io/pkgs/pro/noarch
  - https://repo.continuum.io/pkgs/msys2/win-64
  - https://repo.continuum.io/pkgs/msys2/noarch

Tried it with pip:

(fastai) C:\Users\PJvG\Documents\Fast AI\fastai>pip install sklearn_pandas
Requirement already satisfied: sklearn_pandas in c:\programdata\anaconda3\envs\f
astai\lib\site-packages
Requirement already satisfied: pandas>=0.11.0 in c:\programdata\anaconda3\envs\f
astai\lib\site-packages (from sklearn_pandas)
Requirement already satisfied: scipy>=0.14 in c:\programdata\anaconda3\envs\fast
ai\lib\site-packages (from sklearn_pandas)
Requirement already satisfied: scikit-learn>=0.15.0 in c:\programdata\anaconda3\
envs\fastai\lib\site-packages (from sklearn_pandas)
Requirement already satisfied: numpy>=1.6.1 in c:\programdata\anaconda3\envs\fas
tai\lib\site-packages (from sklearn_pandas)
Requirement already satisfied: python-dateutil>=2 in c:\programdata\anaconda3\en
vs\fastai\lib\site-packages (from pandas>=0.11.0->sklearn_pandas)
Requirement already satisfied: pytz>=2011k in c:\programdata\anaconda3\envs\fast
ai\lib\site-packages (from pandas>=0.11.0->sklearn_pandas)
Requirement already satisfied: six>=1.5 in c:\programdata\anaconda3\envs\fastai\
lib\site-packages (from python-dateutil>=2->pandas>=0.11.0->sklearn_pandas)

I downloaded a Python Wheel file for sklearn_pandas here: https://pypi.python.org/pypi/sklearn-pandas

Tried it with pip:

(fastai) C:\Users\PJvG\Documents\Fast AI\Wheels>pip install -U sklearn_pandas
Requirement already up-to-date: sklearn_pandas in c:\programdata\anaconda3\envs\
fastai\lib\site-packages
Requirement already up-to-date: scipy>=0.14 in c:\programdata\anaconda3\envs\fas
tai\lib\site-packages (from sklearn_pandas)
Requirement already up-to-date: scikit-learn>=0.15.0 in c:\programdata\anaconda3
\envs\fastai\lib\site-packages (from sklearn_pandas)
Requirement already up-to-date: numpy>=1.6.1 in c:\programdata\anaconda3\envs\fa
stai\lib\site-packages (from sklearn_pandas)
Requirement already up-to-date: pandas>=0.11.0 in c:\programdata\anaconda3\envs\
fastai\lib\site-packages (from sklearn_pandas)
Requirement already up-to-date: python-dateutil>=2 in c:\programdata\anaconda3\e
nvs\fastai\lib\site-packages (from pandas>=0.11.0->sklearn_pandas)
Collecting pytz>=2011k (from pandas>=0.11.0->sklearn_pandas)
  Downloading pytz-2018.3-py2.py3-none-any.whl (509kB)
    100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 512kB 4.7MB/s
Requirement already up-to-date: six>=1.5 in c:\programdata\anaconda3\envs\fastai
\lib\site-packages (from python-dateutil>=2->pandas>=0.11.0->sklearn_pandas)
Installing collected packages: pytz
  Found existing installation: pytz 2017.3
    Uninstalling pytz-2017.3:
      Successfully uninstalled pytz-2017.3
Successfully installed pytz-2018.3

But still, in lesson1 the ModuleNotFoundError problem remains for sklearn_pandas.

Am I missing something here? Can someone help me figure out how to solve this problem?