Windows 10 Installation Notes (Windows command and WSL bash)

(Robert Salita) #1

Notes on installing basic set of machine learning tools into Windows 10. Also followed by Lesson1 install notes and a list of current issues. Tested using Fall Creators Edition.

Warning: Windows issues with Lesson 1 notebook currently under investigation:

  1. Windows command line: Pickle section of Lesson 1 is throwing a thread error.
  2. WSL: In training section, pytorch is complaining that an Nvidia driver is missing. I haven’t figured out how to disable GPU access.

Installing to Windows command line:

  1. Anaconda installer is a different animal than usual installer. I had to turn off my antivirus (Webroot) to get it to install properly. Always reboot after Anaconda install/uninstall/reinstall.
  2. Download and install Anaconda for Windows at https://www.anaconda.com/download/. I don’t recommend using any existing Anaconda environment installed by Visual Studio 2017. Start with your own dedicated environment.
  3. When using tools, always use Anaconda Prompt, not default Command Prompt.
    FYI, Anaconda Prompt location: C:\Users\%USERNAME%\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Anaconda3 (64-bit)
    FYI, Anaconda Prompt shortcut: %windir%\System32\cmd.exe “/K” C:\Users\%USERNAME%\Anaconda3\Scripts\activate.bat C:\Users\%USERNAME%\Anaconda3
  4. Install pytorch for Windows, if needed: conda install -c peterjc123 pytorch cuda80 # for both Windows and Linux?
  5. Install additional packages, if needed (e.g. conda install tensorflow-gpu joblib).
  6. If installing tensorflow-gpu (for systems with NVidia GPUs), you must install EXACTLY these two packages.
    a. Install cuda_8.0.61_win10 from https://developer.nvidia.com/cuda-toolkit-archive
    b. Install cudnn-8.0-windows10-x64-v6.0 from https://developer.nvidia.com/rdp/cudnn-download

Installing to Windows Subsystem for Linux/WSL/Ubuntu/bash/openSUSE/SUSE Enterprise.

  1. Follow instructions for installing WSL at Install the Linux Subsystem on Windows 10.
  2. Install Anaconda using wget and latest version number:
    wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh
    bash Anaconda3-5.0.1-Linux-x86_64.sh
  3. I had to install execstack (apt or rpm) to workaround an https error which was blocking additional conda installs.
    sudo apt-get update
    sudo apt install execstack
    execstack -c ~/anaconda3/lib/libcrypto.so.1.0.0
  4. Install pytorch, if needed: conda install pytorch
  5. Install other packages, if needed (e.g. conda install tensorflow joblib keras matplotlib seaborn)

Make sure packages are updated:

conda upgrade conda
conda upgrade --all
conda upgrade jupyter tensorflow-gpu keras matplotlib seaborn

We’ve upgraded the packages that conda knows about but some packages are only known to pip. You might want to update pip packages.

Verify that packages are installed and display version numbers:

python -c "import sys;print (sys.executable)" 
python -c "import sys;print (sys.version)" 
python -c "import jupyter;print (jupyter.__file__)" 
python -c "import torch;print (torch.__file__)"
python -c "import tensorflow;print (tensorflow.__file__, tensorflow.__version__)"
python -c "import keras;print (keras.__file__, keras.__version__)"
python -c "import matplotlib;print (matplotlib.__file__, matplotlib.__version__)"
python -c "import seaborn;print (seaborn.__file__, seaborn.__version__)"

Additional installation for Part 1 v2 Lesson1 Jupyter notebook:

SET PYTHONPATH=<path to fastai directory> # Windows only
export PYTHONPATH=<path to fastai directory> # Linux only
conda install -c anaconda bcolz
pip install isoweek
pip install pandas_summary
pip install tqdm
conda install -c peterjc123 pytorch # for both Windows and Linux?
pip install torchvision
pip install torchtext
pip install graphviz
pip install sklearn-pandas
conda install -c anaconda opencv # Linux only
conda install -c conda-forge opencv # Windows only
execstack -c ~/anaconda3/lib/* # Linux only: I needed this to fix an execute stack error. Not sure why.

After installing, verify that Part 1 v2 Lesson1 requisites actually exist.

python -c "import bcolz;print (bcolz.__file__, bcolz.__version__)"
python -c "import isoweek;print (isoweek.__file__, isoweek.__version__)"
python -c "import pandas_summary;print (pandas_summary.__file__)"
python -c "import tqdm;print (tqdm.__file__, tqdm.__version__)"
python -c "import torch;print (torch.__file__, torch.__version__)"
python -c "import torchvision;print (torchvision.__file__)"
python -c "import torchtext;print (torchtext.__file__)"
python -c "import graphviz;print (graphviz.__file__)"
python -c "import sklearn-pandas;print (sklearn-pandas.__file__)"
python -c "import cv2;print (cv2.__file__, cv2.__version__)"

Screenshot showing all modules working for lesson 1.

6 Likes

Setting up windows environment, torch?
Anyone able to run the Lesson 1 notebook?
Windows setup with CUDA for real
Fastai v0.7 install issues thread
(Davide Boschetto) #2

Ok, now: the usual debate “conda vs pip” regarding native or conda python distribution just evolved with the bash distribution under windows.

I’m usually for PIP into native windows, without conda or virtual environments… But I’m not so sure now! Hope this gets addressed in the upcoming workshop.

1 Like

(Jeremy Howard (Admin)) #3

Thanks for sharing this! :slight_smile:

0 Likes

(Debashish Panigrahi) #4

thanks! But I’m getting stuck at getting headers. Any suggestions?

Reading state information… Done

Install these packages without verification? [y/N] y
0% [Waiting for headers]

0 Likes

(Robert Salita) #5

Did you turn off antivirus? Are you using Windows 10 Fall Creators Edition?

0 Likes

(Debashish Panigrahi) #6

I turned off antivirus/firewall. Still the issue persists. I’m using Windows 10 Enterprise, version 1607, OS: 14393.1715.

0 Likes

(Jeremy Howard (Admin)) #7

Sounds like you can’t access the repo/mirror. Have a look at https://askubuntu.com/questions/918992/waiting-for-headers-error-while-apt-get-update

1 Like

(Robert Salita) #8

Perhaps the issue is that Anaconda isn’t supported on 14393?

1 Like

(Jeremy Howard (Admin)) #9

Thanks for this tip - I was getting stuck there too!

1 Like

(Robert Salita) #10

Happy to return the lift.

0 Likes

(Vraj Shah) #11

hey I have trouble in starting jupyter notebook. I followed jeremy but after the notebook starts , when I copy paste the URL it doesn’t open up anything. Tried several times. thanks in advance. I know this may be in wrong discussion thread still I didn’t want to create a new one.

0 Likes

(Robert Salita) #12

What happens if you use http://localhost:8888/tree

Are you on Fall Creators Update?

Does it work in any other browser?

0 Likes

(ecdrid) #13

Try
jupyter notebooks list
to see which server is currently running…

0 Likes

(Vraj Shah) #14

Yes I am on the fall creators update(Windows version 1709 and OS Build 16299.19)

If have tried http://localhost:8888/tree too.

and on chrome and edge both the browsers. It still doesn’t work.

0 Likes

(Vraj Shah) #15

Thanks. I have tried that on another terminal and it shows 1 server running(the same link is shown)

0 Likes

(Nikhil B ) #16

The notes worked quite well for me, I could get my Jupyter notebook up and running. (Enterprise 1607)

I’m facing issues when I try to install pytorch using conda install pytorch though ( tried the execstack fix and a couple of other things but i eventually see a core dump)

@jeremy Wondering if we need pytorch on our local machine for the course? ( or ssh into servers should be good enough)

0 Likes

(Jeremy Howard (Admin)) #17

No need to have it on your local machine - if you don’t have an Nvidia GPU that’s not going to help much anyway.

BTW there’s a new version of Anaconda out as of yesterday, which fixes the SSL Windows bug.

1 Like

(Pierre Guillou) #18

Hi @bsalita, thanks for your notes !

Just to add one command that was necessary for me before your point 3 (source : How to install execstack On Ubuntu 16.04 Lts?) : sudo apt-get update

So, in order to use execstack to end with the https error which was blocking additional conda installs, I typed the 3 following commands in my Ubuntu Windows 10 :

sudo apt-get update
sudo apt-get install execstack
execstack -c ~/anaconda3/lib/libcrypto.so.1.0.0
0 Likes

(Robert Salita) #19

Merci. I’ve updated the first post.

0 Likes

(Gerardo Garcia) #20

If I’m using the Windows 10 Linux Subsystem with Ubuntu
Is there any way to access the 2 1080 Ti NVIDIA cards that I have?

I’m having issues with the code to execute
I’m getting this error?

image

Should I switch to dual booth to allow me to use my cards?

0 Likes