Fastai on WSL 2 Ubuntu | 0.7.0 or any version

Intro to Machine Learning (2018)
1

Hello guys.

What is WSL you wonder ? It’s Windows Subsystem for Linux. Basically, you run a Linux OS inside windows.
WSL 1 was good, but it had slow read write speeds. Because WSL 2 is basically a VM, it’s much better, other than it’s dynamic IP address that changes every time you close the terminal making some problems with the SSH.

If you are planning to use GPU for Deep learning, please note that:

Its experimental, but here is another thread where they are trying to get CUDA drivers to work on WSL 2: Platform: Windows 10 using WSL2 w/GPU

This is going to be split in 4 parts.

  1. Install/Enable WSL
  2. Install Miniconda
  3. Setup the environment
  4. Setup SSH

1. Install/Enable WSL

To download WSL 2, you need Windows 10 version 2004(Build 19041) or higher. Windows 10 2004 was launched around June 2020. If you have yet to update, get the windows 10 update assistant here https://www.microsoft.com/en-us/software-download/windows10 if you have not updated already.

Here is the Microsoft guide to install WSL 2: https://docs.microsoft.com/en-us/windows/wsl/install-win10

After you install your linux distribution of choice, set your username password. My choice would be the Ubuntu 18.04 LTS or the Ubuntu 20.04 LTS

2. Install Miniconda

Now download and install Miniconda.

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

Full Miniconda Linux Installation guide here: https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html#

3. Setup the environment

Grab the conda cheat sheet here to check stuff when you get stuck.

Open the WSL terminal. Clone the fastai repo.

git clone https://github.com/fastai/fastai
cd fastai

Depending on whether you want to use the GPU environment or the CPU environment, do either:

conda env create -f environment.yml # GPU
conda env create -f environment-cpu.yml # CPU

Now you will have all the dependencies installed except the fastai 0.7.0 library.
So first, activate the conda environment:

conda activate fastai # If you installed GPU environment
conda activate fastai-cpu # If you installed CPU environment

Now, you change directory to the old folder which has fastai 0.7.0. Install fastai and jupyter lab:

cd old
pip install .
pip install jupyterlab

You need to install jupyter lab to view the notebooks and stuff. You can use jupyter too. Just my preference.

After this, I normally add a line in the .bashrc file in the home directory to activate the environment on launch. This is because I use WSL exclusively for the fastai learning stuff. If you use WSL for multiple things, I do not recommend this. Like this:

echo 'conda activate fastai' >> ~/.bashrc
# use fastai for GPU env, fastai-cpu for cpu env

If you are going to be using the computer, I think you are pretty much ready at this point. Otherwise, you might want to setup remote access using SSH if you access your home computer from your laptop on local network like me.

4. Setup SSH

This part is a bit unorganized. I don’t have time to fix it, but I’d appreciate if anybody can sort this out. This is a wiki post.

Okay, the problem comes here. WSL 1 has a static IP address, so you can just login anytime if you remember the IP address.
But WSL 2 has a dynamic IP. So if you close the terminal anytime, the session ends. And you open the terminal again and try to SSH but it does not work because it has a different IP.
What ? Yes, that is the problem
So I either, first open a terminal on the WSL machine and SSH into the Windows machine, get the WSL IP, set up IP forwarding

So I set this up some time back and had to refer some websites. I had them bookmarked then. I paste them here, for others to refer to. I don’t remember the particular order.

  1. SSH into Windows

  2. Key-based Authentication for OpenSSH on Windows - Concurrency

  3. How to use Windows 10’s built-in OpenSSH to automatically SSH into a remote Linux machine - Scott Hanselman

  4. How to Install PowerShell 7 on Windows 10

  5. SSH on Windows Subsystem for Linux (WSL) | Illuminia Studios

  6. A step by step tutorial on how to automatically start ssh server on boot on the Windows Subsystem for Linux running Ubuntu 18.04

After this, you’ll need to use SSH tunnelling to acccess your jupyter notebook running on WSL from another computer. This guide looks fine:https://medium.com/@apbetahouse45/how-to-run-jupyter-notebooks-on-remote-server-part-1-ssh-a2be0232c533

For people wanting to know the status of the WSL 2 Dynamic IP issue on GitHub, you can check it here: https://github.com/microsoft/WSL/issues/4150#issuecomment-504209723

Also, please check the ML lesson 1 Wiki for some stuff you might need : Wiki thread: lesson 1

7 Likes