Setting up fast.ai on a Dell XPS15 9570 with Pop!_OS

Hi folks,

I recorded my experiences setting up fast.ai on an Dell XPS15 9570 with Pop!_OS here:

Initial impression is that my laptop is a bit slower to train the resnet34 model on the cats and dogs exercise, taking 7 minutes versus the 20 seconds describing a Paperspace instance. But still not bad! Hoping to go through the course while taking occasional tea breaks, to let my laptop chew through the notebooks.

I really appreciated the discussions on setting up the fast.ai coursework on Dell XPS15 9560 laptops, so hopefully these notes help someone else to set up on their own machine.

Text of the post, in case anyone doesnā€™t want to click on an external link, and possibly to help the forum search find the relevant sentences. Iā€™ll try to keep both this forum version and my Github notes in sync.

Setting up fast.ai on a Dell XPS15 9570 with Pop!_OS

In these notes, I will attempt to recall the main points of interest
that I encountered as I worked to port over the
fast.ai paperspace setup
on a Dell XPS 15 model 9570.

Laptop Specifications

The specific model that I am using for this writeup is the XPS9570-7085SLV-PUS.
In case the internet loses all recollection of the contents of this model, it includes:

  • IntelĀ® Coreā„¢ i7-8750H Processor 2.2GHz (6 cores)
  • 32GB DDR4 2666MHz RAM
  • 1TB M.2 PCIe Solid State Drive
  • 4GB NVIDIAĀ® GeForceĀ® GTX 1050 Ti Graphics

Installing Pop!_OS as dual boot with Windows 10

I settled on using Pop!_OS as the Linux distribution.

My reasoning for this doesnā€™t really extend beyond my seeing testimonials that Pop!_OS runs on a Dell XPS15 with minimal fuss after installation, especially if the Nvidia version of their ISO images is used to have drivers right after install.
System76, the distro maintainer, also maintains their own apt installs for the NVIDIA CUDA Toolkit. I am also using the 18.04 LTS version, as I saw some complaints that Linux on XPS13/15 installs could get corrupted from upgrading between major versions.

These articles were especially helpful for getting the Pop!_OS disk image to be recognized on boot, along with installation:

Patrick Watersā€™ guide is a great overview of what one can expect when installing a Linux distro on an XPS 15.
In particular, the instructions on how to work with a Bitlocker encrypted Windows 10 install are particularly important.
Save your Bitlocker recovery keys!!! Youā€™ll type them in alot!
However, take a look at the rest of this section to see were I diverged from that overview post.

One panic surprise I encountered was that I couldnā€™t enable Windows Safe Mode after I had switched to AHCI from the BIOS menu.
Peter Pang and chenxiaolongā€™s posts point out that itā€™s easier to enable Safe Mode before enabling AHCI, otherwise Windows will not boot.
Also, you canā€™t use pin or fingerprint login while in Safe Mode, so plan how to log into Safe Mode accordingly.

I second the recommendation of using Etcher for writing a bootable USB stick.

When partitioning for my Linux install, I allocated 1GB for /boot, 32 GB for swap (to match the RAM), and the rest for my root directory.

After Pop!_OS is installed, my favored dual boot setup is to open the BIOS settings again, and to set the Windows boot as the default option. When I want to boot into Pop!_OS, I hold F12 to bring up the boot options.

I havenā€™t totally gone through Patrick Watersā€™ Post-Installation recommendations, as Pop!_OS seemed to work well out of the box. I plan to look over JackHack96ā€™s scripts later to see if there are any interesting tweaks I should pick up.

Setting up fast.ai with the Paperspace script

To set up fast.aiā€™s course repository, I started off with the paperspace script:

However, there were a few tweaks I needed to make, to get it working:

To summarize my divergence:

  • The directory of /etc/apt/apt.conf.d/*.* does not exist, so I skipped that removal step.
  • On both a Ubuntu VM and with this Pop!_OS install, I had trouble using apt install to get the NVIDIA CUDA Toolkit. Specifically I was getting the following error:
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 cuda : Depends: cuda-10-0 (>= 10.0.130) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

My workaround for this dependency error was to use Pop!_OSā€™s maintained version of the NVIDIA CUDA Toolkit:

(Also note that virtual machines do not support GPU passthrough as of this writing. Donā€™t be like me and not realize this when setting up an Ubuntu image on a VMWare player.)

Setting up Jupyter Notebook to run from localhost

To check that my setup is working, I followed the post-paperspace-setup steps from lesson 1:

The following assumes the commands are run in ~/fastai/ :

Do once in awhile:

git pull
conda env update

Because I am running jupyter from my laptop, I need to set the IP to localhost, otherwise I will get the errors:

KeyError: 'allow_remote_access'
...
ValueError: '' does not appear to be an IPv4 or IPv6 address

The solution was found in the discussion at

In a nutshell, I needed to create jupyter_notebook_config.py in ~/fastai/, and to set the IP address to localhost:

touch jupyter_notebook_config.py
jupyter notebook --generate-config jupyter_notebook_config.py ā€œc.NotebookApp.ip = ā€˜127.0.0.1ā€™ā€

Then the notebooks could be hosted locally using the command:

jupyter notebook

Sanity Check: Deep Learning 2018 Lesson 1

The first lesson could then by found at:
http://localhost:8888/notebooks/courses/dl1/lesson1.ipynb

And thankfully, the notebook indicated that cuda and cudann were both enabled (returning true)!

torch.cuda.is_available()
torch.backends.cudnn.enabled()

Because I was not using Paperspace or Crestle, I needed to download the dogs and cats dataset into the ~/fastai/data/ folder:

~/fastai/data$ wget http://files.fast.ai/data/dogscats.zip

So how long did it take to use the resnet34 model for the first dogs/cat exercise?

100%|ā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆ| 360/360 [06:26<00:00,  1.17it/s]
100%|ā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆā–ˆ| 32/32 [00:33<00:00,  1.22it/s]

Epoch
100% 2/2 [00:04<00:00, 2.25s/it]

epoch      trn_loss   val_loss   accuracy                     
    0      0.044299   0.028482   0.989     
    1      0.042639   0.026683   0.99                          

[array([0.02668]), 0.99]

6-7 minutes? Not as fast as the 20 seconds expected runtime from the lesson notes, but pretty good for a laptop!

Wrapup

That about covers all the major obstacles that traumatized me enough to remember them after the entire setup experience.
If you also have an XPS 15 and are having issues setting up a similar environment, feel free to send me a message or open an issue on Github.
I canā€™t promise that Iā€™ll get your issue resolved or that my suggestions wonā€™t nuke your existing Windows install,
but hopefully I can remember something that I forgot to put into this guide.

Best of luck on your own setup!

1 Like

Iā€™m trying to set up on a VM that has Spyder, Anaconda, and Jupyter Notebook already set up. I am not sure what application to run the paperspace set up/your updates in (video runs them in the paperspace web browser). Can you please advise?

Iā€™ve been running the commands in a normal terminal.

What VM player are you using to run the already set up VM? In my research, I didnā€™t find a VM player that supported GPU pass-through. That meant that I couldnā€™t use my discrete GPU in a VM. To work around that, I set up a dual boot partition with a Linux install (specifically Pop OS).

(Also that quote highlights a mistake on my part. I accidentally typed ā€œwgetā€ twice in my writeup. You only need to type ā€œwget http://files.fast.ai/data/dogscats.zipā€)

Ok, so then I can just use the Command Prompt? Iā€™m using a work VM so Iā€™m not sure that I can download something like Ubuntu Terminal, etc. I believe this work VM should already have GPU as required, since it is set up to run Python models.

What operating system are you using on your VM?

My VM uses Win7

While I canā€™t confirm that these instructions work 100% for all lessons, it looks like thereā€™s some existing discussion on how to work with Windows.

However, I would be concerned about whether there are any lessons that would require you have Windows Subsystem for Linux (WSL), such as using Linux utilities, as that is a Windows 10 feature.

Youā€™re also likely going to need to learn how to translate
concepts between Windows console versus Linux terminal commands, as all the lecture videos are going to assume you are on a Linux machine. eg: dir vs ls, etc

Would it be possible for your workplace to reimburse you for Paperspace time? Is this effort for official training time, or are you personal learning on company equipment?

That seems like the best approach. Paperspace is affordable enough to pay out of pocket, if I do not get a reimbursement.

Thank you for trying to help out with my set up, I appreciate it.

HI @sunzenshen . I have a 9570 as well so Iā€™m glad to see your post. However, I dual booted mine with Ubuntu 18 instead of your Pop!_OS. Do you think I can still follow your instructions to train networks locally on my laptop? Which CUDA toolkit should I download?

Iā€™m very annoyed that they just assume we donā€™t have an NVIDIA on our laptop and dont provide instructions for cases that we will have it.

What happens when you run the paperspace script?

http://files.fast.ai/setup/paperspace

Is there a point that your setup fails with those procedures?

One thing to mention in here is that I met the situation where I canā€™t download the dogcat dataset from the file.fast.ai homepage.
What i did is using the Kaggle dataset. Check the link below.
https://www.kaggle.com/takjohn/fastai-dogscats/version/1