Making your own server

Part 1 (2017)
179

First of all, thank you for your answer.

As usual, it seems that Windows is the least recommended option for science…! So, let me redo my initial question. If I wanted to avoid Windows, could I make an Ubuntu installation under VirtualBox work with my GPU (the host system being Windows 10)? If that was not possible, could I follow the course without GPU in a VirtualBox VM with Ubuntu and with the Tensorflow CPU-only installation?

What I want, in the end, is to be able to set up and use my own GPU, as I already have it. My second option would be to know if it is feasible to follow the course with a CPU-only installation, as I’d like to avoid AWS, which would be my last option.

Thanks again for your help!

180

If you are doing part I then windows is usable. It isn’t ideal as you need to change the notebooks a little when it comes to operating system commands via the % symbol.

Part II is not possible with Windows as even more file system issues and missing core
Libraries like tensorflow and pytorch for 3.6.

You can use virtual box but using Ubuntu bash for windows would be better. Although it has some limitations and you can’t use gpu.

Windows cannot pass gpu to Linux but you can pass gpu from Linux to Windows.

The best option if you do not do a dedicated box is to dual boot.

CPU is going to be really slow, costing nearly hundred
hours or more of wasted time.

You can look at Floyd, you get enough free hours to finish the course but it is extremely difficult to use and frustrating. AWS is better choice but will cost a lot more but can be mitigated by using spot images.

2 Likes
181

Hmmm I see… well, then I’m going to try to install Ubuntu alongside Windows, I should have done it years ago. Thanks for all the help!

182

If you can get a spare SSD even a $95 Samsung EVO 250GB you can install it there and it is really easy. Recommend unplugging windows drive during the install to make things easier. You can do two things from there. Setup grub (boot loader) to configure for windows or just use bios boot selection (typically f8,f11,f12 on boot) to select which drive to boot.

183

Thanks again! I can’t afford getting a new SSD, I have already a SSD running W10 and a HDD for data, so my idea is to either make some space in the SSD for Ubuntu (but that would be tight, I don’t have much free space) or make a separate partition in the data HDD and then install Ubuntu there. In either case I’d like to set up a dual-boot with W10 booting by default.

184

@mgarrus and @Estiui here are the steps I did for my 64-bit Win7 with GTX 960 setup:

Now you want to create a GPU environment for Tensorflow and install all these packages into it (and any other packages you want to use):

  • conda create --name tensorflow-gpu python=3.5
  • activate tensorflow-gpu
  • conda install jupyter
  • conda install scipy
  • conda install matplotlib
  • conda install pandas
  • conda install scikit-learn
  • conda install numpy
  • conda install keras
  • conda install pillow
  • conda install bcolz
  • conda install h5py
  • pip install tensorflow-gpu

Note that you HAVE to use Anaconda Python 3.5… Tensorflow will only work with that specific version on Windows (not 2.7, not 3.6).

Whenever you want to use your new setup, pull up a Windows command prompt, and type this into it:

  • activate tensorflow-gpu

Change to whatever directory you want, then ‘jupyter notebook’ to start up Jupyter. It will automatically open a browser window in your default browser.

A few things to note:

  • Tensorflow uses different dimension ordering than Theano:
For 2D data (e.g. image), "tf" assumes (rows, cols, channels) while "th" assumes (channels, rows, cols).
For 3D data, "tf" assumes (conv_dim1, conv_dim2, conv_dim3, channels) while "th" assumes (channels, conv_dim1, conv_dim2, conv_dim3).
  • An alternate “quick and dirty” kludge for the class notebooks is to just specify “th” ordering in the keras.json file (in the .keras directory):

{
“image_dim_ordering”: “th”,
“epsilon”: 1e-07,
“floatx”: “float32”,
“backend”: “tensorflow”
}

  • There are a few things you need to change for Python 3.5 – such as cPickle becomes _Pickle… I don’t remember what they all were, but it something worked in 2.7 but you get an error in 3.5, that’s likely to be the problem.

I’m trying to remember if I missed anything… but this is basically what I did. I hope this helps!

Christina

8 Likes
185

One more thing I did forget… when specifying directories and files in the notebooks, you will have to use the Windows convention, not the linux, such as path = “data\\fish\” instead of path = “data/fish/”.

186

Thanks for your explanation, I’ll have to see which is the easier way to go for me :slight_smile:

187

@Christina

One thing that might make working with windows more palatable is the pathlib module in Python 3 - it reduces a lot of unnecessary text from os, os.path, glob, etc too.

E.g.


from pathlib import Path

basepath = Path('mypath')

# overloads '/' operator
train = basepath / 'train'
# can create directories without raising an error if it already exists
train.mkdir(parents=True, exist_ok=True)
assert train.exists(), 'does not exist'
assert train.is_dir(), 'not a directory'

#recursively search for files matching a pattern
file_gen = train.rglob('*.png')

filepath = next(file_gen)

#only issue is filepath is a Path object, not str
#some modules know how to deal with that
#others require str(filepath)
1 Like
188

Thanks David, great point!

189

What an incredibly useful answer, Chris. Also a fan of your username spoonerism.

I believe you’re referring to the Nvidia NVLink technology here: http://www.nvidia.com/object/nvlink.html

But, as far as I can find, the tech seems to be for servers and workstations. Did not find any information about plans for a consumer version.

1 Like
190

Yes, that’s it. I couldn’t find it quickly when I was on mobile. When I looked I couldn’t tell if it was going to be exclusive to their high end cards or not either. My guess it very well maybe. They already crippled the Titan X and 10xx cards for machine learning, so it is no surprise this would be held back from them as well.

191

:frowning: #deeplearningproblems

192

I found it annoying to have to constantly restart my jupyter notebooks on my Ubuntu server. There are a bunch of ways to do this, but using systemd on Ubuntu you can set up an “always running” jupyter workplace (e.g. a directory with all your projects). Systemd will handle restarting for you after reboots or crashes.

Create the file: /etc/systemd/system/jupyter.service

[Unit]
Description=Jupyter Workplace

[Service]
Type=simple
PIDFile=/run/jupyter.pid
ExecStart=/home/bfortuner/anaconda3/bin/jupyter-notebook --config=/home/bfortuner/.jupyter/jupyter_notebook_config.py
User=bfortuner
Group=bfortuner
WorkingDirectory=/home/bfortuner/workplace
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Then, run the commands

systemctl enable jupyter.service
systemctl daemon-reload
systemctl restart jupyter.service
14 Likes
193

With this code then after a reboot jupyter notebook will just be running on the localhost:8888? And from a remote machine I would be able to use the code you gave to forward it with ssh to my laptop?

ssh -NL 8888:localhost:8888 user@host -p portnumber -i ~/.ssh/privatekey

This is awesome. Can’t wait to set it up! Thanks @brendan.

194

I’ve set up an Ubuntu 16.04 box and configured it for access from my mac laptop via ssh. Everything works well, except sleeping/suspending. After I suspend my box I can use a magic packet to wake it from my phone or laptop either on or off my home lan. But if I wait more than about 15 or 20 minutes neither works.

Has anybody gotten wake on lan to work consistently from outside their home network?

Or should I just leave my rig running 24/7?

195

I wrote up the steps for setting up a server in Ubuntu (Windows dual boot). For the complete Ubuntu/Windows ignoramus, like I am, or was.

Perhaps someone could add it to the Wiki Installation pages?

Start----------------------------------------------------------------------------------------
Setup Ubuntu local server, for beginners (including Windows dual boot)

This guide is geared toward beginners with Ubuntu and Windows, to get you set up for the course with minimal hassles. I will try to clearly lay out every step needed, without assuming that you already have prior skills with these OS’s.

I do assume that you are starting with a PC and an nVidia GPU. This guide is current as of 18-March-2017. The Ubuntu version installed is 16.04.2 LTS. For other Ubuntu versions and and hardware, YMMV.

Making the Ubuntu Installer

In this section, we create the Ubuntu installer on a flash drive, from Windows. If you are not starting from Windows, you will need to create the installer a different way. You will need a USB flash drive, 2 GB or larger.

To create the bootable installer drive, follow the instructions at

Installing Ubuntu

You will be installing Ubuntu in its own partition, separate from the Windows partition. You may find instructions that recommend using the Windows Disk Manager to pre-allocate a partition for Ubuntu. This is neither necessary nor advised. WDM is limited in how much it can shrink the Windows partition because there are system files that cannot be moved while Windows runs. The Ubuntu installer itself can do the partitioning better than WDM because Windows is not running and can be shrunk further. Just know your ideal partition sizes before proceeding.

The Ubuntu installer advises that an internet connection will speed up the installation. Actually, a bug in the installer will cause the installation to fail partway through when there is an internet connection. So disconnect internet during the install, and don’t waste time figuring this out.

Next, boot from the installer. This will likely require entering the BIOS screens to select the USB flash drive as the boot device.

The installer will let you choose “Install them side by side, choosing between them each startup” (for dual boot), and let you size the two partitions.

Once the installer finishes, remove the flash drive and boot from the hard drive. If all has gone well, you will see a menu that allows the choice between Ubuntu and Windows.

Navigating Ubuntu

At the top of the Launcher is a search tile that allows you to find programs quickly. Terminal is one you will be using soon. Note that Ctrl-v does not work in Terminal. You have to right-click and select Paste from the menu.

The next tile below is the file browser with your Home directory as the root. At this point, go to Edit->Preferences->Views. Select “Show hidden and backup files”. You will need to edit some of these later.

System Settings (looks like a gear and wrench) gives access to many settings for personalizing Ubuntu. Be careful with the “Additional Drivers” tab. Ubuntu will offer to install a proprietary driver from nVidia. If you install it, you will be locked out of Ubuntu in a login screen loop. Instructions for escaping this situation are in the notes below.

Shutdown and reboot are found in the gear icon in the upper right corner.

Setting up the GPU and Course Material
Note: do NOT install the nVidia drivers offered in System Settings->Additional Drivers.

Fortunately, several generous experts have created a script that does the setup automatically. The script is found at

If you are easy with git, you can get the script file directly. Or click the Raw link, and using Text Editor simply copy/paste into a local file named install-GPU.sh.

Note that the script has recently been revised to install an older version of Keras, the one that the course’s code was designed for. Later you may want to update to the current Keras 2.

One small problem is that the script calls git, which is not yet installed. To install git, open a Terminal window and enter

sudo apt install git

Now it’s time to run the script. From the Terminal window, enter

bash ‘path-to/install-gpu.sh’
(You may simply type "bash ", and drag the script file to Terminal to enter its path.)

Wait while the script runs.

As part of its operations, the script will install an up-to-date nVidia driver, one that does not have the boot loop problem. At some point, Ubuntu required me to enable Insecure Boot Mode and provided the steps. I do not recall when this occurred.

Reboot now.

Open Terminal, and enter

nvidia-smi

If all has gone well, you will see the GPU and its state.

Running the Notebooks

A configuration change makes this more convenient. Open the .jupyter invisible folder found in Home, and edit jupyter_notebook_config.py. Edit the matching line to

c.NotebookApp.notebook_dir = u’courses/deeplearning1/nbs/’

This way jupyter will always open to the notebooks directory.

Next, enter in Terminal

jupyter notebook

To access the jupyter server, open a browser window and go to

http://localhost:8888

You should see a file list that contains the lesson notebooks for the course. Open one of them and you are ready to work.

Random Notes

  • Inside a jupyter notebook, use shift-enter to run a cell.

  • To monitor GPU load, enter in a second Terminal,
    watch nvidia-smi

  • Run Software Updater to get security updates that were not part of the installer.

  • To escape from the nVidia driver boot loop, at login type Ctrl + Alt + F1. At the login prompt, type your user name all lowercase, then your password. This is called TTY login.

type
sudo apt-get purge nvidia-*

and reboot.

  • To browse root level, type in Terminal

gksu nautilus

End---------------------------------------------------------------------------------------------------------------

Also, feel free to edit and correct.

On to Lesson 2!

Malcolm

2 Likes
196

So I just got my 1080 Ti and did a quick run on CatsDogs. First fit time was 190s vs 245 on my old 1080.

2 Likes
197

Good idea - and thanks! We’ll add it.

198

You should be able to do a lot better than that, I get 229s on a 1070 and when I calculated the cuda stats and terraflops I guestimated you can hit around 140s. I’m waiting for the Gamer X editions to come out with the proper cooling and higher stock clocks.