Installing Tensorflow, Python 3, and friends
So far we’ve been using Python 2 with Theano as our backend for keras. We need to install Python 3 and Tensorflow for part 2 of our course, and make them our default python and keras backend. The approach we’ll be using here is to update our path and keras config so that python 3 and the tensorflow backend will be used from now on - even for existing projects (so you may need to make some changes to existing projects to make them continue to work). If you’d prefer to be able to easily switch back and forth between python versions, you should follow these tips instead.
We could, of course, have simply provided a new AMI so you wouldn’t have to worry about any of this, but we decided on the manual root because:
- We don’t want you to lose your existing work, and
- We think it’s important to learn how to manage your server yourself, so now’s a good time to start if you haven’t done this before - just reply below if you have any questions or issues!
To start, ssh into your p2 instance as we’ve done before. First we’re going to update our Linux libraries and drivers (most importantly, there’s a new Nvidia driver and CUDA version that this will install):
sudo apt update
sudo apt upgrade
Next we’re going to download and install Anaconda’s new python 3.6 distribution:
cd
cd /Downloads
wget https://repo.continuum.io/archive/Anaconda3-4.3.0-Linux-x86_64.sh
bash Anaconda3-4.3.0-Linux-x86_64.sh -b
Next we need to replace our path in our bash configuration file to point to Anaconda 3:
cd
vim .bashrc
Once vim is open, replace the anaconda 2 path with anaconda 3 by simply changing the 2 to 3 (it’ll probably be the last line of the file). As a reminder, press ‘i’ to enter insert (editing) mode, Esc when done editing, and ‘:wq’ to write the file and exit the editor. Next, reboot the instance to ensure the new drivers are loaded:
sudo shutdown -r now
…and ssh back in. Now we’re going to download and install the new version of cudnn:
cd downloads/
wget http://files.fast.ai/files/cudnn-8.0-linux-x64-v5.1.tgz
tar -zxf cudnn-8.0-linux-x64-v5.1.tgz
cd cuda/
sudo cp lib64/* /usr/local/cuda/lib64/
sudo cp include/* /usr/local/cuda/include/
Next we’re going to install the latest versions of tensorflow, bcolz, and keras as well as update all our conda packages.
pip install tensorflow-gpu
conda install bcolz
conda update --all
pip install git+git://github.com/fchollet/keras.git
Now we need to configure keras to use tensorflow as opposed to theano. This can be done with:
echo '{
"image_dim_ordering": "tf",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "tensorflow"
}' > ~/.keras/keras.json
To test our configuration, launch ipython and test importing tensorflow and keras successfully:
ipython
import tensorflow
import keras
Exit with Ctrl+d. Now we’re going to create a unique password for our notebooks, so that we’re not all using the same password like we were in part 1!
python -c "from notebook.auth import passwd; print(passwd())"
Enter a password on prompt, and copy the output.
Next open up the jupyter notebook configuration file:
cd
vim .jupyter/jupyter_notebook_config.py
Scroll to the bottom and replace the previous password with the output you generated in the previous step. Finally we’re going to configure extensions and launch our notebook:
pip install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user
jupyter nbextensions_configurator enable --user
cd nbs
jupyter notebook
Go to the appropriate port and test your new password, as well as testing tensorflow and keras. If everything works, you’re good to go!
