Up & Running Swift for TensorFlow with Jupyter on macOS

(Sina Moeini) #1

Hello everyone,

As you all know Swift for TensorFlow (S4TF) is at its early stages and note fully developed. However, I would love to start using it locally on my mac and get more familiar with syntax and etc. For the last couple of days I struggled to install S4TF alongside jupyter. It took me a while to figure out some hackery to do it. I would like to share it with anyone who has the same issue.

First note that there two repositories on GitHub for S4TF, one maintained by apple as dev branch of Swift:
https://github.com/apple/swift/tree/tensorflow
And the other maintained by TF team:
https://github.com/tensorflow/swift

Installation page on the later provides links to binary releases for S4TF. Using any of the installers marked Xcode, will successfully install S4TF on your mac. However, not all of them are a suitable candidate for jupyter. As of the time of writing this forum the stable release is incompatible with jupyter. For it to be compatible the lldb python package in S4TF toolchain must be configured with python3. This is something that they have considered in the later releases. There is an easy way to check, install one of the release listed in installation and then in terminal execute:

lldb -P

the result will look something like:

/Library/Developer/Toolchains/swift-tensorflow-DEVELOPMENT-2019-08-28-a.xctoolchain/System/Library/PrivateFrameworks/LLDB.framework/Resources/Python

This is the path for lldb python library. Navigate to the library directory by executing:

cd  `lldb -P`/lldb

Now you should make sure that the file _lldb.so is linked to built for python 3.x. Simply

otool -L _lldb.so | grep Python

The output will tell you if lldb is configured for python version 3.x. If that is not the case, you first must uninstall the toolchain by launching Xcode. Click on “Xcode” in menu bar, then select “Preferences…”. Click on “Components”. Right click on the swift toolchain and then hit “move to trash”. Look for another version on installation page and repeat this process until you find one that is configured for python 3.x.

When you find a version that supports python 3.x most likely it is linked to a non-existent file. In my case “_lldb.so” was linked to

/usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/Python (compatibility version 3.7.0, current version 3.7.0)

But fear not! This can be resolved. Find the equivalent file in your python 3 installation and then use install_name_tool command to right this wrong:

sudo install_name_tool -change <wrong_path> <right_path> _lldb.so

For example:

sudo install_name_tool -change /usr/local/opt/python/Frameworks/Python.framework/Versions/3.7/Python /Library/Frameworks/Python.framework/Versions/3.7/Python _lldb.so

Finally let’s make sure that lldb can be loaded in to python 3.

export PYTHONPATH=$(lldb -P):$PYTHONPATH
python3 -c "import lldb"

If this runs without any errors you are good and ready to install S4TF jupyter kernel (assuming that jupyter is already installed on your mac).

In a directory of your choice

git clone https://github.com/google/swift-jupyter.git
cd swift-jupyter
python3 -m venv venv
. venv/bin/activate
pip install -r requirements.txt
python register.py --sys-prefix --swift-toolchain /Library/Developer/Toolchains/swift-latest

Finally run Jupyter

. venv/bin/activate
jupyter notebook

But wait it is not over! Create a new swift notebook and try to execute a line of swift to see if the installation was successful. In my case it was not. Checking the terminal I saw that the error “Exception: Could not start debugger” coming from file swift_kernel.py. After doing some digging I realized that this is probably a bug so I edited the file swift_kernel.py and commented out the following two lines:

    #if not self.debugger:
    #    raise Exception('Could not start debugger')

run jupyter again:

jupyter notebook

And voila we have S4TF running in jupyter on mac.

2 Likes

(Robin) #2

Did you discover if there is a Docker container that will expose Jupyterlab with Swift?

0 Likes

(Vova Manannikov) #3
1 Like