More specifically, this is the error message from the REPL:
Welcome to Swift version 5.0-dev (LLVM dcb9eb74a7, Clang 95cdf7c9af, Swift ec27d9b99d).
Type :help for assistance.
1>
2>
3>
4> import Glibc
5> let h = dlopen("package/.build/x86_64-unknown-linux/debug/libjupyterInstalledPackages.so", RTLD_NOW)
h: UnsafeMutableRawPointer? = (_rawValue = 0x000000000134cb70)
6> import vips
warning: Swift error in fallback scratch context: <module-includes>:1:10: note: in file included from <module-includes>
:1:
#include "/home/pedro/code/s4tf/fastai_docs/dev_swift/SwiftVips/Sources/vips/shim.h"
^
/home/pedro/code/s4tf/fastai_docs/dev_swift/SwiftVips/Sources/vips/shim.h:1:10: note: in file included from /home/pedro
/code/s4tf/fastai_docs/dev_swift/SwiftVips/Sources/vips/shim.h:1:
#include <vips/vips.h>
^
error: /usr/local/include/vips/vips.h:87:10: error: 'glib.h' file not found
#include <glib.h>
^
error: repl.swift:6:8: error: could not build C module 'vips'
import vips
^
note: This error message is displayed only once. If the error displayed above is due to conflicting search paths to Cla
ng modules in different images of the debugged executable, this can slow down debugging of Swift code significantly, si
nce a fresh Swift context has to be created every time a conflict is encountered.
expression failed to parse, unknown error
I’m guessing there are some system module maps that are not being correctly prepared in the install directory. I tried to force it by defining pkg-config dependencies for them, but it didn’t work. I’ll see if I can get some info from the build.db database, maybe we can whitelist them.
So why aren’t those headers available? Shouldn’t they have been compiled in to the swiftmodule somehow? I don’t understand how swiftmodules work - but I thought they had the symbol info in them?..
According to the source, the arguments has to be a list. Is there another way you know of to pass command-line arguments to the swift process? Or is repl_swift perhaps ignoring them?
@jeremy This ugly workaround does indeed work in my system:
cd /usr/include
for f in /usr/include/glib-2.0/*; do sudo ln -s $f .; done
for f in /usr/lib/x86_64-linux-gnu/glib-2.0/include/*; do sudo ln -s $f .; done
Nevertheless we need to figure out how to make it work inside the jupyter kernel, now that we know that there are additional arguments that need to be observed.
It is not possible to pass commandline arguments to the swift compiler in the Jupyter kernel, because it invokes the compiler stages directly without going through any commandline flag processing code. (This “LaunchSimple” method passes commandline arguments to the “Swift program” itself, not to the compiler.)
It is just a few extra -I arguments that we need? The SWIFT_IMPORT_SEARCH_PATH env variable that I recently added (https://github.com/apple/swift-lldb/commit/bdb96e8b352f7cb18e2b3b66f2d3f75d92f81dcd) does the same thing as -I, except that it works in the Jupyter kernel. Unfortunately, it can only do one and it’s already being used for package installation. But I could add support for more pretty easily if that’s all we need.
^ Those are all the thoughts that I have without having actually tried this out myself. I will try it out myself Monday morning and see if I can figure anything out!
From my first glance, I don’t see any easy/obvious ways to extract the extra args from SPM. I’m sure it’s possible in the longer term, maybe by adding some features to SPM. But for a quick fix, I propose that we just run pkg-config ourselves.
There’s another obstacle that I do not know how to add extra include search paths after the kernel starts running. We can work around this by making symlinks from /tmp/xyzxyzxyz/swift-install/modules to all the necessary header files.
I propose combining all these workarounds into a swift-jupyter feature that you use like this:
This feature will run the specified commands, extract the -I flags from them, and create links from the kernel’s “install location” to all the headers.
Check out that branch and add %install-extra-include-command pkg-config --cflags vips to your install cell.
I’m going to test it out on a few more notebooks and systems and add some docs in the readme. Then I will merge it.
By the way, I had to do some extra things not included in the install instructions to get SwiftVips working:
sudo ldconfig
conda install harfbuzz pango. The versions of these libs that were on my Ubuntu system had missing symbols. I’m guessing that conda installs newer versions that have more symbols, though I haven’t looked closely and compared versions and understood what’s actually going on.
FYI, I think @marcrasi’s change has been splatted in all the merge madness. I had to add the extra %install-extra-include-command pkg-config --cflags vips line to the 1st cell, and also include an import TensorFlow in the 2nd cell, to get the nb to run.
Every time running autogen.sh it got it’s panties in a twist and would not generate the deleted files on a remake. The files ltmain.sh and Makefile.in were missing ( not regenerated) by the libtoolize tool.
So my fix was once I had generated the configure file which autogen.sh thankfully did was to run individually from the command prompt :-
libtoolize --copy --force --automake automake --add-missing --copy ./configure make sudo make install
The intuition I got for this was from a 2015 issue #305