I’ve just starting adding libvips for the C interop example, which you can see in nb sox_integration_example.ipynb. First you’ll need to install libvips, following SwiftVips/install.sh docs.
After I import SwiftVips or import vips I get the dreaded expression failed to parse, unknown error. @marcrasi any chance you could take a look since this is a blocker for this part of the next lesson? @pcuenq do you have any insights on what might be happening here?
I got a previous error while linking. String(cString: dlerror()) reports that libvips.so.42 cannot be found, but it does exist in my machine. Did you have that error?
I don’t have any errors using it from a CLI app.
Me neither. But in the notebook I get this error when running the install cell:
Install Error: dlopen error. Run `String(cString: dlerror())` to see the error message.
I didn’t see that. I’ll try updating my swift-jupyter.
I got rid of that but I still get the parse error. What I’ve seen so far is that libvips depends on glib and glib-object. I’ve tried to define explicit system library dependencies for them; or link them inside the vips module map file, but I’ve had no success so far.
Interesting. I had a similar problem when I tried to use a glib-requiring lib a while ago and didn’t use spm - instead I tried to use an objc bridging header. I never figured out how to get it working.
More specifically, this is the error message from the REPL:
Welcome to Swift version 5.0-dev (LLVM dcb9eb74a7, Clang 95cdf7c9af, Swift ec27d9b99d).
Type :help for assistance.
1>
2>
3>
4> import Glibc
5> let h = dlopen("package/.build/x86_64-unknown-linux/debug/libjupyterInstalledPackages.so", RTLD_NOW)
h: UnsafeMutableRawPointer? = (_rawValue = 0x000000000134cb70)
6> import vips
warning: Swift error in fallback scratch context: <module-includes>:1:10: note: in file included from <module-includes>
:1:
#include "/home/pedro/code/s4tf/fastai_docs/dev_swift/SwiftVips/Sources/vips/shim.h"
^
/home/pedro/code/s4tf/fastai_docs/dev_swift/SwiftVips/Sources/vips/shim.h:1:10: note: in file included from /home/pedro
/code/s4tf/fastai_docs/dev_swift/SwiftVips/Sources/vips/shim.h:1:
#include <vips/vips.h>
^
error: /usr/local/include/vips/vips.h:87:10: error: 'glib.h' file not found
#include <glib.h>
^
error: repl.swift:6:8: error: could not build C module 'vips'
import vips
^
note: This error message is displayed only once. If the error displayed above is due to conflicting search paths to Cla
ng modules in different images of the debugged executable, this can slow down debugging of Swift code significantly, si
nce a fresh Swift context has to be created every time a conflict is encountered.
expression failed to parse, unknown error
I’m guessing there are some system module maps that are not being correctly prepared in the install directory. I tried to force it by defining pkg-config dependencies for them, but it didn’t work. I’ll see if I can get some info from the build.db database, maybe we can whitelist them.
On my system that’s in /usr/include/glib-2.0. Maybe we just do a symlink to /usr/include?
Here’s how to get the info with pkg-config: pkg-config --cflags --libs glib-2.0
Well this is interesting:
$ pkg-config --cflags --libs vips
... -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include ... -lgobject-2.0 -lglib-2.0
So why aren’t those headers available? Shouldn’t they have been compiled in to the swiftmodule somehow? I don’t understand how swiftmodules work - but I thought they had the symbol info in them?..
That’s exactly what I thought, I don’t understand why the headers are needed at what should just be runtime.
More info - looking at SwiftVips/.build/debug.yaml I see:
other-args: ["-target","x86_64-unknown-linux","-swift-version","4.2","-enable-batch-mode","-index-store-path","/home/jhoward/git/fastai_docs/dev_swift/SwiftVips/ .build/x86_64-unknown-linux/debug/index/store","-sdk","/","-Onone","-g","-enable-testing","-j64","-DSWIFT_PACKAGE","-DDEBUG","-Xcc","-fmodule-map-file=/home/jhoward/ git/fastai_docs/dev_swift/SwiftVips/.build/x86_64-unknown-linux/debug/CSwiftVips.build/module.modulemap","-I","/home/jhoward/git/fastai_docs/dev_swift/SwiftVips/Sour ces/CSwiftVips/include","-Xcc","-fmodule-map-file=/home/jhoward/git/fastai_docs/dev_swift/SwiftVips/Sources/vips/module.modulemap","-I/usr/local/include","-I/usr/inc lude/glib-2.0","-I/usr/lib/x86_64-linux-gnu/glib-2.0/include","-module-cache-path","/home/jhoward/git/fastai_docs/dev_swift/SwiftVips/.build/x86_64-unknown-linux/deb ug/ModuleCache","-Xfrontend","-color-diagnostics"]
So the -I flags appear there. But no other of the -I flags from pkg-config vips are there. So it seems like these need to be part of the compilation.
I cannot believe this, but the following REPL invocation works:
swift -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
Then dlopen and import vips succeed.
Looking at ways to hack it into the kernel. (I tried to use %install-swiftpm-flags unsuccessfully).
I can’t get it to work inside the kernel.
@marcrasi I have replaced None in the following code with a list of arguments extracted from the yaml file:
self.process = self.target.LaunchSimple(None,
repl_env,
os.getcwd())
According to the source, the arguments has to be a list. Is there another way you know of to pass command-line arguments to the swift process? Or is repl_swift perhaps ignoring them?
@jeremy This ugly workaround does indeed work in my system:
cd /usr/include
for f in /usr/include/glib-2.0/*; do sudo ln -s $f .; done
for f in /usr/lib/x86_64-linux-gnu/glib-2.0/include/*; do sudo ln -s $f .; done
Nevertheless we need to figure out how to make it work inside the jupyter kernel, now that we know that there are additional arguments that need to be observed.
It is not possible to pass commandline arguments to the swift compiler in the Jupyter kernel, because it invokes the compiler stages directly without going through any commandline flag processing code. (This “LaunchSimple” method passes commandline arguments to the “Swift program” itself, not to the compiler.)
It is just a few extra -I arguments that we need? The SWIFT_IMPORT_SEARCH_PATH env variable that I recently added (https://github.com/apple/swift-lldb/commit/bdb96e8b352f7cb18e2b3b66f2d3f75d92f81dcd) does the same thing as -I, except that it works in the Jupyter kernel. Unfortunately, it can only do one and it’s already being used for package installation. But I could add support for more pretty easily if that’s all we need.
^ Those are all the thoughts that I have without having actually tried this out myself. I will try it out myself Monday morning and see if I can figure anything out!
Yes. These extra args should be figured out from SPM, or using the same underlying method SPM uses.