Lesson 14 (2019) discussion and wiki

Lesson resources

Software requirements

  • You need to install swift for TensorFlow and swift-jupyter, see Jeremy’s install guide.
  • s4tf download
  • For help installing s4tf, please ask on the above thread. Don’t ask install questions in this lesson discussion thread please!

Links

Notes

13 Likes

Thank you a lot Chris, Jeremy and Rachel! I’m sure we all are extremely grateful to you for this amazing opportunity!

15 Likes

And there is enough study material for the year ahead.

3 Likes

WOW amazing C integration, we wrangle with C all the time in genomics for speed ups

9 Likes

Go Sox! (says guy from Boston)

4 Likes

One language to rule them all :wink:

6 Likes

Using Swift to glue together C and Python libraries sounds crazy and awesome!

7 Likes

Is it possible to achieve similar results with things like Cython? Though it probably more difficult and require some manual wrapping.

2 Likes

Aren’t the #if lines being interpreted by the specific compiler? Would the C library need to be written to be compiled by a certain compiler?

1 Like

The #if macros in the C-header files are preprocessed with Clang, LLVM’s C-language-family frontend.

So would Sox in this example have to be Clang compatible?

Yes, that’s exactly right. It needs to be compilable with Clang. Fortunately, Clang is pretty good. :slight_smile:

1 Like

Link to the Skip the FFI: Embedding Clang for C Interoperability lecture slides Chris was talking about.

Does anyone know if this has a video lecture somewhere?

Are the c libraries dynamically linked or statically linked .
or it is actually compiled from source to be available in swift.

2 Likes

Is it possible to do all this C-related imports without knowing a lot of C?

2 Likes

My impression is that almost all of the examples of the (future) power of Swift seem to rely not on the abstraction to higher levels, but on the diving into lower level details.

As a data scientist I try to avoid doing this. I only go low if I know there’s a big performance gain to be had.
Does Swift maybe help motivate me to dig into the weeds?
Does it help me recognize opportunities to improve my code that demand that I go low?

And, actually, all the discussion of search and genetic algorithms and differentiable programming suggests that the ideal is that I shouldn’t have to become expert in the details of multiple languages.

So…will adopting Swift make it necessary for me to learn more about the decades of C code that is available (on top of 100+ deep learning papers published every day).
Will it help me to make substantial improvements despite my (vast) ignorance?

Probably just showing my ignorance here, but I’m struggling to see how this makes my life easier or more effective…

13 Likes

You’re not alone, I personally feel myself being a bit overwhelmed with Swift/C/C++/Python/PyTorch/TF/S4TF/etc. too :smile:

15 Likes

Not much C knowledge is needed at all!

Here’s a nice guide for importing a C library in Swift.

As an example, to import the C library libcmark, all you need is:

  1. A single-line C shim header (#include <cmark.h>).
  2. A modulemap file.
module Ccmark [system] {
    header "shim.h"
    link "cmark"
    export *
}

Then, you can import Ccmark in Swift!

import Ccmark

let markdown = "*Hello World*"
let cString = cmark_markdown_to_html(markdown, markdown.utf8.count, 0)!
defer {
    free(cString)
}
let html = String(cString: cString)
print(html)
17 Likes

This is a fantastic question! The power of Swift is that because as little as possible is built into the language (or the Swift for TensorFlow framework), this means that you can build or mix-and-match your use of libraries yourself, instead of relying on a pre-built “framework”.

The key advantage of this approach is that you can then use the best tool for the job, instead of being forced to use what’s currently available. While I expect we’ll have common sets of libraries that are designed to work well together, if you’re blazing the trail on a new domain, you aren’t stuck!

10 Likes