Has anyone had sucess building a full stack deeplearning based app/webapp in just(or mostly) swift yet?
1 Like
Sure, but not with Swift for TensorFlow. 
A few people have built their own neural networking libraries in Swift from scratch, including training etc. I do deep learning in Swift for a living, but I’m using low-level GPU libraries and writing GPU shaders by hand etc. The nice thing about S4TF (if it lives up to the hype) is that it will make a lot of that stuff easier. 
9 Likes
As Matthijs points out, there are a number of examples of Swift-based applications using trained networks for inference. These are usually aimed at iOS, given the head start that platform had with Swift and its popularity. The iOS ones tend to use Core ML for deployment, although a few of us have built our own inference frameworks using Metal or OpenGL. Swift for TensorFlow doesn’t yet target iOS, so other solutions are used for deployment.
Matthijs might be too modest to point to his own work in this area, so I’ll highly recommend his articles on the subject which contain some of the best overviews of iOS-based neural network inference (and relevant Swift code). His treatise on one-shot object detectors alone will keep you occupied for an afternoon.
In partnership with local agricultural experts, I built and released a Swift-based iOS application that performs live object detection to diagnose common Midwestern crop diseases (blog post about the development process). It uses a custom Swift inference framework I built using Metal and OpenGL, but the image classification and object detection network training was done in Caffe. Since then, we’ve been slowly transitioning our training pipeline from Caffe to TensorFlow 2.0 (Python).
Ideally, it’ll be great when the training piece can be handled in Swift for TensorFlow to provide an end-to-end process in Swift. There’s work to be done to make that happen, but as you can see from the forums and elsewhere rapid progress is being made towards that.
13 Likes
Wow!!! You are both doing some truly amazing work. Your app is truely outstanding! Your website, logo, and video demo are exactly what I thought every app should do. Well done!!!
I love this idea; so much that I started tinkering with it a couple of days ago 
I figure we should be able to make the bear detector web app from lesson 2 (?) pretty easily entirely in Swift. I’ve just been playing with Vapor to get a basic web app up and running. In theory it should be pretty straightforward to show the whole pipeline from training to “production” in one language in a couple of modules.
Also, +1 to @machinethink’s work; I’ve bought both his books about CoreML (the ray wenderlich one and his own one) and found them both very useful in learning CoreML. (Thanks for your great work and community efforts Matthijs!)
5 Likes
+1, also playing with Vapor As I understand it doesn’t support Swift v5 yet (let alone s4tf). E.g. default project generated by vapor cli doesn’t compile because of sqlite dependency. So far I’ve managed to fix swift-jupyter so Vapor can be installed there, so I can play with it in the notebook (which is silly I know :)).
3 Likes
Hi!
I’ve got working example of vapor serving S4TF model via HTTP, and it’s running on heroku!
Please read my post here if you’re interested:
https://groups.google.com/a/tensorflow.org/forum/?nomobile=true#!topic/swift/yTi73Q9XCUI
The model is separate swiftpm package and weights are stored as TF checkpoint. It’s compiled together with vapor and weights are loaded on start up.
Would love to hear your thoughts,
Thanks!
9 Likes