Lesson 13 (2019) discussion and wiki

maxim.pechyonkin · April 24, 2019, 1:19am

Last time I checked, it wasn’t possible to choose Swift in Colab drop down menu, the only way to get it in Colab was to clone an existing Swift Colab notebook. Is that going to be fixed?

saeta · April 24, 2019, 1:20am

Great question! This is something we will fix soon.

PierreO · April 24, 2019, 1:33am

I’m curious, how many people are working full time on S4TF at Google right now?

saeta · April 24, 2019, 1:37am

We have about 8 people on my team, but there are a number of very close friends across Alphabet.

deena-b · April 24, 2019, 1:37am

A follow-up question from Jeremy’s comments at the end of last class:

Why has fast.ai chosen Swift over Julia??

hiromi · April 24, 2019, 1:44am

sgugger · April 24, 2019, 1:47am

Here is the blog post Jeremy is mentioning.

miwojc · April 24, 2019, 1:48am

i get the impression v4 of the course (or 2020 version) will be in Swift 4 TF…

PierreO · April 24, 2019, 1:50am

How long will fastai support PyTorch?

harikrishnanrajeev · April 24, 2019, 1:50am

google needs it … its says all … so Swift is the way forward

sgugger · April 24, 2019, 1:50am

There is no short term plan to discontinue support for PyTorch. The idea is to have fastai sitting on different languages.

BrianHenryIE · April 24, 2019, 1:51am

Google have a long write up on why they chose it:

github.com

tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md

# Why *Swift* for TensorFlow?

* Date: April 2018

The core [graph program extraction algorithm](GraphProgramExtraction.md), [automatic differentiation](AutomaticDifferentiation.md), and [Python language interoperability](PythonInteroperability.md) features of Swift for TensorFlow can be implemented for other programming languages, and we are occasionally asked why we didn’t use some other one for this project.

The engineers on the project were previously familiar with Swift (and several other languages), but the choice was guided by the goals of our project, which imposed specific technical requirements (explained below).  This choice was also discussed extensively, debated with coworkers and other interested engineers, and we concluded that Swift was the best direction.  In this document we're sharing our deliberation process with the community to help explain our decisions.

That said, while our choice of language was guided by our specific project goals, we would love to see wider application of these techniques and ideas in the context of other programming languages!  If you are interested in pursuing a similar project, please reach out to us and we will happily share our expertise.

### How we got here

As discussed in the [design overview document](DesignOverview.md) our project goal is to improve usability of TensorFlow.  We quickly realized that our core [static analysis-based Graph Program Extraction algorithm](GraphProgramExtraction.md) would not work well for Python given its highly dynamic nature.  This led us down the path of having to pick another language to work with, and we wanted to approach this methodically.  As such, we defined goals for the project, explored which properties of a programming language are important to achieve those goals, and then evaluated a lot of languages against these properties.  You already know the outcome--we eventually settled on Swift.

Below we explain [our project goals](#project-goals), discuss the [programming language properties](#properties-of-programming-languages) that contribute to these goals, provide a short [evaluation of specific languages](#which-languages-fit-our-project-requirements) against those goals, and discuss the [pros and cons of Swift](#evaluating-swift-against-our-language-properties) specifically.

## Project goals

TensorFlow is a world-class machine learning framework used by a wide range of different people for lots of different things.  Our project goal is to provide a new interface to TensorFlow that builds on TensorFlow’s power and capabilities, while taking its usability to a whole new level.  Our aim is to make machine learning researchers (both theoretical and applied), production engineers deploying at scale, and anyone else using TensorFlow more productive and joyful without sacrificing anything that already makes TensorFlow great.

This file has been truncated. show original

magiclantern · April 24, 2019, 1:51am

From the class lecture:

Julia is too mature; Swift is new and maleable

S4TF is support by Google, so they have to make sure it works.

Jeremy likes the direction S4TF is going better than the direction Julia is going

miwojc · April 24, 2019, 1:53am

Will S4TF support other than Nvidia GPUs?
like AMD Radeon 7, which has 2x the memory of 2080 which costs similar money.

PierreO · April 24, 2019, 1:53am

Do you already see other big OS projects / groups like Google start to work on Swift / S4TF as well? Or are Google and fastai the only ones right now?

rachel · April 24, 2019, 1:55am

Here is Jeremy’s post that Chris just quoted: https://www.fast.ai/2019/01/10/swift-numerics/

saeta · April 24, 2019, 1:56am

S4TF is designed to support many types of accelerators, including NVIDIA GPUs, Cloud TPUs, and more. Additional platforms in the short term will come from hardware support added to compiler technologies like: XLA (in the short term) and MLIR (in the long term).

harikrishnanrajeev · April 24, 2019, 1:57am

How long do you think will take for “swift” be ready for use for real production projects ?

ThomM · April 24, 2019, 1:57am

What’s the roadmap/relationship for S4TF regarding mainstream Swift? Is there a world in which S4TF is just Swift in a few years?

radikubwa · April 24, 2019, 1:58am

Maybe it’s me but swift reminds me of scala, R and python. ALL TOGETHER! Woah!