Sticking to functional programming during the API design

SwiftAI
1

Pure functional programming gives you an easy way to reason about the code but immutability consumes a lot of memory and is sometimes hard to work with. I know we are trying to stick to that paradigm as much as reasonable in python but I have a feeling that we could do a lot better in swift.

I remember a few harder to track issues caused by the mutability of learners or models. So I would love to have callbacks API that does not mutate the learner or immutable RNNs that keep state separate from the function.

Do you have other examples where mutability caused issues?

3 Likes
2

This doesn’t answer your question about “problems with mutability”, but here’s some more info about mutability in Swift, in case you’re curious. :smiley:

Swift supports value types: these include struct types, enum types, tuples, and standard library collections like Array and Dictionary. By default, value-typed function parameters cannot be mutated within the function body. To enable mutation, the parameters must be explicitly marked as inout:

struct Parameters {
    var w, b: Float
}

// Example function with `inout` value type parameter.
func update(_ params: inout Parameters, with gradients: Parameters) {
    // `params` can be mutated.
    params.w -= 0.1 * gradients.w
    params.b -= 0.1 * gradients.b
}
var params = Parameters(w: 1, b: 1)
let gradients = Parameters(w: 0.5, b: 0.5)
update(&params, with: gradients)
print(params)
// Parameters(w: 0.95, b: 0.95)

Swift advocates value types and value semantics for safety. Here’s an excerpt from a decent explanation of value types vs reference types:

The Role of Mutation in Safety

One of the primary reasons to choose value types over reference types is the ability to more easily reason about your code. If you always get a unique, copied instance, you can trust that no other part of your app is changing the data under the covers. This is especially helpful in multi-threaded environments where a different thread could alter your data out from under you. This can create nasty bugs that are extremely hard to debug.

Additionally, Swift has key paths, which are a statically-typed mechanism for referring to the properties of a type. Non-writable and writable key paths have distinct types, which is important (KeyPath<Root, Value> vs WritableKeyPath<Root, Value>).

Parameter optimization in Swift is implemented using key paths and the KeyPathIterable protocol:

struct Parameters : KeyPathIterable {
    var w, b: Float
    // Compiler synthesizes:
    // var allKeyPaths: [PartialKeyPath<Parameters>] {
    //     return [\Parameters.w, \Parameters.b]
    // }
}

// Perform update by iterating over recursively all key paths to
// `Float` members.
func update(_ params: inout Parameters, with gradients: Parameters) {
    for kp in params.recursivelyAllWritableKeyPaths(to: Float.self) {
        params[keyPath: kp] -= 0.1 * gradients[keyPath: kp]
    }
}

Here are some docs regarding parameter optimization, if you’d like to learn more:

4 Likes