Default to open APIs rather than closed

Where possible, please try to keep your APIs more open, rather than more closed, e.g.:

  • Make methods and properties public not private
  • Use var not let

We can close things later if needed - but when we try to use something that’s closed, we get blocked. For instance, lr and eps in Optimizer are let, but we need to set them, since they both can get annealed (although beta is var, which is good). init in Delegate is private so we had to subclass to use it. model.AllDifferentiableVariables is private, so we can’t do layer freezing/unfreezing. The initial version of Learner had many private properties. And so forth.

Generally, you can’t rely on your intuition to guess ahead of time what people will change. E.g. many people think of Adam.eps as a thing that’s just there for numerical stability and never needs to change. But actually you can reverse-anneal it to great effect!

Also, things that are closed are harder or impossible for students to examine and experiment with, or for researchers to tweak and improve.

I know that this is different to how things tend to be done in the wider swift world, and in your final version of s4tf once everything is working nicely, you may decide to close things then as you wish. And you’ll be able to see at that point where it’s going to cause problems, and you need to provide alternative methods, since you’ll see what breaks when you do that.

3 Likes

FYI: updating the Swift APIs for optimizers: https://github.com/tensorflow/swift-apis/pull/81

1 Like

@saeta just wanted to remind you open this “request for openness”. We just tried to use the new resetToZero method and found it explicitly marked fileprivate. Obviously it’s easy for us to recreate it (which we’re doing), but it would be much better if we could all use the helpful code that the s4tf team is creating! :slight_smile:

1 Like