I’m trying something a bit different in terms of software development process - here’s my attempt to document it.
What we’re doing is to create notebooks which start with a fairly specific solution to a particular problem, and then we show how to gradually refactor and generalize that solution. At the end of the notebook, we hope to have a solution to the class of problems of which the initial notebook section showed one specific instance. Also, the last section of the notebook should show that different types of model/dataset/etc can be tackled with only a small amount of simple code for the user.
Then when that all looks pretty good, I download the notebook as a python file, remove all the code that isn’t necessary for the final approach, and save that as a module. The notebook names start with a number (eg
002), and the python module created from that notebook is then named eg
When I start the next notebook (eg 003), I start by importing everything from (eg)
The net result of all this is that we build up over a few notebooks a complete solution to some problem (such as “create a training loop”, or “do data augmentation”). Once we’ve got a good solution to a whole problem domain (like computer vision), we can combine the various modules build along the way into a well-designed set of one or more modules, which we can then add tests and docs to.
Some reasons for this approach include:
- I work best in an interactive notebook environment
- We can include pictures, tables, etc so that people understand exactly what’s going on, and we can have visual “tests” of correctness
- I’m planning for the next version of fast.ai part 2 to be a “bottom up” approach, where students learn how fastai is made and why it’s made that way. The notebooks we create as we build fastai_v1 will form the basis of this.