Following Jeremy’s Response to Karpathy’s tweet a few months ago: https://twitter.com/jeremyphoward/status/1528992716407066624?s=20&t=vdL9aMUgTeHDUK0eQCe9qA,
I got curious about people’s approach to creating training scripts after you are done experimenting in a notebook.
So, during the course, Jeremy teaches in notebooks which are very convenient in experimentations and trying things out like different hyperparameters and ways to improve a model.
When I started working at my current company and also going through open source projects on github, I often notice people having a file like traininig.py
which they conveniently just call using:
python training.py --epochs 5
While I do not wish to bring up the hot debate of Notebooks in Production, I was wondering how other people approach this. If I am done experimenting and I have a working combination of a model and hyperparameters and I, for example, want to train the model for a couple of days and leave it running, how should I approach this, do you still use your notebooks and leave them running for all this time or develop a script.
In the response, Jeremy mentions two fastai projects: fastscript and fastgpu which I am currently looking at to see if they will fit my use case. If anyone has an example of how to combine both of these two projects, this would help.
My current approach is to use nbdev + metaflow like this example repo here: GitHub - jimmiemunyi/nbdev-metaflow-example: An example nbdev-metaflow workflow integration but it is very clunky and hacky as I described in this issue here: Custom Directives and Example: Integration with MetaFlow · Issue #1019 · fastai/nbdev · GitHub