My primary place for deep learning experiments & development is Colab. It’d have been a great trouble if I was willing to build a library and
nbdev wasn’t there. It has augmented my productivity by a lot. I’m currently working on a project called
fast_impl which is the collection Deep Learning applications and implementation of papers; I’m building a utility library on top of fastai2 along the way.
These are some of the features I’d like to see in the
nbdev. Comments and criticism is welcome
Support for notebooks in sub-directories
You can look at the current hierarchy of my repository. A better example could be http://d2l.ai 's hierarchy.
nbdevwas meant to enable development through notebooks and organizing them in sub-directories is always a good idea when no. of notebooks grow tremendously. This also enables keeping associated files in that directory, for instance a ‘.csv’ file, images that are related to a group of notebooks only.
Ability to maintain separate README for PyPi package and the repository
I’m in the process of implementing research papers, tutorials etc. and I want to maintain a comprehensive README that will help you navigating the repository, but I don’t want to include all these details on PyPi description.
Update index.ipynb ← README.md
There’s a script to update notebooks from source code (
nbdev_build_libif I’m not wrong), I’m also looking doing the same with markdown files. Personally, I like to use separate editor for markdown files, but updating a README with this doesn’t change the index.ipynb, rather overwrites the README when I run
(Consider the following as secondary requirements and putting them here just for the discussion)
Automated symlinks in the subdirectories
When sub-directories will be supported, the next challenge would be the need to create symlinks to library in each and every-folder. Also, I’m not sure if images from the subdirectories will get copied to the docs folder and if it seems easier to copy them from one place, we need a symlink for ‘images’ folder as well. A handy
nbdev_scriptfor this would be appreciated!
I’ve seen repositories maintaining
__init__.pyto enable “import *” but I’m not sure why fastai2 uses
all.py. If this is the necessity, can this be automated ? Like generating
all.pyfor your library and a tag to exclude any module you don’t want to be imported with that.
Cleanup certain cells from notebook itself (for colab)
Working on a colab notebook and using that with git repository is already a quite tedious process. I’m not complaining about that, but there’re some shell commands, scripts, pip installs that I need to do every time before starting to work. I need to maintain those cells on colab notebook, but are not required in the notebook that I’ll put on git repository, so every time when I download that notebook from colab to upload on github, I need to open that notebook, remove certain cells and then commit the changes. I’m aware of
#hidetag that skip such cells for docs, but I need them to be removed from notebook itself