nbdev is a great tool.
However some of you may not be comfortable with the default fastai codestyle, or maybe your team is already using a different one.
This post will show you how to integrate your nbdev workflows with industry standard tools like:
- black , The uncompromising code formatter
- isort , A Python utility / library to sort imports.
- mypy , Optional static typing for Python
- pre-commit , A framework for managing and maintaining multi-language pre-commit hooks.
An astute reader may notice that one can
black_formatting = True in
- this will only format the code in the module, not the code in the notebook.
- this does not allow you to use other tools like
use nbqa to format and verify code in your notebooks
nbqa is a tool that allows you to run any tool (formatter/linter) on your notebooks, without having to worry about the details.
Black is a python code formatter that is becoming the de-facto standard
pip install nbqa nbqa black nbs/
isort will sort your imports, and mypy will check your type annotations
pip install isort mypy nbqa isort nbs/ nbqa mypy nbs/
But you are already using nbdev command-line arguments like
nbdev_test, who wants to remember more commands?
Make is a great tool for automating your workflows, usually comes pre-installed on most unix systems.
Makefile in the root of your project and add the following lines:
# Makefile format: nbqa black nbs/ nbqa isort nbs/ nbqa mypy nbs/
Now you can run
make format to format your notebooks.
Let’s add more commands to our
format: nbqa black nbs/ nbqa isort nbs/ nbqa mypy nbs/ export: nbdev_export mypy: nbqa mypy nbs/ --ignore-missing-imports --check-untyped-defs test: nbdev_test --n_workers 4 prepare: # Export, test, and clean notebooks, and render README if needed nbdev_prepare docs: nbdev_docs
And now the cherry on top: Makefile supports dependecies, so we can chain commands together.
add this to your
all: format prepare mypy
make all will format your notebooks, export them, run mypy, run tests and update README.md
Want to automate your formatting even more?
Force your team to format their notebooks before committing?
pip install pre-commit # install pre-commit install # enable for this repo
repos: - repo: https://github.com/fastai/nbdev rev: 2.2.10 hooks: - id: nbdev_clean - id: nbdev_export
Add nbqa hooks
- repo: https://github.com/nbQA-dev/nbQA rev: 1.5.3 hooks: - id: nbqa-mypy args: ["--ignore-missing-imports", "--check-untyped-defs"] additional_dependencies: ["mypy", "types-toml", "types-requests"] - id: nbqa-black - id: nbqa-isort
control your hooks behaviour
- repo: https://github.com/nbQA-dev/nbQA rev: 1.5.3 hooks: - id: nbqa-mypy args: ["--ignore-missing-imports", "--check-untyped-defs"] # pass args to mypy exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting additional_dependencies: ["mypy", "types-toml", "types-requests"] # install additional dependencies - id: nbqa-black exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting - id: nbqa-isort exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting
Sometimes you may run into reproducibility issues, where your notebook works on your machine, but not on your team’s.
The best way to avoid it is to use a dependency manager, such as:
But how do you integrate it with nbdev, since it uses
settings.ini to manage dependencies?
In this post I show an example of how to use Pipenv with nbdev.
I hope you find this post useful.
These changes may make nbdev more friendly to a team environment.
Please don’t hesitate to post any questions or comments below.