Introducing nbdev-extensions

Hi all, been a bit since I’ve written a post over here :smile: I absolutely love nbdev v2 and my creative juices are now flowing once again post the unconference :slight_smile: As a result, I’m happy to introduce nbdev-extensions

pip install nbdev-extensions

This will serve as a conglomerate of various ideas I try out in nbdev, as an easy to use package for you :smiley:

Currently there are two major features in it:

new_nb

One thing I find sometimes annoying is having to write out the boilerplate for new nbdev notebooks. This simple CLI interface let’s you create them on the fly, and customize them in any which way you’d like:

Afterwards, a simplistic notebook is created that you can immediately explore:

Code Notes

Arguably my favorite of the two extensions, Code Notes are a new annotation tool for, well, annotating code!

It works by having two cells, the first being a code cell with any code, and the following cell(s) be a markdown cell that contains a special #| explanation tag.

You specify what code should be highlighted, and the explanation that follows underneath (this can be anything, even LaTeX!), and then the documentation will render this as a two-part table. One tab will just have the code, and the second tab contains the explanations and code snippets you specified:

I hope some of you find these tools useful, let me know if there are any other ideas that this gets you thinking of :hugs:

13 Likes

This is really great! and I hate to be that guy, but there’s a typo in the spelling of explanation.

1 Like

Mike, this typo will follow me to my grave. It is easily the hardest word for me, and I have no clue why :laughing:

2 Likes

You know what, I noticed because I make the same typo all the time … :sweat_smile:

1 Like

Fixed on pypi now :slight_smile:

2 Likes

Cool stuff. Will you add lib2nbdev to this repo or will it remain extra? Also, it needs to be updated for nbdev v2, I guess?

1 Like

Thank you for making this.

1 Like

We just merged Experimental: Users can provide extra processors via the `procs` key in `settings.ini` by seeM · Pull Request #1157 · fastai/nbdev · GitHub which means you can use @muellerzr’s custom processor quite easily, and create your own processors too! Here’s a very quick guide that will eventually make its way into the docs.

Creating a processor plugin

To develop your own processor plugin, sub-class Processor and override any methods you need to. Check out the nbdev.processors module for examples – all of nbdev’s processing is implemented with pluggable processors.

Once its ready, upload your package to PyPI and/or conda.

For example, the nbdev-extensions package provides the nbdev_extensions.codenotes.NoteExportProc processor, which allows you to annotate code snippets with a new #| explain directive.

Using a processor plugin

First install the package containing the processor. Continuing with the nbdev-extensions example which is hosted on PyPI:

pip install nbdev-extensions

Add the import path to your processor under the procs key of your settings.ini. The import path should follow the same syntax as setuptools console scripts (without the name and equals sign). For example:

procs = nbdev_extensions.codenotes:NoteExportProc

The processor will now be applied in your nbdev project!

5 Likes

@muellerzr Was very excited to use this today but ran into some problems. Mainly I couldn’t ever get it to create both the module entry and the title entry simultaneously. Only one or the other. Even when I simplly ran new_nb test it created a nb with a title but no module cell.

It might be important to note that I’m using Paperspace and I have added the following to my settings.ini :

Here area screenshots of what is happening:

This error appears each time I open a notebook that was created using new_nb:
Screen Shot 2022-11-16 at 1.54.46 PM


1 Like

Thanks for the report! Will take a look at this today :slight_smile:

1 Like

Hi guys, just wanted to mention here what I wrote in another post, regarding an extension I developed for using pytest with nbdev:

and to freeze and unfreeze code cells to avoid re-running them, while preserving their outputs. I hope to add more things in the future.

I am also compiling an awesome-nbdev page to gather extensions and other resources as I become aware of them:

Thanks for the awesome work!

3 Likes

BTW you should follow the in-progress guide here so you can turn this into an extension! (And can hook into something like nbdev_build_lib) Or just give guides on how to make it used :smiley: jk just saw the other post

Great work!

1 Like

That’s awesome, thank you @muellerzr. I need to catch up with the latest tutorials in nbdev, so much great material

2 Likes

Hi!

You might want to check out the nbdev-mkdocs, an extension of nbdev enabling you to use Material for Mkdocs for documentation generation. A tutorial and also an example can be found here:

Thanx :slight_smile:

3 Likes

Great. I’ve seen Material for Mkdocs many times but didn’t have the opportunity to go through it yet. Would you mind to share what are the important differences from your point of view (or pros/cons) wrt quarto? Had a look at your first page but didn’t find any comment about it?

In any case, I just included your extension in this awesome nbdev :slight_smile:

3 Likes

Thanx :slight_smile:

Internally, we use Quarto to generate Markdown files and then generate documentation using Material for Mkdocs. Material for Mkdocs is probably the most popular tool for documentation for Python open-source projects, used by tools such as FastAPI and Pydantic. There is a large ecosystem of plugins for it and one such is automatic API Reference generation from docstrings, a very big selling point for us. I personally like look and feel of it, but that’s a question of taste.

1 Like