Upcoming changes in v2.3 (edit - now released)

nbdev
1

v2.3 will be released soon-ish, and you can play with the upcoming functionality by installing from master. Here’s a summary of some of the key changes to know about.

New processing system

The big change is that now rendering your docs is much faster. The time for the notebook processing stage for nbdev’s own notebooks has gone down from 24 seconds to 1 second! In addition, the latest quarto prerelease has also been sped up. The combination should dramatically cut your doc rendering time, possibly by even an order of magnitude!

The reason that nbdev’s notebook processing is faster is that we’ve written our own parallel processing system, instead of relying on quarto’s ipynb-filters. This system is automatically run whenever you call nbdev_docs or nbdev_preview. It processes each of your notebooks, and places the result in a directory called _proc in the root of your repo. These processed notebooks are exactly what quarto uses as input. Therefore, you can cd into that directory and run quarto render, quarto publish, etc, and it’ll all “just work”. This is particularly useful if you use something other than Github Pages for publishing your docs, since quarto publish supports numerous other services.

When you run nbdev_preview, we now run our own watchdog server which watches for any changes, so you get live/hot reloading in the browser. Try changing and saving any notebook whilst preview is running, and you’ll see the results reflected in your browser immediately without any manual refresh required.

New _quarto.yml approach

Previously, nbdev auto-generated your _quarto.yml for you and updated it based on changes to settings.ini. In v2.3 we still auto-generate it if it’s missing, but we no longer auto-update it. Instead, there’s a small nbdev.yml that’s auto-updated from settings.ini and is included into your _quarto.yml. But other than that, you’re free to customise _quarto.yml however you like, and your changes won’t ever get trampled by nbdev.

New quarto sidebar auto generation

Previously, nbdev auto-generated your sidebar for you, assuming you had custom_sidebar = True in your settings.ini (which was the default). However, quarto (v1.2, currently in prerelease) now provides its own automatic sidebar system. Therefore, we’ll be deprecating custom_sidebar = True, and you should switch to using Quarto’s approach. We’ll be providing a way to automate this change so most users won’t have to do anything manually except run a single line in the shell.

nbdev’s own docs are now using this approach, so take a look at the nbdev repo if you’re curious about how any of is working on the nbdev site.

PS: You may also be interested in checking out info about the changes in the v2.2 series.

11 Likes
2

Another interesting upcoming change is that you can create rendered .py scripts. These are regular python .py files, except that:

  • Rather than just having the extension .py, they’ll have an extension like .qmd.py
  • They contain a module docstring containing frontmatter (i.e three hyphens on a line, then some yaml, then another three hyphens on a line).

These scripts are run when your site is rendered. Anything that they print to stdout becomes a new file in your site. The name of the file is the same as the name of the .py script, but without the .py extension. For instance, in nbdev itself, the home page is created by a script called index.qmd.py, which you’ll find here.

You’ll also see that file includes the line:

from nbdev import qmd

nbdev.qmd is a small module that has some convenient functions for creating .qmd documents, such as div and img. You can see examples of their use in index.qmd.py.

Hot/live reloading even works with these .py scripts – so as soon as you save the script, you’ll see the new output in your web browser! I haven’t tested everything yet, but you should be able to use this approach for creating any kind of file - e.g. you could create an SVG file dynamically.

PS: Once you’ve run nbdev_preview or nbdev_docs you’ll find your rendered document in the _proc directory, along with all of your processed notebooks. This might be helpful for debugging.

4 Likes
3

2.3 has now been released.

4 Likes
4

Request: I currently am not a fan of their autogenerated sidebar because it breaks a flow of nbdev I’ve had set since the dawn of nbdev, which is my nbs are one folder full of 00_{title}, 01_, 01a_, … and each of those groups are designed to be a particular place in the sidebar.

With the current new setup I can’t even use a custom sidebar because they all need to be in a folder for it to be considered “nested”, which slows my own workflow down. I took a look at this briefly tonight but got a bit too far deep for me to really know quite where to go for a processor that needs to create a folder and then set metadata in that notebook, but I think this should be in there because it’s a breaking change from v1 now of something that we could do before and no longer can :slight_smile: (And I doubt I’m the only person used to that workflow).

Proposal:

In your _quarto.yml you define the sidebar such as:

  sidebar:
    style: "floating"
    contents:
      - section: Section 1
        contents: 
         - auto: "01*.ipynb"
      - section: Section 2
        contents:
         - auto: "02*.ipynb"

nbdev then parses the contents and understands that a folder called lesson_1 should be created and it should contain the information of notebooks 01a_ to 01zzz_ (for example) in the order of a → zzz. So then, the data inside of each notebook should have added in the order the notebook should be read in for the sidebar. And it should be any legal glob, rather than assuming just 01 and 02 are separate sections.

Thanks :slight_smile:

5

Sorry @muellerzr I’m not following your request or understanding the issue you’re raising. Sorry if I’m being slow!

Could you try to explain it in a way that even I can understand?..

1 Like
6

No worries! Basically given a file setup of:

- nbs
  - 00a_testA.ipynb
  - 00b_testB.ipynb
  - 01a_something.ipynb

Currently quarto will assume they are all the same “level” if you autogenerate them i.e.:

  • testA
  • testB
  • something

What I would like:

  • Section 1: (or something based on the section title as described from earlier)
    • testA
    • testB
  • Section 2:
    • something

Does this help a bit more? :slight_smile: Because otherwise I have to setup my notebooks such as:

- nbs
  - Section 1 (folder)
    - 00a_testA.ipynb
    - 00b_testB.ipynb
  - Section 2
    - 01a_something.ipynb

Does that help slightly more?

7

With the flat file structure, could you add this to your _quarto.yml to get what you want:

website:
  sidebar:
    style: floating
    contents:
      - section: Section 1
        contents:
          - 00a_testA.ipynb
          - 00b_testB.ipynb
      - section: Section 2
        contents:
          - 01a_something.ipynb

Or are you saying that this used to be automated?

1 Like
9

For some reason I thought this wasn’t possible, apologies @seem and @jeremy, the docs were a bit confusing on how to get it done because to do “auto” you can either do auto or be specific with a glob. So this is the solution:

  sidebar:
    style: "floating"
    contents:
      - index.ipynb
      - section: Section 1
        contents: "00*.ipynb"
      - section: Section
        contents: "01*.ipynb"

Sorry for the noise :sob:

2 Likes