Migrating from nbdev1 to nbdev2

I’ve created a migration guide to show how to upgrade from nbdev1 to nbdev2:

Let me know how it goes if you try it out.



To see a list of all the commands available in nbdev2, run nbdev_export

is it nbdev_help ? Looks like it.

PR sent.


Merged, thank you! :slight_smile:

1 Like

if my libname isn’t the same as the reponame, what should LIBNAME in the tutorial be?
my repo is github.com/$USER/nbplanetary
but I export and publish it on pypi as planetarypy for org processing reasons.
So, my git repo looks like:

❯ exa nbplanetary -l --no-user
.rw-r--r--@ 4.6k 13 Oct  2021 07_database.ipynb
drwxr-xr-x@    - 16 Nov  2021 build
.rw-r--r--@ 2.8k 16 Feb 08:21 CHANGELOG.bak
.rw-r--r--@ 2.9k 16 Feb 08:21 CHANGELOG.md
drwxr-xr-x@    - 31 Oct  2021 conda
.rw-r--r--@ 2.3k 28 Jan  2021 CONTRIBUTING.md
.rw-r--r--@ 1.5k 16 Nov  2021 cse.xml
drwxr-xr-x@    - 16 Feb 18:48 dist
.rw-r--r--@  946 28 Jan  2021 docker-compose.yml
drwxr-xr-x@    - 27 Apr 01:27 docs
.rwxr--r--@ 1.1k 24 Jan 01:31 LICENSE
.rw-r--r--@  545 31 Oct  2021 Makefile
.rw-r--r--@  152 12 Oct  2021 MANIFEST.in
drwxr-xr-x@    - 26 Apr 21:13 notebooks
drwxr-xr-x@    -  1 Aug 10:53 planetarypy
drwxr-xr-x@    - 15 May 10:30 planetarypy.egg-info
.rw-r--r--@  14k 27 Feb 19:10 README.md
.rw-r--r--@   40 24 Jan 01:31 requirements.txt
.rw-r--r--@  798 22 Feb 06:47 settings.ini
.rw-r--r--@   29 17 Mar 03:49 setup.cfg
.rw-r--r--@ 2.0k 12 Oct  2021 setup.py
.rw-r--r--@ 5.3k 24 Jul  2021 test_nbs.py
.rw-r--r--@   41  7 Feb 22:41 token
.rw-r--r--@  16k 12 Jul  2021 toml_tools.ipynb

(Notebooks are in notebooks apart from a few stray testing notebooks)

For a different repo I run into the fact that the given perl commands don’t work if the notebooks are in the root folder:

[[ $SHELL = *bash ]] && shopt -s globstar

perl -pi -e 's/#\|\s*hide_input/#| echo: false/' **/*.ipynb
perl -pi -e 's/#\|\s*hide_output/#| output: false/' **/*.ipynb
perl -pi -e 's/#\|\s*skip/#| eval: false/' **/*.ipynb
perl -pi -e 's/from nbdev.export import notebook2script/from nbdev import nbdev_export/' **/*.ipynb
perl -pi -e 's/notebook2script/nbdev_export/' **/*.ipynb
perl -pi -e 's/#\|?\s*all_slow/#| nbflags skip_exec/' **/*.ipynb
bash-5.1$ exa -l --no-user --no-time
.rw-r--r--@  14k 00_io.ipynb
.rw-r--r--@  23k 01_opusapi.ipynb
.rw-r--r--@  15k 02_pipeline.ipynb
.rw-r--r--@  37k 03_ringutils.ipynb
.rw-r--r--@ 7.5M 04_plotting.ipynb
.rw-r--r--@  19k 05_index.ipynb
.rw-r--r--@ 4.4k 06_isis.specialpixels.ipynb
.rw-r--r--@ 3.8k 07_download.ipynb
.rw-r--r--@  13k 08_isis.cubefile.ipynb
.rw-r--r--@  11M 09_ringcube.ipynb
drwxr-xr-x@    - build
drwxr-xr-x@    - conda
.rw-r--r--@ 2.3k CONTRIBUTING.md
drwxr-xr-x@    - dist
.rw-r--r--@  946 docker-compose.yml
.rw-r--r--@ 1.3k index.ipynb
.rw-r--r--@  11k LICENSE
.rw-r--r--@  111 MANIFEST.in
drwxr-xr-x@    - pyciss
drwxr-xr-x@    - pyciss.egg-info
.rw-r--r--@ 5.0k README.md
.rw-r--r--@  688 settings.ini
.rw-r--r--@ 2.5k setup.py
.rw-r--r--@  324 styles.css
.rw-r--r--@ 1.4M test_save.html
.rw-r--r--@  976 to_be_implemented.ipynb
bash-5.1$ ls **/*.ipynb
ls: **/*.ipynb: No such file or directory

IIUC set LIBNAME to your library name as it’s imported in Python (planetarypy). The repo name shouldn’t matter. At the end of the migration, you can doublecheck your settings.ini. I think this should work:

git_url = https://github.com/user/nbplanetary
lib_path = planetarypy  # path to library from root of your repo
lib_name = planetarypy  # name of your package on pypi and conda

You might also find this useful: nbdev - read

This is because you’re using bash which doesn’t support **. The first command is supposed to enable it, but it doesn’t seem to work for all builds of bash (particularly for macOS). Sorry about that, we’ll get those commands updated to work on all shells. In the meantime, it should work if you change the perl commands to use just *.ipynb instead (or notebooks/*.ipynb if they’re in the notebooks folder)

1 Like

ok, did that fix.
Now at the point of nbdev_preview I get asked for a password? Password of what?

Your current user password. It’s installing Quarto which requires sudo (aka admin) permissions. See code here for more: nbdev/shortcuts.py at cbf91055327e2975986e94e9476bc47ea4b4d003 · fastai/nbdev · GitHub

If you’re not comfortable with that, you could install Quarto manually yourself and then rerun nbdev_preview

1 Like

ironically, i switched to bash because of the export statement in the guide.
in my standard fish shell it would have worked. :slight_smile:

1 Like

Ok, now we are getting into the weeds of nbdev, with an issue with either nested packages and/or relative imports.

nbdev is replacing my absolute imports that I use in notebooks:

from planetarypy import utils


from ... import utils

Note that this code comes from a sub-package notebook planetarypy/pds/utils.ipynb, while I also have a planetarypy/utils.ipynb. (Could that be a problem? Worked before ™ :wink: )

Now, a 3rd notebook is importing things from the pds.utils notebook and balks when importing via the installed python library file:

✦ ❯ nbdev_preview
Preparing to preview
Traceback (most recent call last):
  File "/Users/klay6683/miniforge3/envs/py310/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3398, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-2eb0543a6bcb>", line 3, in <cell line: 3>
  File "/Users/klay6683/Dropbox/src/nbplanetary/planetarypy/pds/indexes.py", line 17, in <module>
    from ... import utils
ImportError: attempted relative import beyond top-level package

For clarification, the involved library files are these (among others):


and the directive in pds.utils was changed to:

#| default_exp pds.utils
#| default_cls_lvl 3

@michaelaye can you share your project would be helpful for debugging, if you cannot share your project a minimal example would also be great

cc: @jeremy

yes, here it is:


@michaelaye could you please push the branch with the nbdev2 migration as well?


1 Like

A migration question: is it still possible to have multiple notebooks export to the same python file? (E.g. have several notebooks that define different types of datasets, all of which export to datasets.py?)

After migrating to v2, running nbdev_export seems to overwrite the rest of the file with the last notebook it exports. Is this intended – perhaps a consequence of the new two-way sync between notebooks and source code?

I love using nbdev, by the way – very excited to take advantage of all v2 has to offer!

Not at the moment, but we do plan to add back that functionality soon.

I followed the migration tutorial and it all seems to work well until the point where I typed nbdev_preview.
It looks like a version conflict but I’m not sure where it stems from. Any ideas?

(myenv) eta1si@mycomp:~/dev/01_libs/q8p$ nbdev_preview
Preparing to preview
Traceback (most recent call last):
  File "/home/eta1si/mambaforge/envs/myenv/bin/nbdev_filter", line 8, in <module>
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/fastcore/script.py", line 116, in _f
    return tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/nbdev/cli.py", line 127, in nbdev_filter
    with redirect_stdout(dn): filt.nb_proc(nb).process()
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/nbdev/process.py", line 130, in process
    for cell in self.nb.cells: self._process_cell(cell)
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/nbdev/process.py", line 111, in _process_cell
    cell = opt_set(cell, proc(cell))
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/nbdev/processors.py", line 34, in add_links
    nl = NbdevLookup()
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/nbdev/doclinks.py", line 167, in __init__
    self.entries = {o.name: o.load() for o in pkg_resources.iter_entry_points(group='nbdev')
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/nbdev/doclinks.py", line 167, in <dictcomp>
    self.entries = {o.name: o.load() for o in pkg_resources.iter_entry_points(group='nbdev')
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2457, in load
    self.require(*args, **kwargs)
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2480, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages/pkg_resources/__init__.py", line 788, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (pydantic 1.9.0 (/home/eta1si/mambaforge/envs/myenv/lib/python3.10/site-packages), Requirement.parse('pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4'), {'spacy'})

So I created a brand new repo and copied files over bit by bit and it appears to work now. Perhaps I missed out doing a pip install -e . or something.

Another question: Is there an equivalent of the #exports directive in the new nbdev?
I have hard-coded a dictionary in my library and I would like the documentation to show as I have typed it (show_doc looks rather ugly for dicts!)


I tried #| echo: fenced and #| echo: false, none of them worked. As a workaround I am copy-pasting the code in another cell so that it renders the same way I type it :exploding_head: .