Nbdev_readme vs. nbdev_test

nbdev
1

I’m running into an issue that’s related to
file dunder attribute works in .py, not in notebook, but different. I think it shows some differences between nbdev_test and nbdev_readme that I don’t understand.

I have created a minimal reproducible example here. The details (also written in the README there):

Intended behavior:

Copy some data from the library itself (the assets folder) into the user’s current working directory.

Cell 1:

from shutil import copytree
from pathlib import Path

Cell 2: __file__ when run in a script gives the script’s path. In a notebook, we can use the current working directory (cwd).

#| hide
__file__ = (Path().cwd() / "00_core.ipynb").as_posix()

Cell 3: Copy the folder

src = Path(__file__).parents[1] / "assets"
dest = Path().cwd() / "output"
copytree(src, dest, dirs_exist_ok=True)

This succeeds:

  • 00_core.ipynb runs fine
  • nbdev_test 00_core.ipynb succeeds
  • the script 00_core.py created by nbdev_export runs fine

This fails:

  • nbdev_readme fails with an error referencing 00_core.ipynb:
NameError: name '__file__' is not defined
  • If I set __file__ to cwd without the #| hide directive:
    • 00_core.ipynb still runs
    • nbdev_readme fails because it cannot find the assets folder. This happens because cwd gives the root of my repo (where I ran nbdev_readme) rather than the nbs folder.

Underlying questions:

It looks like nbdev_readme runs notebooks besides index.ipynb, in this case 00_core.ipynb, in iPython.

  • How is this iPython run different from nbdev_test?
  • What’s the best way to execute stuff in nbdev_readme that we need in iPython context but don’t want to run in the script, such as setting __file__?
2

I kind of figured out what’s happening in this specific case.

nbdev_readme runs proc_nbs. This filters the notebooks before doing exec_nb via serve_drv.

Since exec_nb runs ipython, __file__ isn’t available by default. However, since the notebook has already been filtered, I think the cells with the #|hide directive are already gone.

nbdev_test strips out cells with #|eval:False", but I think it leaves in the #|hide` cells before also running execnb.

This means there doesn’t seem to be a directive to hide a cell for a .py but keep it in for any ipython run.

I can think of a couple ways to deal with this:

  • put in a “try… except” block for __file__
  • use in_ipython from fastcore to test if we’re in ipython

Both of these ways will let me set __file__ if we’re in ipython.

But there’s another problem – if I set it using Path.cwd(), I think* this changes between nbdev_test (seems to be the notebook’s directory) and nbdev_readme / nbdev_proc_nbs (seems to be my cwd in the terminal). I haven’t confirmed this with print statements or such yet.

My third solution, which seems to work:

  • Put a version of copytree into a separate module
  • Import this module and use its __file__ attribute to set the src for my copy:
if in_ipython():
    copy_root_dir = Path(copytree.__file__).parents[1] 
else:
    copy_root_dir = Path(__file__).parents[1]

I hope I didn’t annoy anyone with this verbose thread!

2 Likes
3

Thank you, this helps me understand a little better how nbdev_readme works. And I really appreciate your reproducible example and verbose explanations (neither of which I’ve fully absorbed, to be honest, given my use case, but I’m glad they’re there if I did need them).

1 Like