Nbdev : A few things/tricks I learned by trial & error

I read all (most) of the documentation a few times, but couldn’t see clearly how all works out or what translates to what.

I have never built a python module before using nbdev, so there is a knowledge gap I need to fill.

So here are my thoughts / experiences / lessons to self.


Running the notebook exports Python modules because of this cell.

Yes and no.

If you run that cell in the notebook, the corresponding module (and all other modules in that project) will be recreated, BUT only if the notebook file has changed. And with changed I mean ‘you change it and save the file’.

  1. Do changes
  2. Save file
  3. run the cell
  4. .py module file is recreated

I guess it applies

if the file hasn't changed from last time:
    do nothing
else:
   recreate modules

If you want to see the notebook and the corresponding module_name.py side by side in Jupyter.

notebooks in:  
project-name/nbs/00_module1.ipynb

module files in:  
project-name/project_name/module1.py

[Jupyter Lab]
An update in module1.py file does not update what you see on the screen (right side) automatically.

You need to use this “Reload python file from disk”


Another option is to “Open in new browser tab” and press F5 to refresh it.


1 Like

If you make changes in the Notebook that don’t make sense, running this from the Notebook won’t alert you that something is wrong.


Q: Will the module_name.py still be recreated?
A: Yes, even if there is something wrong in it.

Q: How do I know what is wrong?
A: One way is to run, on the console/terminal nbdev_prepare while you are inside project-name directory.

Q: Is there a way to get notified within the Notebook?
A: Yes, run nbdev_prepare from it.
WARNING! Read below why this is a BAD TERRIBLE IDEA.

1 Like

On the image you can see ‘what happens if’ for some cases.


Q: Do I need to use #| export on import statements?
A: Yes


dumb vs no-dumb ( left: notebook, right: module.py )

image


  • Do NOT assume that nbdev will figure out something for you.
  • If you want it IN the module use #|export or #|exporti

Example: import statements


This doesn’t make sense unless you also export / exporti the internal_function.

#| export
var_from_internal_function=internal_function()

More dumb stuff yet to come …

2 Likes


I have been trying to figure out why nbdev would eat all the RAM I can give to it…

It turns out I left that line with
! nbdev_prepare
in the test notebook, which I added just for the photo and question above.

And whenever I run on the Terminal
nbdev_prepare
it just never ends (eating all the RAM until the machine crashes).

Troubleshooting
Running one by one all the pieces:

  • nbdev_export
  • nbdev_test
  • nbdev_clean
  • nbdev_readme

I found out that nbdev_test was the problem.

Then I used
# nbdev_test --do_print --n_workers 1
to find out on which Notebook it was … forever.

Starting /home/jupyter/p/n1.ipynb
- Completed /home/jupyter/p/n1.ipynb
Starting /home/jupyter/p/n2.ipynb
- Completed /home/jupyter/p/n2.ipynb
...

Why?
It runs all the cells in the Notebook, being the last one (here is the dumb idea) a call to itself (i.e. do it again, and again, and again …).

Lesson:
Don’t add the line nbdev_prepare OR nbdev_test in the notebook.

2 Likes

I created this little table to help me separate the concepts of ‘what goes where’. Comments/corrections welcome.


---- DOCUMENTATION Visibility ----	cell/line/code/output/etc. 
#|hide
#|hide_line
#|filter_stream <keyword> ...

#|echo: <true|false>
#|output: <true|false|asis>
#|code-fold: <show|true>

#|exec_doc

---- Python MODULE specification/details ----
#|default_exp <name>
#|export
#|export <some.thing>
#|exporti
#|exports

---- TESTING specification/details ----
#|eval: <true|false>
1 Like

Don’t put anything after
#| export
as
#| export # Goes to module

or you will get a

TypeError: _export_() takes from 2 to 3 positional arguments but 15 were given

when running
nbdev_export

I guess it because of this other export
#|export <some.thing>
which has a parameter.

import statements:

  • When generating Docs: Make the cell that contains it, to Run ALWAYS.
  • When generating Modules: Appear in the module only if you used #| export*

I don’t believe that’s correct. You do indeed have to save the notebook for the export to see changes, but there’s no conditional of the type you mentioned.

1 Like