Super exciting stuff! The new tutorial is amazing too. I feel like anyone can write their own libraries effortlessly with nbdev. It’s so magical and makes everyone a magician of making libs.
2 Likes
00:00 - Welcome
01:00 - Getting started and What are those super amazing things that can be done with nbdev?
02:44 - What does Jeremy think of nbdev? What’s Jeremy’s user experience of it?
03:37 - What’s the plan for today’s session?
Turn a github repo into a nbdev repo
04:39 - What’s the first step? Creating a repo
05:34 - Make this repo into a nbdev repo: how to install nbdev, how lightweight is nbdev
06:48 - What are nbdev commands? type nbdev_ How to use them? type nbdev_help
07:25 - How nbdev makes writing docs super easy lazy?
08:17 - How to find out the docs of a nbdev command like nbdev_new? type nbdev_new -h
08:37 - What does nbdev_new give us? What is settings.ini? Why it is so important to help you lazy?
09:56 - What are the stuff generated by nbdev_new on your cloned repo for you? Which ipynb is your home for docs? (index.ipynb) Which ipynb is your library? (00core.ipynb)
10:54 How should you name ipynbs for your library so that you and people understand how the lib is written? Jeremy showed us how he got started on nbdev with execnb and showed the history of how he explored and invented nbdev
13:36 - Create two ipynbs as two modules card and deck
14:09 - How to name your module with #| as in quarto? How and when to hide your cell?
15:32 - What does the heading 1 and heading 2 do? Have a look at execnb as an example
Create a Card module
18:02 - How did Jeremy create a module Card from scratch in nbdev?
19:01 - Creating suits: How did Jeremy solve the card emoj in python problem live?
21:02 - Creating ranks: based on the codes from the book author
21:23 - Creating a class by starting to create the __init__ function : what is fastai/Jeremy’s coding style?
24:28 - How Jeremy wrote docs of the explorative experiment before starting to build Card class?
26:54 - How to overwrite the __str__ and __repr__ to display the class instance nicely? Also writing docs for the class along the way.
29:03 - How to write docs for input argument (to make docs for the class more clean and short) in the source and how to display docs with show_doc? Can you even show_doc other libraries not written in nbdev?
33:13 - Create tests for the class: How to import test from fastcore for testing with *? This way of wild card import require __all__. nbdev takes care of it and makes it easy.
35:26 - run test_eq? and test_eq?? or test_eq or just test_eq( + tab can give you all the info you need for a class or function
36:26 - How did Jeremy build tests for the Card class live with different test functions?
37:06 - How to define equality __eq__ for the Class for the test functions to pass?
38:04 - How to give indication or set the types of the input arguments in the class? def __eq__(self, a: 'Card'): ...
38:32 - How to define a function outside of a class Card with @patch from fastcore.utils in an explorative and experimentative way? Now, you can just def __eq__(self:Card, a:Card): .... Also the docs for this function is generated automatically too.
40:08 - Let’s create __lt__ and __gt__ with @patch and more tests with assert
42:18 - How to export your Card py file with nbdev_export and try out your own single module library with pip install -e . And then import the lib inside index.ipynb to try out while writing your front page there.
44:37 - to copy the description of the library from setttings.ini to index.ipynb and write about the inspiration of this lib.
46:21 - Write about the how to install cell: Note that no code quote is used in the cell. The first middle tricky issue: the lib name can’t have a _ underscore, so it has to be nbdev-Cards even though the pyfile name and its path name is using nbdev_cards. Also we need make them clear in settings.ini file. Jeremy also mentioned how to use the % in the settings.ini?
48:33 - How to preview your documentation? nbdev_preview which also install quarto for you. How Jeremy preview both source file and documentation webpage at the same time to make adjustment and see them in real time.
51:58 - How Jeremy handles functions like __eq__ etc? Do we need to show_doc them? How to organize these functions?
53:24 - So, you decide which cell in the notebook to show to your readers with #|hide. For example, you can just show readers one test example and hide the rest without overwhelming them
54:22 - nbdev can also generate advanced webpage like the homepage of nbdev.fast.ai in a single notebook.
55:02 - How to do local test? nbdev_test to see whether all your tests pass or not. How helpful can nbdev_test be when a test fails? You can also check out all the functionalities of nbdev_test by nbdev_test -h
56:52 - How to do debugging in real life in nbdev with hotkey for reload and allrun? in the source notebook, 00 to reload or restart kernel, edit the keyboard shortcut in the notebook for running all cells. Then you can put %debug in the failed cell to debug what’s wrong.
59:18 - Start writing up the next module to create a deck of cards. Thanks to pip install -e . your lib will update itself whenever you make changes to it
1:00:51 - Which codes should be exported with #|export? We can double check the exported pyfile to see what is actually exported with __all__
Creating Deck module
1:02:09 - Write up the Deck class: use ctags -R and vim to search and jump through the source codes: /card to search, ctrl + ] to jump to the defintion of card in a different file if needed
1:03:51 - How Jeremy figure out a problem of not exporting a line of import and test and fix it live?
1:05:24 -
What is the golden rule of making library? Don’t mix import with other things in a single cell.
1:06:41 - How to overwrite the __len__ and __contains__ and how to use them? len(deck) and Card(1,1) in deck
1:08:19 - Start creating a pop method: Who (R) likes and who (quarto) does not like a space between #| and export? How to create the function pop and write docs and default value for arg idx:int=-1): # docs for this arg here? Then write a test for it right away. And nbdev_export, nbdev_test and nbdev_preview to check the site
1:15:22 - Card in the markdown cell will be automatically turned into a link to the class Card in the website. This features work for all kinds of libraries.
1:17:00 - Start writing remove method for the Deck class: copy and write the code, write a test, run tests, check with the preview to see whether exported properly
1:18:48 - Creating a function draw_n cards: also add another method shuffle inside the class source and we can show_doc(Deck.shuffle) at where the shuffle’s tests are.
1:21:04 - More techniques or features in written tutorials later
Pushing back to github
1:21:50 - Sending to github: first, nbdev_clean to clean unnecessary metadata; git status to see what will be going to github; git add -A to add all the things seen above to be ready to send to github; you can run export, clean and test in one go by nbdev_prepare; then git commit -am "..." and git push to the github.
1:23:45 - What actions does nbdev ask github to do for it? Check them in .github/workflows. Then Jeremy showed how to use gh pages to to display the website and how github running the two actions.
1:25:59 - Now enjoy your library website
1:26:19 - You should be absolutely amazed with what nbdev has done for you.
1:27:21 - run nbdev_docs to put the homepage inside README.md for you.
Release Your Library
1:28:47 - Use nbdev_pypi or nbdev_conda or nbdev_release to upload. How to deal with the two versions of the pypi library created? use git diff to see the differences between the two versions; then git commit -am bump and then git push
1:30:16 - How nbdev makes PR easier for all? and what’s more you can do with nbdev.
10 Likes
Could anyone explain what’s the purpose of bump in this line of code? Thanks
Try man git-commit, look for -m, and tell us what you find!
1 Like
Thanks Jeremy.
I found out that -m is an option to allow user to write git commit message, and there is no option called bump. (This is where I was confused about)
So bump is actually an message without quotation marks.
Reading the man git-commit is very instructive. Thanks!
2 Likes
Jeremy posted a sped up tutorial for nbdev2.
4 Likes
Thanks Sarada, I finished the longer one and didn’t feel much pause during those 90 minutes. So, I couldn’t believe there are 30 minutes can be cut due to “silence” 
2 Likes
I was experimenting fastcore with nbdev. At first, nbdev_preview works fine. Now I finished some experiments and I can run nbdev_prepare with success. but when I want to enjoy a nice nbdev_preview, it throws me the following errors which I have no idea how to fix.
Where should I go to fix it? Thanks
(base) 22:35 ~/Documents/debuggable > nbdev_prepare
Success.
(base) 22:36 ~/Documents/debuggable > nbdev_preview
Preparing to preview
Traceback (most recent call last):
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3398, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-1-b58b38ab58c3>", line 2, in <cell line: 2>
NameError: name 'mid' is not defined
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/Natsume/mambaforge/bin/nbdev_filter", line 8, in <module>
sys.exit(nbdev_filter())
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/fastcore/script.py", line 116, in _f
return tfunc(**merge(args, args_from_prog(func, xtra)))
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/nbdev/cli.py", line 125, in nbdev_filter
with redirect_stdout(dn): filt.nb_proc(nb).process()
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/nbdev/process.py", line 168, in process
for cell in self.nb.cells: self._process_cell(cell)
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/nbdev/process.py", line 149, in _process_cell
cell = opt_set(cell, proc(cell))
File "/Users/Natsume/mambaforge/lib/python3.9/site-packages/nbdev/processors.py", line 145, in __call__
if self.k.exc: raise Exception(f'Error: cell {cell.idx_}:\n{cell.source}') from self.k.exc[1]
Exception: Error: cell 11:
#| echo: false
show_doc(mid)
Is mid defined in the notebook? Does the cell defining mid have the correct directive? I think it should be #| export, but not 100% sure without seeing the notebook.
Details: The NameError means that nbdev_preview is trying to access mid (via show_doc(mid)), but it isn’t defined in the session. That usually happens when the directives stop the cell that defines mid from being executed.
1 Like
Thanks Wasim!
It seems the problem is not with the space between #| and export, also Jeremy mentioned that quarto likes no space.
Right now, I get nbdev_preview to work without error. The way I did is to add #|export to every function the error reported as “not defined”. I have no idea why I should export those functions even though I don’t intend them to be inside my library. I probably need to read nbdev docs to learn more of how and when to use #|export.
Another troublesome issue is that when I ctrl + z the preview and start preview again. I have the following error message in the image below and the only way to solve it for me is to quit and restart browser and terminal. Is there a better solution? Thanks.
So far, what I figured out the use of #|export:
- export the functions you write specifically for your library purpose.
- export the func so that
show_doc(func)can run. (this is the place I fixed the error fromnbdev_preview
You shouldn’t be running show_doc on any function you don’t export - since otherwise your users will see documentation for a function that you didn’t actually export for them to use!
2 Likes
ctrl+z suspends a job but doesn’t cancel it - it’s still there in the background (type jobs to see it). You should use ctrl-c to cancel it.
1 Like
Thank you Jeremy! This is helpful. Please let me summary what I learnt by experimenting nbdev and from this forum:
On the use of show_doc in nbdev:
- Only use
show_docon functions you export for your library, becauseshow_docis intended for users to check the docs of your library functions and classes. - Also, if you didn’t export e.g., func1 and run
show_doc(func1), nbdev will throw an error. - If for special cases you really want to display
show_doc(func1)on your preview whenfunc1is not to be exported to your library, then you should runshow_doc(func1), and restart the notebook and comment outshow_doc. Since you still have things like<nbdev.showdoc.BasicMarkdownRenderer at 0x10fbaea60>, you can render theshow_doceffect in nbdev_preview to display.
One more thing to add - you generally don’t need to manually add show_doc, since it’s automatically called for you for any function or class you #|export. Have a look at some of the nbdev or fastai source notebooks for examples.
1 Like
Thanks Jeremy, you are absolutely right, it works the same on my nbdev project too.
I have another quick question about describing the following code on @delegates in English: Should I say:
func
middelegateslowto decide which args to be included in**kwargs
def low(a, b): pass
@delegates(low)
def mid(c, d, **kwargs): pass
I don’t actually understand what that sentence is trying to say - sorry!
1 Like
IIUC, correct English is “mid delegates to low”.
But you wouldn’t need to say that, because the point of delegates is that the decorated function signature has kwargs replaced with to’s, so the user will already see the effect of that in show_doc
2 Likes
Hi @seem
It seems nbdev only allows headings to extend from level 1 to level 3. Can I customize it to extend to level 4 or more? Where should I look? Thanks
Could you please share more about your use-case and the specific issues nbdev is giving? Are you referring to the table of contents in Quarto? What does your notebook contain?
1 Like