I’ve a couple questions about nbdev and don’t know if there’s any separate thread to post these, so asking here:
- How can I export a cell to a specific module (other than
default_exp)
- Any guidelines on customizing sidebar? what are naming conventions for notebooks?
- I want to have my custom tag for colab which will be used to remove those cells from notebook, since there are many Colab specific instructions I need to execute every session (Like script to install fastai2 and dependencies, cloning my repo, etc.), is there any similar functionality available in nbdev that I’m unaware of?
Be sure to look at the docs as all your answers are in there:
-
#export/exports/exporti module_name
- Edit the
sidebar.json in the docs folder. Look at fastai2 for some examples on how things are done. Notebooks should be named with the module names they create if you want the automatic linking with entity between ` to work.
-
#hide remove cells from notebooks for documentation.
2 Likes
Do the numbers at the beginning are ignored?
But they remain in the notebook. I want them to be removed from notebook itself since those are platform specific instructions
I did. in fact, I was trying to replicate that for a while but wasn’t able to have everything as expected. Here’s my sidebar.json
{
"Overview": "/",
"Tutorial": {
"Overview":"/tutorial",
"Explore Architectures": "/tutorial.arch_explore"
},
"Core": "/core"
}
and this is generated doc Core and Overview is not working
Yes the numbers at the beginning are always ignored.
If you want cells removed from a notebook… remove them before committing. There is no mechanism in nbdev to do this since the goal is to share the notebook.
For the sidebar, look more closely at the fastai2 sidebar.json. As soon as you have subcategories, you need something in each of your main categories to please Jekyll, so
{
"Getting started": {
"Overview": "/"
},
"Tutorial": {
"Overview":"/tutorial",
"Explore Architectures": "/tutorial.arch_explore"
},
"Core": "/core" --> also needs something inside.
}
2 Likes
This is an ignorant question, but here goes: I forked someone else’s Python package by creating a Github repository that I had set up using the nbdev template. On my local machine, I created a repository in a subdirectory of a folder that I had also set up using nbdev (more or less by mistake). When I push a commit from my local machine to my Github repository, the author of the commit shows as ‘Jeremy Howard’. Yet I can’t find any mention of him, and ‘author’ is set to my name in ‘settings.ini’. Does anyone know what’s going on? I also live in fear of committing to nbdev itself by accident. Is there some hidden nbdev settings file that I don’t know about?
How to update index.ipynb from README.md ?
CI step Check if all notebooks are cleaned always fails:
Process exits abruptly with following output
echo "Check we are starting with clean git checkout"
if [ -n "$(git status -uno -s)" ]; then echo "git status is not clean"; false; fi
echo "Trying to strip out notebooks"
nbdev_clean_nbs
echo "Check that strip out was unnecessary"
git status -s # display the status to see which nbs need cleaning up
if [ -n "$(git status -uno -s)" ]; then echo -e "!!! Detected unstripped out notebooks\n!!!Remember to run nbdev_install_git_hooks"; false; fi
shell: /bin/bash -e {0}
env:
pythonLocation: /opt/hostedtoolcache/Python/3.6.10/x64
Check we are starting with clean git checkout
git status is not clean
##[error]Process completed with exit code 1.
Link to the repository: https://github.com/Kshitij09/fast_impl
I realized what was wrong. I forgot to push changes in _nbdev.py 
1 Like
Is it possible to propograte README.md changes to index.ipynb? Just like we do with library changes to notebook (script2notebook).
I prefer writing markdown pages with an editor rather than putting it in a notebook.
Also, is it possible to have different PyPi package description and repository’s README ?
I figured out the answer to my own question, which was kind of obvious: Github uses your email to assign your name to a commit. When I set up my cloud VM, I forgot to change the github email and so when I created a project using the nbdev template, the associated email was “info@fast.ai,” which is linked to Jeremy Howard’s name. To fix it:
git config --global user.email # See what email is being used now
git config --global user.email "email@example.com” # Re-set it
@sgugger I’d like to suggest removing the default email from the nbdev template so Jeremy isn’t the default author of all commits…if that’s possible.
This information is not embedded in the template by itself, so there is no way to remove it. I guess GitHub puts that as a default if it doesn’t have another one because Jeremy was the one who created the template, but that’s just a guess.
OK thanks Sylvain. I don’t know if anyone else will be as dumb as I was, but it might be worth a line in the documentation like “Don’t forget to set your Github email address if creating a repository from an nbdev template on a VM.”
What can I do to apply nbdev_build_docs recursively on subdirectories ?
1 Like
I noticed one problem with nbdev_build_lib. When I export a method to some module other than default, and if I change that method a bit and run nbdev_build_lib again, rather than updating the previous method, nbdev adds the duplicate method in that file, so you get two methods with same name, one with the changes and one older version.
Also, before that, by simply doing an export to other module caused diff in the notebook and library, thus failed the CI pipeline. I’m not sure what does the output of nbdev_diff_nbs means, but it shows some diff and causes CI to fail.
No, it’s not because you export to a module different than the default, it’s because you export to a module that was not created by nbdev (e.g., it does not have a notebook where it’s the default module). nbdev can thus not know it has to reset this notebook each time you export. You should have a prior notebook that creates this module or just leave it as a module only that never gets anything exported from any notebook (like the imports.py in nbdev).
It’s not the case. I do have a notebook that creates this module and I’m just exporting this single method to that.
This is the the notebook creating a module called transforms
This one is exporting a method to that module, look for to_grayscale and the end
Yes but you’re using subfolders which are not supported, and non-numbered notebooks, so the order in which the export is done is probably wrong.
Okay, so the numbers determine the order of export? Also, in case of subfolders, none of the nbdev instructions are executed on them. I manually execute it for each notebook using --fname parameter, so the order in which I do the execution will make any difference? or is it like both the notebooks are not aware of each other and thus affect the results ?
Yes, only the global nbdev_build_lib command will properly reinitialize all the modules created by nbdev to avoid having the same function twice.
1 Like
I finally decided keep all the notebooks under same directory as it’s getting cumbersome. But now I’m facing an issue of overwriting the file contents.
Converted 00_core.ipynb.
Converted 02_preprocessing.ipynb.
Converted 03_tutorial.arch_explore.ipynb.
Converted 04_tutorial.stanford_dogs_wrangling.ipynb.
Converted 05_classification.AlexNet_cifar10.ipynb.
Converted 06_classification.NiN_cifar10.ipynb.
Converted 07_visualize.cam.ipynb.
Converted 08_visualize.grad_cam.ipynb.
Converted 09_CNN_Interpreter.ipynb.
Converted applications.ipynb.
Converted index.ipynb.
Converted paper_impl.ipynb.
Converted tutorial.ipynb.
The notebooks were converted in the above order. 08_visualize.grad_cam.ipynb is the first creating visualize module and has exported 2-3 methods/classes. The very next notebook 09_CNN_Interpreter.ipynb adds bunch of functionalities to the same visualize module. So both the notebooks have default_exp set to visualize and the 2nd one is overwriting contents of visualize.py instead of appending new functionalities.
You can have a look at this branch for reference