Questions on nbdev

I think the latest PR merged is going to fix that issue for you, so use a dev install of the current master branch.

Cool ! will try to do so.

It did solve overwriting issue, but I’m getting this error when I call nbdev_build_lib

Traceback (most recent call last):
  File "/home/kshitij/anaconda3/envs/fastai2/bin/nbdev_update_lib", line 8, in <module>
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastscript/", line 73, in _f
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/", line 87, in nbdev_update_lib
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/", line 125, in script2notebook
    [ _script2notebook(f, dic, silent=silent) for f in files]
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/", line 125, in <listcomp>
    [ _script2notebook(f, dic, silent=silent) for f in files]
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/", line 108, in _script2notebook
    l = nb['cells'][i]['source'].split('\n')[0]
IndexError: list index out of range

Also, there’s a diff in notebook, so CI is failing

diff -ru /tmp/tmpd62fbtg1/ /tmp/tmp81wg_ehv/
--- /tmp/tmpd62fbtg1/	2020-05-29 21:41:58.000000000 +0530
+++ /tmp/tmp81wg_ehv/	2020-05-29 21:47:32.000000000 +0530
@@ -1,4 +1,4 @@
-# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/08_CNN_Interpreter.ipynb (unless otherwise specified).
+# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/07_visualize.grad_cam.ipynb (unless otherwise specified).
 __all__ = ['show_heatmap', 'create_batch', 'GuidedBackprop', 'generate_cam', 'CamImage', 'get_at', 'show_at',
            'show_cam_batch', 'BaseInterpreter', 'batch_none', 'CamInterpreter']

I couldn’t find how to document about arguments of a method in nbdev docs. Also, is there anything we need to do to make documentation visible? like, calling show_doc every time after the #export cell or so.

What could be the reason for such messy table of contents?


This comes from jekyll not really nbdev: you need to have a headers in order: each function has an header of level 4 and each class a header of level 2 so you need to put a level 2 header before your first function to have a ToC fixed.


Yes. I figured it out with some experiments

What about this? I’ve seen in fastai2 notebooks, you’ve written about them in markdown but this way it won’t appear in the docstring of that method. So is there any syntax I’ve to follow to document about the arguments ?

You can follow the markdown syntax inside your document and it will be properly displayed in your docs.

1 Like

I’m bit confused about nbdev_test_nbs. Cells having the flags we pass-in to the nbdev_test_nbs are skippd from test or only those cells are tested ? I more interested in achieving the former behavior :slightly_smiling_face:

Looks like comments above the function are being removed in latest version nbdev (from sources).

# Adopted from 
def convert_MP_to_blurMP(model, layer_type_old):
    "Convert MaxPool/AvgPool to MaxBlurPool"

How can I make sure they get’s exported ? Copying it manually will be preserved ?

I’ve opened #167 to track this.

1 Like
def tsne(x:Tensor,k=2,seed=47):
  "TSNE embeddings using `sklearn`"

I’ve patched tsne to the Tensor class and it works fine with the code. However, when I try to run nbdev_build_docs, I’m getting “name ‘Tensor’ is not defined” error

converting: nbs/04_utils.ipynb
An error occurred while executing the following cell:
show_doc(Tensor.tsne, default_cls_level=2)

NameErrorTraceback (most recent call last)
<ipython-input-11-deee42e37eb3> in <module>
----> 1 show_doc(Tensor.tsne, default_cls_level=2)

NameError: name 'Tensor' is not defined
NameError: name 'Tensor' is not defined

Traceback (most recent call last):
  File "/home/kshitij/anaconda3/envs/fastai2/bin/nbdev_build_docs", line 8, in <module>
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastscript/", line 73, in _f
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/", line 162, in nbdev_build_docs
    notebook2html(fname=fname, force_all=force_all, n_workers=n_workers)
  File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/", line 537, in notebook2html
    raise Exception(msg + '\n'.join([ for p,f in zip(passed,files) if not p]))
Exception: Conversion failed on the following:

Can you take a look at
hopefully this replicates your issue - and also suggests a work-around.

I’ve raised to see if this is an issue or intended bahavior

1 Like

By default, all code cells are run during testing. Putting a test flag (e.g. #slow) in a code cell means the cell will not be run unless you: nbdev_test_nbs --flags slow.
nbdev_test_nbs --flags slow will run all code cells without test flags and all code cells with the #slow flag.

so … If you want flagged cells to be skipped during testing, don’t pass the flag. e.g.

  • you have tst_flags=slow|gpu|broken in settings.ini
  • you want to run all test that are not flagged with broken (i.e. skip the broken test steps)

nbdev_test_nbs --flags "slow gpu"

hope this helps (o:


Thanks @pete88b ! I’m not sure I understood the workaround correctly. But I tried to add

from fastai2.torch_basics import *
from nbdev.showdoc import *

these lines in the imports cell at the top of notebook. It didn’t help.

Also, one thing I’m finding difficult is nbdev_build_docs is becoming extreme RAM hogger. I tried setting --n_workers according to the no. of cores in my system, but eventually it acquires almost whole RAM and freezes my laptop.
I’ve intel i7 7th gen processor with 8GB RAM and I’m finding this insufficient for generating the docs. With normal applications opened, my RAM usage is around 2.5 GB but when I execute nbdev_build_docs, it almost every time finds the RAM insufficient and eventually freezes the system.

I think you need to try this instead, Tensor is specifically designated in torch_core: This worked for me and did not throw the error when building docs.

from fastai2.torch_core import Tensor

As for the memory issues, I noticed the same and I had to just stop and restart the kernel when my computer started slowing down.

I would also add to save on RAM only import packages that you need instead of using import *

Interesting! will try that. Hope someone will build a tool to convert all import * to only used packages :stuck_out_tongue: Just like InteliJ’s optimize imports.

I guess Tensor is not defined in the fastai2.torch_core, rather it’s the original Tensor class from PyTorch. So I’ve tried to do from torch import Tensor but this didn’t help.

don’t worry about the workaround (o: I’m about to make a PR to fix this issue

i’ll try to take a look at memory use in the next few days - are you able to share the project you’re working on? it’d really help when i’m trying to replicate issues

I see but it did work for me, Tensor is imported from and this is imported into torch_core

1 Like

pull the latest when get accepted - this should fix your problem

if you can’t afford to wait, try adding a cell containing:
_all_ = ['Tensor']
run nbdev_build_lib then i think nbdev_build_docs will work.
please let me off if this doesn’t work - it’s just a quick idea i’ve not had time to try yet

1 Like

Can you please specify your solution. I have the same problem. Did you mean, that you forgot to push your local changes to your remote repository? Not sure what you did!