I think the latest PR merged is going to fix that issue for you, so use a dev install of the current master branch.
Cool ! will try to do so.
It did solve overwriting issue, but Iām getting this error when I call nbdev_build_lib
Converted transforms.py.
Converted core.py.
Traceback (most recent call last):
File "/home/kshitij/anaconda3/envs/fastai2/bin/nbdev_update_lib", line 8, in <module>
sys.exit(nbdev_update_lib())
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastscript/core.py", line 73, in _f
func(**args.__dict__)
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/cli.py", line 87, in nbdev_update_lib
script2notebook(fname=fname)
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/sync.py", line 125, in script2notebook
[ _script2notebook(f, dic, silent=silent) for f in files]
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/sync.py", line 125, in <listcomp>
[ _script2notebook(f, dic, silent=silent) for f in files]
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/sync.py", line 108, in _script2notebook
l = nb['cells'][i]['source'].split('\n')[0]
IndexError: list index out of range
Also, thereās a diff in notebook, so CI is failing
diff -ru /tmp/tmpd62fbtg1/visualize.py /tmp/tmp81wg_ehv/visualize.py
--- /tmp/tmpd62fbtg1/visualize.py 2020-05-29 21:41:58.000000000 +0530
+++ /tmp/tmp81wg_ehv/visualize.py 2020-05-29 21:47:32.000000000 +0530
@@ -1,4 +1,4 @@
-# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/08_CNN_Interpreter.ipynb (unless otherwise specified).
+# AUTOGENERATED! DO NOT EDIT! File to edit: nbs/07_visualize.grad_cam.ipynb (unless otherwise specified).
__all__ = ['show_heatmap', 'create_batch', 'GuidedBackprop', 'generate_cam', 'CamImage', 'get_at', 'show_at',
'show_cam_batch', 'BaseInterpreter', 'batch_none', 'CamInterpreter']
I couldnāt find how to document about arguments of a method in nbdev docs. Also, is there anything we need to do to make documentation visible? like, calling show_doc every time after the #export cell or so.
What could be the reason for such messy table of contents?
This comes from jekyll not really nbdev: you need to have a headers in order: each function has an header of level 4 and each class a header of level 2 so you need to put a level 2 header before your first function to have a ToC fixed.
2 Likes
Yes. I figured it out with some experiments
What about this? Iāve seen in fastai2 notebooks, youāve written about them in markdown but this way it wonāt appear in the docstring of that method. So is there any syntax Iāve to follow to document about the arguments ?
You can follow the markdown syntax inside your document and it will be properly displayed in your docs.
1 Like
Iām bit confused about nbdev_test_nbs. Cells having the flags we pass-in to the nbdev_test_nbs are skippd from test or only those cells are tested ? I more interested in achieving the former behavior 
Looks like comments above the function are being removed in latest version nbdev (from sources).
#export
# Adopted from
# https://discuss.pytorch.org/t/how-can-i-replace-an-intermediate-layer-in-a-pre-trained-network/3586/7
def convert_MP_to_blurMP(model, layer_type_old):
"Convert MaxPool/AvgPool to MaxBlurPool"
...
How can I make sure they getās exported ? Copying it manually will be preserved ?
Iāve opened #167 to track this.
1 Like
@patch
def tsne(x:Tensor,k=2,seed=47):
"TSNE embeddings using `sklearn`"
...
Iāve patched tsne to the Tensor class and it works fine with the code. However, when I try to run nbdev_build_docs, Iām getting āname āTensorā is not definedā error
converting: nbs/04_utils.ipynb
An error occurred while executing the following cell:
------------------
show_doc(Tensor.tsne, default_cls_level=2)
------------------
---------------------------------------
NameErrorTraceback (most recent call last)
<ipython-input-11-deee42e37eb3> in <module>
----> 1 show_doc(Tensor.tsne, default_cls_level=2)
NameError: name 'Tensor' is not defined
NameError: name 'Tensor' is not defined
Traceback (most recent call last):
File "/home/kshitij/anaconda3/envs/fastai2/bin/nbdev_build_docs", line 8, in <module>
sys.exit(nbdev_build_docs())
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastscript/core.py", line 73, in _f
func(**args.__dict__)
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/cli.py", line 162, in nbdev_build_docs
notebook2html(fname=fname, force_all=force_all, n_workers=n_workers)
File "/home/kshitij/anaconda3/envs/fastai2/lib/python3.7/site-packages/nbdev/export2html.py", line 537, in notebook2html
raise Exception(msg + '\n'.join([f.name for p,f in zip(passed,files) if not p]))
Exception: Conversion failed on the following:
Can you take a look at https://github.com/pete88b/decision_tree/blob/master/71_tensor_patch.ipynb
hopefully this replicates your issue - and also suggests a work-around.
Iāve raised https://github.com/fastai/nbdev/issues/171 to see if this is an issue or intended bahavior
1 Like
By default, all code cells are run during testing. Putting a test flag (e.g. #slow) in a code cell means the cell will not be run unless you: nbdev_test_nbs --flags slow.
nbdev_test_nbs --flags slow will run all code cells without test flags and all code cells with the #slow flag.
so ā¦ If you want flagged cells to be skipped during testing, donāt pass the flag. e.g.
- you have tst_flags=slow|gpu|broken in settings.ini
- you want to run all test that are not flagged with broken (i.e. skip the broken test steps)
nbdev_test_nbs --flags "slow gpu"
hope this helps (o:
2 Likes
Thanks @pete88b ! Iām not sure I understood the workaround correctly. But I tried to add
from fastai2.torch_basics import *
from nbdev.showdoc import *
these lines in the imports cell at the top of notebook. It didnāt help.
Also, one thing Iām finding difficult is nbdev_build_docs is becoming extreme RAM hogger. I tried setting --n_workers according to the no. of cores in my system, but eventually it acquires almost whole RAM and freezes my laptop.
Iāve intel i7 7th gen processor with 8GB RAM and Iām finding this insufficient for generating the docs. With normal applications opened, my RAM usage is around 2.5 GB but when I execute nbdev_build_docs, it almost every time finds the RAM insufficient and eventually freezes the system.
I think you need to try this instead, Tensor is specifically designated in torch_core: This worked for me and did not throw the error when building docs.
from fastai2.torch_core import Tensor
As for the memory issues, I noticed the same and I had to just stop and restart the kernel when my computer started slowing down.
I would also add to save on RAM only import packages that you need instead of using import *
Interesting! will try that. Hope someone will build a tool to convert all import * to only used packages 
Just like InteliJās optimize imports.
I guess Tensor is not defined in the fastai2.torch_core, rather itās the original Tensor class from PyTorch. So Iāve tried to do from torch import Tensor but this didnāt help.
donāt worry about the workaround (o: Iām about to make a PR to fix this issue
iāll try to take a look at memory use in the next few days - are you able to share the project youāre working on? itād really help when iām trying to replicate issues
I see but it did work for me, Tensor is imported from torch_imports.py and this is imported into torch_core
1 Like
pull the latest when https://github.com/fastai/nbdev/pull/173 get accepted - this should fix your problem
if you canāt afford to wait, try adding a cell containing:
_all_ = ['Tensor']
run nbdev_build_lib then i think nbdev_build_docs will work.
please let me off if this doesnāt work - itās just a quick idea iāve not had time to try yet
1 Like
Can you please specify your solution. I have the same problem. Did you mean, that you forgot to push your local changes to your remote repository? Not sure what you did!