Nbdev discussion

nbdev
145

Note to people using a dev install of nbdev: I changed the clean notebooks command to make it faster (trying to commit on a repo with lots of notebooks like fastai v2 was proving veeeery long) so you will need to run again pip install -e . at the root of the nbdev repo otherwise things will stop working.
For other folks, there is going to be a new release soon with that functionality (once it’s tested a but more).

146

this is a great idea and implementation is certainly coming along!

how do I / we help? are there issues posted somewhere or a dev mailing list?

cheers!

147

Is it possible to explicitly define __all__? If I add a cell like:

#export
__all__ = ['function_name']

then build the library, the autogenerated python file has something like:

__all__ = ['__all__', 'function_name', 'other_function']

# Cell
__all__ = ['function_name']

I could explicitly prevent the function from being included by prepending _ to the name, but I’d also like the possibility of setting __all__ myself

edit: this is with nbdev version 0.2.4

148

You can’t define your own __all__ using nbdev. If you want to exclude some function, give it a private name, it won[t be picked automatically.
Lines like _all_ = ["name1", "name2"] will add things to your __all__ if they are not picked automatically.

1 Like
149

Hi,
I’m getting error when trying to run command nbdev_build_lib saying ‘config.yml’ not found. Can someone help pls?

2 Likes
150

Hi @theaws, I also encountered this problem and have logged it as an issue on GitHub: https://github.com/fastai/nbdev/issues/34

The solution for me was to copy the _config.yml file from the nbdev repo

into the docs/ folder of my nbdev project

151

I have a few questions about the workflow for contributing pull requests to nbdev. After making some changes to index.ipynb I ran the following commands:

  1. nbdev_build_lib
  2. nbdev_clean_nbs
  3. nbdev_build_docs

This produced the expected changes in index.html and README.md, but it also make a lot of small changes to other .html files in the docs/ directory.

My questions are:

  1. Is the above the correct sequence of CLI commands to run before making a PR?
  2. Are the changes to the additional .html files the expected behaviour and if yes, should these changes also be included in the pull request?
152

If you run nbdev_install_git_hooks once after cloning, then you shouldn’t need to manually run nbdev_clean_nbs - it’ll be done when you commit. Otherwise, yes, that’s the steps.

We generally prefer a minimal diff so it’s easier for us to understand the PR. What additional changes are you seeing exactly?

Thanks for asking! :slight_smile:

153

This has been solved by latest release normally.

154

Great!

155

Thanks for the answer! Here’s a snippet of the changes I’m seeing in the HTML files:

diff --git a/docs/clean.html b/docs/clean.html
index 071872e..d5882cf 100644
--- a/docs/clean.html
+++ b/docs/clean.html
@@ -159,7 +159,7 @@ summary: "Strip notebooks from superfluous metadata"
 <div class="output_markdown rendered_html output_subarea ">
 <h4 id="clean_nb" class="doc_header"><code>clean_nb</code><a href="https://github.com/fastai/nbdev/tree/master/nbdev/clean.py#L36" class="source_link" style="float:right">[source]</a></h4><blockquote><p><code>clean_nb</code>(<strong><code>nb</code></strong>, <strong><code>clear_all</code></strong>=<em><code>False</code></em>)</p>
 </blockquote>
-<p>Clean <code>nb</code> from superfulous metadata, passing <code>clear_all</code> to <a href="/clean#clean_cell"><code>clean_cell</code></a></p>
+<p>Clean <code>nb</code> from superfulous metadata, passing <code>clear_all</code> to <code>clean_cell</code></p>
 
 </div>

For some reason the reference "/clean#clean_cell" is being removed, but curiously if I discard the changes and rerun nbdev_build_docs the diffs do not reappear. It could well be something on my end, so let me try to investigate further.

156

I’m really enjoying nbdev, thank you for developing it :slight_smile:

I’m trying to add a package to requirements which isnt on pypi but is on github. The package I’m trying to add is https://github.com/shchur/gnn-benchmark and I’ve tried the following (and a bunch of variants to no success)

requirements = networkx numpy matplotlib scipy git+git://github.com/shchur/gnn-benchmark.git

I got this string from https://stackoverflow.com/questions/16584552/how-to-state-in-requirements-txt-a-direct-github-source since the settings.ini file says I should use the same format as setuptools requirements. I get the following error when I try and do pip install . (the CI gives the same when I push to GitHub):

ERROR: Command errored out with exit status 1:
 command: /anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/8s/pw80r_bj4tv87rqttkkk90200000gp/T/pip-req-build-joogxlf6/setup.py'"'"'; __file__='"'"'/private/var/folders/8s/pw80r_bj4tv87rqttkkk90200000gp/T/pip-req-build-joogxlf6/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/8s/pw80r_bj4tv87rqttkkk90200000gp/T/pip-req-build-joogxlf6/pip-egg-info
     cwd: /private/var/folders/8s/pw80r_bj4tv87rqttkkk90200000gp/T/pip-req-build-joogxlf6/
Complete output (1 lines):
error in grapht setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Invalid requirement, parse error at "'+git://g'"
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Would anyone know how to fix this? Cheers!

158

@lewtun Many Thanks for your prompt response. Works fine now

159

Quick question:

Say I want to just export an example notebook (without a cfg etc) would this be possible? For exact reasoning, for one of my lessons I want to show how to use #export to help build a library (and convert our ipynb to a .py file). How would I go about this? As I cannot run notebook2script without a Configuration file of sorts. Thanks!

1 Like
160

hi @hennesseeee, i think you need to add the git repo via the dependency_links argument of setup tools: https://python-packaging.readthedocs.io/en/latest/dependencies.html#packages-not-on-pypi

1 Like
161

There is no script that does that directly, no (though having a config file is a very minimal requirement).

1 Like
162

Hi @lewtun, thanks for your response and suggestion. I tried adding the following line

dependency_links=[‘https://github.com/shchur/gnn-benchmark/tarball/master#egg=gnnbench-0.1.0’],

in the setup.py along with gnnbench in requirements (I also tried gnnbench==0.1.0) but both give me the following

ERROR: Could not find a version that satisfies the requirement gnnbench (from grapht==0.0.1) (from versions: none)
ERROR: No matching distribution found for gnnbench (from grapht==0.0.1)

As a workaround I’ve changed

requirements = cfg.get(‘requirements’,’’).split()
to
requirements = cfg.get(‘requirements’,’’).split() + [‘gnnbench @ https://github.com/shchur/gnn-benchmark/tarball/master#egg=gnnbench-0.1.0’,]

which seems to work. It looks like with later versions of pip you should be able to have a single string in the requirements which is what my workaround is based (https://stackoverflow.com/a/54216163/2453167). The parser which parses settings.ini seems to have issues if I put gnnbench @ https://github.com/shchur/gnn-benchmark/tarball/master#egg=gnnbench-0.1.0 directly in requirements

Update: this fix doesn’t quite work… its install the base gnnbench package but not the submodules…

163

Thanks :slight_smile: I figured it out:

create_config('nbdev', user='muellerzr', path='.', cfg_name='settings.ini')
cfg = Config(cfg_name='settings.ini')
from nbdev.export import reset_nbdev_module, notebook2script
reset_nbdev_module()
notebook2script('testingNBDev.ipynb')
2 Likes
164

Hello! First of all thanks a lot for all the work that went into this. I am really excited about nbdev. But there is one problem I have. It seems the notebook has to get autosaved in order to build the package correctly.
This is annoying because sometimes notebook2script() has no effect if the notebook did not get saved.
You can test this by modifying a nb with an export and an notebook2script() and execute it in short time intervals.

Is there any special configuration I can enable to make sure this compiles better and more immediate ??? Has anybody else noticed the same problem?

3 Likes
165

Hello all - it’s a fantastic and interesting approach, thanks to all contributors!

Is there a good way to break “big cells” (e.g. classes) into multiple cells so py file and related documentation can still be generated. I am aware of this “hack”

class MyClass():
    def method1(self):
        print("method1")

and in the next cell:

class MyClass(MyClass):
    def method2(self):
        print("method2")

Does anyone have a better idea or approach ? Many thanks!