@lewtun Many Thanks for your prompt response. Works fine now
Say I want to just export an example notebook (without a cfg etc) would this be possible? For exact reasoning, for one of my lessons I want to show how to use #export to help build a library (and convert our ipynb to a .py file). How would I go about this? As I cannot run
notebook2script without a Configuration file of sorts. Thanks!
hi @hennesseeee, i think you need to add the git repo via the
dependency_links argument of setup tools: https://python-packaging.readthedocs.io/en/latest/dependencies.html#packages-not-on-pypi
There is no script that does that directly, no (though having a config file is a very minimal requirement).
Hi @lewtun, thanks for your response and suggestion. I tried adding the following line
setup.py along with
gnnbench in requirements (I also tried
gnnbench==0.1.0) but both give me the following
ERROR: Could not find a version that satisfies the requirement gnnbench (from grapht==0.0.1) (from versions: none)
ERROR: No matching distribution found for gnnbench (from grapht==0.0.1)
As a workaround I’ve changed
requirements = cfg.get(‘requirements’,’’).split()
requirements = cfg.get(‘requirements’,’’).split() + [‘gnnbench @ https://github.com/shchur/gnn-benchmark/tarball/master#egg=gnnbench-0.1.0’,]
which seems to work. It looks like with later versions of pip you should be able to have a single string in the requirements which is what my workaround is based (https://stackoverflow.com/a/54216163/2453167). The parser which parses
settings.ini seems to have issues if I put
gnnbench @ https://github.com/shchur/gnn-benchmark/tarball/master#egg=gnnbench-0.1.0 directly in requirements
Update: this fix doesn’t quite work… its install the base gnnbench package but not the submodules…
Thanks I figured it out:
create_config('nbdev', user='muellerzr', path='.', cfg_name='settings.ini') cfg = Config(cfg_name='settings.ini') from nbdev.export import reset_nbdev_module, notebook2script reset_nbdev_module() notebook2script('testingNBDev.ipynb')
Hello! First of all thanks a lot for all the work that went into this. I am really excited about nbdev. But there is one problem I have. It seems the notebook has to get autosaved in order to build the package correctly.
This is annoying because sometimes
notebook2script() has no effect if the notebook did not get saved.
You can test this by modifying a nb with an export and an
notebook2script() and execute it in short time intervals.
Is there any special configuration I can enable to make sure this compiles better and more immediate ??? Has anybody else noticed the same problem?
Hello all - it’s a fantastic and interesting approach, thanks to all contributors!
Is there a good way to break “big cells” (e.g. classes) into multiple cells so py file and related documentation can still be generated. I am aware of this “hack”
class MyClass(): def method1(self): print("method1")
and in the next cell:
class MyClass(MyClass): def method2(self): print("method2")
Does anyone have a better idea or approach ? Many thanks!
hey @hennesseeee, i’m not sure whether this will solve your problem but have you tried upgrading setuptools as follows:
pip install setuptools -U
I’ve sometimes found that this was the main source of errors with pip install
@Andreas_Daiminger Well yes, you need to save the notebooks for the changes to take effect, the same way you would need to save your python files. Note that in command mode, you just need to hit
S to save.
@tinoka You can use the patch decorator from fastcore to achieve this.
This is remarkable. As an extension we should be able to export few selected cells from a single notebook into a script using
#export? It would be wonderful if we could put in
notebook2script() as the last cell of a notebook with necessary imports (but without all the other bells and whistles and repo cloning) and we get a .py with only those export cells as output.
That’s essentially what happens, we need a configuration file to note where everything is being done to but otherwise that’s exactly what happens
See my solution a few posts down
Tried with your approach but could not get it to work. However, here is a similar effort. I now have the code listed in the link in my project directory and then I call the
notebook2script function from the
export_nb_cells.py file using
from export_nb_cells import notebook2script.
This is a rather neat solution. Will explore this approach a bit more; conceptually, I find the ability to experiment using a notebook and once I figure out how to do things, export the selected cells from the notebook to a script very powerful.
I made sure my setuptools and pip were up to date. I think it may actually be a issue with the package because I can’t seem to install it from command line as I normally do (I have to git clone and then install from the downloaded repo).
Regarding other packages installed from github such as pygsp (which is on pypi but the github version is more up to date) I’ve managed to get it to install with the nbdev package (and pass CI) if I edit
setup.py and change the line
requirements = cfg.get(‘requirements’,’’).split()
requirements = cfg.get(‘requirements’,’’).split() + [‘pygsp @ https://github.com/epfl-lts2/pygsp/tarball/master’]
I’m not sure if there is a cleaner was to do this by just editing
Thanks for your help!
I’m getting a similar error to Marco. I’m trying to run tests after installing fastai2. Before installing, I delete my fastai2 miniconda directory, restart the terminal, and check
pip list and
conda list in my base conda environment to confirm nbdev, fastai2, and fastcore aren’t there. I tried a number of combinations (for nbdev, fastai2, and fastcore) of editable and packaged (conda/pip) installs. All tests fail because
nbdev_test_nbs can’t find
ModuleNotFoundError: No module named 'fastai2'
What I did to check this as cleanly as possible was:
git clone https://github.com/fastai/fastai2 cd fastai2 conda env create -f environment.yml source activate fastai2 pip install -e '.[dev]' nbdev_test_nbs
Looks like because of MacOS’s zsh, the
.[dev] has to be in quotes.
@sgugger does this seem familiar? I’ll update if I get it working.
nbdev_test_nbs works inside the nbdev root folder, after running a dev-install, via:
pip install packaging git clone https://github.com/fastai/nbdev pip install -e nbdev
«cd to /nbdev»
nbdev_test_nbs didn’t work in /fastai2 after the above, and I wasn’t able to run it again in the /nbdev root folder afterwards, though my terminal history seems to point there. It did successfull run in /nbdev/nbdev. Are the tests supposed to only work there?
update: I tried running
nbdev_test_nbs in /fastai2 with a fresh git-clone and package-install of everything in a new conda environment – no luck. Maybe… the tests aren’t supposed to be working yet? But that’d mean the nbdev I have isn’t working at all if it can’t find fastai2 … guess I’ll find out next.
No the tests are supposed to work everywhere. It looks like you have a problem in your enviromnent so I’d suggest creating a new one and reinstalling.
So… I got it working, but the way to do it was very weird. I’ll explain it here in case anyone else with a long-term conda system on MacOS has similar problems.
I’ve been reinstalling fresh conda environments before each attempt, before my last post. I later tried deleting and reinstalling Miniconda itself twice. I finally got the tests working (it’s able to import fastai2) by deleting every hidden file/folder I could find relating to jupyter, conda, or ipython; ontop of a fresh conda reinstall.
nbdev_test_nbs tests pass in /nbdev and /fastcore. 17 notebooks fail in /fastai2 → these seem more ’normal’: the result of CUDA tests on a cpu-only system, or maybe import failures from wrong versions (“
No module named 'wandb’”, “
cannot import name 'PILLOW_VERSION' from 'PIL’”, and “
Torch not compiled with CUDA enabled” for example).
This is with a total clean install of miniconda — my base env doesn’t even have Jupyter on it yet. To get it working (along with a bunch of restarts to be on the safe side) what I did was:
- delete my Miniconda root directory
- delete every hidden folder and file I could find in my Home directory that seemed relevant:
- .jupyter; anything with ‘ipython’, ‘conda’, etc.
I already commented out any conda-related lines in my .zshrc (MacOS equivalent of .bashrc or .bash_profile). Then redownload Miniconda, verify, install, and (since it isn’t configured to write to Zsh) copy its conda-initialization lines from ~/.bash_profile to ~/.zshrc. Checking
pip list and
conda list after restarting Terminal, the base conda env only has a handful of packages.
Then I installed fastai2 to its own env, along with fastcore. I don’t remember if I used packaged or editable, I think I tested both.
nbdev_test_nbs worked, with the new errors mentioned above; I dev-installed nbdev and its own tests passed.
This method deletes all Jupyter customizations and conda environments.
Where I think the issue was:
I still had errors even after deleting and reinstalling conda on my system. I noticed in Jupyter there were old ‘named’ kernels available in the dropdown menu. Kernels for deleted environments. This likely meant ipython/ipykernel had a config file pointing there, and was being used by new Jupyter & etc installations. I only had one for “fastai” and a couple unrelated kernels like Scala … so I don’t know how this tripped up
nbdev_test_nbs into being unable to import fastai2.
What contributed confusion early in this process was not knowing how “developer”/“editable” pip/conda/python installs work. I’m used to the pre-0.7 course-v1 workflow of freely editing files in a fastai folder; and I wasn’t sure if dev-installs went to the base/system python ‘env’ or to the active env (it’s the active conda env). Also, MacOS’s switch to ZShell as its default from Bash means that the
pip install -e .[dev] line doesn’t work → the brackets must be in quotes (double or single):
pip install -e '.[dev]', and I didn’t know the significance of “.[dev]” since
pip install -e . appaeared to run just fine — so I thought “.[dev]” could’ve been some informal programmer shorthand. Not the case.
Hopefully this helps anyone facing a similar problem.
Note: looks like the new PIL (7.0.0) doesn’t work with Torchvision, as of 3 January, → but the PyTorch devs are aware and plan to update (PyTorch and Torchvision) to fix it this week (the PIL & PILLOW_VERSION error). Until then this:
pip install "pillow<7" seems to work.
edit: is fastai2 currently not meant to work on CPU-only systems? After installing a few dependencies (seemingly not taken care of by the environment.yml fastai2 env repo install):
pip install wandb conda install -c fastai -c pytorch fastai pip install tensorboard
the only tests that fail, seem to all be CUDA related. Only 6 fail now:
Interesting, since Apple has dropped all CUDA support… I wonder if it’s possible to install a cuda-gpu version of PyTorch on MacOS just for code compatibility. I’ll update here if I try it out.
Hi, this seems great so far! I was just wondering:
- How do you get the [source] link to show up beside your classes and functions in the docs? I tried comparing my notebook file with the tutorial one but could not find how it was done.
- Is there a way to customize the style the way that the nbdev docs are done (the blue lines between headers etc.)
I’ve been using nbdev since the release on my laptop, and so far it’s been great.
I’ve just pulled my repo on another computer after installing nbdev as well, and it seems the behaviour of nbdev_build_lib has changed.
in my notebooks i have a bunch of
import mylib.foo as foo,
mylib being the lib_name from settings.ini
on my laptop, (with nbdev installed very early at the release, and not updated since), the generated code is stayed untouch as
import mylib.foo as foo and it works great
on the fresh install, the code is modified to
import .foo as foo and my python (3.7) is complaining with a
SyntaxError: invalid syntax error message. It doesn’t like the .foo part.
Did you guys change how the imports are processed? How can I solve this syntax error?
This problem seems to have been introduced with fix #13