Test individually PASS, but not in a group (when doing nbdev_prepare)

nbdev
1

I don’t understand how is it possible that testing fails/pass depending on whether you run it alone or in a group.


# jupyter [x] (master)$ nbdev_test --do_print --n_workers 1 --file_re translation
Starting /home/jupyter/x/nbs/translation.ipynb
- Completed /home/jupyter/x/nbs/translation.ipynb
Success.

Right after the test above I run the following (below).

# jupyter [x] (master)$ nbdev_test --do_print --n_workers 1 
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting ...
- Completed ...
Starting /home/jupyter/x/nbs/translation.ipynb
AssertionError in /home/jupyter/x/nbs/translation.ipynb:
===========================================================================

While Executing Cell #100:
--------------------------------------------------------

....


nbdev Tests Failed On The Following Notebooks:
==================================================
        translation.ipynb

And YES. If I run the individual test again it does PASS.

# jupyter [x] (master)$ nbdev_test --do_print --n_workers 1 --file_re translation
Starting /home/jupyter/x/nbs/translation.ipynb
- Completed /home/jupyter/x/nbs/translation.ipynb
Success.

The failure comes from a test_eq but that is not the point (is it?).

/opt/conda/lib/python3.7/site-packages/fastcore/test.py in test_eq(a, b)
     35 def test_eq(a,b):
     36     "`test` that `a==b`"
---> 37     test(a,b,equals, '==')
     38 
     39 # %% ../nbs/00_test.ipynb 25

/opt/conda/lib/python3.7/site-packages/fastcore/test.py in test(a, b, cmp, cname)
     25     "`assert` that `cmp(a,b)`; display inputs and `cname or cmp.__name__` if it fails"
     26     if cname is None: cname=cmp.__name__
---> 27     assert cmp(a,b),f"{cname}:\n{a}\n{b}"
     28 
     29 # %% ../nbs/00_test.ipynb 16
2

This keeps happening and driving me nuts.

What is the difference between
‘test (one notebook) alone’ vs ‘batch test (of all the notebooks)’ ?

3

I also ran into this when using nbdev_prepare

The workaround for me was to run the commands separately, i.e.

nbdev_export
nbdev_test
nbdev_readme
nbdev_clean