Failed GitHub CI when following End-To-End Walkthrough

As per the title, GitHub actions fail when following the tutorial ( Error: Process completed with exit code 1.).
https://github.com/FerranC96/VRland/actions/workflows/test.yaml has all (failed) runs, and I am unable to get it working.
Originally, not even the github page related actions were running, but I managed to fix that by starting from scratch and by focing the conda channel for nbdev in my conda environment.yml

Does anyone have any ideas? Ta!

Hey @ferranc,

Welcome to the forums and thanks for posting :smiley:

Within your CI fail run I can see on line 281 thereā€™s a warning mentioning Detected unstripped out notebooks

On the following line thereā€™s a reminder ā€˜Remember to run nbdev_install_hooksā€™. Run that and make a new commit to have some clean notebooks in your commit to re-trigger your CI.

Hopefully that will clean up your notebooks before you push and your CI will pass, pending hitting any other errors. Hope that helps! Let me know if you hit any other errors.

3 Likes

Funny thing, sometime back (I think last week) I was getting the same error even after manually cleaning before pushing. The CI was failing with the same error about cleaning.

If you look at his notebook files in the repo, they look stripped and cleaned off metadata so I am wondering what the issue could be.

In my case, I ended up just writing the code in a traditional IDE since I was behind my deadlines and had no time to investigate if the error I was getting was coming from my side, but seeing @ferranc having the same issue gets me wondering if it is a problem from the nbdev side of things. I will setup a basic repo tonight and see if that is the case.

2 Likes

gh ci proably has different (more up to date) version of nbdev. try updating nbdev locally.

2 Likes

Hi @miwojc , Can confirm it works now. It was probably a version issue as you mentioned. Thanks

3 Likes

yeah i was ā€˜debuggingā€™ this for hours last time it happened to me lol

4 Likes

Thanks everyone! I could have sworn I tried nbdev_install_hooks, nbdev_clean and even experimented with pre-commit hooks yesterday while trying to troubleshoot it.
Following your comments I tried to update the nbdev package through conda (already had the latest version so nothing changed here), run nbdev_install_hooks once again (it seemed to not do anything), and run nbdev_clean too. The later changed 2 tracked files in the git repo, and upon pushing the changes the 2 CI actions on GitHub run without issue!

So thanks again for everything! Iā€™m still unsure if it was just bad timing re. versions or if there is something that could be changed within the End-To-End Walkthrough (perhaps some kind of disclaimer/warning in case people encounter similar issues?)

2 Likes

So, turns out things are still not working 100%ā€¦

Now CI fails again during deploy. You can see the log here. Tried to troubleshoot the isssue by running export+test+clean+readme (instead of prepare) and found that it would fail at nbdev_readme with the message below:

Traceback (most recent call last):
  File "/Users/ferran/miniconda3/envs/VRland/bin/nbdev_filter", line 10, in <module>
    sys.exit(nbdev_filter())
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/fastcore/script.py", line 119, in _f
    return tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/cli.py", line 46, in nbdev_filter
    with redirect_stdout(dn): filt(nb)
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/processors.py", line 243, in __call__
    def __call__(self, nb): return self.nb_proc(nb).process()
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/process.py", line 126, in process
    for proc in self.procs: self._proc(proc)
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/process.py", line 120, in _proc
    if hasattr(proc,'end'): proc.end()
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/frontmatter.py", line 66, in end
    self.fm.update({'output-file': _nbpath2html(Path(self.nb.path_)).name})
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/fastcore/basics.py", line 247, in __getattr__
    def __getattr__(self,k): return self[k] if k in self else stop(AttributeError(k))
  File "/Users/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/fastcore/basics.py", line 216, in stop
    raise e
AttributeError: path_

Sorry to be a pain, but does anyone have any ideas as to what might be happening now (i.e. what is it I am doing wrong :sweat_smile:)

Just to add after some further tests: nbdev_readme still fails with the same error message when using different conda environments.
Consequently, the github CI deploy action fails at the same step when attempting to run this command (https://github.com/FerranC96/VRland/actions/runs/4467864203/jobs/7847862647)

These are the versions of the relevant(?) pacakges installed through conda:

  • fastcore fastai/noarch::fastcore-1.5.28-py_0
  • nbdev fastai/noarch::nbdev-2.3.12-py_0

hey @ferranc, Iā€™m assuming youā€™ve installed nbdev and fastcore from conda. Can you install nbdev from pypi and try again?

To do so, run the following commands,

conda remove nbdev
conda remove fastcore
pip install nbdev

Hi all,

I am getting the same error as @ferranc mentions at the beginning of the thread. I am just following the tutorial step-by-step, installed everything using pip on Python 3.9.
I ran both nbde_install_hooks and nbdev_clean, but nothing changed so far.

Any ideas on what could have gone wrong?

1 Like

Ok, just answering my own question here.

After running pip install --upgrade fastcore pandas nbdev it seems to work fine

Itā€™s strange though, because I installed nbdev like 30 minutes before thatā€¦

1 Like

Thanks Deven!
Unfortunately Iā€™ve installed the packages using pip just now and the issue still persistsā€¦

Pip install output:
Successfully installed astunparse-1.6.3 execnb-0.1.5 fastcore-1.5.28 ghapi-1.0.3 nbdev-2.3.12 watchdog-3.0.0

nbdev_readme error:

Traceback (most recent call last):
  File "/var/home/ferran/miniconda3/envs/VRland/bin/nbdev_filter", line 8, in <module>
    sys.exit(nbdev_filter())
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/fastcore/script.py", line 119, in _f
    return tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/cli.py", line 46, in nbdev_filter
    with redirect_stdout(dn): filt(nb)
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/processors.py", line 243, in __call__
    def __call__(self, nb): return self.nb_proc(nb).process()
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/process.py", line 126, in process
    for proc in self.procs: self._proc(proc)
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/process.py", line 120, in _proc
    if hasattr(proc,'end'): proc.end()
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/nbdev/frontmatter.py", line 66, in end
    self.fm.update({'output-file': _nbpath2html(Path(self.nb.path_)).name})
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/fastcore/basics.py", line 247, in __getattr__
    def __getattr__(self,k): return self[k] if k in self else stop(AttributeError(k))
  File "/var/home/ferran/miniconda3/envs/VRland/lib/python3.10/site-packages/fastcore/basics.py", line 216, in stop
    raise e
AttributeError: path_

Also, checked and all dependencies are working fine and so is quartoā€¦

The only thing remaining to check is to nuke the whole directory and start a new repo from scratch without using conda, but that wouldnā€™t really work for my project since I need conda and want to avoid using pip as much as possible

Hey all,

Iā€™m having what I think is the same problem. Just started following the walkthrough for the first time. Followed the instructions precisely, and yet after I commit for the first time, the actions fail.

Things seem to go awry somewhere around:

This error can also happen if you are using an older version of nbdev relative to what is in CI. Please try to upgrade nbdev with the command Requirement already satisfied: nbdev in /opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages (2.3.12)

Iā€™m definitely using the same version of nbdev.

You can see the run here:

Thanks for any help!

I was running into an error when deploying to GitHub pages and was having a very hard time figuring out what the problem was. The pages built fine locally using nbdev_preview but were giving me a weird unhelpful error message in the logs when GitHub actions was run. I eventually figured out that it was the Gradio widget being displayed inline in the notebook that was causing the issue. I didnā€™t really care whether or not it was displayed in the doc pages so I cleared the cell and added these descriptors to the top of the cell which resolved the issue.

#| eval: false
#| output: false

Hopefully this helps someone else with this issue. It took me several hours to figure it out.

1 Like

Sorry for the confusion here! These CI issues have been a huge pain point, so weā€™ve made the following improvements:

  • Weā€™ve made the ā€œunstripped notebooksā€ error message suggest to upgrade nbdev locally (#1308): ā€œThis error can also happen if you are using an older version of nbdev relative to what is in CI. Please try to upgrade nbdev with the command pip install -U nbdevā€.
  • Fixed nbdev_newā€™s initial notebooks to also match the latest nbdev behaviour (#1311). This meant that following the tutorial exactly would lead to the above error.
  • In v2.3 we moved away from using Quartoā€™s notebook filters. However, attempting to do so leads to the confusing error message: AttributeError: path_ (#1298). Weā€™ve replaced this with a friendlier error message.
3 Likes