tylere
August 11, 2022, 11:58pm
1
I am trying to get nbdev v2 to generate docs that include Jupyter widgets. Is this expected to work?
The Quarto docs show that it does support Jupyter widgets.
I my experiments, notebooks containing Jupyter widgets render as expected with the classic Jupyter Server and JupyterLab. However, when I run nbdev_preview, the code cell outputs do not appear in pages served. I don’t see any errors reported in the terminal running nbdev_preview or in the browser console.
2 Likes
hamelsmu
August 12, 2022, 12:45am
2
I’m trying this now I’ll let you know
1 Like
hamelsmu
August 12, 2022, 4:30am
3
I am having the same problems. We have reported the issue to Quarto. Thanks for letting us know
tylere
August 15, 2022, 6:22pm
4
Thanks for investigating and reporting to Quarto. Is there a quarto-dev issue to follow for this?
1 Like
hamelsmu
August 15, 2022, 6:38pm
5
Thanks @tylere for following up on this. We have heard back from Quarto that we actually need to have execution turned on (which we turn off by default in nbdev) in order to render the widget.
Let me discuss this with @jeremy and @seem and get back to you
1 Like
tylere
August 19, 2022, 5:42am
6
I’ve tried specifying cell execution in the YAML front matter of the notebook that contains Jupyter widgets:
execute:
enabled: true
Unfortunately this addition causes nbdev_preview to fail while interpreting the comment directives:
$ nbdev_preview
**Preparing to preview**
**[1/1] 00_core.ipynb**
Starting python3 kernel...Done
Executing '00_core.ipynb'
Cell 1/7...ERROR:
None of the known patterns match for default_exp core
Let me know if there something I can do to help debug (providing a detailed reproducible bug report, etc.).
hamelsmu
August 21, 2022, 4:34am
7
tylere:
execute:
enabled: true
Thanks, I’m looking into this
tylere
September 6, 2022, 6:23pm
8
Any update on having an option to turn cell execution on?
Fwiw, adding the following notebook frontmatter, as described in the nbdev doc’s Automatic Cell Execution section, also doesn’t seem to have any effect:
---
exec_all: true
---
If it would be helpful, I’d be happy to create a bug with a full repo for reproducing the issue.
jeremy
September 7, 2022, 2:46am
9
Could you try it again with the current master branch? Setting enabled: true should work now I think.
tylere
September 8, 2022, 5:41am
10
I tried that, but I am still having issues. I created a few bugs with details:
opened 05:29AM - 08 Sep 22 UTC
## Test environment
Setup:
`conda create -n test-evaluate-code-2 -c conda-fo… rge -c fastai python=3.10 gh jupyterlab jupyterlab-git ghapi ipywidgets`
`conda activate test-evaluate-code-2`
`pip install -U git+https://github.com/fastai/nbdev.git@08462b3da90c86a06cd62159babc69c346d21f88`
`cd test-evaluate-code-2`
`nbdev_install_quarto`
`nbdev_new`
`nbdev_install_hooks`
Library versions:
```
(test-evaluate-code-2) tylere-macbookpro5:test-evaluate-code-2 tylere$ pip list | grep nbdev
nbdev 2.3.0
(test-evaluate-code-2) tylere-macbookpro5:test-evaluate-code-2 tylere$ quarto -v
1.1.189
```
## To replicate:
1. In `index.ipynb`, add a raw format cell as the first cell of the notebook with the following frontmatter content:
```
---
execute:
enabled: true
---
```
2. Add two code cells to print the current time:
```
import datetime
```
```
print(f'time = {datetime.datetime.now()}')
```
3. Start the local docs preview server.
```
nbdev_preview
```
4. Make a trivial change to `index.ipynb` and save it.
5. Note that the output of nbdev_preview shows the page is continually refreshed. (The preview content at localhost:3000 looks correct, it just keeps refreshing itself.)
A repository with this content is available at: https://github.com/tylere/test-evaluate-code-2
opened 05:38AM - 08 Sep 22 UTC
## Test environment
Setup:
`conda create -n test-evaluate-code-2 -c conda-fo… rge -c fastai python=3.10 gh jupyterlab jupyterlab-git ghapi ipywidgets`
`conda activate test-evaluate-code-2`
`pip install -U git+https://github.com/fastai/nbdev.git@08462b3da90c86a06cd62159babc69c346d21f88`
`cd test-evaluate-code-2`
`nbdev_install_quarto`
`nbdev_new`
`nbdev_install_hooks`
Library versions:
```
(test-evaluate-code-2) tylere-macbookpro5:test-evaluate-code-2 tylere$ pip list | grep nbdev
nbdev 2.3.0
(test-evaluate-code-2) tylere-macbookpro5:test-evaluate-code-2 tylere$ quarto -v
1.1.189
```
## To replicate:
1. In `index.ipynb`, add a raw format cell as the first cell of the notebook with the following frontmatter content:
```
---
execute:
enabled: true
---
```
2. Add two code cells to print the current time:
```
import datetime
```
```
print(f'time = {datetime.datetime.now()}')
```
3. Prepare the changes for a push to GitHub.
```
nbdev_prepare
```
4. Push the changes to GitHub.
5. Wait for the GitHub Actions to run.
6. Note the error indicating that the `hide` directive in the first code cell fails.
https://github.com/tylere/test-evaluate-code-2/runs/8242110591?check_suite_focus=true#step:2:387
> Executing 'index.ipynb'
> Cell 1/5...
>
> None of the known patterns match for hide
> Quarto is not installed. We will download and install it for you.
> Installing or upgrading quarto -- this requires root access.
> Error: Process completed with exit code 1.
A repository with this content is available at: https://github.com/tylere/test-evaluate-code-2
tylere
September 12, 2022, 9:03pm
11
Filed another issue related to widget rendering:
opened 08:54PM - 12 Sep 22 UTC
*(This is a follow up to the [Nbdev v2 & Jupyter Widgets](https://forums.fast.ai… /t/nbdev-v2-jupyter-widgets/98259) discussion started on the nbdev forum.)*
I am trying to get ipywidget output to render in nbdev_preview, with an eventual goal of getting it to render on GitHub pages. I am using (hopefully correctly) the master versions of nbdev, quarto, fastcore, execnb.
# To reproduce:
## Clone the test repository
https://github.com/tylere/test-evaluate-code-3
## Setup a Local Environment setup
```
conda create -n test-evaluate-code-3 -c conda-forge python=3.10 gh jupyterlab jupyterlab-git ipywidgets ipyleaflet
conda activate test-evaluate-code-3
cd test-evaluate-code-3
# Install fastai dependencies
pip install -e ../../fastai/fastcore/
pip install -e ../../fastai/execnb/
pip install -e ../../fastai/nbdev/
nbdev_new
nbdev_install_hooks
# Install dev version of quarto
# https://github.com/quarto-dev/quarto-cli#development-version
ln -s /Users/tylere/Documents/GitHub/quarto-dev/quarto-cli/package/dist/bin/quarto /Users/tylere/miniconda3/envs/test-evaluate-code-3/bin/quarto
pip install -e '.[dev]'
```
## Prerelease dependency versions
```
(test-evaluate-code-3) tylere-macbookpro5:test-evaluate-code-3 tylere$ (cd ../../fastai/fastcore && git pull && git log -1)
Already up to date.
commit 0ee5b7f52a6b8d81691046d73555d56a25fbafac (HEAD -> master, origin/master, origin/HEAD)
Merge: f7e81b8 8fdfd81
Author: Jeremy Howard <j@fast.ai>
Date: Mon Sep 12 08:59:13 2022 +1000
Merge pull request #486 from seeM/fix-http-exceptions
fix: error in IPython while handling `HTTP4xxClientError`
```
```
(test-evaluate-code-3) tylere-macbookpro5:test-evaluate-code-3 tylere$ (cd ../../fastai/execnb && git pull && git log -1)
Already up to date.
commit 25f3a2f94eb0ea5b9c88303dfd6b9130caa7a029 (HEAD -> master, tag: 0.1.3, origin/master, origin/HEAD)
Author: Jeremy Howard <j@fast.ai>
Date: Sat Sep 3 13:13:53 2022 +1000
release
(test-evaluate-code-3) tylere-macbookpro5:test-evaluate-code-3 tylere$ (cd ../../fastai/nbdev && git pull && git log -1)
Already up to date.
commit 8b354dbed937005b500197431cc43c85cdf37d09 (HEAD -> master, origin/master, origin/HEAD)
Merge: 98fa120 6699aee
Author: Hamel Husain <hamel.husain@gmail.com>
Date: Sun Sep 11 20:27:55 2022 -0700
Merge pull request #1039 from fastai/nb-add
refactor best practices intro
```
```
(test-evaluate-code-3) tylere-macbookpro5:test-evaluate-code-3 tylere$ (cd ../../fastai/nbdev && git pull && git log -1)
Already up to date.
commit 8b354dbed937005b500197431cc43c85cdf37d09 (HEAD -> master, origin/master, origin/HEAD)
Merge: 98fa120 6699aee
Author: Hamel Husain <hamel.husain@gmail.com>
Date: Sun Sep 11 20:27:55 2022 -0700
Merge pull request #1039 from fastai/nb-add
refactor best practices intro
```
```
(test-evaluate-code-3) tylere-macbookpro5:test-evaluate-code-3 tylere$ (cd ../../quarto-dev/quarto-cli/ && git pull && git log -1)
Already up to date.
commit cb5343c248387e7e0c46b5e90c72479d0615a59b (HEAD -> main, tag: v1.2.95, origin/main, origin/HEAD)
Author: Charles Teague <cteague@gmail.com>
Date: Mon Sep 12 13:44:01 2022 -0400
Pass meta to shortcodes without messing with it
Fixes #2407 (part 1 - pass meta through cleanly)
```
## View the output in JupyterLab
<img width="1432" alt="Screen Shot 2022-09-12 at 1 43 35 PM" src="https://user-images.githubusercontent.com/92862/189755337-3e6f3af1-d5e9-40ab-97fd-2f263d5be0da.png">
## Preview the output
```nbdev_preview```
Note that the datetime printout rendered correctly, but the two ipywidgets do not render.
<img width="1432" alt="Screen Shot 2022-09-12 at 1 43 38 PM" src="https://user-images.githubusercontent.com/92862/189755671-27c8e33e-1af4-4fa1-8438-e97e33ffa685.png">
## Clues
The nbdev_preview output warns **"Malformed Jupyter Output Display Data found"**
```
(test-evaluate-code-3) tylere-macbookpro5:test-evaluate-code-3 tylere$ nbdev_preview
Check file:///Users/tylere/Documents/GitHub/quarto-dev/quarto-cli/src/quarto.ts
Preparing to preview
[1/1] 01_test_ipywidget_evaluate.ipynb
Starting python3 kernel...Done
Executing '01_test_ipywidget_evaluate.ipynb'
Cell 1/6...Done
Cell 2/6...Done
Cell 3/6...Done
Cell 4/6...Done
Cell 5/6...Done
Cell 6/6...Done
WARNING: Malformed Jupyter Output Display Data found:
WARNING: {"model_id":"1149bbef449c4481bf183fb503c7079d","version_major":2,"version_minor":0}
WARNING: Malformed Jupyter Output Display Data found:
WARNING: {"model_id":"a4ac7997ebed4a8b969b5827cb3af393","version_major":2,"version_minor":0}
Watching files for changes
Browse at http://localhost:3000/01_test_ipywidget_evaluate.html
GET: /01_test_ipywidget_evaluate.html
```
1 Like
jeremy
September 14, 2022, 5:23am
12
@tylere have you tried this in JupyterLab?
IIRC with this you shouldn’t need to do anything special to have widget state appear in quarto.
tylere
September 14, 2022, 9:25pm
13
@jeremy I hadn’t, but now I have enabled the Save Widget State Automatically (SWSA) option, I still can’t view the widget output with nbdev_preview.
The SWSA option doesn’t seem to work as I would expect. Here is my mini-experiment.
Verify that the SWSA option is enabled:
Restart the kernel and run the cells so it displays the widget output in JupyterLab:
Change the state of the widget slider output (to 42):
Save the notebook. Close the notebook tab and kernel:
Reopen the notebook. Note that the widget is not fully rendered, and the text representation shows a different state (value=0) than when I previously saved the notebook.
I also note that the JupyterLab interface indicates the notebook is untrustednbdev_trust --force_all
The classic notebook shows the same lack of trust:
Perhaps there is a problem with indicating that the notebook is trusted, which prevents saving the widget output?
I found some related discussions within the JupyterLab channels:
Not sure if this is a bug or just a problem I am having. I didn’t have this problem before Jupyterlab 3.0. With JupyterLab 3.0, I did a fresh install, but now all notebooks seem to be untrusted after a Jupyter restart. For example, I make a new...
Reading time: 1 mins 🕑
Likes: 3 ❤
opened 11:18PM - 09 Feb 21 UTC
bug
tag:Security
## Description
In https://discourse.jupyter.org/t/jupyterlab-not-remembering-… trust-for-notebooks/7869, a user told about how they had difficult persisting trust state in a new notebook. I tried trusting a new notebook, and could not get JLab to do so.
## Reproduce
In JupyterLab 3.0.7 in a clean environment (but also in latest JLab 2 or JLab 1, from my experiments):
1. Create a new notebook and evaluate a few cells and have a few blank cells
2. Check the trust status in the status bar. For me, it says for example "Notebook not trusted: 3 cells of 4 cells trusted", where I have 4 cells and one of them is blank.
3. Run the "Trust notebook" command from the command palette, and see that the trust status bar icon still says the same thing.
4. Close and shutdown the notebook, then open it again. None of the cells will be trusted (which IIRC is expected behavior if at least one cell was untrusted when saved)
5. Run all the cells and delete the empty cells. Then the trusted status bar icon shows the notebook is trusted.
## Expected behavior
Running all the cells in a notebook should trust all the cells (and thus trust the notebook as a whole). Probably blank cells (both input and output) should be considered trusted by default.
## Context
- Operating System and version: macOS Catalina
- Browser and version: Firefox 85
- JupyterLab version: 3.0.7
It does not look like there are any relevant errors in the js console or in the server log.
My next step will probably be to explore more about how notebook trust is stored in the metadata…
jeremy
September 15, 2022, 12:01am
14
Thanks for checking. @seem has also noted that jupyter trust seems pretty broken in general at the moment. It’s not done with nb metadata iiuc, but using a separate database:
https://jupyter-notebook.readthedocs.io/en/stable/security.html#security-in-notebook-documents
seem
September 15, 2022, 1:24am
15
This is now being tracked as a quarto bug: Embedded `ipywidgets` are not rendered · Issue #2445 · quarto-dev/quarto-cli · GitHub
Some other points worth noting:
I don’t think trust affects embedded widgets. I think it only affects whether widgets are rendered in the Jupyter Lab interface
nbdev_clean currently removes widget state from the notebook, even if you enable the “Save Widget State Automatically” option in Jupyter Lab. We might want to change that (or add a setting)
However, execute: {enabled: true} reruns widgets on quarto render which recreates widget state in the notebook file, so it’s an ok workaround
2 Likes
jeremy
September 15, 2022, 6:00am
16
I’d guess we definitely want to change that. AFAICT you’d never want to remove widget state if it’s there, since it’s removed by default – if it’s there then someone has made a conscious decision to make that happen!
1 Like
seem
September 15, 2022, 6:55am
17
1 Like
