Jupyter Notebook Enhancements, Tips And Tricks

[Daniel (深度碎片)]

How to add a keyboard shortcut to insert a code snippet

Inspired by https://stackoverflow.com/a/51719689/9201239
and some help from @stas

If you first enter a cell with either code mode or markdown mode, like the following

Then you press Ctrl + Shift + M, you will get the following

To make this happening, you just need to

1. go to custom.js whose path can be found by running

echo $(jupyter --config-dir)/custom/custom.js

2. add the following codes into custom.js and save.

Jupyter.keyboard_manager.edit_shortcuts.add_shortcut('Ctrl-Shift-M', {
                              help : 'add details drop',
                              help_index : 'zz',
                              handler : function (event) {
                              var target = Jupyter.notebook.get_selected_cell()
                              var cursor = target.code_mirror.getCursor()
                              var before = target.get_pre_cursor()
                              var after = target.get_post_cursor()
                              target.set_text(before + '[/details][details=""]' + after)
                              cursor.ch += 20 // where to put your cursor
                              target.code_mirror.setCursor(cursor)
                              return false;
                              }}
                                                        );

3. finally, go back to Jupyter Notebook and refresh your page, and you are ready to go.

[Daniel (深度碎片)]

How to create a multi-line code snippet for Jupyter notebook

First, install nbextensions

conda install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user

Second, go to your jupyter notebook and open nbextensions settings to add features

Third, open custom.js file and add the following codes

require(["nbextensions/snippets_menu/main"], function (snippets_menu) {
        console.log('Loading `snippets_menu` customizations from `custom.js`');
        var horizontal_line = '---';
        var my_favorites = {
        'name' : 'My favorites',
        'sub-menu' : [
                      {
                      'name' : 'most_jupyter_magics',
                      'snippet' : ['%reload_ext autoreload',
                                   '%autoreload 2',
                                   '%matplotlib inline',
                                   'from IPython.core.interactiveshell import InteractiveShell',
                                   'InteractiveShell.ast_node_interactivity = "all"',],
                      },
                      {
                      'name' : 'kaggle download links',
                      'snippet' : ['from IPython.display import FileLinks',
                                   'FileLinks(\'.\')',],
                      },
                      {
                      'name' : 'details_drop',
                      'snippet' : ['[/details][details=""]',],
                      },
                      {
                      'name' : 'find snippet custom js',
                      'snippet' : ['echo $(jupyter --config-dir)/custom/custom.js',],
                      },
                      ],
        };
        snippets_menu.options['menus'].push(snippets_menu.default_menus[0]);
        snippets_menu.options['menus'][0]['sub-menu'].push(horizontal_line);
        snippets_menu.options['menus'][0]['sub-menu'].push(my_favorites);
        console.log('Loaded `snippets_menu` customizations from `custom.js`');
        });

Finally, refresh your notebook page, and you shall see the following

---

reference:
Jupyter docs on snippet

[stas (Stas Bekman)]

Getting the notebook name automatically

# cell 1
%%javascript
IPython.notebook.kernel.execute('nb_name = ' + '"' + IPython.notebook.notebook_name + '"')
# cell 2
nb_name

Same without js magic:

# cell 1
from IPython.display import display, Javascript
Javascript("IPython.notebook.kernel.execute('nb_name = ' + '\"' + IPython.notebook.notebook_name + '\"')")
# cell 2
nb_name

In either case, JS is async - so it may not run right away and therefore it’s not guaranteed nb_name will be set right away.

I was looking for this feature, since I wanted to replace hardcoded cells in each dev_nbs such as:

!./notebook2script.py 02_fully_connected.ipynb

with something similar that will extract the nb name automatically.

So I wrote a little helper function:

from IPython.display import display, Javascript
def nb_auto_export():
    display(Javascript("if (IPython.notebook.kernel) {IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name )}"))

(it looks like JS is the only way to get the nb name in jupyter )

notes:

• it assumes the nb name has no spaces in it, add quotes if it does
• had to bracket the code with if (IPython.notebook.kernel) {} since the browser will attempt to run this js code automatically on jupyter nb load and fail, since IPython.notebook.kernel hasn’t been defined yet. It only happens if there is ! in the js code - odd.

and now there is no need to hardcode the nb name, just have the last cell say:

nb_auto_export()

that is if you imported nb_auto_export from somewhere you saved it.

Or alternatively without needing to import anything, you could just have the last cell of each notebook:

%%javascript 
if (IPython.notebook.kernel) {
    IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name)
}

Also note that it’s async, so it usually takes a sec or so to start running once the nb finished running.

Or the messier approach w/o js magic:

from IPython.display import display, Javascript
display(Javascript("if (IPython.notebook.kernel) {IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name)}"))

You might be able to drop display from it, but it seems not to work w/o it for me.

Finally, you shouldn’t rely on auto-save, since you’re likely to miss recent changes, so why not tell this code to save the notebook first, resulting in this code:

from IPython.display import display, Javascript
def nb_auto_export():
    display(Javascript("if (IPython.notebook) { IPython.notebook.save_notebook() }; if (IPython.notebook.kernel) { IPython.notebook.kernel.execute('!./notebook2script.py  ' + IPython.notebook.notebook_name )}"))

Ideas for this solution came from SO: 1, 2

[t-v (Thomas)]

With a bit of Python import glue, you could also skip the export step and just import the notebooks (I used a %lib magic to indicate the things actually I want to show up in the library, but you could remove that and take it all). Maybe that can be even more user friendly than having to export things.

Best regards

Thomas

[Interogativ (Bart Fish)]

Interactive Labeling:

I have some python code that I would like to use for interactive labeling of sections in a matplotlib plot using Jupyter Notebook. The code seems to work in python 2.7 under GUI, but will not run properly in a notebook. The problem is that the matplotlib close_event never leaves the cell containing the plot.

I’ve put together an example to demonstrate. If anyone has some working Jupyter Notebook code that can be used to label sections of a plot interactively please let me know.

Here is the sample code:

The notebook is here

[stas (Stas Bekman)]

Please do not use this thread for questions or problem reports, instead post them here:

Jupyter Notebook Enhancements [Discussion]

This thread is dedicated for discussing Jupyter Notebook Enhancements. If you have an issue with one of the recipes posted here please discuss it here. If you have a complete recipe to contribute please post it here. Thank you.

as explained in the first post. Thank you.

[nok]

AutoScroll and go to current cell function are now included in the extension. The extension is in master branch but a new pip version is not released yet.

For now, you can install through

pip install https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tarball/master

The PR for go to current cell

You can customize the shortkey yourself if needed.

Go to current running cell

Jupyter Notebook Enhancements [Discussion]

Interesting, I do experience some lag of * sometimes but have not observe it would stuck forever, will check out your notebook later. I didn’t say forever, I said for the duration of the notebook run - i.e. as long as there is at least one cell running, those earlier cells report being busy. And since you’re going after the active cell, it will be the wrong cell - it’ll be the first stuck cell. Do let me know if you can reproduce it on your side - thank you! The only culprit with this appro…

[stas (Stas Bekman)]

Undo cell delete

Rachael Tatman (rctatman)

[dipam7 (Dipam)]

Hey, I tried doing this and it gives me an error. I don’t understand why this is? Please help.

I did a simple

%%script false
for i in range(10): print(i)

[stas (Stas Bekman)]

Indeed, I’m able to reproduce this.

 %%script false

no longer works.

Since that behavior was never documented (or was intended to work) they must have dropped it in the recent versions of ipython.

I’m not sure why I no longer am able to edit my posts here, but here are workarounds, based on programs ignoring their arguments when you tell them not to expect any. Here are some easy examples:

Perl:

%%perl -e0
​
for i in range(10): print(i)

Here you’re running: perl -e '0' cellcontents

A more memorable version:

%%perl -eat
​
for i in range(10): print(i)

Here you’re running: perl -e 'at' cellcontents

Bash:

%%bash -c :

for i in range(10): print(i)

‘:’ is a noop in bash, so you’re running: bash -c : cellcontents

I haven’t looked at the external magic implementation code, but I’m pretty sure “cellcontents” are passed as arguments and won’t be interpreted by shell by mistake, say if you were to include ‘;’ in them and accidentally inject some bad code. But I can’t guarantee you that.

I’m sure you can come up with other creative solutions, by looking at the supported programs here: https://ipython.readthedocs.io/en/stable/interactive/magics.html#cell-magics

p.s. perhaps a forum admin with edit rights could tweak this comment to indicate that it no longer works and send readers to this post instead. Thank you.

p.p.s. I discovered the original hack here.

[jeremy (Jeremy Howard)]

perhaps a forum admin with edit rights could tweak this comment to indicate that it no longer works and send readers to this post instead. Thank you

I’ve made the earlier post a wiki so you should be able to edit it.

[stas (Stas Bekman)]

I’ve made the earlier post a wiki so you should be able to edit it.

Thank you, Jeremy. I now was able to update the outdated comment.

[stas (Stas Bekman)]

@stas

%%javascript IPython.notebook.kernel.execute('nb_name = ’ + ‘"’ + IPython.notebook.notebook_name + ‘"’)

Grr, async js isn’t always a great solution. e.g. I was trying to use this to get the nb name for logging, and it doesn’t work with Run-All, so luckily others found a non-js script solution (with a few minor limitations):

from notebook import notebookapp
import urllib, json, os, ipykernel

def ipy_nb_path():
    """Returns the absolute path of the Notebook or None if it cannot be determined
    NOTE: works only when the security is token-based or there is also no password
    """
    connection_file = os.path.basename(ipykernel.get_connection_file())
    kernel_id = connection_file.split('-', 1)[1].split('.')[0]

    for srv in notebookapp.list_running_servers():
        try:
            if srv['token']=='' and not srv['password']:  # No token and no password, ahem...
                req = urllib.request.urlopen(srv['url']+'api/sessions')
            else:
                req = urllib.request.urlopen(srv['url']+'api/sessions?token='+srv['token'])
            sessions = json.load(req)
            for sess in sessions:
                if sess['kernel']['id'] == kernel_id:
                    return os.path.join(srv['notebook_dir'],sess['notebook']['path'])
        except:
            pass  # There may be stale entries in the runtime directory
    return None

Since I only wanted the nb name, I tweaked it a bit (1) to just return the filename w/o ext (2) throw an exception instead of returning None if something went wrong.

from notebook import notebookapp
import urllib, json, os, ipykernel, ntpath

def ipy_nb_name():
    """ Returns the short name of the notebook w/o .ipynb
        or get a FileNotFoundError exception if it cannot be determined
        NOTE: works only when the security is token-based or there is also no password
    """
    connection_file = os.path.basename(ipykernel.get_connection_file())
    kernel_id = connection_file.split('-', 1)[1].split('.')[0]

    for srv in notebookapp.list_running_servers():
        try:
            if srv['token']=='' and not srv['password']:  # No token and no password, ahem...
                req = urllib.request.urlopen(srv['url']+'api/sessions')
            else:
                req = urllib.request.urlopen(srv['url']+'api/sessions?token='+srv['token'])
            sessions = json.load(req)
            for sess in sessions:
                if  sess['kernel']['id'] == kernel_id:
                    nb_path = sess['notebook']['path']
                    return ntpath.basename(nb_path).replace('.ipynb', '') # handles any OS
        except:
            pass  # There may be stale entries in the runtime directory
    raise FileNotFoundError("Can't identify the notebook name")

[msm1089 (Mark McPherson)]

@stas I searched for a very long time to find a way to get notebook name from within a notebook. I had tried ipyparams, but being JS based it had same issue with async execution.

So I modified your code just a little and created a package.

Get the notebook name:

import mynameis
nb_fname = mynameis.what()

Get the full path to the notebook:

import mynameis  
nb_path = mynameis.what(path=True)

To install it, just do

pip install mynameis

[stas (Stas Bekman)]

Fantastic, that is very useful to have it in a package, @msm1089! Thank you!

I’d just recommend to rename the package to something that’s indicative of what kind of name it is getting, e.g. ipynbname - which now tells the user that it’s about jupyter notebooks.

[msm1089 (Mark McPherson)]

Good shout @stas, that name makes a lot more sense.

[muellerzr (Zachary Mueller)]

Today @KevinB and I went on a mission to try showing the output from ? and ?? into a cell rather than a pop up. Nice if you were for instance doing a blog. Thanks to fastcore and the @patch functionality it’s as simple as this:

@patch
def pinfo_to_str(self:Inspector, obj, oname='', formatter=None, info=None, detail_level=0, enable_html_pager=True):
       info = self._get_info(obj, oname, formatter, info, detail_level)
       return info['text/plain']

To use it, do:

from IPython.core.oinspect import Inspector
insp = Inspector()

o = insp.pinfo_to_str(Learner, detail_level=1)

print(o) will then return the same as what Learner?? did (also if you just want the doc string rather than source code do detail_level=0)

[KevinB (Kevin Bird)]

Here is a slight update that reduces the complexity a bit:

print(get_ipython().inspector._get_info(SGD, detail_level=1)['text/plain'])

In use:

def show_doc(func_name):
    print(get_ipython().inspector._get_info(func_name, detail_level=1)['text/plain'])
show_doc(SGD)

output:

[Pomo (Malcolm McLean)]

Can any of you fine developers show me how to implement this?

When any cell is run, apply the function showType to the final value and display the resulting string in the output cell (along with the normal output).

Yes, I could study a pile of docs and examples to get it done, but maybe the method will be obvious to you.

[msm1089 (Mark McPherson)]

I’m starting a blog using the fast.ai template for GitHub pages. Thanks, this might be useful