Jupyter Notebook Enhancements, Tips And Tricks

With a bit of Python import glue, you could also skip the export step and just import the notebooks (I used a %lib magic to indicate the things actually I want to show up in the library, but you could remove that and take it all). Maybe that can be even more user friendly than having to export things.

Best regards

Thomas

2 Likes

Interactive Labeling:

I have some python code that I would like to use for interactive labeling of sections in a matplotlib plot using Jupyter Notebook. The code seems to work in python 2.7 under GUI, but will not run properly in a notebook. The problem is that the matplotlib close_event never leaves the cell containing the plot.

I’ve put together an example to demonstrate. If anyone has some working Jupyter Notebook code that can be used to label sections of a plot interactively please let me know.

Here is the sample code:

The notebook is here

Please do not use this thread for questions or problem reports, instead post them here:


as explained in the first post. Thank you.

AutoScroll and go to current cell function are now included in the extension. The extension is in master branch but a new pip version is not released yet.

For now, you can install through

pip install https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tarball/master

The PR for go to current cell

You can customize the shortkey yourself if needed.

1 Like

Undo cell delete

2 Likes

Hey, I tried doing this and it gives me an error. I don’t understand why this is? Please help.

I did a simple

%%script false
for i in range(10): print(i)

1 Like

Indeed, I’m able to reproduce this.

 %%script false

no longer works.

Since that behavior was never documented (or was intended to work) they must have dropped it in the recent versions of ipython.

I’m not sure why I no longer am able to edit my posts here, but here are workarounds, based on programs ignoring their arguments when you tell them not to expect any. Here are some easy examples:

Perl:

%%perl -e0
​
for i in range(10): print(i)

Here you’re running: perl -e '0' cellcontents

A more memorable version:

%%perl -eat
​
for i in range(10): print(i)

Here you’re running: perl -e 'at' cellcontents

Bash:

%%bash -c :

for i in range(10): print(i)

‘:’ is a noop in bash, so you’re running: bash -c : cellcontents

I haven’t looked at the external magic implementation code, but I’m pretty sure “cellcontents” are passed as arguments and won’t be interpreted by shell by mistake, say if you were to include ‘;’ in them and accidentally inject some bad code. But I can’t guarantee you that.

I’m sure you can come up with other creative solutions, by looking at the supported programs here: https://ipython.readthedocs.io/en/stable/interactive/magics.html#cell-magics

p.s. perhaps a forum admin with edit rights could tweak this comment to indicate that it no longer works and send readers to this post instead. Thank you.

p.p.s. I discovered the original hack here.

I’ve made the earlier post a wiki so you should be able to edit it.

1 Like

Thank you, Jeremy. I now was able to update the outdated comment.

Grr, async js isn’t always a great solution. e.g. I was trying to use this to get the nb name for logging, and it doesn’t work with Run-All, so luckily others found a non-js script solution (with a few minor limitations):

from notebook import notebookapp
import urllib, json, os, ipykernel

def ipy_nb_path():
    """Returns the absolute path of the Notebook or None if it cannot be determined
    NOTE: works only when the security is token-based or there is also no password
    """
    connection_file = os.path.basename(ipykernel.get_connection_file())
    kernel_id = connection_file.split('-', 1)[1].split('.')[0]

    for srv in notebookapp.list_running_servers():
        try:
            if srv['token']=='' and not srv['password']:  # No token and no password, ahem...
                req = urllib.request.urlopen(srv['url']+'api/sessions')
            else:
                req = urllib.request.urlopen(srv['url']+'api/sessions?token='+srv['token'])
            sessions = json.load(req)
            for sess in sessions:
                if sess['kernel']['id'] == kernel_id:
                    return os.path.join(srv['notebook_dir'],sess['notebook']['path'])
        except:
            pass  # There may be stale entries in the runtime directory
    return None

Since I only wanted the nb name, I tweaked it a bit (1) to just return the filename w/o ext (2) throw an exception instead of returning None if something went wrong.

from notebook import notebookapp
import urllib, json, os, ipykernel, ntpath

def ipy_nb_name():
    """ Returns the short name of the notebook w/o .ipynb
        or get a FileNotFoundError exception if it cannot be determined
        NOTE: works only when the security is token-based or there is also no password
    """
    connection_file = os.path.basename(ipykernel.get_connection_file())
    kernel_id = connection_file.split('-', 1)[1].split('.')[0]

    for srv in notebookapp.list_running_servers():
        try:
            if srv['token']=='' and not srv['password']:  # No token and no password, ahem...
                req = urllib.request.urlopen(srv['url']+'api/sessions')
            else:
                req = urllib.request.urlopen(srv['url']+'api/sessions?token='+srv['token'])
            sessions = json.load(req)
            for sess in sessions:
                if  sess['kernel']['id'] == kernel_id:
                    nb_path = sess['notebook']['path']
                    return ntpath.basename(nb_path).replace('.ipynb', '') # handles any OS
        except:
            pass  # There may be stale entries in the runtime directory
    raise FileNotFoundError("Can't identify the notebook name")

@stas I searched for a very long time to find a way to get notebook name from within a notebook. I had tried ipyparams, but being JS based it had same issue with async execution.

So I modified your code just a little and created a package.

Get the notebook name:

import mynameis
nb_fname = mynameis.what()

Get the full path to the notebook:

import mynameis  
nb_path = mynameis.what(path=True)

To install it, just do

pip install mynameis

1 Like

Fantastic, that is very useful to have it in a package, @msm1089! Thank you!

I’d just recommend to rename the package to something that’s indicative of what kind of name it is getting, e.g. ipynbname - which now tells the user that it’s about jupyter notebooks.

1 Like

Good shout @stas, that name makes a lot more sense.

1 Like

Today @KevinB and I went on a mission to try showing the output from ? and ?? into a cell rather than a pop up. Nice if you were for instance doing a blog. Thanks to fastcore and the @patch functionality it’s as simple as this:

@patch
def pinfo_to_str(self:Inspector, obj, oname='', formatter=None, info=None, detail_level=0, enable_html_pager=True):
       info = self._get_info(obj, oname, formatter, info, detail_level)
       return info['text/plain']

To use it, do:

from IPython.core.oinspect import Inspector
insp = Inspector()

o = insp.pinfo_to_str(Learner, detail_level=1)

print(o) will then return the same as what Learner?? did (also if you just want the doc string rather than source code do detail_level=0)

2 Likes

Here is a slight update that reduces the complexity a bit:

print(get_ipython().inspector._get_info(SGD, detail_level=1)['text/plain'])

In use:

def show_doc(func_name):
    print(get_ipython().inspector._get_info(func_name, detail_level=1)['text/plain'])
show_doc(SGD)

output:

4 Likes

Can any of you fine developers show me how to implement this?

When any cell is run, apply the function showType to the final value and display the resulting string in the output cell (along with the normal output).

Yes, I could study a pile of docs and examples to get it done, but maybe the method will be obvious to you. :slightly_smiling_face:

I’m starting a blog using the fast.ai template for GitHub pages. Thanks, this might be useful :slight_smile:

Hi @stas - I have refactored and renamed the package I put on PyPi, to make it easy to get either the current notebook name or path:

pip install ipynbname

Get the notebook name:

import ipynbname
nb_fname = ipynbname.name()

Get the full path to the notebook:

import ipynbname
nb_path = ipynbname.path()
2 Likes

You rock, @msm1089 - thank you for doing the work!

I can’t help but want to tell that the b (break) command in Pdb is super super helpful to debug/experiment the source code.

Before I usually copy/paste the source code in jupyter notebook to experiment (by running line by line or adding pdb.set_trace() ) . If doing so is not evident, I would open VSCode and quickly going through chain of definitions and guessing what the code is doing. Sometimes it doesn’t work. For example, when I want to check the definition of self.after_batch(b) at line 53 in data/core.py I get “no definition found”

With the b command, I can set break point to any function that I want in Jupyter Notebook. It’s super handy and save me a lot of time

Below is an example of getting inside the cluster_columns function in fastbook