Hey @stas, thanks for this. This is amazing!
I took what I learned here, combined with what I learnt through fast ai and daily use of Jupyter notebook into this one place: https://github.com/NirantK/best-of-jupyter
Hope you find this useful!
3 Likes
For those coming here for the first time, here are some direct links to what you do better in your Jupyter usage:
Contents
- Getting Started Right
-
Debugging - IPython debugger and
set_trace() -
Programming Sugar - using shell commands with Python from within notebook and other hacks
- Search Magic - search across several notebooks for a code snippet
-
Jupyter Kungfu - jupyter specific tips such as looking up docs in several ways
- Sanity Checks
-
️ nbdime - Better git difffor Jupyter - Markdown Printing - Use formatted Markdown in your print statements
- Skip Cell Execution - Jupyter magic to mark code as work in progress
- Find currently running cell - javascript snippet which adds a keyboard shortcut to find currently executing cell
- Better Mindset - covers broader Python recommendations
- Plotting and Visualization Tips
️
15 Likes
Tell the notebook to save itself now
juputer notebook by default autosaves itself every 5 min or so if you haven’t changed the defaults.
But if you want to make sure the notebook is saved at the end of the run, you can just insert a new cell at the end of your notebook and make sure you run it:
%%javascript
IPython.notebook.save_notebook()
now your notebook will be always saved as soon as it’s done running.
This is useful if you’re then immediately needing to commit the change to git.
credit: the idea came from here.
3 Likes
Python (not Jupyter) trick, but still useful for our long running Jupyter Notebooks:
When the code takes extremely long to run and I don’t want to be staring at it all the time but want to know when it is done.
How can I make the (Python) code sort of sound an “alarm” when it is done?
The following answer works only on server side. So it is useful only when you have a local server:
On Windows
import winsound
duration = 1000 # millisecond
freq = 440 # Hz
winsound.Beep(freq, duration)
Where freq is the frequency in Hz and the duration is in milliseconds.
On Linux (and Mac)
import os
duration = 1 # second
freq = 440 # Hz
os.system('play --no-show-progress --null --channels 1 synth %s sine %f' % (duration, freq))
In order to use this example, you must install sox .
On Debian/Ubuntu/LinuxMint you need to run in your terminal:
sudo apt install sox
Here is the macports way of doing that…run this is your terminal:
sudo port install sox
Speech on Mac
And something really cool if you’re using a mac in terminal, maybe can do the same in windows, but I only know for sure for mac, this will tell you it’s done:
import os
os.system('say "your program has finished"')
Speech on Linux
import os
os.system('spd-say "your program has finished"')
You need to install the speech-dispatcher package in Ubuntu (or the corresponding package on other distributions):
sudo apt install speech-dispatcher
Source:
https://stackoverflow.com/questions/16573051/sound-alarm-when-code-finishes/16573339#16573339
4 Likes
Make Esc always switch to Command mode
update: this problem should be fixed in the recent version of navigation-hotkeys.
If you use a very handy jupyter_contrib_nbextensions/nbextensions/navigation-hotkeys it introduces a very bad idea of changing the behavior of Esc to always switch to the opposite mode, making Esc unreliable as a precursor for a command mode shortcut call. Which leads to a lot of unpredictable behavior. So the following takes the goodness of the navigation-hotkeys extension and avoids the undesirable.
- disable
navigation-hotkeys - add to ~/.jupyter/custom/custom.js the following:
/* start a custom version of jupyter_contrib_nbextensions/nbextensions/navigation-hotkeys to remove Esc=> Edit mode toggle as it interferes with the workflow - Esc needs to be deterministic - switches to Command mode */
var add_command_shortcuts = {
'home' : {
help : 'go to top',
help_index : 'ga',
handler : function() {
IPython.notebook.select(0);
IPython.notebook.scroll_to_top();
return false;
}
},
'end' : {
help : 'go to bottom',
help_index : 'ga',
handler : function() {
var ncells = IPython.notebook.ncells();
IPython.notebook.select(ncells-1);
IPython.notebook.scroll_to_bottom();
return false;
}
},
'pageup' : {
help : 'page up',
help_index : 'aa',
handler : function() {
var wh = 0.6 * $(window).height();
var cell = IPython.notebook.get_selected_cell();
var h = 0;
/* loop until we have enough cells to span the size of the notebook window (= one page) */
do {
h += cell.element.height();
IPython.notebook.select_prev();
cell = IPython.notebook.get_selected_cell();
} while ( h < wh );
var cp = cell.element.position();
var sp = $('body').scrollTop();
if ( cp.top < sp) {
IPython.notebook.scroll_to_cell(IPython.notebook.get_selected_index(), 0);
}
cell.focus_cell();
return false;
}
},
'pagedown' : {
help : 'page down',
help_index : 'aa',
handler : function() {
/* jump to bottom if we are already in the last cell */
var ncells = IPython.notebook.ncells();
if ( IPython.notebook.get_selected_index()+1 == ncells) {
IPython.notebook.scroll_to_bottom();
return false;
}
var wh = 0.6*$(window).height();
var cell = IPython.notebook.get_selected_cell();
var h = 0;
/* loop until we have enough cells to span the size of the notebook window (= one page) */
do {
h += cell.element.height();
IPython.notebook.select_next();
cell = IPython.notebook.get_selected_cell();
} while ( h < wh );
cell.focus_cell();
return false;
}
}
};
var add_edit_shortcuts = {
'alt-add' : {
help : 'split cell',
help_index : 'eb',
handler : function() {
IPython.notebook.split_cell();
IPython.notebook.edit_mode();
return false;
}
},
'alt-subtract' : {
help : 'merge cell',
help_index : 'eb',
handler : function() {
var i = IPython.notebook.get_selected_index();
if (i > 0) {
var l = IPython.notebook.get_cell(i-1).code_mirror.lineCount();
IPython.notebook.merge_cell_above();
IPython.notebook.get_selected_cell().code_mirror.setCursor(l,0);
}
}
},
'shift-enter' : {
help : 'run cell, select next codecell',
help_index : 'bb',
handler : function() {
IPython.notebook.execute_cell_and_select_below();
var rendered = IPython.notebook.get_selected_cell().rendered;
var ccell = IPython.notebook.get_selected_cell().cell_type;
if (rendered === false || ccell === 'code') IPython.notebook.edit_mode();
return false;
}
},
'ctrl-enter' : {
help : 'run cell',
help_index : 'bb',
handler : function() {
var cell = IPython.notebook.get_selected_cell();
var mode = cell.mode;
cell.execute();
if (mode === "edit") IPython.notebook.edit_mode();
return false;
}
},
'alt-n' : {
help : 'toggle line numbers',
help_index : 'xy',
handler : function() {
var cell = IPython.notebook.get_selected_cell();
cell.toggle_line_numbers();
return false;
}
},
'pagedown' : {
help : 'page down',
help_index : 'xy',
handler : function() {
var ic = IPython.notebook.get_selected_index();
var cells = IPython.notebook.get_cells();
var i, h=0;
for (i=0; i < ic; i ++) {
h += cells[i].element.height();
}
var cur = cells[ic].code_mirror.getCursor();
h += cells[ic].code_mirror.defaultTextHeight() * cur.line;
IPython.notebook.element.animate({scrollTop:h}, 0);
return false;
}
},
'pageup' : {
help : 'page down',
help_index : 'xy',
handler : function() {
var ic = IPython.notebook.get_selected_index();
var cells = IPython.notebook.get_cells();
var i, h=0;
for (i=0; i < ic; i ++) {
h += cells[i].element.height();
}
var cur =cells[ic].code_mirror.getCursor();
h += cells[ic].code_mirror.defaultTextHeight() * cur.line;
IPython.notebook.element.animate({scrollTop:h}, 0);
return false;
}
},
'ctrl-y' : {
help : 'toggle markdown/code',
handler : function() {
var cell = IPython.notebook.get_selected_cell();
var cur = cell.code_mirror.getCursor();
if (cell.cell_type == 'code') {
IPython.notebook.command_mode();
IPython.notebook.to_markdown();
IPython.notebook.edit_mode();
cell = IPython.notebook.get_selected_cell();
cell.code_mirror.setCursor(cur);
} else if (cell.cell_type == 'markdown') {
IPython.notebook.command_mode();
IPython.notebook.to_code();
IPython.notebook.edit_mode();
cell = IPython.notebook.get_selected_cell();
cell.code_mirror.setCursor(cur);
}
return false;
}
}
};
Jupyter.keyboard_manager.edit_shortcuts.add_shortcuts(add_edit_shortcuts);
Jupyter.keyboard_manager.command_shortcuts.add_shortcuts(add_command_shortcuts);
/* end a custom version of jupyter_contrib_nbextensions/nbextensions/navigation-hotkeys */
1 Like
Restoring Jupyter client’s GUI (scores and graphs…etc) after client disconnects for long time
Requirements:
- we should use tmux to keep the jupyter server running, even after ssh disconnects. Without tmux, jupyter server will be killed after ssh connection disconnects.
- The client (e.g., your pc/laptop showing the jupyter notebook) may go to hibernation or sleep without problem, but the tab of the browser showing the jupyter clients should not be closed. Only that tab that began running the notebook will be able to restore the whole running GUI elements of the notebook.
- Increase 2 parameters in the jupyter config file:
nano /home/<username>/.jupyter/jupyter_notebook_config.py
if config file is not available, we should generate it by:
jupyter notebook --generate-config
ctrl+w: search for c.NotebookApp.iopub_data_rate_limit
Uncomment and change these values into something very large:
c.NotebookApp.iopub_msg_rate_limit = 100000000
c.NotebookApp.iopub_data_rate_limit = 10000000000
2 Likes
Automatic Notebook Halt / Restart cell
When included at the bottom of a notebook, it will stop the notebook at the end of Run All, and pop up a modal dialog letting the user know that the notebook was auto-stopped at the end of the run. The dialog offers a “Kernel Restart” button which restarts the notebook.
%%javascript
require(
["base/js/dialog"],
function(dialog) {
dialog.modal({
title: 'Notebook Halted',
body: 'This notebook is no longer running; the kernel has been halted. Close the browser tab, or, to continue working, restart the kernel.',
buttons: {
'Kernel restart': { click: function(){ Jupyter.notebook.session.restart(); } }
}
});
}
);
Jupyter.notebook.session.delete();
Use plotly instead of matplotlib to get better-looking and interactive chart
For those used to matplotlib , all we have to do is add one more letter ( iplot instead of plot ) and we get a much better-looking and interactive chart! We can click on the data to get more details, zoom into sections of the plot, and as we’ll see later, select different categories to highlight.
- One-line charts for rapid exploration
- Interactive elements for subsetting/investigating data
- Option to dig into details as needed
- Easy customization for final presentation
That’s just as simple:
df['claps'].iplot(kind='hist', xTitle='claps', yTitle='count', title='Claps Distribution')
Try it now with nbviewer live demo:
The plotly Python package is an open-source library built on plotly.js which in turn is built on d3.js . This code is using a wrapper on plotly called cufflinks designed to work with Pandas dataframes. So, our entire stack is cufflinks > plotly > plotly.js > d3.js which means we get the efficiency of coding in Python with the incredible interactive graphics capabilities of d3.
source:
P.S.: For jupyter Lab you should install also:
jupyter labextension install @jupyterlab/plotly-extension
3 Likes
-
Shift + Tabcycles through help info like docstring -
Ctrl + Shift + -splits the current cell from cursor -
Ctrl + /comments out selected lines in the cell
1 Like
How to add a keyboard shortcut to insert a code snippet
Inspired by https://stackoverflow.com/a/51719689/9201239
and some help from @stas
If you first enter a cell with either code mode or markdown mode, like the following
Then you press Ctrl + Shift + M, you will get the following
To make this happening, you just need to
- go to
custom.jswhose path can be found by running
echo $(jupyter --config-dir)/custom/custom.js
- add the following codes into
custom.jsand save.
Jupyter.keyboard_manager.edit_shortcuts.add_shortcut('Ctrl-Shift-M', {
help : 'add details drop',
help_index : 'zz',
handler : function (event) {
var target = Jupyter.notebook.get_selected_cell()
var cursor = target.code_mirror.getCursor()
var before = target.get_pre_cursor()
var after = target.get_post_cursor()
target.set_text(before + '[/details][details=""]' + after)
cursor.ch += 20 // where to put your cursor
target.code_mirror.setCursor(cursor)
return false;
}}
);
- finally, go back to Jupyter Notebook and refresh your page, and you are ready to go.
1 Like
How to create a multi-line code snippet for Jupyter notebook
First, install nbextensions
conda install jupyter_contrib_nbextensions
jupyter contrib nbextension install --user
Second, go to your jupyter notebook and open nbextensions settings to add features
Third, open custom.js file and add the following codes
require(["nbextensions/snippets_menu/main"], function (snippets_menu) {
console.log('Loading `snippets_menu` customizations from `custom.js`');
var horizontal_line = '---';
var my_favorites = {
'name' : 'My favorites',
'sub-menu' : [
{
'name' : 'most_jupyter_magics',
'snippet' : ['%reload_ext autoreload',
'%autoreload 2',
'%matplotlib inline',
'from IPython.core.interactiveshell import InteractiveShell',
'InteractiveShell.ast_node_interactivity = "all"',],
},
{
'name' : 'kaggle download links',
'snippet' : ['from IPython.display import FileLinks',
'FileLinks(\'.\')',],
},
{
'name' : 'details_drop',
'snippet' : ['[/details][details=""]',],
},
{
'name' : 'find snippet custom js',
'snippet' : ['echo $(jupyter --config-dir)/custom/custom.js',],
},
],
};
snippets_menu.options['menus'].push(snippets_menu.default_menus[0]);
snippets_menu.options['menus'][0]['sub-menu'].push(horizontal_line);
snippets_menu.options['menus'][0]['sub-menu'].push(my_favorites);
console.log('Loaded `snippets_menu` customizations from `custom.js`');
});
Finally, refresh your notebook page, and you shall see the following
reference:
Jupyter docs on snippet
4 Likes
Getting the notebook name automatically
# cell 1
%%javascript
IPython.notebook.kernel.execute('nb_name = ' + '"' + IPython.notebook.notebook_name + '"')
# cell 2
nb_name
Same without js magic:
# cell 1
from IPython.display import display, Javascript
Javascript("IPython.notebook.kernel.execute('nb_name = ' + '\"' + IPython.notebook.notebook_name + '\"')")
# cell 2
nb_name
In either case, JS is async - so it may not run right away and therefore it’s not guaranteed nb_name will be set right away.
I was looking for this feature, since I wanted to replace hardcoded cells in each dev_nbs such as:
!./notebook2script.py 02_fully_connected.ipynb
with something similar that will extract the nb name automatically.
So I wrote a little helper function:
from IPython.display import display, Javascript
def nb_auto_export():
display(Javascript("if (IPython.notebook.kernel) {IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name )}"))
(it looks like JS is the only way to get the nb name in jupyter 
)
notes:
- it assumes the nb name has no spaces in it, add quotes if it does
- had to bracket the code with
if (IPython.notebook.kernel) {}since the browser will attempt to run this js code automatically on jupyter nb load and fail, sinceIPython.notebook.kernelhasn’t been defined yet. It only happens if there is!in the js code - odd.
and now there is no need to hardcode the nb name, just have the last cell say:
nb_auto_export()
that is if you imported nb_auto_export from somewhere you saved it.
Or alternatively without needing to import anything, you could just have the last cell of each notebook:
%%javascript
if (IPython.notebook.kernel) {
IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name)
}
Also note that it’s async, so it usually takes a sec or so to start running once the nb finished running.
Or the messier approach w/o js magic:
from IPython.display import display, Javascript
display(Javascript("if (IPython.notebook.kernel) {IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name)}"))
You might be able to drop display from it, but it seems not to work w/o it for me.
Finally, you shouldn’t rely on auto-save, since you’re likely to miss recent changes, so why not tell this code to save the notebook first, resulting in this code:
from IPython.display import display, Javascript
def nb_auto_export():
display(Javascript("if (IPython.notebook) { IPython.notebook.save_notebook() }; if (IPython.notebook.kernel) { IPython.notebook.kernel.execute('!./notebook2script.py ' + IPython.notebook.notebook_name )}"))
2 Likes
With a bit of Python import glue, you could also skip the export step and just import the notebooks (I used a %lib magic to indicate the things actually I want to show up in the library, but you could remove that and take it all). Maybe that can be even more user friendly than having to export things.
Best regards
Thomas
2 Likes
Interactive Labeling:
I have some python code that I would like to use for interactive labeling of sections in a matplotlib plot using Jupyter Notebook. The code seems to work in python 2.7 under GUI, but will not run properly in a notebook. The problem is that the matplotlib close_event never leaves the cell containing the plot.
I’ve put together an example to demonstrate. If anyone has some working Jupyter Notebook code that can be used to label sections of a plot interactively please let me know.
Here is the sample code:
The notebook is here
Please do not use this thread for questions or problem reports, instead post them here:
as explained in the first post. Thank you.
AutoScroll and go to current cell function are now included in the extension. The extension is in master branch but a new pip version is not released yet.
For now, you can install through
pip install https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tarball/master
The PR for go to current cell
You can customize the shortkey yourself if needed.
1 Like
Undo cell delete
2 Likes
Hey, I tried doing this and it gives me an error. I don’t understand why this is? Please help.
I did a simple
%%script false
for i in range(10): print(i)
1 Like
Indeed, I’m able to reproduce this.
%%script false
no longer works.
Since that behavior was never documented (or was intended to work) they must have dropped it in the recent versions of ipython.
I’m not sure why I no longer am able to edit my posts here, but here are workarounds, based on programs ignoring their arguments when you tell them not to expect any. Here are some easy examples:
Perl:
%%perl -e0
for i in range(10): print(i)
Here you’re running: perl -e '0' cellcontents
A more memorable version:
%%perl -eat
for i in range(10): print(i)
Here you’re running: perl -e 'at' cellcontents
Bash:
%%bash -c :
for i in range(10): print(i)
‘:’ is a noop in bash, so you’re running: bash -c : cellcontents
I haven’t looked at the external magic implementation code, but I’m pretty sure “cellcontents” are passed as arguments and won’t be interpreted by shell by mistake, say if you were to include ‘;’ in them and accidentally inject some bad code. But I can’t guarantee you that.
I’m sure you can come up with other creative solutions, by looking at the supported programs here: https://ipython.readthedocs.io/en/stable/interactive/magics.html#cell-magics
p.s. perhaps a forum admin with edit rights could tweak this comment to indicate that it no longer works and send readers to this post instead. Thank you.
p.p.s. I discovered the original hack here.
I’ve made the earlier post a wiki so you should be able to edit it.
1 Like