Progress bars in IPython Notebooks

The tqdm like progress bar is not appearing in my ipython notebook, however the same code when run on jupyter is shows the progress bars. Is there any easy way to display those progress bars in ipython?

In the previous version of fastai, the progress bar worked very well in both ipython and jupyter. Can this be supported? Or is there any easy hack?

Normally, if the notebook progressbar isn’t selected, it falls back on a console bar that prints the text (like if you execute with python script.py). There may be two causes for this bug:

  • does it print the epochs stats? If that’s the case, it may be because of my test to check if the script is redirected to a file
  • or doesn’t it print anything? If that’s the case, it’s the test to detect we’re in a notebook that is wrong.

This is what it prints when on ipython:

VBox(children=(HBox(children=(IntProgress(value=0, max=1), HTML(value=''))), HTML(value='epoch  train loss  valid loss<p>')))

Screenshot of the same:

Same thing in Jupyter:

I am not sure of why this is happening, as it seems to be printing the epoch stats, but I don’t see any file being created

Ok, it’s the test to determine if we’re in a notebook that doesn’t work. Will try to see why.

It seems to be working when invoked using python qnet_main.py, however using ipython console it gives error. I am guessing this is related to fast_progress here https://github.com/fastai/fast_progress/blob/master/fast_progress/fast_progress.py#L4

IN_NOTEBOOK returns true when invoked from jupyter or ipython console, returns false when invoked using the python command.

Yes, it’s hard to distinguish between the different IPython. Are you using Spyder or something else here?

Nope, simply ipython on my terminal.

A quick fix I am using at the moment is to clone the fast_progress repo
and do an editable install and use this code https://stackoverflow.com/questions/15411967/how-can-i-check-if-code-is-executed-in-the-ipython-notebook/39662359#39662359 to determine if it is a notebook.

It works now:

And also in the jupyter notebook:

To clarify: the top of the fast_progress.py file now looks like this:

I can send a pull request to fast_progress repo if you need.

Ok. This fix doesn’t catch Spyder though (just tried it on my laptop). I’m implementing it because it’s better than nothing as well as a function to force console_behavior for spyder users.

Pushed a few changes that I’ll send on Pypi later today for v0.1.4. It replaces IS_NOTEBOOK by your test and adds a function force_console_behavior that can force the behavior of console (as the name indicates) for users of Spider or qtconsole (I’ll fix this problem later if necessary). Usage:

from fast_progress import force_console_behavior
master_bar, progress_bar = force_console_behavior()

Also @radek wanted an option to have no bar at all and only the epochs stats printed, this behavior can now be obtained with

from fast_progress import force_console_behavior
import fast_progress
fast_progress.fast_progress.NO_BAR = True
master_bar, progress_bar = force_console_behavior()
2 Likes

Great. Can you check the output for spyder console for shell = get_ipython().__class__.__name__. That should be different from the normal ipython and python outputs and another if else statement should do the job. Though its not the most elegant solution.

Nope, this one gives the same as in a notebook. I have seen solutions for Spyder and QtConsole, it’s jsut going to take a bit of time and we have other priorities right now :wink:

1 Like

Important note: we have removed the package fast_progress and replaced it with fastprogress because conda didn’t like the _.
Don’t forget to update your code accordingly!

Sure. Thanks for the heads up. Might want to change them on the readme files as well :slight_smile:

1 Like

Good point, thanks for telling me!

fastprogress link in readme file of fastai should redirect to fast_progress repository, or may be you would want to change the name of the repo

I ran this with fastai_pytorch and fastprogress 0.1.5 and works great :slight_smile: Thx again Sylvain

Fastprogress is not working for me in Jupyter Lab but works in Jupyter notebooks with the new release.

There is an additional step to have it properly working in jupyter lab: you must have the jupyter widgets extension properly installed, which I think is done with:

jupyter labextension install @jupyter-widgets/jupyterlab-manager

Let me know if it solves your issue.

5 Likes

Hi @sgugger can you clarify how I can use the force_console_behavior to change the default display of progress bars with the fastai library. Once I have the master_bar and progress_bar how do I tell fastai to use them?

Once you have done that, every time you call fit or a function that uses progress bars, you’ll see the text rendering of a console.
Then you can use progress_bar on anything you want to track the progress of (see the fastprogress repo for more examples).