Jupyter Notebook vs IDE

Part 2 (2017)
1

Ive been doing software development + some machine learning for a few years using eclipse + pydev + webstorm for js. For small projects ill ocasionally use pycharn or vs code.

The way i have been working with the jupyter notebooks for part 1 and 2 this and last year is to convert them to .py files then wrap sections in functions so i can step though the nb in 4-10 steps in a workflow.

Advantages: ide search, browsing of source code for libraries, auto completion, awesome debugger in pydev.

Disadvantages: cant pause / replay a cell like you can in jupyter nb - can use debugger but is gonna really slow intensive code down.

I have really been trying to use jupyter nb the last few days but find lact of being able to one click go to definition and browse around pretty debilitating.

I have tried:

Pycharm + embedded jupyter notebook: really sliw, kludgy, need to set inteactive mpl on to get plots to work, doesnt cut it for me.

Vs code + jupyter extension- need to work in .py files, can run cells but hard to keep track of where you are, hard to see the run cell button: not for me.

Wing ide : too complicated to get notbook running inside.

Atom + hydrogen: couldnt get to work

Spyder ide: dev seems to be stalled

Rodeo: couldnt install, dead project.

So that leaves me back with either jupyter lab/notebook and eclipse/pycharm.

May be that i write in notebook and use an ide with source cose for fastai/pytorch open on another screen so i can work out whats going on.

Anyone else have workarounds for limitations of jupyter notebook?

2

I don’t really have a problem with the limitations of jupyter, but I can see how its about what you’re used to.
it may be that back in the day I did a lot of matlab - so I’m used to that type of environment (I’ve used IDEs too)

this helped me get the most out of jupyter notebook and its extensions. I haven’t tried labs yet

I don’t really understand ide search advantage. jupyter lets you search code only, not output (esc f)
??function() lets you easily get to code (ok a little more work to nest). or can enter debugger and step through code to view. (there’s possibly an ‘open file’ shortcut from within that I’ve never needed)
the debugger is sufficient for my needs

3

Thanks, some good info in the link. Ive been trying out the extensions. I think its a matter of persistence, and getting used to the notebook workflow.

Saying that, I have been using PyCharm with an embedded notebook today and am getting acceptable performance with only minor issues:

Increased heap size and startup memory size in pycharm–version–/bin/pycharm64.vmoptions:

-Xmx2048m  
-Xms1024m

To ensure plots display in a pycharm .ipynb at start of the notebook I set

plt.ion()

Then after each plot I set

plt.show()

sometimes I also need to add

plt.savefig('temp.png')

and for some reason the plot then displays.

Another annoying feature on the pycharm ipynb is that the last output of learn.fit() is not printed. I added the following to the end of the function model.print_stats() to force flush the buffer

def print_stats(epoch, values, decimal=6):
    ...
    sys.stdout.flush()

The advantages of having autocompletion after typing n characters and nested one click go to definition and browse around for me when learning the fastai package and pytorch, outweigh the annoying ‘features’ of PyCharm (like lack of notebook shortcuts), at least until I am better versed in the fastai code. (Once I have the skeleton written out and just need to make some tweaks, or run something overnight I can just run the notebook through Jupyter if necessary).

Pycharm also sometimes ‘formats’ .ipydb files in such a way that they wont open fully anymore in jupyter notebook. However jupyter lab has been able to open these files ok.

1 Like
4

Just wanted to let you know that it is now possible to jump to the definition of a variable in JupyterLab with an extension of mine: jupyterlab-go-to-definition. It may not be as powerful as the PyCharm’s one, but should be good enough for a start.

Sorry for reviving an old thread, it’s just that I’ve seen this topic a while ago now, and it convinced me that working on this extension would be useful for others. Hope this will work for you too.

1 Like
5

This is really handy. For me alt+lmb doesn’t work but CTL+alt+b works fine (Ubuntu Linux).

(I actually gave up using pycharm for .ipynb’s a while ago and either use Jupiter lab or notebook. - but probably just lab now with this jump to feature).