PyCharm setup help

(Jason Patnick) #4

do you still have the steps to set up PyCharm?

0 Likes

(Malcolm McLean) #5

Hi Patty. It has been a number of months, but I will try to retrieve the steps from my current setup.

First, make a File->New Project. Mine was initially set to ~/fastai/courses/dl1. At this point, the dialog wants a Project Interpreter. Choose Existing Interpreter, …, Conda Environment, …, Select Python Interpreter.

Here, mine is set to ~anaconda3/envs/fastai/bin/python

I hesitate because I recall that instead of selecting the interpreter while creating the project, I went next to File->Settings->Project:dl1->Project Interpreter. Then in the popup or in “Show All…” that interpreter was already listed to be simply selected. And my philosophy when using complex systems I don’t understand is to let the automatic process do it, rather than risk messing something up manually. Good luck!

A couple of notes…

  • Pycharm will open a Jupyter notebook and display it, but it won’t run or debug. Furthermore, it will damage the notebook when automatically resaving any edits. Then the notebook no longer opens in Jupyter. Whoops!

  • What I do is, from inside the notebooks, first clear all output, and File->Download As->Python(.py). Then work in PyCharm with that copy.

  • The Jupyter download is pretty half-baked, messing up comments and leaving a bunch of “In[] :” to clean up.

  • You will have to remove the Jupyter magic commands and put print() around any bare expressions you’d like to see in the console.

  • I think I had to tweak the inline plotting commands, The upside is that plots end up in separate windows, where they can be examined in detail.

  • And you can now make use of source code navigation, debugging, and tracing.

HTH you to get started. Mostly I have used PyCharm to explore fastai code and to run my own simple experiments.

If anyone has found a way to invoke the PyCharm or VS Code debugger/navigation from within a Jupyter notebook, I would love to hear about it.

0 Likes

(Jason Patnick) #6

Thanks for writing all that out! unfortunately im still stuck. so i have the the fastai environment set up to use with jupyter notebook, but all of the locations and project interpreters to choose from in pycharm start with C: and not ~. maybe theres something easy, but i cant figure out how to get pycharm to look somewhere besides C: or how to go from C: to the fastai environment. maybe its because of where i had pycharm installed? thanks for your help!

0 Likes

(Malcolm McLean) #7

Hmm, you are working in the Windows world, and my experience is with Linux.

The tilde represents the home folder, which in Windows is usually C:\Users\yourName. But I am neither a Windows expert nor know where Anaconda, fastai, and PyCharm are installed there.

I suggest looking for the corresponding anaconda3 folder mentioned above, either in C:, or in your home folder. Beyond that, there’s google and any Windows gurus on this forum. Sorry i can’t help more in this case.

0 Likes

(Jason Patnick) #8

So i tried setting it up through a conda environment and i located it through C:. it let me apply the location and conda executable and then it gave me details that said it cant be run because its not a Win32 program. could this be because its set up through the Ubuntu subsytem? it wont let me get to python by setting it up as a virtualenv or a system interpreter.

0 Likes

(Malcolm McLean) #9

This problem is completely outside my expertise. However, there is a whole forum area devoted to Windows installation. Perhaps someone there could help.

My own solution was to convert my Windows installation to dual boot Ubuntu/Windows. (There are many tutorials online on how to make this conversion.) Ubuntu has proven to be trouble free; besides, more students are using Linux, therefore more support and advice is available.

0 Likes

(Jason Patnick) #10

figured it out, i just put the fastai repo in my user folder and then a python.exe in the fastai in fastai folder. then in pycharm the location for the project can be where ever, and then I chose existing interpreter as the python.exe from C:\Users\me\fastai\fastai\python.exe
and it automatically loads in all the libraries/dependencies for fastai into pycharm

and i went through the very beginning of lesson 1 (the imports, torch.cuda.is_available(), and torch.backends.cudnn.enabled) without any errors and they both returned True

0 Likes

(adrian) #11

A trick i use to overcome pycharm .ipynb corruption issues is to rysnc my source code directory to a copy of it just for pycharm. I can then jump to source from within the pycharm nb editor. Just need to run rsync each time you change original notebook.

0 Likes

(Brian Perkins) #12

If you are using linux it would be more efficient to use a symlink instead of copying the repo. The link points back to where the real install of fastai’s packages are. For instance, I have a pyCharm project where fastai’s stuff is 3 directories above me. At a command prompt, I go in the project directory and type
ln …/…/…/fastai ./fastai
This creates a link back to the fastai source with no copying.
Presto! Everything works!

1 Like

(Iaroslav Russkykh) #13

debug notebook from PyCharm

tested on Ubuntu 16.04 LTS, PyCharm 2018.3 Pro(me) PyCharm Community(@Pomo). Python 3.6.x. fast-ai 1.0.28.dev0(me). My opinion about other Python/OS usage here
PyCharm and Jupyter must use the same python environment (not sure about it, but I use this setup)

  1. move code which you want to debug to separate module
  2. start Jupyter server in terminal window:
cd <project dir>
source activate <env>
jupyter notebook
  1. in PyCharm
  1. in Jupyter:
  • create notebook
  • import module
  • run code
    Screenshot%20from%202018-11-23%2002-55-34
  1. profit
2 Likes

Jupyter Notebook Enhancements, Tips And Tricks
Pycharm can not open .ipynb file
(Malcolm McLean) #14

Thanks so much for this! It works perfectly in same configuration with PyCharm Community after doing the connection error fix.

To be more specific, you can create an empty project with its Project Interpreter set to the desired fastai conda environment. Then set breakpoints at External Libraries->Python 3.6->site-packages->fastai anywhere. Attach to the Jupyter process as @elruso shows above.

Then as notebook cells are run, PyCharm will break and allow tracing through the fastai library. In this simple use, there’s no need to place code into a separate module.

What a relief - Jupyter notebook with a fully functional debugger and source navigator.

1 Like

(Haider Alwasiti) #15

@Pomo @elruso
What are your OS and Python versions?

Seems the attach to local process is failing in Ubuntu 18 and Win 10 with Python 3.7

Bug report
https://youtrack.jetbrains.com/issue/PY-30064


Unfortunately, it seems with every new Python version, seems this would not work. And needs several months to debug it and make it work again. Pycharm is relying on this open source project.

0 Likes

(Malcolm McLean) #16

Ubuntu 16.04/Python 3.6.7

0 Likes

(Haider Alwasiti) #17

Yep. I tested it on my older fastai 0.7 with Python 3.6.6 environment (Win 10) and it worked without any issue. And as you said, I can create a python file and call a simple function inside this file that has a breakpoint, and it will stop right there if I call it from a jupyter cell. Not necessary to be part of fastai library.

It seems I have to wait for Python 3.7 support. The new fastai v1 upgraded my Python 3.6 to Python 3.7.

Do you use fastai v1 with Python 3.6 ?

0 Likes

(Iaroslav Russkykh) #18

16.04 LTS Python 3.6.x
CAVEAT I use compiled from the source OpenCV for my needs. I you can work inside the world of precompiled packages this opinion can be irrelevant to you

  • About Ubuntu 18.04. I’m not recommend to use 18.04, you’ll face in dependences hell.
  • About Python 3.7. I had problems with libs, I would have waited while all underling libraries will start to support it before switching to it. Currently one will got a lot of headache and no benefits from 3.7.
  • About Windows. IMO it’s a bad idea to use it. If you will need to compile something from source you’ll enter in the house of pain
0 Likes

(Iaroslav Russkykh) #19

Do you use fastai v1 with Python 3.6 ?

Yes. All working without a problems with fast-ai 1.0.28.dev0

0 Likes

(Iaroslav Russkykh) #20

How it’s possible? Can you provide more details? Python libs can’t update environment Python version as I know.

0 Likes

(Haider Alwasiti) #21

All my conda environments are created with python 3.6 by default. When I created a new fastai-v1 conda env it was python 3.6, but when I have installed fastai:
conda install -c fastai fastai
It installed and updated its dependencies.

One of the things it did, updated python 3.6 to 3.7.1.

For the last fastai update (v1.0.37 I think) it even downgraded python from v3.7.1 to 3.7.0

Maybe because you are cloning the repo and using the dev branch of fastai, so you are keeping python version 3.6?

But it is interesting that it is not necessary to use python 3.7 and maybe I will try to downgrade it to Python 3.6.

Pycharm debugging is awesome and it worth it…

However, it is clear that there is a reason Jeremy made python dependency version changes (to 3.7.1 then to 3.7.0). Maybe there will be something relevant that I will face if I use 3.6. Or maybe he is just trying to make fastai compatible with the most recent 3.7 and should work fine with 3.6.

Agree. But the forum threads for installing fastai v1 was encouraging, and I haven’t seen any issues so far with all the lessons of fastai. I do not have any other major work on Ubuntu other than fastai. And the GUI of v18 is so much better. Especially with multi-monitor setup.

So when do you think it is time to upgrade to Ubuntu 18.0.4 ?

I have other working Windows pc that is essential for my work with a GPU, so I am interested in Windows too. But agree that Ubuntu is much better for DL.

0 Likes

(Malcolm McLean) #22

Just downgrade to Python 3.6. Both @elruso and I have verified that fastai v1 works correctly.

0 Likes

(Haider Alwasiti) #23

Yep. And I will switch to Ubuntu 16.0.4.

I’ve just now seen that even Jeremy uses Python 3.6 with Pytorch v1 stable:

0 Likes