Using PyCharm, fastai hangs loading data [SOLVED]

fastai
1

I’ve been using PyCharm for many months to trace and debug with fastai. The use is: Run->Attach to process…->attach to the notebook’s .json process. Set breakpoints in fastai, and it just works.

Starting a couple of days ago, PyCharm causes fastai to hang once “Attach to process” has been done. This may have started after installing fastai 1.0.47.post-1 and still happens with 1.0.48-1. The problem may not even be in fastai, only coincidental to the update. Maybe PyCharm updated itself silently - who knows?

Simplest case is:

data = (ImageList.from_csv(DATA,'train_labels.csv', folder='train', suffix='.tif', label_cls=FloatList)
           .split_by_rand_pct(valid_pct=0.1)
           .label_from_df()
           .add_test_folder('test')
           .databunch(bs=BATCH_SIZE)).normalize(imagenet_stats)

b1 = data.one_batch()

At this point, the notebook hangs (whenever PyTorch attach to process has been done previously). Interrupting the kernel in the notebook gives:

Traceback (most recent call last):
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 621, in trace_dispatch
    self.do_wait_suspend(thread, frame, event, arg)
Traceback (most recent call last):
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 125, in do_wait_suspend
    self._args[0].do_wait_suspend(*args, **kwargs)
  File "/snap/pycharm-community/117/helpers/pydev/pydevd.py", line 877, in do_wait_suspend
    time.sleep(0.01)
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 621, in trace_dispatch
    self.do_wait_suspend(thread, frame, event, arg)
KeyboardInterrupt
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 125, in do_wait_suspend
    self._args[0].do_wait_suspend(*args, **kwargs)
Traceback (most recent call last):
Traceback (most recent call last):
  File "/snap/pycharm-community/117/helpers/pydev/pydevd.py", line 877, in do_wait_suspend
    time.sleep(0.01)
KeyboardInterrupt
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 621, in trace_dispatch
    self.do_wait_suspend(thread, frame, event, arg)
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 621, in trace_dispatch
    self.do_wait_suspend(thread, frame, event, arg)
KeyboardInterrupt
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 125, in do_wait_suspend
    self._args[0].do_wait_suspend(*args, **kwargs)
  File "/snap/pycharm-community/117/helpers/pydev/pydevd.py", line 877, in do_wait_suspend
    time.sleep(0.01)
  File "/snap/pycharm-community/117/helpers/pydev/_pydevd_bundle/pydevd_frame.py", line 125, in do_wait_suspend
    self._args[0].do_wait_suspend(*args, **kwargs)
  File "/snap/pycharm-community/117/helpers/pydev/pydevd.py", line 877, in do_wait_suspend
    time.sleep(0.01)
KeyboardInterrupt

To further confuse the issue, I have a different notebook that does not hang! Can anyone offer fix or a direction? This problem is a total showstopper for my learning.

Thanks for any help.

2

The problem is having a PyCharm breakpoint set at FloatList.get(). PyCharm apparently stops at this breakpoint but never displays its debugger, showing instead that the notebook is still running.

Maybe it’s related to multiple processes spawned by fastai calling get(), and PyCharm not dealing with them correctly?

What relief to find a simple cause!