(fyi, I moved your report and my followup to the thread where it belongs so that we don’t discuss off-topic things there.)
Yup, I’ve been battling with this one for a while.
It seems to have to do with python threads. There is a peak memory manager python thread and there is the ipython callback thread . I need to be able to do an atomic check and quit the thread if that check fails, but I’m not sure how one goes about this in python threads. What happens now is that it intermittently fails at:
where it succeeds at the conditional check, and gets immediately yilded to the main process, and when it comes back
do_something fails, because the condition is no longer true.
Because of two overlapping contexts - cell-level and notebook level, which reference each other in order to avoid circular references and have
del exp do the right thing, I use a weakref proxy - that’s where it fails, since the proxy is gone and the thread still wants to run. so I’m not quite sure how to resolve this race condition.
And I can’t kill the ipython thread, that would be a disaster.
And if I keep the real parent object and not a proxy the sub-system will prevent it from being destroyed, which defeats the purpose of the experiment.
Perhaps I made a design mistake and it needs to be redone.
If you have the know-how please have a look.