Parallel(...) hangs when used with lambda function. Why?

fastai
1

This hangs:
foo = parallel(lambda p,i: getPathDict(p), paths, max_workers=4

But this works:
def gpd(p,i): return getPathDict(p)
foo = parallel(gpd, paths, max_workers=4)

Why? And thank you!

Jupyter notebook/fastai 1.0.57/Python 3/Ubuntu.

2

Lambda functions can’t be pickled and multiprocess relies on that AFAIK.

1 Like
3

Thank you. That makes sense.