How to use all CPU cores to run fastai to speedup training

fastai
1

Hi,

There is a CPU which has 10 CPU cores(20 CPU threads) on my Debian PC.
I installed fastai for CPU build.

When I run the sample code learn.fit_one_cycle(4) in lesson1-pets.ipynb to train the model,
I found my CPU usage is just about 20% rather than 100%. And only 4 CPU threads is fully running and other 16 CPU threads is idle.

Can someone know how to enable all my 20 CPU threads running ?

2

I solve this by change torch.set_num_threads(4) to torch.set_num_threads(20) in fastai/torch_core.py.
Now my PC can train much faster than before :grin:

2 Likes
3

I found fasfai not only open thread for torch, but also for other task which maybe image transform tast by pillow. So I reduce torch thread count from 20 to 9 which can further improve performance.