As a follow up to the original Productionizing models - I have encountered an issue:
I’m trying to build a server which trains models. Currently using CPU. When training in Jupyter notebook, I can see the training process is using “600%” CPU, meaning it’s taking advantage of all cores. But when running inside starlette, It’s only getting to 67% CPU.
I tried to fiddle with torch.set_num_threads=18
but didn’t seam to help.
Any ideas?