Get_preds() GPU utilization problem

fastai
1

Hey everyone,

We are experiencing a bottleneck with the get_preds() function while using a custom model for an image classification problem. In short, the GPU usage is too low.

Our workflow is the following:
imgs = get_image_files(image_dir)
learn = load_learner(f'{modelname}.pkl')
dl = learn.dls.test_dl(imgs, bs=batch_size, device=torch.device('cuda'))
predictions = learn.get_preds(dl=dl)

VRAM usage increases with the batch size, but GPU utilization is typically at 1%. Any suggestions about why this might be the case?

Thanks!

2

Try with ‘cpu=False’ in load_learner.

2 Likes
3

Oh, that did it, thanks! :slight_smile:

4

Going to ping this again, because I found that although it works perfectly with Ubuntu, this does not work on Windows.

When you run learn.get_preds on Windows, it first uses CPU ram and then runs the analysis using the GPU, which makes it much slower than on Ubuntu (where everything occurs in the GPU). Any suggestions why this might be different between the two OS?

Thanks!