I have a
unet model for which I am trying to speed up inference.
After reading the wonderful Speeding Up fastai2 Inference - And A Few Things Learned article, this is what I attempted.
learner.model.eval() loaders = learner.dls.test_dl(items) for batch in loaders: with learner.no_bar(), learner.no_logging(), torch.no_grad(): inputs = batch masks = learner.model(inputs) masks = learner.dls.decode_batch((inputs, masks))
The batch size i am using for the
32 during inference.
The shape of the inputs and masks after the invocation of
torch.Size([32, 3, 240, 320]) as expected.
However, when I try and decode the predictions to get the
TensorMask outputs by calling:
masks = learner.dls.decode_batch((inputs, masks))
I get a
L list type with
9 fully decoded
TensorMasks (not 32).
Am i doing something wrong ? Why am i not getting the fully decoded results for the full batch ?