Speeding up inference in a `unet` model

Part 1 (2020)
1

I have a unet model for which I am trying to speed up inference.

After reading the wonderful Speeding Up fastai2 Inference - And A Few Things Learned article, this is what I attempted.

learner.model.eval()
loaders = learner.dls.test_dl(items)
    for batch in loaders:
        with learner.no_bar(), learner.no_logging(), torch.no_grad():
            inputs = batch[0]
            masks = learner.model(inputs)
            masks = learner.dls.decode_batch((inputs, masks))

The batch size i am using for the DataLoader is 32 during inference.
The shape of the inputs and masks after the invocation of learner.model(...) is torch.Size([32, 3, 240, 320]) as expected.

However, when I try and decode the predictions to get the TensorMask outputs by calling:

masks = learner.dls.decode_batch((inputs, masks))

I get a L list type with 9 fully decoded TensorMasks (not 32).

Am i doing something wrong ? Why am i not getting the fully decoded results for the full batch ?

Thanks,
Rahul

1 Like
2

D’oh. @rahulrav there’s a max_n parameter for decode_batch that you should pass along with it

1 Like
3

:man_facepalming:

Doh indeed. Thanks again @muellerzr

4

The question would be why it returned 9? instead of “other number”? I guess a default value?

5

See my comment on max_n. It’s default is 9 (n being the number used)

6

@muellerzr I am trying to perform batch inference for unet super-resolution model. Any documentation that I can refer to?

7

Hi @rahulrav. I am trying to perform an unet batch inference and I am not able to achieve that. Will you be able to share some documentation or repo where you have done unet inference?