Inference using load_learner

fastai
1

I’ve trained a u-net for semantic segmentation and now I want to try making predictions on a set of test images contained in a separate folder.

After much searching I came across the article “Deep Learning on a Shoestring” (https://docs.fast.ai/tutorial.resources.html)

This seemed to have exactly what I wanted - the ability to save a trained network and then load it again explicitly for inference, with the suggested code being this:

# end of training
learn.fit_one_cycle(epochs)
learn.freeze()
learn.export()
learn.purge()

# beginning of inferences
learn = load_learner(path, test=ImageItemList.from_folder(path/'test'))
preds = learn.get_preds(ds_type=DatasetType.Test)

However, when I run this I get the following error:

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/opt/conda/lib/python3.6/site-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs)
    537         x = ds[0]
--> 538         try: x.apply_tfms(tfms, **kwargs)
    539         except Exception as e:

/opt/conda/lib/python3.6/site-packages/fastai/core.py in apply_tfms(self, tfms, **kwargs)
    156         "Subclass this method if you want to apply data augmentation with `tfms` to this `ItemBase`."
--> 157         if tfms: raise Exception(f"Not implemented: you can't apply transforms to this type of item ({self.__class__.__name__})")
    158         return self

Exception: Not implemented: you can't apply transforms to this type of item (EmptyLabel)

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
<ipython-input-57-a6feb4485995> in <module>()
      1 # beginning of inferences
----> 2 learn = load_learner('/kaggle/working', test=ImageItemList.from_folder(path_test_img))
      3 preds = learn.get_preds(ds_type=DatasetType.Test)

/opt/conda/lib/python3.6/site-packages/fastai/basic_train.py in load_learner(path, fname, test)
    502     model = state.pop('model')
    503     src = LabelLists.load_state(path, state.pop('data'))
--> 504     if test is not None: src.add_test(test)
    505     data = src.databunch()
    506     cb_state = state.pop('cb_state')

/opt/conda/lib/python3.6/site-packages/fastai/data_block.py in add_test(self, items, label)
    508         if isinstance(items, ItemList): items = self.valid.x.new(items.items, xtra=items.xtra).process()
    509         else: items = self.valid.x.new(items).process()
--> 510         self.test = self.valid.new(items, labels)
    511         return self
    512 

/opt/conda/lib/python3.6/site-packages/fastai/data_block.py in new(self, x, y, **kwargs)
    573     def new(self, x, y, **kwargs)->'LabelList':
    574         if isinstance(x, ItemList):
--> 575             return self.__class__(x, y, tfms=self.tfms, tfm_y=self.tfm_y, **self.tfmargs)
    576         else:
    577             return self.new(self.x.new(x, **kwargs), self.y.new(y, **kwargs)).process()

/opt/conda/lib/python3.6/site-packages/fastai/data_block.py in __init__(self, x, y, tfms, tfm_y, **kwargs)
    546         self.y.x = x
    547         self.item=None
--> 548         self.transform(tfms, **kwargs)
    549 
    550     def __len__(self)->int: return len(self.x) if self.item is None else 1

/opt/conda/lib/python3.6/site-packages/fastai/data_block.py in transform(self, tfms, tfm_y, **kwargs)
    663         _check_kwargs(self.x, tfms, **kwargs)
    664         if tfm_y is None: tfm_y = self.tfm_y
--> 665         if tfm_y: _check_kwargs(self.y, tfms, **kwargs)
    666         self.tfms,self.tfmargs = tfms,kwargs
    667         self.tfm_y,self.tfms_y,self.tfmargs_y = tfm_y,tfms,kwargs

/opt/conda/lib/python3.6/site-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs)
    538         try: x.apply_tfms(tfms, **kwargs)
    539         except Exception as e:
--> 540             raise Exception(f"It's not possible to apply those transforms to your dataset:\n {e}")
    541 
    542 class LabelList(Dataset):

Exception: It's not possible to apply those transforms to your dataset:
 Not implemented: you can't apply transforms to this type of item (EmptyLabel)

It appears to be wanting to apply a transform or add labels to the test data - I wouldn’t want either of these things to happen, since I want the un-altered image to go through the network and for this to produce the label.

Any pointers to how I could fix this would be most appreciated!

6 Likes
2

The Learner you export contains the transforms you passed to the training and validation set. For inference, it applies the transforms you had set on the validation set, and I’m guessing those are non-empty.
Beware you will need some post processing to match your results to your images if that’s the case, and that some transforms aren’t reversible.

2 Likes
3

I am also trying to do inference on a trained unet model for semantic segmentation. I can perform inference using a workflow of loading the previously saved Learner with load_learner and then calling predict on single images at a time. I am interested in being able to do this with batches of images on the GPU to get better performance.

However when I try to add a test DataBunch to the bunch associated with the (restored) Learner using bunch.add_test an exception is thrown:

Exception: It’s not possible to apply those transforms to your dataset:
Not implemented: you can’t apply transforms to this type of item (EmptyLabel)

I would appreciate some guidance on the recommended way to do batch processing inference on a folder of test images (without labels). I am using fastai 1.0.46

A feature request would be to make batch inference processing as straight-forward and easy as it now is for single images. The predict(image) method seems to handle all the logic related to transforms, normalization, etc.

Thank you.

1 Like
4

Be very careful when applying transforms at inference time for segmentation (or any task that modifies the images): your prediction is going to be for the transformed image, not the original image. That’s why there is this error.
You can avoid it by monkey-patching EmpyLabel like this:

def no_tfms(self, x): return x
EmptyLabel.apply_tfms = no_tfms

but that’s not normally a good idea.

5

Thank you for the information. Rather than apply a hack, is there an alternative approach for restoring a Learner and then batch processing inferences you could suggest more in keeping with best practices on how to use fastai? I would guess that the use case of wanting to predict on many images at once is fairly common vs. the use case of responding to single requests from web clients.

2 Likes
6

The regular approach does that. it’s just that if your images need to be resized, then your predictions will be on that resized image and will need post-processing. So you should manually resize the images, keep track of their original size, then apply your deployed learner on batches of resized images (without transforms) then do the inverse resize on your predictions.

7

It seems I must be missing something… can you please explain a bit more on what you mean by the “regular approach”.

I’ve previously tried the approach described at https://docs.fast.ai/tutorial.inference.html,

    item_list = ItemList.from_folder(input_dir, extensions=['.png'], recurse=True)
    learn = load_learner(model_root, fname=model_name,  test=item_list) 
    preds,y = learn.get_preds(ds_type=DatasetType.Test)

When I try this I also get an exception on the call to load_learner related to transforms on EmptyLabel

Exception: Not implemented: you can't apply transforms to this type of item (EmptyLabel)
2 Likes
8

Yes, because you can’t apply transforms to your images for reasons I explained earlier, otherwise your predictions will be off.

1 Like
9

I understand that transforms should not be applied during inference.

The issue I’m facing is that it does not even seem possible to add a Test DataSet to a Learner that has been restored through a call to load_learner.

Using load_learner with a test ItemList as input fails immediately. Using load_learner without providing a test ItemList works, but later attempts to add a Test DataSet to the Learner’s DataBunch also fail.

Looking at the values of the restored Learner and DataBunch I find that bunch.dl_tfms = [ ].

I would appreciate the following clarifications:

  1. Where in the restored Learner are the transforms that are causing the exception? Is it bunch.dl_tfms or other?

  2. Does fastai support (without modifying source code) adding a Test DataSet to a Learner restored from load_learner? If yes, is there documentation somewhere explaining this?

  3. If the answer to 2. is Yes – Does fastai support (without modifying source code) doing batch inference for semantic segmentation?

  4. Performing inference on single images through learn.predict(img) does work after calling load_learner (without providing a test ItemList). How does this workflow avoid the problems of doing batch processing? Can this be replicated to the batch case?

2 Likes
10
  1. The transforms are applied at the dataset level, which is why you can’t find them in dl_tfms. They are inherited from what you set during training, and you can modify them with data.test_ds (once you set it), attributes are going to be things like tfms, tfms_y, tfm_y…
  2. Yes, that is shown in the first example in the inference tutorial.
  3. Yes, as long as you pre-process your images to be all of the same size, otherwise they can’t be collated in a batch without being transformed, which will make your predictions wrong (and you will have to apply a post-process to your predictions afterward)
  4. This workflow avoid the problem of resizing because you only have one image and don’t need to collate it in a batch.
1 Like
11

@sgugger thanks for all your help in this thread. I had most of the same issues as @turntwo463. One clarifying question about his final question:

Can this be replicated to the batch case?

Is there a way to apply the transforms that are done when calling learner.predict on an inference batch (potentially using learner.pred_batch)? Thanks in advance for any guidance you can provide

12

That’s done automatically, the problem here is that you can’t apply those transforms to your labels. You can pass tfm_y=False when adding your test data to avoid having those transforms (like resize) applied there but your predictions won’t match your original images (if you resize for instance, they will have the resized size).

1 Like
13

I’m not using any labels and approaching the problem from the perspective of using load_learner but having no access to the code/data used during training. I tried to mimic the behavior of .predict on .pred_batch instead but wasn’t able to match the one_item data method behavior:

from PIL import Image
from fastai.vision import open_image, pil2tensor, load_learner
import torch
import numpy as np

learner = load_learner(...)

# .predict behavior
img = open_image(...)
batch = learner.data.one_item(img) # first line of .predict to transform data correctly

# .pred_batch behavior
img2 = Image.open(...)
tensor = pil2tensor(img2, dtype=np.float32).div_(255) # mimic open_image behavior
tfm_tensor, _ = learner.data.valid_dl.tfms[0]((tensor, torch.zeros(0))) # apply validation transforms

assert(np.array_equal(batch[0], tfm_tensor))
# AssertionError

The final tensors are very nearly equal with the exception that they are slightly different sizes. In this specific case the training data is resized on input (src.transform(..., size=src_size)) and that transform isn’t applied.

14

You’re passing dataloader transforms and not dataset transforms. Resize is a dataset transform (needs to be applied before you collate your samples in a batch).

1 Like
15

Is there a way to not apply any transformation for validation, such that load_learner would not raise the Not implemented exception?

1 Like
16

You can always remove the transforms applied in the add_test method by passing tfms=None. Note that you will need to have all your images of the same size otherwise you won’t be able to collate them in batches.

17

Thank you, i think that did it. I actually planned to do inference on images with arbitrary input sizes.

18

According to the documentation, at https://docs.fast.ai/data_block.html#LabelLists.add_test, add_test function should be able to accept tfms and tfm_y arguments. However, i was prompted the error below. Is this the correct behaviour for the function?

data = (SegmentationItemList
    .from_folder(images, presort=True)
    .split_by_rand_pct(seed=seed)
    .label_from_func(get_y_fn, classes=codes)
    .add_test(tests, tfms=None))

TypeError Traceback (most recent call last)
in
11 .label_from_func(get_y_fn, classes=codes)
12
—> 13 .add_test(items=tests, tfms=None)
14

TypeError: add_test() got an unexpected keyword argument ‘tfms’

19

You need a dev install for this, it’s very recent.

20

Hello All,

Thanks for sharing your findings. Unfortunately I have a similar problem for inference:

After loading the learner using load_learner without specifying test set: learn = load_learner(test_dir, ‘learner.pkl’).load(model), I intend to make a prediction using one of the images from validation set:

learn.predict(data.valid_ds[0][0]). But I got an error from FlowField in the function ‘create’ in class ‘ImageBBox’: “shape ‘[-1, 2]’ is invalid for input of size 4725”. The problem is it reads the whole prediction (189 * 25) as bounding boxes. The 189 is the number of boxes in total and the ‘25’ consists of 21 classes and 4 coordinates per box.

Could you give me some hint what could go wrong? Thanks!