Error-It's not possible to apply those transforms to your dataset?

Part 1 (2019)
1

I am trying to add a test dataset to the databunch and getting the following error. There are 2 other questions (1, 2) similar to this with no clear answer.
Now, how can I add this test data to the data bunch and eventually run the trained model on test data? thanks

Code

data = (SegmentationItemList.from_folder(path/'train/images')
       .split_by_rand_pct(0.3)
       .label_from_func(get_y_fn, classes=codes)
        .transform(get_transforms(), size=224, tfm_y=True)
        .add_test((path/'test/images').ls(), label=None)
        .databunch(path=Path('.'), bs=32)
        .normalize(imagenet_stats))

Error

    ---------------------------------------------------------------------------
    Exception                                 Traceback (most recent call last)
    /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs)
        590         x = ds[0]
    --> 591         try: x.apply_tfms(tfms, **kwargs)
        592         except Exception as e:

    /home/gbogu17/.local/lib/python3.7/site-packages/fastai/core.py in apply_tfms(self, tfms, **kwargs)
        187         "Subclass this method if you want to apply data augmentation with `tfms` to this `ItemBase`."
    --> 188         if tfms: raise Exception(f"Not implemented: you can't apply transforms to this type of item ({self.__class__.__name__})")
        189         return self

    Exception: Not implemented: you can't apply transforms to this type of item (EmptyLabel)

    During handling of the above exception, another exception occurred:

    Exception                                 Traceback (most recent call last)
    <ipython-input-41-bd55d1f668b6> in <module>()
          8        .label_from_func(get_y_fn, classes=codes)
          9         .transform(get_transforms(), size=224, tfm_y=True)
    ---> 10         .add_test((path/'test/images').ls(), label=None)
         11         .databunch(path=Path('.'), bs=32)
         12         .normalize(imagenet_stats))

    /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in add_test(self, items, label, tfms, tfm_y)
        560         elif isinstance(items, ItemList): items = self.valid.x.new(items.items, inner_df=items.inner_df).process()
        561         else: items = self.valid.x.new(items).process()
    --> 562         self.test = self.valid.new(items, labels, tfms=tfms, tfm_y=tfm_y)
        563         return self
        564 

    /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in new(self, x, y, tfms, tfm_y, **kwargs)
        627         tfms,tfm_y = ifnone(tfms, self.tfms),ifnone(tfm_y, self.tfm_y)
        628         if isinstance(x, ItemList):
    --> 629             return self.__class__(x, y, tfms=tfms, tfm_y=tfm_y, **self.tfmargs)
        630         else:
        631             return self.new(self.x.new(x, **kwargs), self.y.new(y, **kwargs), tfms=tfms, tfm_y=tfm_y).process()

    /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in __init__(self, x, y, tfms, tfm_y, **kwargs)
        599         self.y.x = x
        600         self.item=None
    --> 601         self.transform(tfms, **kwargs)
        602 
        603     def __len__(self)->int: return len(self.x) if self.item is None else 1

    /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in transform(self, tfms, tfm_y, **kwargs)
        722         if tfm_y is None: tfm_y = self.tfm_y
        723         tfms_y = None if tfms is None else list(filter(lambda t: getattr(t, 'use_on_y', True), listify(tfms)))
    --> 724         if tfm_y: _check_kwargs(self.y, tfms_y, **kwargs)
        725         self.tfms,self.tfmargs  = tfms,kwargs
        726         self.tfm_y,self.tfms_y,self.tfmargs_y = tfm_y,tfms_y,kwargs

    /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs)
        591         try: x.apply_tfms(tfms, **kwargs)
        592         except Exception as e:
    --> 593             raise Exception(f"It's not possible to apply those transforms to your dataset:\n {e}")
        594 
        595 class LabelList(Dataset):

    Exception: It's not possible to apply those transforms to your dataset:
     Not implemented: you can't apply transforms to this type of item (EmptyLabel)
2

I think you need .add_test((path/'test/images').ls(), label=None, tfms=None)

Edit: .add_test((path/'test/images').ls(), label=None, tfm_y=False)

3

I am getting the same error even after using tfms=None

data = (SegmentationItemList.from_folder(path/'train/images')
       .split_by_rand_pct(0.3)
       .label_from_func(get_y_fn, classes=codes)
        .transform(get_transforms(), size=224, tfm_y=True)
        .add_test((path/'test/images').ls(), label=None, tfms=None)
        .databunch(path=Path('.'), bs=2)
        .normalize(imagenet_stats))

---------------------------------------------------------------------------
  Exception                                 Traceback (most recent call last)
  /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs)
      590         x = ds[0]
  --> 591         try: x.apply_tfms(tfms, **kwargs)
      592         except Exception as e:

  /home/gbogu17/.local/lib/python3.7/site-packages/fastai/core.py in apply_tfms(self, tfms, **kwargs)
      187         "Subclass this method if you want to apply data augmentation with `tfms` to this `ItemBase`."
  --> 188         if tfms: raise Exception(f"Not implemented: you can't apply transforms to this type of item ({self.__class__.__name__})")
      189         return self

  Exception: Not implemented: you can't apply transforms to this type of item (EmptyLabel)

  During handling of the above exception, another exception occurred:

  Exception                                 Traceback (most recent call last)
  <ipython-input-15-c57b83c45e6c> in <module>()
        8        .label_from_func(get_y_fn, classes=codes)
        9         .transform(get_transforms(), size=224, tfm_y=True)
  ---> 10         .add_test((path/'test/images').ls(), label=None, tfms=None)
       11         .databunch(path=Path('.'), bs=2)
       12         .normalize(imagenet_stats))

  /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in add_test(self, items, label, tfms, tfm_y)
      560         elif isinstance(items, ItemList): items = self.valid.x.new(items.items, inner_df=items.inner_df).process()
      561         else: items = self.valid.x.new(items).process()
  --> 562         self.test = self.valid.new(items, labels, tfms=tfms, tfm_y=tfm_y)
      563         return self
      564 

  /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in new(self, x, y, tfms, tfm_y, **kwargs)
      627         tfms,tfm_y = ifnone(tfms, self.tfms),ifnone(tfm_y, self.tfm_y)
      628         if isinstance(x, ItemList):
  --> 629             return self.__class__(x, y, tfms=tfms, tfm_y=tfm_y, **self.tfmargs)
      630         else:
      631             return self.new(self.x.new(x, **kwargs), self.y.new(y, **kwargs), tfms=tfms, tfm_y=tfm_y).process()

  /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in __init__(self, x, y, tfms, tfm_y, **kwargs)
      599         self.y.x = x
      600         self.item=None
  --> 601         self.transform(tfms, **kwargs)
      602 
      603     def __len__(self)->int: return len(self.x) if self.item is None else 1

  /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in transform(self, tfms, tfm_y, **kwargs)
      722         if tfm_y is None: tfm_y = self.tfm_y
      723         tfms_y = None if tfms is None else list(filter(lambda t: getattr(t, 'use_on_y', True), listify(tfms)))
  --> 724         if tfm_y: _check_kwargs(self.y, tfms_y, **kwargs)
      725         self.tfms,self.tfmargs  = tfms,kwargs
      726         self.tfm_y,self.tfms_y,self.tfmargs_y = tfm_y,tfms_y,kwargs

  /home/gbogu17/.local/lib/python3.7/site-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs)
      591         try: x.apply_tfms(tfms, **kwargs)
      592         except Exception as e:
  --> 593             raise Exception(f"It's not possible to apply those transforms to your dataset:\n {e}")
      594 
      595 class LabelList(Dataset):

  Exception: It's not possible to apply those transforms to your dataset:
   Not implemented: you can't apply transforms to this type of item (EmptyLabel)
4

Sorry, my mistake. Try tfm_y=False instead of tfms=None.

1 Like
5

Great! Thank you very much. It is working.
I am wondering now how the output would look like? Does it print dice scores of both test data and the validation data that was randomly split from training data?

6

I am wondering now how the output would look like? Does it print dice scores of both test data and the validation data that was randomly split from training data?

For starters, you won’t be able to get a dice score for your test data because you do not have labels for your test data. During training you can see training and validation loss as well as any other metrics like dice score that you add.

As for the output of your model, that depends on your input. If you’re working on a problem similar to camvid then your input will be an RGB image of shape (3,height,width) and your output will be a mask of shape (n_classes, height, width). I think you could use argmax to squish this into a shape of (1,height,width) in which each pixel contains the most likely class for that pixel.

1 Like
7

That means adding test data to the databunch is of no use as long as they don’t have labels, correct? If yes, I guess the only use of having test images with no labels is we could just open a random test image and apply learn.predict to see how well the prediction model works like in camvid?

8

That means adding test data to the databunch is of no use as long as they don’t have labels, correct?

Well, adding test data to the databunch allows you to generate predictions for your test set. You could use preds, _ = learn.get_preds(ds_type=DatasetType.Test). So for something like a Kaggle competition you would then submit these predictions (maybe after more processing) to Kaggle and they would give you a score. If you were making a self-driving car system, you might use get_preds() to segment live pictures from a camera and then use those to make decisions on whether to turn left, go straight etc.

Typically the reason we use .add_test() is so we generate predictions against a test set in batches. This is much faster than generating predictions for single images, one at a time.

1 Like
9

Is this the command to save the predictions from the test data set?
Yes, I am using data from kaggle segmentation challenge and the final goal is to submit pred scores for test data.

10

Is this the command to save the predictions from the test data set?

Yes it is, there’s some more information here that might help you too: https://docs.fast.ai/tutorial.inference.html

1 Like
11

One last thing, is it possible to save these pred scores by each folder in the test dataset (in my case I have 10 samples/folders in test dataset and each folder contains around 1k images)?