Tfm_y only on some transforms?

I have the following problem: I want to apply several transforms in a segmentation scenario. However, I do not want to apply all the transforms on the masks. Normally, by setting tfm_y to True, all transforms are applied to the masks. But how can I configure that for some transforms, these are also applied to the mask, and for some transforms not? I appreciate any help.

If you don’t want to apply these transforms to the mask, why not applying these transforms before the others and only on your images?

Each epoch it should be a different transform, so doing it before is not reasonable :frowning:

1 Like

I require similar functionality and implemented it the following way:
I changed the image transform functions to filter the y transforms for an attribute skip_y which defaults to False. I can then create my transforms this way:

tfm_train, tfm_val = get_transforms()
tfm_train += [
    cutout(n_holes=(1,4), length=(1, hp['im_sz']//8)),
    RandTransform(TfmPixel(crappify_randomly), {"max_noise_std": .05})
]
for t in tfm_train[-2:]: t.skip_y=True
# ... In your data blocks pipeline:
data = (.....).transform((tfm_train, tfm_val), tfm_y=True)

Check out in my fork: https://github.com/xetaiz/fastai (more specifically this commit)

@sgugger Is this something you would want in a PR? If yes, suggestions to improve this?

I am bit confused by the use case: how do you predict something that has no link to your initial image?

Imagine the auto encoder from the course. you may want to crappify your input images x (additionally to some rotations etc.) and have the same rotation augmentations on your labels images y, except for crappification. Then reconstruct the original images.

Ok, but in that case shouldn’t you use a custom ImageList for your inputs that does that crappification on the fly? That way transform still behave the same way.

That would force me to have crappification as first step (which I do not necessarily want in my experiments).

I mean yea, the use cases are not exactly endless. I will use my changes for my experiments anyways… Just thought I could contribute it if there’s interest. I’d also be willing to integrate this a little nicer, however I won’t bother unless you guys think its useful.

In general I feel like having the option to use different transforms for x and y is useful, as there is currently only the option to use the same transforms or no transforms.
If you think too few people use it, I do understand it’s just yet another parameter somewhere :smiley:

Not saying this is useless, I’m just trying to understand how important this is. In general not applying your transforms to your targets at the same time as your inputs would be bad and we don’t want beginners to accidentally use this by msitake.
Your implementation seems safe enough and doesn’t change a lot however, so you can suggest a PR with it.

It should behave the very same if not explicitly setting this skip_y.
I was wondering if it makes sense to have this parameter passed via RandTransform or so?
And I wondered If you came up with a better name? Currently this and related pieces of code have tfm_y (data_block.py LabelList), do_run and skip_y (image.py RandTransform).
I was a bit confused when I wrote this, especially because of do_run which might sound like the opposite of skip_y. Also I kinda don’t like the fact that skip_y is only used with not :smiley:
How is use_on_y?

I would then add the skip_y (or whatever name) to Transform.__call__ as well, so that the option is available as follows:

tfm_train, tfm_val = get_transforms()
tfm_train += [
    cutout(n_holes=(1,4), length=(1, hp['im_sz']//8), skip_y=True),
    crappify_randomly(max_noise_std=.025, skip_y=True)
]

PR: https://github.com/fastai/fastai/pull/2002

I like use_on_y better, now that you mention it. And maybe it’s best to have it as a parameter of the transform because it’s easier to use. Can you amend your PR for that?