Quick question: resize of target image?

fastai
1

I am building an autoencoder with the ImageImageLIst class
I am able to create the data but I don’t get resizing of the output.

pct = 0.1
sample = 5000
src = (ImageImageList.from_df(df=df_.sample(sample), path=PATH, folder=TRAIN_AUG, suffix='.png')
       .random_split_by_pct(pct, seed=42)
      )

def get_data(sz=64, bs=64):
    data = (src.label_from_func(lambda x: x)
            .transform(tfms, size=sz, tfms_y=True)
            .databunch(bs=bs).normalize(stats, do_y=True)
           )
    data.c = 3
    return data

This only applies tfms and not the resize to output.

data = get_data(64, 2)
x,y = next(iter(data.train_dl))
>>(torch.Size([2, 3, 64, 64]), torch.Size([2, 3, 512, 512]))
2

Could you share how you defined your src? Just tested it and I got the right sizes.

3

Thanks for looking, you can check the full nb here:


I noticed the problem when I saw the images with:

data.show_batch

and then torch.shape shows the issue.

4

Any ideas @sgugger ?? I am in 1.34 btw.

5

There is no s in tfm_y=True. You just made a typo :wink:

6

LOL! why there is no warning or error? There is no auto complete or shift+tab magic with data block API. That’s something I miss. Actually, this line works:

    data = (src.label_from_func(lambda x: x)
#             .transform(tfms, size=sz, tfms_y=True)
            .transform(superluly, trump_donald=sz, sgugger=True)
7

Note that you get the tab completion if you do it bits by bits (instead of in one line of code). As for transform, it accepts any kwargs for now, which is something I’m not sure how to fix (since there are several things that can be passed along).

Edit: It’s actually easier than I thought, preparing a fix for this.

8

Tacking on to this – with you the tfm_y=True – are the transforms only being applied to the training data during a train loop and not the validation data?

Thanks.

9

You specify to lists of transforms, one for the training set, one for the validation/test set.

10

I will do more tests tomorrow.

11

How can I resize x to size, and y to 2*size? I’m doing that for super resolution and I’m not using the unet, so I don’t need an upscaled input.

12

That’s a good question btw. Don’t know if this can be done on the fly, in lesson superres they built a low_res folder and a high_res labels.

13

Yup, and the low res gets upscaled to your target size, while high gets downscaled to it. Say, I have a folder with 96x96ish images, and a folder of original 3000x3000ish or something.
If I set size=256, lowres folder will be upscaled to 256, highres will be downscaled to 256. But what if I need low res to stay 96ish, and highres get downscaled to 256?
I just don’t want to write a dataloader from scratch, especially if I’m going to use all the data augmentation transforms.

14

Hi, I’m having trouble with ImageImageList It is setting the inputs and output to the same size.
@sgugger

src = ImageImageList.from_folder(path_lr).split_by_rand_pct(0.1, seed=42)
def get_data(bs,size):
    data = (src.label_from_func(lambda x: path_hr/x.name)
           .transform(get_transforms(max_zoom=2.), size=size, tfm_y=True)
           .databunch(bs=bs).normalize(imagenet_stats, do_y=True))

    data.c = 3
    return data

data = get_data(bs, sz_lr*scale)

data

ImageDataBunch;

Train: LabelList (6651 items)
x: ImageImageList
Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192)
y: ImageList
Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192)
Path: /root/.fastai/data/oxford-iiit-pet/small-96;

Valid: LabelList (739 items)
x: ImageImageList
Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192)
y: ImageList
Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192),Image (3, 192, 192)
Path: /root/.fastai/data/oxford-iiit-pet/small-96;

Test: None

15

Because ImageImagelist now ties the size of both x and y my SuperRes model output is scaled by a factor of two so there is a mismatch.

Using a target size (torch.Size([16, 3, 96, 96])) that is different to the input size (torch.Size([16, 3, 192, 192])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.

16

@Prince if you notice you’re using 200% zoom (which 2*96 is exactly the 196). You should probably not use it or ensure another size inside of it so they’re the same in and out (just for an idea of where to start debugging). You can also check the sizes quickly by looking at one batch (IE databunch.one_batch()[0].shape())

17

I did that, it is still the same.

I even ran the 2019 course on Super Resolution, it is also having the same bug.

Something is wrong inside the library.

18

ImageImageList is behaving the wrong way,
It is resizing both the input and output image to the same size in the transform()
that the big problem. Because it is supposed to keep the size of the images loaded in src.

19

If I don’t pass size for the High Resolution image it sort of behaves as expected but then the size of the original images are too big to what my model will output.

20

You specify tfm_y=True, so yes it is applying transforms on the input and targets. Remove the tfm_y=True to not apply them to the target.
And if you don’t want transforms applied to your inputs, you need to use a custom class you have to write since your inputs are not traditional images using data augmentation. It’s not because ImageImageList does not behave the way you want, that you have to say it behaves the wrong way.