In transforms.py, crop is applied after rescale

#1

Hi @jeremy
First , since it’s the first time I ping you, thanks for the great course :slight_smile: . the generated abstracts in lesson 4v2 are absolutely mind blowing ! I can’t wait to try by myself.

for now I’m still working on image classification, on kaggle’s IEEE camera recognition and I think I face a bug in fastai.
I want to use a random crop on the images but without downscaling to keep native resolution (get 299*299 out of the picture, even if it’s a small part). I found out that the crop is applied after the scaling due to this code in transforms.py:

self.tfms = tfms + [crop_tfm, normalizer, channel_dim]

so I end up with rescaled images and augmented images which are quite close from one another.
in my case I patched this to self.tfms = [crop_tfm] + tfms + [normalizer, channel_dim]
but that would break the intuitive default behavior for crop_type=CENTER, for most image recognition problems I guess.

I think the clean way to fix this is to add an optional crop_size parameter that would be either an int (nb of pixels to keep) or a float ( % of the image to keep ) on top of the current crop_type. and apply the crop before tfms.

What do you think?
If you agree with the approach, I can submit a PR.

0 Likes

#2

looking at this again it looks like the crop feature is more intended to fit the model input shape than data augmentation, am I right ? so, instead of modifying it, I should have added another random crop transformation, placed first in tfms_aug?

or maybe even pre-cropped the jpeg files offline - but i’d lose the randomness -, since right now i have a CPU bottleneck on opening these big jpeg files just to discard 90%+ of the pixels.

0 Likes

(houda kaddioui) #3

Wait does it mean that the images are cropped instead of being resized (like downsized)?

0 Likes

#4

yes, the images are resized. which I guess is the right behaviour for a vast majority of image recognition tasks.

my problem was that in this particular case i didn’t want the rescaling, i just wanted a crop (even if it meant discarding 90% of the image) which was not possible since the images were first resized so that the smallest dimension is 299px, and then cropped randomly.

0 Likes