DataBlock: how to crop borders in item_tfms

fastai
1

Image segmentation with Unet. I make a big Pandas dataframe with paths to all images and masks, along with other image attributes. The DataBlock is like this:

src_datablock = DataBlock(
    blocks=(ImageBlock, MaskBlock),
    getters=[make_image, make_mask],
    splitter=TrainTestSplitter(stratify=src_df_big["dataset"].to_list()),
    item_tfms=[
        Resize(size=(960, 720), method="squish"),
        # I need to crop borders here
        Resize(size=224, method="squish")
    ],
    batch_tfms=None,
)

The getters simply ensure that all images and masks are 8 bits per pixel, but they make no other changes, so I’m not including them here.

Most images are 960x720. But a few are much bigger than that. So in item_tfms I first resize to 960x720 to ensure everyone is on the same page. Then I resize to the size that the model needs.

But in between those two Resize calls I need to crop borders - crop out a few dozen pixels from each border, because most images have a frame around them that I don’t need. Basically, I need to keep an inner box from each item and discard the outer region. I know the size and coordinates of the box, they are the same for all images.

Perhaps I’m missing something obvious - but I can’t figure out what is the right method there to crop out the border.

If all this is too hard, perhaps I can move all the resizing / cropping / resizing to the getters, but first I would like to try doing it with item_tfms. This is to improve my understanding of fast.ai. Also, I think it’s better code structure to keep low level processing in the getters, while doing higher level processing in item_tfms.

Thank you.

2

Hi @FlorinAndrei!

Would the CropPad transform (link) included with fastai work for your situation? It center-crops an image to the specified size (assuming the target size is smaller than the current image).

3

No, the border widths are all different. The box is not centered. I need to specify my own custom box.

4

In that case, you should be able to make a custom transform to crop out the inner box. I’m about to head to bed, but I can post something tomorrow if you’d like. I’ve done something similar recently but would need to put the code in a custom transform class.

Do you have a sample image I could test on?

5

Something like in this notebook?

6

An example would be very helpful, thank you.

I cannot share the dataset, unfortunately. But literally any random PNG would work.

7

Ah, yes, that notebook looks very promising.

I also need to head to bed, I will try it tomorrow. Thanks a lot!

8

No problem!

9

@cjmills There’s a bug that’s puzzling me, I’ve noticed it when I tried to implement my own crop, and it’s the reason I’ve opened this thread. It also occurs with the method you’ve proposed. Take a look at this notebook:

Most images are 960 x 720. One is much bigger than that. If I apply crop in between the two Resize calls in the DataBlock, it seems like the first resize does not happen and cropping is applied to the full size image. If the bug is not triggered, re-run that cell a few times until it happens.

The test images are also in that folder in the repo, so you can download the whole folder and test it quickly, it uses relative paths so it should work anywhere.

EDIT: Cropping in my test notebook doesn’t do anything on the correct-size images, because the box size is the same as the image size. But after the first resize, all images should be at the correct size. Yet that doesn’t happen.

10

I think you can take a look at the albumentation transform: Crop transforms (augmentations.crops.transforms) - Albumentations Documentation . There is plenty of cropping method and I guess there is one that solves your problem. There is a tutorial how to adapt albumentation of the fastai DataBlock API in fastai - Custom transforms

11

The dimensions look swapped for the crop transform in that notebook compared to the first resize method.

Per the documentation (link) for the PIL.Image.crop() method, the box dimensions should be (left, upper, right, lower). The first resize method has 960 as the height, whereas the crop method has it as the width.

12

I swapped the dimensions for the first resize method.

13

I’m testing something in the notebook right now

14

I’ve swapped the dimensions but it doesn’t seem to do anything.

As far as I understand the output of crop_datablock.summary(crop_df) the first resize does not seem to get invoked at all.

15

So, I don’t think fastai likes having two transforms of the same function/class name in the item_tfms list. I replaced the first resize transform with a custom one and it seems to do what you want. Perhaps move the second resize to the batch_tfms. Although, it would probably be more straightforward to have a custom resize transform like I did in the updated notebook.

16