Different size for item_tfms in every batch

Hi guys,

I am trying to run an experiment to check how the size of input could affect the training procedure at a batch level.

I have a fully convolutional neural network which is doing multi-class classification problem. I am grouping the images having similar aspect ratios in a batch and then what I want to do is

from all the samples in that batch
figure out the smallest image resolution (wmin, hmin)
resize all images to (wdash, hmin) [Where wdash is computed as per corresponding image's aspect ratio]
resize the batch to a square dimension of (hmin, hmin)

I want to check with this experiment the effect of aspect ratio and image resolution on training. *How can I build a dataloader which can allow me to apply custom item_tfms during every batch of an epoch?

Any suggestion is welcome and if there’s any research you’ve come across which talks about this, kindly let me know in the replies.

Thanks! :slight_smile:

Was able to do this with before_batch callback in dataloaders by defining a custom function as follows

class lowest_resize(ItemTransform):
    def encodes(self,samples, pad_idx=1, pad_fields=0, pad_first=False, backwards=False):
        widths = []
        # Figure out the minimum height
        for item in samples:
            ip, op = item
            _, _, w = ip.shape
        min_w = min(widths)
        # Define a resize function based on the minimum height
        rsz_func = Resize(min_w, method = ResizeMethod.Pad, pad_mode= PadMode.Zeros)
        # Using the resize method above, transform the images thus obtained into 
        # a small sample size and then collate them together
        final_samples = []
        for item in samples:
            ip_image, target_label = item
            # Here's where the transformation happens
            pilimage = to_image(ip_image)
            resized_image = rsz_func(pilimage)
            tensor_image = TensorImage(resized_image).unsqueeze(0).transpose(0,-1).squeeze(-1)
            final_samples.append((tensor_image, target_label))
        return final_samples
    def decodes(self, o):
        return o

Used this link as a template which is straight out of fastai source code

Thanks! :slight_smile: