Opencv based image pre-processing step

I would like to apply a pre-processing step on all train and test images. This step is based on some opencv transformation functions. Is there any way I can directly use it with the databunch?

I can pre-process the data and save it on the disk and load these images, but my data is huge, and I don’t have enough space. Any help would be appreciated.

1 Like

If you can’t preprocess them, you can create a custom transformation with opencv for python.

Take a look at:

In the multi spectrogram notebook I’ve created a custom function and appended it to the tfms array.

1 Like

will these applied transforms also work on the test data?

I think they will be applied if you add a test set and use TTA. If you’ll manually feed the model you need to apply them by hand.

Check “add test” to have an idea of the workflow.

Thanks for the links. I am stuck on the transforms part. I have the function ready, which takes in an Image object and then do some transforms on the data and then returns another Image object. I have changed the original transforms array like this-

tfms = vision.get_transforms(flip_vert=False)
tfms = [[clahe_simplewb_tfms]+tfms[0], [clahe_simplewb_tfms]+tfms[1]]

data = (
    vision.ImageList.from_df(train_labels, path, folder="train")
    .split_by_rand_pct(seed=1995)
    .label_from_df(cols="category_id")
    .transform(tfms=tfms, size=size)
    .databunch(bs=bs)
    .normalize(vision.imagenet_stats)
)

The function clahe_simplewb_tfms is my transformation function. I am getting the following error (stacktrace is incomplete)-

/opt/anaconda3/lib/python3.7/site-packages/fastai/data_block.py in _check_kwargs(ds, tfms, **kwargs)
    570         try: x.apply_tfms(tfms, **kwargs)
    571         except Exception as e:
--> 572             raise Exception(f"It's not possible to apply those transforms to your dataset:\n {e}")
    573 
    574 class LabelList(Dataset):

Exception: It's not possible to apply those transforms to your dataset:
 'function' object has no attribute 'tfm'

When I am looking at your code, I can’t see any tfm attribute defined. Do you know what’s the issue here?

The tfms var should be a list of list because it holds the list of transformations of train set and the list of transformations of validation set.
Remember that usually we apply data augmentation ONLY to train set.

Btw when you got an error like this: try to test your transformation chains (both train list of tfms and valid) to be sure that they’re doing what you expect.

on my Notebook:

Look for function “applyAllTfms” that let you “try” all transformations to be sure that everything is working correctly.

I also would like to implement a custom transform using opencv. I already have the function say:

def customtf(img):
	[...transform...]
	return result

with img and result being images as used in opencv. How do I create a fastai transform with that and how do I add it to the data augmentation pipeline?

Thanks.

I couldn’t get any of the methods given by Stefano to work for me. I was getting the same error I have replied with in the previous comment. Later I pre-processed the images and saved them on disk. If you are successful then please post it here.

So after much searching, debugging and wrangling, I managed to find a solution. The following works for me:

def custom_transform(x): # you get a PyTorch tensor as input, not a fastai Image object
    npim = image2np(x)*255 # convert to numpy array in range 0-255
    npim = npim.astype(np.uint8) # convert to int
	
    # If your image is a regular colour image, remember it will be in RGB so convert to BGR if needed
    img_bgr = cv2.cvtColor(npi, cv2.COLOR_RGB2BGR)
	
    # transform using opencv
    # ...
    # don't forget to convert back to RGB if needed

    # assuming result image is stored in result_img
    return pil2tensor(result_img, dtype=np.float32)/255 # convert back to tensor and return

custom_trans = TfmPixel(custom_transform) # wrap in TfmPixel to create a valid transform
tfms = [[custom_trans()],[]] # set your custom transform for training dataset only

# View a couple of results when applying your custom transform on a test image
def get_ex(): return open_image('testimage.jpg')
def plots_f(rows, cols, width, height, **kwargs):
    [get_ex().apply_tfms(tfms[0], **kwargs).show(ax=ax) for i,ax in enumerate(plt.subplots(
        rows,cols,figsize=(width,height))[1].flatten())]

plots_f(2, 4, 12, 6)

HTH

4 Likes

I used the same TfmPixel function to convert the function into a transform, but I was giving me errors. Only difference I can see here is you are returning pil2tensor's output and I was returning an Image object. Nice work!

@tminima, @Tamori:

other way around performing custom image transformation using opencv can be done as below:

    from fastai import *
    #define function    
    def _load_format(path, convert_mode, after_open)->Image:
                image = cv2.imread(path)
                image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
                image = crop_image(image) #call image tranformation function
                    
                return Image(pil2tensor(image, np.float32).div_(255)) #return fastai Image format

    vision.data.open_image = _load_format

Hi @Tamori,

You wrote is will only apply for training dataset.
Is it possible to apply it for valid & test dataset ?

Thanks

You can of course apply your custom transform to the validation set as well. Just put your function in the second element of the tfms tuple:

tfms = [[custom_trans()],[custom_trans()]]

For test sets, I’m not sure. Fast AI handles those differently, as I understand.

1 Like

I tried to turn my images into greyscale but they turned out to be green instead :sweat_smile:

Thank you