Normalization for inference

Part 2 (2019)
1

I apologized in advance if this was already asked but I couldn’t find any solution.

I have trained a classification model and I got a pretty good accuracy on the validation dataset (>97%).
Now I found out that during inference, I am getting really bad results.
I think this is due to the fact that for training I am performing normalization of the data (2d images) and before doing inference I am not doing it which cause a discrepancy.

Could you tell me the best way to normalize data for inference please ?
I am using the last version of Fastai (2.7.10)

The current source code I am using for inference is the following:

image_item = cv.resize(image_item, (inference_width, inference_height))
model_classification_results.append(classification_model.predict(image_item))

Thank you.

2

Hey do you have a colab notebook or some more sample code to look at? Its a little tricky to only see your .predict() call and see how that compares to your training inputs to your model. From how you describe the problem, it sounds like you’re correct that your normalising/transforming your data in some way that is different to your inference time data.

If you post something I can run/replicate your problem, I’d be happy to have a look and try to help :slight_smile:

3

@nglillywhite
Thank you really much for your reply.
Unfortunately, I can’t provide a notebook.

For training, I am using:

images_folder = "***dataset***"
dataloader = ImageDataLoaders.from_folder(images_folder, 
                                          bs=64, 
                                          valid_pct=0.2, 
                                          num_workers=0, 
                                          batch_tfms=[Normalize.from_stats(*imagenet_stats)])
learner = vision_learner(dataloader, resnet18, metrics=error_rate)
learner.model = torch.nn.DataParallel(learner.model)

learner.lr_find()
learner.fit_one_cycle(10, lr_max=1.2e-3)

Then for inference, I am using the following code:

image_item = cv.resize(image_item, (200, 200))
				cv_imwrite_unicode("***/temp.png", image_item)
				image_item = image_item.astype("float32")
				image_item = image_item / 255.0
				normalized_data = torch.reshape(transforms.Compose([transforms.ToTensor(), normalization])(image_item), (1, 3, inference_height, inference_width)).float().cuda()
				
temp = normalized_data.cpu().numpy()
model_classification_results.append(classification_model.predict(temp))

Please let me know if you have any ideas about what could be wrong.
I went inside the source code of FastAi library but couldn’t find any difference in values of the data.

Thank you.

4

Hey @bennnum,

Sorry it took me a couple days to get back to you, is there a reason you’re not calling .predict() directly with your filepath? Similar to the following?

learner.predict(images_folder / "inference_image.jpg")

Or is this because you’re not writing the image to disk and instead have it in memory as “image_item” for your application?

I’m wondering if you could write your normalisation process/function as a method and reference it at both training and inference time so that you know the same process is running for both instances.

If you could make a colab notebook or some usable example I could debug I reckon I could knuckle down your problem, its definitely harder with just snippets but I understand thats not always possible.

5

@nglillywhite .
Thank you very much for your help and your reply.
The reason I didn’t use directly the path was because for inference I have an image with a high resolution. I crop it, make patches then inference.
I was used to previous FastAi versions and I didn’t know that they changed the way to use transformations. I found out that I was doing too many pre-processing that are now being handled automatically. Now, after correction of my source code, the validation results and the inference results are matching in terms of ratio (the confusion matrix is pretty beautiful and consistent).
Thank you for helping me.
I wish you a good weekend.

1 Like