Is there a way in fast.ai library to do some occlusions to make sure that the model really uses object of interest in the image for classification and not just its surroundings?
For example, in lesson 2, teddies are often seen in contexts of homes, people, children or just stand alone with the white background. Black bears and grizzlies are often seen in outside, with trees, and other nature background components. How do we check, and, most importantly make sure, that our model truly localizes and recognizes object of interest in the image? What if our object of interest is quite small on the image, so the model will probably be more inclined to look for some patterns, textures, etc in the rest of the image not occupied by the object of interest.
Another example I do not remember where I heard it is that huskies were only classified correctly, because the majority of images had snow on the background, so it was snow and not huskies themselves that were used for this breed classification.
See the heatmap of the predictions. It would show which parts of the image your model preferred more than others. There is plot_top_losses function in fastai, that shows a heatmap along with the image.
Then the first thing would be to try for different images of the same class and see if you still get the same result.
If yes, then either your training data was imbalanced in which the number of examples for that class is small and the model couldn’t learn them or your model has not finished training. (It is assuming you choose your losses for the work correctly)
Another thing to so is to augment the data by changing the background, though it is more complicated data augmentation issue. The way it works is by cutting out your objects of interest, and pasting it into a non typical environment. Like a teddy bear in the woods and not in a home.
The paper I cited above offers another way to augment, just to introduce object of interest occlusions. So perhaps augmenting by introducing random occlusions could help? I wonder if we have this type of augmenting in fast.ai library?
Finally an answer to the problem that I am facing right now. I am using tensorflow 2.1 and I know very well that this is an fastai forum, but I don’t know whom else to ask. I hope that you reply to this message. Regarding your comment:
Then the first thing would be to try for different images of the same class and see if you still get the same result.
If yes, then either your training data was imbalanced in which the number of examples for that class is small and the model couldn’t learn them or your model has not finished training. (It is assuming you choose your losses for the work correctly)
I am training synthetic MNIST and my model looks like this:
My model’s accuracy for 50 epoch is ~97%. The classes are perfectly balanced, even after the split into training (8000 images i.e. 800 images per class) and validation set (2000 images i.e. 200 images per class). I also augmented data and trained the model. The different augmentation that I used were:
The accuracy of the model is pretty good, but still, I get the same results on different images of the same class. It isn’t giving me the desired results. Any idea what could be the cause? or what else I can try?
