How do I modify siamese network for minor modifications of image?

I want to do quality assurance for food items let say Pizza (Pepperoni Pizza). The goal is to check if the food that I made closely matches with the set metric. I think Siamese network algorithm fits my need.

Suppose I have trained my model to detect Pepperoni pizza with normal toppings. But later the customer wants to do a small customization ( replace topping A with B or add a new topping C).

How can I most effectively handle this scenario? Now my idea is to reduce the threshold for acceptance in these type of custom cases. Also, is Siamese network the most effective approach for this problem? All ideas are welcome.