Reading this thread got me confused about a totally different topic. Your case here shows that the training set needs to be quite varied for your model to perform well, as @jeremy confirmed. Now, if that is the case, how is data augmentation technique helping us with bigger datasets and improving the accuracy? After all they are the very same images which we modify a bit by zooming or rotating or flipping. Am I missing something here?