Trying to do the Dogs vs. Cats Redux completely on my own and I am getting 1.9 loss when I submit my results to Kaggle. Looking for some ideas on what I’m doing wrong. Here is my repository with the code:
If I am missing anything here, let me know. All I did was take the VGG16 application built into Keras and popped off the top layer and added a fully connected layer as my output that has a sigmoid activation. The loss seems really good and in line with everyone else, but when I submit, I am just not getting good results. I trimmed the edges so the high was 98% and the low was 2% which did help me because before I have an 8 or 9 loss. So I’m going in the right direction, but I’m just a ways off where I want to wrap this one up.
You are overfitting quite much. When you are close to 100% accuracy, it is often useful to compare error rates: 98.27 test accuracy vs 97.28 validation accuracy doesn’t seem that big difference, but if you compare error rates, it is actually 1.73% test error vs 2.72% validation error. It’s 50% difference.
Likely there is a class of images that are causing some problems in your train/test/valid split. In Cats vs. Dogs, there are a few images that are icons or logos, there are images in which the animal is behind bars (in a cage). These typically caused problems, if the train/test/validation split happened to have a bit of skewed selection, when I tried to get the best possible result in Cats Vs Dogs.
It might help if you visualize and analyze images that are misclassified in your validation set.
I finally figured out what my issue was and it is one that I’ve had before. I was using flow_from_directory to pull my test images into batches and I didn’t say shuffle=False. Then I tried to still connect it when I was generating my file using tst_batch.filenames[i]. Since the default for flow_from_directory is shuffle=True, the images were all shuffled up, but the filenames weren’t because the filenames don’t shuffle when the images shuffle. After changing that and adding shuffle=False, I get 0.10950 which is much better. I still have a lot I can do to improve this, but I’m at least in the ballpark at that point.