First and foremost - thank you so much for putting this course together. It is awesome. I have been studying/playing with DL for a long time and for the first time I feel like I am starting to feel like I am getting it. (It feels like I am gaining a super power!).
I have a question on overfitting. I am running a CNN on private data to try to identify which pictures are the most clicked on based on just the image.
I ran a standard VGG16 BN and ended up with a test accuracy of .60 and a validation accuracy of 0.65.
I re-ran it after removing drop out and now have a test accuracy of .85 and a validation accuracy of 0.74!.
I am very excited as the validation accuracy is much much higher, but it now is an overfitting model. My question:
Given the second model is overfitting in a big way- is it still better than the first model or does the fact it significantly overfits outweigh the fact the validation accuracy is much higher?
I would assume the validation accuracy is what matters in the end of the day and that overfitting only matters in that it means your model cannot continue to improve with future epochs but just wanted to make sure.
Also - given I am now overfitting, should I re-introduce a smaller dropout?