Systematically lower accuracy e.g. on State Farm


#1

Hi everyone,

I’m really enjoying the course, but I’ve been struggling with my (validation) accuracies being systematically lower than they were for Jeremy. Between 20% (for preliminary sample testing) and 10% (for my final model) lower.
I started shift-entering through the notebooks (forbidden, I know…) to try to identify what the problem was. I still got much lower accuracies despite using exactly the same code. (Included is the script from the State Farm sample testing that I saved to illustrate this).

This issue has applied to both Cats and Dogs and State Farm.
Regarding State Farm, I have checked that the validation and training sets contain different drivers.
Generally, I’d love to know if anyone has any thoughts / had a similar experience.
Is there some change from Keras 1 to Keras 2 (which I’m using) that would make my results different?

Many thanks for any help!


(Dana Ludwig) #2

Hi @ganesha123 anesha123, I’m doing the 2018 course with the fastai/pytorch models, and I’m getting about 10% lower accuracy / loss than Jeremy’s video on validation set too. I’m running my own DL box with a 1080 GPU (not 1080Ti). nvidia-smi shows driver version 384.130.

The good part is that all the code runs, and better code runs better, but it’s always a little worse than Jeremy’s results.