Curtis Northcutt from MIT took the time time to benchmark all image models available in Keras and PyTorch and created a ranked list.
Now you can benchmark and compare your best image model against all verified SOTA scores of all standard image models. Nice.
Find the benchmark, data, and results here:
Something that was really needed.
Thanks for posting this!
Nice to see that ResNet152 is the leader for PyTorch. I’ve certainly had strong results from it.
I’m surprised the #1 rank is 3% higher though, that’s a fairly large difference. Have not heard of nasnetlarge, but wonder if a wide resnet would beat it (no wides were shown in this list).
*Found the background for nasnet - https://arxiv.org/abs/1707.07012
I’ve found it too memory intensive and slow to be useful. Been a while since I tried though.
Looking at the NAS cell, that would definitely make sense regarding slow and memory intensive. Each normal cell is running 8 convolutions inside…it’s like a mini CNN in each cell:
(image from their paper)
Thanks for the feedback on it!