Benchmark scores for all image models in Keras and PyTorch

Curtis Northcutt from MIT took the time time to benchmark all image models available in Keras and PyTorch and created a ranked list.

Now you can benchmark and compare your best image model against all verified SOTA scores of all standard image models. Nice.

Find the benchmark, data, and results here:


Something that was really needed.

Thanks for posting this!

Nice to see that ResNet152 is the leader for PyTorch. I’ve certainly had strong results from it.

I’m surprised the #1 rank is 3% higher though, that’s a fairly large difference. Have not heard of nasnetlarge, but wonder if a wide resnet would beat it (no wides were shown in this list).

*Found the background for nasnet -

I’ve found it too memory intensive and slow to be useful. Been a while since I tried though.

1 Like

Looking at the NAS cell, that would definitely make sense regarding slow and memory intensive. Each normal cell is running 8 convolutions inside…it’s like a mini CNN in each cell:

(image from their paper)

Thanks for the feedback on it!

this is excellent one