Positive signs with Ranger + Mish for EfficientNet-b3, 1-run test set accuracy of 93.9% for Stanford cars with EfficientNet-b3 after 40e. Their paper quoted 93.6% for b3. Note I’m training on the full training set here, using the test set for validation.
I didn’t play around with the hyperparameters at all, just took what seemed to work well for Ranger:
40 epoch
lr=15e-4
start_pct=0.10
wd=1e-3,
Will kick off 4 additional runs so I can get a 5 run average, but its slow going, 2h20m per run 
