Meet Mish: New Activation function, possible successor to ReLU?

I used the pretrained models from @lukemelas’s EfficientNet-Pytorch github and get nice bump in accuracy, even beating the EfficientNet paper’s b3 result on Stanford Cars : [Project] Stanford-Cars with fastai v1

Looking super promising using it with pretrained b7 too

1 Like