Imagenette/ Imagewoof Leaderboards

ducha-aiki · January 13, 2020, 10:10pm

Here is my (architecture) improvement of great https://github.com/lessw2020/Ranger-Mish-ImageWoof-5 submission.

Specifically, replace all the pooling layers from AvgPool (inside net) or MaxPool (after stem) to MaxBlurPool2d from " Making Convolutional Networks Shift-Invariant Again" paper https://arxiv.org/abs/1904.11486

github.com

ducha-aiki/Ranger-Mish-ImageWoof-5/blob/master/mxresnet.py#L121


                   conv_layer(nh, nf, 3, zero_bn=True, act=False)
        ] if expansion == 1 else [
                   conv_layer(ni, nh, 1),
                   conv_layer(nh, nh, 3, stride=stride),
                   conv_layer(nh, nf, 1, zero_bn=True, act=False)
        ]
        self.sa = SimpleSelfAttention(nf,ks=1,sym=sym) if sa else noop
        self.convs = nn.Sequential(*layers)
        # TODO: check whether act=True works better
        self.idconv = noop if ni==nf else conv_layer(ni, nf, 1, act=False)
        self.pool = noop if stride==1 else kornia.contrib.MaxBlurPool2d(3, True)#nn.AvgPool2d(2, ceil_mode=True)


    def forward(self, x): return act_fn(self.sa(self.convs(x)) + self.idconv(self.pool(x)))


def filt_sz(recep): return min(64, 2**math.floor(math.log2(recep*0.75)))


class MXResNet(nn.Sequential):
    def __init__(self, expansion, layers, c_in=3, c_out=1000, sa = False, sym= False):
        stem = []
        sizes = [c_in,32,64,64]  #modified per Grankin
        for i in range(3):

The rest of entry is exact clone from Ranger-Mish.

Improvement:

acc = [0.76 0.768 0.762 0.746 0.742]
acc_mean = 0.7556
acc_std = 0.009911619

vs original

acc = [0.708 0.74 0.738 0.756 0.734]
acc_mean = 0.7352
acc_std = 0.01552288

Link to the repo: https://github.com/ducha-aiki/Ranger-Mish-ImageWoof-5

Originally posted in wrong branch