Meet Mish: New Activation function, possible successor to ReLU?

[Diganta]

Triplet Attention

[triplet] The preprint for our paper " Rotate to Attend: Convolutional Triplet Attention Module" is now out on arXiv (link). This was in collaboration with Trikay Nalamada (IIT-Guwahati), @iyaja (University of Illinois, Urbana Champaign) and Qibin Hou (National University of Singapore). We have open sourced the code on Github (link to our repository). If you’re into attention mechanisms in computer vision and heard about squeeze-and-excitation networks and their contribution in Efficient Netw…

This is our new work (not the one I mentioned in this thread earlier). Would love some feedback. @muellerzr @morgan @LessW2020 @ilovescience