Spherical CNNs

ICLR 2018 best paper.

https://arxiv.org/abs/1801.10130

Read on if you are interested.

  • Paper summary:

Spherical CNNs is a convolutional network for spherical signals. This work introduces a trainable signal representation for spherical signals (functions defined in the sphere) which are rotationally equivariant by design, by extending CNNs to the corresponding group SO(3). The method is implemented using fast Fourier Transforms on the sphere and illustrated with tasks such as 3D shape recognition and molecular energy prediction.

This line of work was very much inspired by Geoff Hinton and Co’s Capsule ideas, among others. They now have a general theory of equivariant networks that describes convolutional Capsules in terms of tensor fields, and also covers spherical CNNs.

I think this is a well-written paper, which demonstrates the usefulness of group invariance/equivariance beyond the standard Euclidean translation group.

  • Paper implementation:

Code: https://github.com/jonas-koehler/s2cnn

s2cnn is an equivariant CNNs for spherical signals. Possible use cases are, analyzing spherical images, omni-directional vision for autonomous cars, robots, and drones, molecular energy prediction, and other problems.

The authors develop a novel scheme for representing spherical data from the ground up, and test it on different empirical tasks: Spherical MNIST, 3D-object recognition.

Their models use about 1.4M parameters. On a machine with 1 Titan X GPU, training the SHREC17 model takes about 50 hours, while the QM7 model takes only about 3 hours.

I am interested to hear what your thoughts about this paper. Thanks.

4 Likes

This is really cool. Does anyone have experience plugging the code in that repo into FastAI?