Temporal Convolution Networks in FastAI


(Theodoros Galanos) #1

Hello everyone,

I just wanted to drop an idea here for a network architecture that could be included in FastAI. I hope this isn’t redundant (i.e. it has already implemented).

Recently, there have been some implementations of CNNs that are (at least from the published research) seemingly outperforming RNN models in different tasks. These so-called Temporal Convolutional Networks has various advantages over RNNs (faster, longer memory, training, etc.). The most famous perhaps architecture of this is WaveNet (https://arxiv.org/pdf/1609.03499.pdf).

I would say that, at least for someone at my level, it is not the easiest architecture to decipher and certainly not the easiest to write in code which I feel makes it a good candidate, apart from the perfomance, for FastAI.

There are a few implementations out there, the most popular pyTorch seems to be this one: https://github.com/locuslab/TCN/. There is also a higher level API for Keras, which I think is quite amazing. It offers simple calls that generate different types of TCNs with various input parameters for the user to use (https://github.com/philipperemy/keras-tcn). It might be a good example to base a FastAI implementation on.

Hope this is interesting to some of you!

Kind regards,
Theodore.