SoTA for self-supervised sequence recognition?

Deep Learning
1

Hi,
I’m starting to engineer a solution for the following. A few architectures come to mind, but I’m not sure what would be the best. If this forum doesn’t know, I’m not sure who will :wink:

Problem is: I’m receiving a stream of {ID, data} where ID is a user ID, and data is a list of a few hundreds floats (e.g., data.shape = (200,) ). I would like to model the data sequence of each user over time so that I can spot “anomalies”.

Ideas. The first ideas that pop up are:

  1. “classic” multilayer LSTM, interleaved with various dropouts and a single FC layer at the end for multi-class prediction
  2. Using LSTM autoencoders
  3. The famous “taxi prediction” (https://arxiv.org/pdf/1508.00021.pdf) often discussed here.
  4. Recurrent highway networks?
  5. …

Obviously, I’d handle the online learning by:

  1. predict ID given data, and look at the logprob of the “correct” ID
  2. then, train/update the model by providing ID and data on a batch of the last few {ID, data} points.

Am I missing any recent superfancy arch? :slight_smile:
Thanks!

ps: after part1v1, part2v1 i’m now following part1v2. It’s never enough!