Hi Maxime,
I’m just partway through Part 1 right now, so I don’t know much about recurrent networks. But reading your description of the problem as a 2D signal tracked over time, I was reminded of the blog post by @gesman where he classified users based on mouse movements over time. The time series data could be represented in an image, with a CNN applied to extract features. Just an idea!
Here’s a link to his post on the forum: