Moovement classification using ml

Hello everyone,
I am currently working on computer vision, more especially in object tracking.
I need to classify the movement of objects with three categories: in, out, none
I 've made datasets containings centroids of objects, in lists that are and classified with the three categories.
In my dataset, one line = one object position + object id + frame_number (start from 0 for every serie)
I am looking for a simple model to take those series (that have variables sizes), train, then classify new series that I could feed him with.
I was thinking of something like a RandomForestClassifier, with series.
Can someone guide me threw this?