Predicting next image in timelapse

Hi! I have a bunch of timelapses of natural phenomena, and would like to build a model that can predict the next image (ideally based on input of a single image).

Glad to hear about anyone’s experience with similar ideas!


1 Like

wonderful, I want to follow this.

I would look to use the ImageImageList capability for next frame prediction. Conceptually similar to super-res or colorisation examples, predicting one image from another.

Depending on what phenomena you are trying to predict, using a few time-steps from the past can improve the performance. For example if you are predicting a moving object location, with just one time-step the results are probably not good, with two time-steps the model can learn the velocity of the object, with 3 time-steps or more it can also learn the acceleration of the object.

ConvLSTM were designed for this purpose:

Thanks, I’ll check it out!

Yeah I also came across that, have you used it before?

I’ve tried it once but ended up using regular 3d convolutions. It worked well for 3d segmentation with a U-Net structure similar to this:

Do you have more info on this subject, or a recommended repo?
I have want to implement image/video forecasting.