Hi! I have a bunch of timelapses of natural phenomena, and would like to build a model that can predict the next image (ideally based on input of a single image).
Glad to hear about anyone’s experience with similar ideas!
Cheers
1 Like
wonderful, I want to follow this.
I would look to use the ImageImageList capability for next frame prediction. Conceptually similar to super-res or colorisation examples, predicting one image from another.
Depending on what phenomena you are trying to predict, using a few time-steps from the past can improve the performance. For example if you are predicting a moving object location, with just one time-step the results are probably not good, with two time-steps the model can learn the velocity of the object, with 3 time-steps or more it can also learn the acceleration of the object.
ConvLSTM were designed for this purpose:
Thanks, I’ll check it out!
Yeah I also came across that, have you used it before?
I’ve tried it once but ended up using regular 3d convolutions. It worked well for 3d segmentation with a U-Net structure similar to this: https://deepmind.com/documents/239/De%20Fauw%20et.%20al%20Nature%20Medicine%202018%20.pdf
Do you have more info on this subject, or a recommended repo?
I have want to implement image/video forecasting.