Hey every1,
I created a small repo implementing Image Time series forecasting using Recurrent COnvolutional networks, it is actually using a ConvGRU cell with some succes.
I am very interested on this subject of moving object on video, image sequences so It may be useful for someone else.
It is not yet finished, but you can start playing with the Data and the simple model.
Thomas
8 Likes
This is really great work!
I am unable to make show_results work.
My problem is that the input is a sequence of images ant eh output should also be a sequence of images.
So I modified the loss function to compute the loss per image, but I am unable to make the show method work on the output as it stacks the tensors along the batch size. Here you get the idea:
I have to create the
ImageSeq object manually after recovering and aligning the predictions. Any suggestions on how to deal with these?
- Any tips to get sharper results with MSE loss also?
I updated the repo, added training with CrossEntropy (converting BW images to 0-1) with better results.
I heavily updated the repo to include the results form this paper that caught my attention:
-
PhyDNet - Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction
It has a semi working repo that I refactored to make it work with fastai.