Thanks Jeremy!
I’ve performed some additional tests (including resnet 18 as you suggested) to have a more complete picture. Here’s the summary.
I’ve added added this to the notebook gist so that it’s clearer.
Sorry for the lengthy post!
Experiment:
I’ve run an experiment with 75 variations of encoders, image size and model architecture in the OliveOil dataset (UCR univariate dataset).
Note: the notebook is built in such a way that anyone can use any of the 85 available UCR univariate datasets.
Results:
Key findings:
- Performance of images generated by encoders is higher than the raw image.
- Performance increases with the number of encoders used in the image (an encoder per image channel). Rich-content images perform better than simpler images.
- None of the raw time series techniques perform well.
- RecurrencePlots (RP) and Markov Transition Field (MTF) tend to perform better than Gramian Angular Field techniques (GASF/ GADF) in this dataset.
- The 3 main techniques seem to complement each other. Best absolute performance is achieved when one encoder from each type (RP, MTF, GAF) is used.
- Some of images generated by 3 encoders have a performance close to state of the art (actually in some of the runs, they reached state of the art. The numbers shown represent averages of 3 runs).
- Overall Resnet50 delivers the best result. However, that is only true when images are very large (570x570, which correspond to the time series length). This result is very consistent across all encoders.
- With smaller size images both Resnet18 and Resnet34 perform even better than Resnet50.
- Fastai general transfer learning can be successfully applied to images not present in the the initial training set (Imagenet).
Conclusion:
These results indicate that general transfer learning applied to time series encoded as images may be a useful classification technique.
Next steps:
- Apply these findings to other time series datasets.
- Modify the technique so that it can be applied to multivariate time series.
- Explore other state of the art multivariate time series techniques, and see how they can be implemented with fastai.
I’d like to thank all of those who have provided feedback/ ideas, in particular @mb4310, @marcmuc, @antorsae and Jeremy.