As some of you may be interested/ work in a particular area of deep learning, it might be useful to have a place in the forum where we can group ourselves by areas of interest, in a similar way to what we do with time zone/ geography study groups.

I have thus created this thread so that those interested in time series (TS) can share our experience, ideas, blogs, notebooks, libraries, articles, etc about how fastai can be leveraged in this area.

There is no particular TS section in the program, but most of the topics covered in this course can be applied to time series data.

For example, TS can be handled as:

Images (Vision applications)

Bag of words (Text applications)

Structured data (Tabular data applications)

(I’m not so sure about collaborative filtering, but we’ll see when we get there

As Jeremy says: ‘Be bold and creative! What ideas can you come up with which will make your learning journey better, and help you connect with like-minded people?’

It shows one way in which a univariate TS dataset (OliveOil) -from the UCR time series datasets- can be transformed into images (using Gramian Angular Field), and then modelled following the general transfer learning approach we used in lessons 1-2.

The results surprised me (very close to state of the art!) considering:

How small the train sample is (30 samples only)

These GAFD images are very different from those in Imagenet.

I was just applying the standard fastai method, only tuning epochs and lr. I’m sure there is room for improvement.

If you are interested in applying DL to time series, there is a super helpful library named pyts I have recently discovered. pyts is a Python package providing implementations of state-of-the-art algorithms for time series transformation and classification.
You can learn more about it in this website, and it’s Github repo.

Thanks for highlighting this @keijik. That’s an excellent performance.
This makes me wonder what is the real value of transforming a time series using any encoding technique like Gramian Angular Field, Markov Transition Field, etc?
I’m other words, how can you get a higher performance:

Generating images from raw time series data, or

Transforming the time series using an image encoding technique
I’ll try to answer that question in the dataset I used before, but it’d be good if you can confirm it on your data.

I have found my result pretty surprising as I was expecting probably 10-20% error rate. The local result here seems to check out so I think ultimately I need to collect bigger dataset to validate.

I have done something very similar to you using energy data. I have not gotten around to post it and am still looking for public datasources to replicate this so I can show the stuff as the data is private.

But things I have found re this method:

Plotting filled curves instead of just a line (area plot, area under the line filled) gave better results then just the line (probably easier for the cnn to spot patterns when more of the image is colored)

It works even using multivariate timeseries : I used the 3 color channels for the 3 phases of electricity. This gave better results than using just the univariate sum time series! So if you have multivariate <= 3 this is worth experimenting with. Maybe even using more then 3 in different colors and transparency, but haven’t tried that myself.

Depending on your dataset it is worth experimenting with y axis limits when creating the images from the timeseries. Depending on the usecase that helps a lot. Sometimes fixing one or both ylims is better, sometimes letting it zoom just to the present range may be better.

experimenting with temporal resolution is interesting: I have found that hourly time series gave nearly the same accuracy as 15min time series, but of course needs only 1/4 of the data/ pic size so is much more efficient.

I am very happy about this thread, thanks @oguiza for the initiative, looking forward to more of these already intersting insights here!

Ah one more : Using simple heatmaps is very useful and gave me good classification results, but that was for monthly data. But humans can easiliy spot time based e.g. weekly/daily patterns in heatmaps, so CNNs can too!

Thanks a lot Marc for sharing all this!
They are very interesting insights.
I will certainly incorporate some of them in some further experiments I’m planning to carry out and will report back on this thread.

Time Series Data Library created by Rob Hyndman, Professor of Statistics at Monash University, Australia. This one contains thousands of ts datasets, but they are not built as ts classification problems, so you will need to build the train, val and test sets.

Are you guys mainly interested in classification applications or also forecasting? Can anyone here recommend good resources for ts forecasting or also multivariate forecasting, or forcasting univariate but using multivariate inputs with LSTMs , NNs etc.? There is so much stuff out there for ARIMA etc…

I am working on Predictive Maintenance use cases where we try to predict the failure of various kinds of devices in advance (days to weeks). It’s mostly a multivariate classification problem with tabular data where based on various factors (Temporal Data : devices performance parameters on a particular day, days since last repair, age of the device, external weather condition etc. Static Data : Model of the device, geographical location of the device etc), the model tries to predict if the device is going to fail within next X days (Label : 1 ) or not (Label : 0). Would be interested to understand how Deep Learning can be used for such use case. Not sure if there is any comparative study about Deep Learning vs Tree based approaches for such use case. Will be happy to hear from others.

I have not gone into the details, but Uber has a nice blog about the very same topic (Link). The result is little surprising:

…pure machine learning and neural network (NN) methods performed worse than standard algorithms like ARIMA or Exponential Smoothing (ES), and still worse against various combinations of these base statistical methods. However, the winner of the competition, with a solid margin, was Slawek’s hybrid Exponential Smoothing-Recurrent Neural Networks (ES-RNN) method. It mixes hand-coded parts like ES formulas with a black-box recurrent neural network (RNN) forecasting engine.

Electrical energy consumption and generation data for different households, businesses, offices, industries as well as PV, wind and CHP generation on an at least 15min readings level, although as mentioned 1h would also be okay for demonstration purposes. I am aware of the London households datasets, and then there are several datasets from utilities or whole countries for data consumption, but I am looking for installation level / site level data.

Very good thread! Very interesting to learn about using CNNs to classify time-series from images, will definitely give that a go myself.

Personally I work with financial data through my institution and cannot share any of the datasets, but since watching v2 of this course some time ago I have adopted Jeremy’s ULMFiT approach to time-series classification. Briefly, the idea behind ULMFiT is before training a classifier (on some NLP task), one trains a language model (essentially a forecaster, a model which takes a sequence as input and tries to predict the next item in the sequence). Once the LM is strong enough, one slaps on a linear classifier to the end and fine tunes it for the classification task.

This method works exactly the same way for time series and I have used the method for improving classifiers that were trained from scratch in the domain of risk management (probability of default on loan, data consists of individuals deposit activity); asset pricing, and cash-flow optimization. I will try to take a look at some of those datasets in the link @oguiza was kind enough to share and put together a notebook illustrating the procedure.

I think it would be interesting if we try to put together a notebook as a team that compares some of these different methods:

i) CNN classification of un-altered time-series image
ii) CNN classification of transformed time-series image (Gramian Angular Field or maybe a 2d plot in time-frequency domain after wavelet transform… just a thought, see https://www.mathworks.com/examples/wavelet/mw/wavelet-ex11554099-continuous-wavelet-analysis for an example of what this might looks like)
iii) Direct RNN-based approach without ULMFiT pretraining
iv) RNN-based approach with ULMFiT
v) CNN based approach directly on time series with 1d convolution along the temporal dimension

Happy to add more to the list! A good place to start would be agreeing on one or two datasets/tasks and implementing one (or more) of the above methods and sharing a jupyter notebook here, then as we go someone can combine all these into one notebook and once there we an use this to quickly pit these methods against each other on wide range of tasks.

There was a competition in DrivenData (in collaboration with Schneidier) for forecasting energy consumption (Link). As per the data description, it had site level data at an interval of 15 minutes. But, unfortunately, the competition is closed now and the dataset is not also available. Not sure if there is any other source to get the data.