Amazing @marcmuc, thank you! I will work through the solutions.
Hi there. Did you ever make any progress on this? I have a multivariate time series with oh, about 10,000 variables so Iām interested. 
2 Likes
Hi, has anyone ever tried/considered a ResNet-like model in combination with a WaveNet-like model for time series forecasting/time series data in general?
This idea stems from the success I have seen other people have had with CNN-LSTM stacked ensemble method (because typical CNN models are great for learning local features, while RNNās like LSTM are great for time dependent trends or long-term trends) but LSTMās take way too long to train so I was thinking of replacing it with something like a WaveNet (although, I am not entirely sure how long WaveNet-like algorithms take to train).
The second reason for my ResNet-WaveNet model suggestion came about after I had read this article (article argument is that there are other models better than LSTMās such as ones that use attention or WaveNet like structures): https://towardsdatascience.com/the-fall-of-rnn-lstm-2d1594c74ce0
Iām also curious, does anyone know how long wavenets take to train compared to RNNās or ordinary 1-D CNNās?
@ oguiza Do you have any experience with this?
FYI: This was just released by Andrew Ngās deeplearning.ai today. I have not done it yet and cannot provide actual insight/feedback re: the course, so no āendorsementā, just an info. It uses TensorFlow, but the concepts are transferable of course.
4 Likes
This is a great thread, just spent way too much time reading through all the posts. I havenāt come across any fastai/pytroch examples for creating an ItemBase, ItemList, or Dataset when you have multiple time series with additional features. Below is an example of the data I am looking to create a sequential forecast for:
I have a couple thousand correlated series and would like to produce a forecast in the future given the series and other features. Some features I have values for in the future, like the day of the week, so want to consider using those too.
Any advice on setting these up would be great, thanks!
Iām working on a multi-variate time series. To get multiple timesteps into fastaiās tabular dataset, Iāve extended my dataframe sideways by lagging and merging previous timesteps.
Does this make sense?
Yes it is certainly one method 
I have done this with some sensor/movement data I was working on and I saw good results with it
1 Like
Thanks! I guess there must be something else wrong with my architecture then 
Back to staring at my code againā¦
Not necessarily. Iād focus more on the data at first and make sure itās all good on that end and optimized. Also how long are you training for? How is your validation set? Is your test set representative?
Also did you perform any feature engineering?
Hi! I think I want to tackle the same problemā¦I remember from one of the fastai videos that there was a function to extract the seasonal information of a time series (day of week, month and so onā¦). Am I wrong?
Hey Victor, that used to be part of the āoldā fastai (0.7), but the trick used there is basically this: create a dataframe, make the df.index into a datetime type, then you can do the following:
dt_attributes = ['Year', 'Month', 'Day', 'Dayofweek', 'Dayofyear', 'Week', 'Hour']
for att in dt_attributes:
df['datepart_' + att] = getattr(df.index, att.lower())
There are more attributes you can use, check out here:
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DatetimeIndex.html
This lets you go from one timestamp to a lot of feature-columns in a few lines of code without installing fastai0.7 along v1.
Thank you so much!
By the way, Iāll use this thread to ask for some advice for my task:
- I have hundreds of records of multivariate time series, each of them with 8 variables, most of them biosignals (e.g. ECG). I want to use that data to predict a target value in a 2D-space. I could approach the problem as a regression task, or maybe I could discretize the target space and go for ordinal regression or for classificationā¦A parallel objective is to be able to interpret the results of the prediction model.
Anyway, what I do not really know is how I could apply deep learning in this regard. Iāve read some work related to the application of autoencoders for learning representation of multivariate time series -SOM-VAE, but I am not sure of how to use that as an input of a prediction model.
Is there anything if fast.ai that can be helpful here? Iād love to use this library as much as possible.
For me I created a custom Dataset, each time it grabs previous samples as well. This way we donāt need to create new columns in dataframe.
2 Likes
Hey Zachary,
Thanks for the advice. Doing the standard 4 epoch training with fit_one_cycle, and stopped cos validation loss was pretty stagnant after that.
Just did a 80-20 split for my validation with shuffle=False to avoid data leakage. Frankly not sure if its representative. But using different timesteps (from 0 to 3) havenāt really improved my validation loss.
In terms of feature engineering, not that much, trying out fractional differentiation of the features to get more juice out of my data, but now that I think about it, probably start with some basic ones as well, like ratios etc.
What do you suggest I should do next?
Thanks in advance, Neo
Hello Dear community, i have one question : is there a good way to handle multi step time series prediction (about 200 steps in the future). it is a multivariate time series but only one variable is to be predicted.
thanks in advance
Have you thought about using unsupervised networks for example self-organizing-map (SOM)?
I saw example in matlab using it for time series (https://de.mathworks.com/videos/maglev-modeling-with-neural-time-series-tool-68797.html)
Someone experienced in SOM? Is it worth using instead of the feature extraction parts which are implemented in fast.ai so far?
In generel how do you know you have to continue with a time series instead of not putting it like Jeremy says into just data with another feature which is for example a day in a week?
Hi all.
Maybe you guys can point me in the right direction?
Iām currently investigating which method to use to create a timeseries translation model. Basically, I do have weather data (5-7 features plus data) for some stations and only weather prediction data for a subset of those. I want to train the model to map from one time series to the other. When the model has learned this translation I want to use it to derive future prediction data based on information of nearby stations for a station that does not have prediction data.
Iām not sure what kind of architecture Iād need for that. Conditional variations autoencoder? Seq2seq?
Cheers,
C
Hi Christian,
My two cents is that your plan is over complicated. Why not omit the intermediate step of mapping time series to time series? Use as input the features for those stations that have them, and directly predict the weather at the stations that lack feature inputs. Any seq2seq model with multiple inputs and outputs would do.
Please pardon if Iāve misinterpreted the needs of the problem.
Malcolm
Thanks for your input.
My fear is that this will not give me the best result (especially further down the end of the timeline).
Basically, I want to translate one time-series into another. Simplistically, think of a temperature record from the valley and the temperature record on a nearby hill. People currently simply make a correction of the valley temp. with height (i.e. -0.65 deg C per 100m altitude diff). However, this is really crude and Iād like to learn other variables, tooā¦
I suppose this mustāve been done 100 times already. Just lacking the right search termsā¦
The model can be tuned to a given set of inputs (must not generalise to other sites).
Guys, I put this repo with the implementation on fastai of the paper: Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline
Do you think we can improve the resnet results using the xresnet that showed @jeremy ? I have not been able to make much progress with it. Apparently the batch size / kernel size is what has more impact. I am able to replicate the papers results (or mostly) using fit_one_cycle with only 40 epochs (the paper uses 1500)
I would really like to understand what is the best resnet for timeseries.
8 Likes