I got it to work. Problems seemed to araise from to few samples, and the batac size paramater beeing mismatched.
Another paper, this time from IBM research, that showcases the booming of transformers across different data types:
2 Likes
I once worked for a customer on monitoring data looking for unusual events (anomaly dectection).
I tried to use ElasticSearch that has some machine learning and anomaly detection. Studied hard on the theory of seasonality. But at the end i decided to go for a simple solution based on common sense.
Read more about it in my blog:
2 Likes
question on cleaning up the datasets:
Hello guys, I’m trying to get into the world of time series classification, and the forum here has been amazing and enriching for me so far! Thanks so much for that!
I am currently working on a classification project, and with the help of inceptionTime I am getting high success rates on the validation set, but always fail when it comes to the test set.
My hypothesis is that there is “noise” in the training set: mislabled data or data that can not be determined at all to which class it belongs.
My question, regardless of whether my hypothesis is correct is - is there a way to detect such noise in the datasets, and clean it up? In image training, for example, a quick human look at the data can do the job. But it is much more complicated in our case.
I guess there may be a technique of modeling the distance between examples in a dataset - and thus get a picture of the latent space, and then it will be possible to detect anomalies in it. Do you know such a thing?
Works like https://www.researchgate.net/publication/332989762_TimeCluster_dimension_reduction_applied_to_temporal_data_for_visual_analytics can help you out in visualizing a 2d picture of your time series dataset.
Regarding your drop in performance,I would ensure that your validation set reflects the characteristics of the test data, to minimize the surprise you get when moving the model to test
1 Like
Hi all,
I just wanted to let you know that during the last few weeks I’ve been updating the timeseriesAI/tsai library to make it work with fastai v2 and Pytorch 1.7. I’ve also added new functionality and tutorial nbs that may address some of the issues/ questions raised in this forum.
These are the main changes made to the library:
- New tutorial nbs have been added to demonstrate the use of new functionality like:
- Time series data preparation
- Intro to time series regression
- TS archs comparison
- TS to image classification
- TS classification with transformers
- Also some tutorial nbs have been updated like Time Series transforms
- More ts data transforms have been added, including ts to images.
- New callbacks, like the state of the art noisy_student that will allow you to use unlabeled data.
- New time series, state-of-the-art models are now available like:
- XceptionTime
- RNN_FCN (like LSTM_FCN, GRU_FCN)
- TransformerModel
- TST (Transformer)
- OmniScaleCNN
- mWDN (multi-wavelet decomposition network)
- XResNet1d
- Some of the models (those finishing with a Plus) have additional, experimental functionality (like coordconv, zero_norm, squeeze and excitation, etc).
The best way to discover and understand how to use this new functionality is to use the tutorial nbs. I encourage you to use them!
You can find the tsai library here: https://github.com/timeseriesAI/tsai
You’ll be able to clone the repo or pip install the library.
I hope you’ll find it useful.
25 Likes
Wow this is really a good gift for the singles day!!! 
Thank you so much @oguiza!!! I can’t wait to have a look to all those notebooks.
Best!
3 Likes
Look at this paper
Although it does not provide a comparison to state of the art in the field, the idea of doing time series forecasting as a computer vision task looks quite funny and interesting!
2 Likes
Hi vrodriguezf hope you are having a beautiful day!
Great post!
The above sentence made me laugh as it seems that we at fastai love it, when we can turn things into an image classification problem. 
I don’t know if its because its generally the first model we create or if we as humans just love images.
I wonder if Jeremy had taught GANS as the first model, would GANS would be as popular as image classification is on this forum.
Cheers mrfabulous1
Can anybody help me?
I’m new to deep learning and right now I’m stuck in a project I’m doing alone where I want to predict in an EEG pattern if the eyes are open or closed. The problem is that the accuracy is the same from start to end at 64.5%. I’ve tried to change the loss and the bs number but nothing changed. I have no idea what to do. Please help me thanks.
here’s the code(click on it):
2 Likes
I think you have time-series here, but us a tabular learner. This one is, if I understood that correctly, mainly learning embeddings but not the dependencies between the timesteps. If you look carefully your training loss goes down but valid goes up quite a bit. I think you use the wrong model, so it does not work.
The classic thing to try first here is an LSTM model I guess. I did not find a good fast.ai resource, only pure PyTorch, for doing that with sensor data (I have some PyTorch code here: https://github.com/joergsimon/xai-tutorial-april-2020 ). So maybe some more fast.ai experts might help you here.
As a side note: You did put a learning rate more or less at the end of the plot, where we also have a valley. For this tabular learner, you might also try with lr of f.e. 1e-4. But again, I think this is not the right model for the task anyway.
1 Like
Interesting new (to me at least) package below, which is an implementation of the popular prophet but with pytorch, and the AR-net is using fastai2.
2 Likes
Hi @simone.parvizi,
If I interpret your data correctly, you have an EEG with 14 channels and around 15k long. There’s a column for the output. I guess the data is equally spaced and index in time.
If that is the case, the choice you made is not very helpful.
It would be better to use a time series model that takes subsequences of the entire sequence.
But the first thing you need to do is to convert the data you have into samples that can be processed by a time series model.
You may want to take a look at data preparation in the tsai library. You would need to create an array with shape: [n samples x channels x timesteps] using the SlidingWindow function. There are examples that show how to use this function.
That output is something you can then use to create a TSDataLoaders and a model like InceptionTime. These approach tends to work much better. Please, let me know if that works.
5 Likes
This tsai preparation library is amazing. This type of library has noodled in my head for over a year. I can’t believe you have done this. Really amazing.
2 Likes
Anyone interested to join https://www.kaggle.com/c/jane-street-market-prediction? It looks to me like a time series classification contest.
Yeah, I am interested. I have seen similar competitions on Kaggle before, but I feel the Deep Learning community was never in a better position to solve sequential data.
1 Like
I would like to participate
1 Like
Jane Street is happy with the performance of its existing trading model for this particular question.
Ha ha. They could be a bit happier I’ll bet.
So Jane Street Group harvests our skills and best ideas world-wide for a mere $70,000?
Why don’t we form our own coalition and use any profits to give scholarships to ML students, or for something else socially useful?
3 Likes
I am interested.
1 Like