I was trying to use the fastai library with some eeg data in trying to predict when a person is thinking about closing its hand. I used the lesson3-rossman and the lecture 4 in youtube as guides and manage to write some test code.
However it seems that no training is taking place since the loss is not decreasing.
I would be very grateful if @jeremy or anyone could have a look in my code and results, and hopefully give me some advice.
Thanks
P.S. Here is a github repo hosting the code and datasets
thank you - at first glance - you only have numbers (which are all neural nets ever want ) and you try to model using a categorical approach, which might not be ideal?
I will write more later, but for now you could think about that
Well, I used this categorical approach because this is the way lesson 4 shows how to treat structured data. But in the code you can see that I created a dummy variable consisted of only 0’s as to be treated as the categorical one. Wouldn’t this be the same as to not have any categories to begin with?
I think not - The model still tries to align the values in the feature with its predictions for the dependent variable and while I understand the idea, you don’t need the categorical variable.
I am currently struggling with a similar experiment - I have only categorical variables and am trying to predict a category… Somehow this is harder than expected
For your experiment - have a look at your learning rate - is there something weird going on in the plot?
Yes, I understand that I don’t need the categorical variable. The problem is that I couldn’t find a way of using structured data without having at least one categorical variable. The fastai library does not let me create a ColumnarModelData object without a categorical variable. I already asked this another topic but got no answers. So if you know about any better way to work with structured data I am all ears
Bellow is the plot you mentioned. As we can see the loss is not decreasing, showing that no learning is tacking place. Regarding the learning rate I don’t understood why you mentioned it. I used the usual approach shown in the first 4 lessons of the course which is first using lr_find() to find an optimal learning rate. Since we cannot pick a learning rate from the plot I select 1e-3 once it seems to be a good choice for most general applications. Is there any erros in that approach?
Ok so I had another look at the data and tried a traditional ML approach with XGBClassifier and a simple LogisticRegression - both have ~49% accuracy which means they are really bad at predicting the outcome.
Plotting the data revealed that there is no relationship between the response and the individual values - additionally, I checked for pearson correlation of the values to the response and the strongest! was 0.08, where 1 is high and -1 is negatively correlated.
The reason, why the NN did not learn anything might have been because its fed ‘random’ noise and cant make sense of that.
edit: I will however still try to fit a neural net to the data, just for the sake of trying - lets see what happens
I think I am now getting closer to understanding what the purpose is - the experiment is something like a time-series where 14 individuals measurements are made while someone supposedly thinks about closing their hands? That means! you first should do feature selection! - you need to figure out which of the sensors captures the response best! -> something like this and then look for the appropriate channel - however! you only have 1? individual tracked - you need more! like actually way more, 10-100-1000’ish the more the merrier.
This is expected. Thinking about moving a body part has the particular effect of diminishing the magnitude of frequencies among de 8-13 Hz range. The overall magnitude is not affected.
Correct. Each time series come from a sensor attached to different positions in the person head as shown in the figure bellow.
I already have a working method that uses traditional ML approach and it has a lot of featuring engineering/selection. I hoped that by using deep learning I could improve this aspect.
I can get more data since I have access to the EEG equipment. Maybe things will turn out better once have more measurements.
) and you try to model using a categorical approach, which might not be ideal?
