1D FCN Overfits!

Hi everyone.

My Time-Series is a 90 x 100 table representing points from three different types of graphs: Linear, Quadratic, and Cubic Sinusoidal. Thus, there are 30 Rows for Linear Graphs, 30 for Quadratics, and 30 for Cubics. I have sampled 500 points from every graph. Here’s an image to illustrate my point:

I’m very confused why my 1D FCN is performing so badly. I’ve been debugging it for the last two weeks – changing the training set, changing the # classes, changing the architecture … nothing works.

  • Confusion Matrix shows only 42% validation accuracy.
  • Loss Function also looks disappointing – it seems to be overfitting after only 10 epochs (I also tried 100 and 1000 epochs – same results)
  • Probability Graph shows that the Network is extremely confident about the Quadratic Class, but very confused about the Linear and Cubic graphs.

I’ve tried to fix this problem by changing my training set many times, but to no avail. Currently, I’m trying to debug the problem by scaling down my training set, so it has only two classes. I’m not sure if the architecture of the FCN is causing the problem. Here is the code: 1D CNN Time Series Classifier.

I would be very grateful if anyone could help me resolve this issue. Thank you.

Below are some metrics that may be helpful, including the Loss Function, Confusion Matrix, Architecture, and Probability Graph.



LossFunction

Hi Refath,

Welcome to the forums, where some of us who have been around longer try to answer questions as best we can.

Since this is a time series classification problem, you may get better answers in a different part of the forums:

You will find the latest and most effective techniques there.

As for suggestions, first run one_batch() on the Learner to make absolutely sure that the proper batch is being sent to the model. Remember that conv1d expects a channel dimension to be part of the input tensor.

Other ideas:

  • 100 points is not many to characterize a semi-periodic time series. You would need enough to specify the shape of each period in order to distinguish their shapes. If your eye cannot see the differences from 100 points, the model is unlikely to learn them.

  • FCN may not be smart enough to make the classification. Linear+ReLU at best makes a piecewise linear approximation. You are asking it to discern between periodic time series with very similar waveform shapes. FCN might not be able to “see” the difference between cubic and quadratic.

  • It certainly looks like overfitting. Another way to think of it is that generalization is as good as it gets, and the model has gone on to memorize the training set of only 90 members.

My suggestion is to start with an example of the FCN model that works exactly as expected. Then in small steps modify the input data to be what you want it to classify. You might find the specific cause of bad performance that way.

HTH a little,
Malcolm
:slightly_smiling_face: