I have been thinking about whether a computer can do math like a human without using the computing component(e.g. CPU). So I tried to build a Neural Network to do simple math like plus, multiply, and square.
It turned out that the model can do ‘plus’ efficiently without overfitting, though the predictions for the test set that was out of the range of the training set was not as accurate as I expected.
However, I found that it’s really hard to fit a quadratic function (y = x^2) with Neural Network. I randomly generate 1000 numbers for input X, and calculate Y simply with the square of X, then fit the data into a NN model with 2 hidden layers. I tried several hyperparameters and normalizations but the model still could not make a satisfying prediction.
Did anyone try this successfully? What I expect was that after training the NN model will become very close to a quadratic function in all range of numbers.
Here is what I thought when I start this test: if a NN model can’t even learn a simple quadratic function, how can we trust it will do other things right?