04_mnist_basics: Why example curves do not overlap in the end

Part 1 (2020)
1

Hi,
I’m experimenting with 04_mnist_basics notebook (more precisely with An End-to-End SGD Example, where we are looking for quadratic function to approximate points randomly scattered around another quadratic function). And to my surprise they don’t seem to get as close to each other as I expected
screen

No matter how many iterations I do, the left part of the curve stays far away…
I thought that parameters after training will become very close to those used for points initialization (speed = torch.randn(20)*3 + 0.75*(time-9.5)**2 + 1)

Is this an expected result or I’m doing something wrong? (I ran original code from notebook but got same results)

Thanks in advance

2

Due to the noise involved (torch.randn), it is impossible for the two curves to match because a neural net can’t approximate random numbers (I guess they theoretically can since the numbers are pseudorandom, but that’s a seperate topic). They should, however, be very close, which they are in your case, so don’t worry about it.

Have a nice weekend!

3

Thanks,
But I thought that a, b, c will eventually become smth like 0.7, -12.6, 58.7 and curve will become smth like
image

I guess I’m doing a mistake somewhere…

4

And it did) My step was too small and I did not enough iterations

1 Like