I’m on lesson 4 of the course and have been back through the same lesson a few times now but I don’t think I’m quite getting a few things. I want to have an understanding of what’s going on before moving on.
This lesson is on MNIST - and Jeremy is recognising 3s vs 7s. So he has 12000 images (ignoring mini-batches) of about 800 pixels each, and his tensor has a shape of (12000, 800). He randomly generates 800 weights, 1 for each pixel, plus a bias (which I don’t quite understand the purpose of either).
So then for the first image, he multiplies the image’s value by the weight for that pixel. So if the pixel value is 0.9 and the weight is 0.1, 0.9*0.1 = 0.09. Or if the pixel value is 0.5 and the weight is 0.6, 0.5*0.6 = 0.3.
(and then a bias is added, and this is done for every image)
I don’t understand what the purpose or benefit is of multiplying these numbers together. If I wanted a score of how accurate is our weight to the pixel value, I’d do abs(value - weight).
And then adding these up would give me a score of how accurate my weights are.
The idea of deep learning is that you have a value (a single pixel which has a value of 0-255 but we divide by 255 to give 0 to 1). You want to create a matrix w and a vector b so that value*matrix w + vector b returns an indications (say probability) that the number is either 3 or 7. You repeat for every pixel. So you are producing a compromise matrix w and vector v which gives the best value to indicate 3 or 7 for ALL the pixels. You start with a matrix w and vector b of random values and calculate the indication of 3 or 7. You check the answer and depending how wrong you are you change the matrix w and vector b a little bit. You then try again. Eventually you get it right 95% of the time and you declare 95% is the best in the world.
So to change the question
Say I buy 3 apples and 2 pears and it comes to 7 cents.
We can say (3,2) X price matrix = 7. How do I calculate the price.
My guess is random 1 and random 2. say 5 and 9. 3x5+2x9=33 too high for 7.
7 is a quarter of 33 roughly so let us try 5/4 and 9/4
35/4 + 29/4 = 7.25 lucky guess. So lets try …
1 and 2. 3 * 1+2 * 2 = 7. 100% correct.
Deep learning does the matrix guessing until you get a good enough answer.
Regards Conwyn
Hi Andrew,
Share a question I posted about bias. Hope that can clarify the purpose of a bia.
And Conwyn gives a very interesting example, it’s similar with my understanding while learning the lessons. Multiplying weights by pixels and sum them up is for constructing a “equation (set)”, but we don’t knonw the parameters(that is wights) of every variable and the constant(that is bias) . So we initiate a group of random parameter and random constant. And fill solutions(that is train data set, in the MNIST example, it is pixels of a image) into the equation and check if the “equation” is perfect enough, if not, adjust the parameters and then fill solutions, and check again …