Consider the most extreme case - reconstructing the matrix using an outer product of two vectors. Clearly, in most cases we won’t be able to reconstruct the matrix exactly.
Why would we not be able to reconstruct the matrix exactly? Are approximate matrix decomposition more common than exact decomposition?
Also the Motivation mentioned is the motivation for NMF, SVD, topic modeling with NMF and SVD or bag of words approach? I have re watched the first 30 minutes of the video on 5 different days but I am still not able to tell what the motivation part was referring to.
The image below shows the result of an outer product of two vectors can not be any arbitrary set of values but rather should follow a specific relation. The proof is for the case of 2 by 2 matrix and can be extended to any n dimensional square matrices.
We are talking about matrix decomposition. Saw we decompose the term-document matrix A into 2 matrices U and V using SVD. Now, we are saying that we cannot reconstruct A using U and V by using the outer product of U and V. Because outer product is a specific formula? Then why do they say “that outer product would be as close as we can get.”?
Also seems like the motivation part is motivation for why we are looking at bag if words approach for matrix decomposition.
Video 2 is pretty fast-paced, and in video 3 I review the same material (as well as offer some new perspectives and answer questions that have come up), so definitely be sure to check out Video 3 even if you are still stumped on video 2.
Vikram is correct that the outer product of two vectors can’t be an arbitrary set of values. Another way to state it is that an exact decomposition is not always possible. For instance, there may not exist any matrices A and B such that M = A B. In that case, we often try to find the A and B that will give us the closest answer (i.e. that minimizes | M - A B |, since we can’t get M - A B = 0)
“that outer product would be as close as we can get” is referring to this idea of choosing A, B such that | M - AB | is minimized.
By “Motivation”, I meant an idea of why decompositions and clustering may be related (and thus why looking for a decomposition could be a way to find topics/clusters).
