I believe that trying to explain something makes your understanding better. As I am doing this course alone I am writing after every video so that I can arrange my thoughts. Perhaps this may turn out to be useful to others too later.
Video 2 was about taking an example of matrix factorization (topic modeling) and exploring that via using 2 techniques - SVD and NMF. It was not about understanding how SVD and NMF works from a very detailed Maths perspective but an overview to show how to use them.
News group dataset was used through out this video to demonstrate that. This was my first time seeing clustering happening via code. So that is first use case of Matrix factorization that I can see which I think has many use cases. It would be interesting to come back to this at the end of this course to see if I can actually use this for some actual problem.
I found some interesting things that I can probably do
The first one will take some time so I will do it later but will try and do the other two as it does not seem like they will take up a lot of time. The face dataset would be interesting as that is something that I had looked at earlier but did not completely understand. Also I think I heard eigen values (maybe that was Video 1) which I heard in last video which I want to understand what is as that I have heard in other places too many times now.
The parts where Rachel mentioned that SVD is characterized by trying to make things orthogonal and NMF is by the non-negative constraint that was helpful. Because if I look at wikipedia for any Maths article they are usually way too complicated to understand. There are so many variations of an algorithm that knowing what characterizes a family of algorithm becomes hard to find in all the equations. That is the one thing that I have found off-putting about going through the Maths while trying to go through ML/DL.
I am not 100% sure what each row/column of the decomposed matrices mean exactly right now. Probably need to read up a bit more and play around on that.
At the end of video 1 I have watched it completely more than 1 time. I am going to try not to be hung up on the things if I don’t understand it 100% unless that is stopping me from going through the rest of the stuff completely. Going through half of Jeremy’s course I have understood that is how I can expect fast ai courses to be on a per-video basis. Go through the next videos and I will probably understand. So this time I am going through a different way. I am making a spreadsheet containing things that I do not understand and deciding when I should come back to them at a later point in time.
To anyone reading this. Go through Video 3 before you decide whether you understood the material in Video 2 or not. Video 3 is almost all a review of Video 2.
Thank you for this! I’m currently on the second half of vid 2 and feeling super overwhelmed, almost started questioning my ability to actually understand all the materials.
Video 3 and Notebook 2
The whole focus of Video 3 was a review of Video 2 and lot of what was already covered. But with different perspective. I found it useful to not decide whether I understood everything in Notebook 2 till I completed following the sequence
- Video 2 watch
- try out notebook 2 till randomized SVD as much as you can understand and don’t worry if you didn’t understand it completely
- Video 3 watch
- video 4 watch till 24:42
- do watch video of Pytorch mentioned in notebook (Optional but I would recommend it and don’t worry if you don’t understand all of Pytorch syntax as of now)
- try out notebook 2 completely
- do homework 1
This video starts with a new perspective for matrix multiplication i.e. considering it as a linear combination of columns. This makes much more concise to talk about. Then we go back to considerations for algorithms. Then we went to the excel which adds a different perspective on TF-IDF, NMF, SVD. We moved to block matrices which is a topic that will have to wait till Video 4 probably for me to understand it properly. Then moving on to NMF the most interesting point that I got was Jeremy’s point about how not having ordered diagonals like in SVD makes it hard to understand which topic is most important. Then we went and started talking PyTorch which makes me believe that I will need to spend time during review of notebook on that as that is emphasized a lot as the basic framework and alternative to numpy. At this point most of the review is done. Randomized SVD is introduced but still not completely.
Note after doing all this so that you don’t feel that you are not getting it
- Even if you did not understand all of syntax of PyTorch that should be fine. Rachael mentioned that. I guess as the whole course uses PyTorch we will gradually spend more time on that.
- Randomized SVD even after this is bit of something that was looked up from the paper and you may not understand the formula. LU factorization is something that Rachael mentioned will be introduced in video 4
Todo after this
- Wait for further Videos to understand block matrix
- Understand what Racheal and Jeremy were talking about using “norms” to understand the topic importance in case of NMF
- Read the paper “Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions” as much as I can understand
Video 4 after what I have already described is all about background removal using PCA and Robust PCA. There is helpful advice about how to best use your time to read research papers. There is some other pragmatic advice in this that is helpful. LU factorization was not covered in this so that will be in Video 5.
To anyone getting lost when Rachael was explaining the algorithm I got lost too. I have not done the notebook yet but I still will move forward with the notebook because that’s how fast ai courses are. There are missing pieces but if you keep going forward and then do another pass you will understand much more. So I will probably do a second pass over all of the material after I am done with all the videos. If someone has different advice on this please do share. I am writing these post because I want to hear other people’s advice.