<<< Wiki: Lesson 4 ｜ Wiki: Lesson 6 >>>
Lesson resources
 Lecture 5 notes from @timlee
 You can download an arxiv dataset using this project
 The language model dataset is wikitest2
Links to more info
 Jacobian and Hessian in the Deep Learning book: section 4.3.1 (page 84)
 Backpropagation as a chain rule by Chris Olah
 Another explanation about the chain rule from Andrej Karpathy
 Why you should understand backpropagation
 Fun with small image dataset by @beecoder
 Make Neural Networks from Scratch
 An overview of gradient descent optimization algorithms
 Add SGDR, SGDW, AdamW and AdamWR
 Fixing weight decay regularization in Adam
 Deep recommender models using PyTorch
 Initialization Of Deep Networks Case of Rectifiers
 What are hyperparameters in machine learning?
Other datasets available
 Predict the happiness
 Netflix prize
 Kaggle  Movies dataset
 Amazon reviews
 State of the Art benchmarks with datasets
Video timeline

00:00:01 Review of students articles and works

00:07:45 Starting the 2nd half of the course: what’s next ?
MovieLens dataset: build an effective collaborative filtering model from scratch 
00:12:15 Why a matrix factorization and not a neural net ?
Using Excel solver for Gradient Descent ‘GRG Nonlinear’ 
00:23:15 What are the negative values for ‘movieid’ & ‘userid’, and more student questions

00:26:00 Collaborative filtering notebook, ‘n_factors=’, ‘CollabFilterDataset.from_csv’

00:34:05 Dot Product example in PyTorch, module ‘DotProduct()’

00:41:45 Class ‘EmbeddingDot()’

00:47:05 Kaiming He Initialization (via DeepGrid),
sticking an underscore ‘_’ in PyTorch, ‘ColumnarModelData.from_data_frame()’, ‘optim.SGD()’ 
Pause

00:58:30 ‘fit()’ in ‘model.py’ walkthrough

01:00:30 Improving the MovieLens model in Excel again,
adding a constant for movies and users called “a bias” 
01:02:30 Function ‘get_emb(ni, nf)’ and Class ‘EmbeddingDotBias(nn.Module)’, ‘.squeeze()’ for broadcasting in PyTorch

01:06:45 Squeashing the ratings between 1 and 5, with Sigmoid function

01:12:30 What happened in the Netflix prize, looking at ‘column_data.py’ module and ‘get_learner()’

01:17:15 Creating a Neural Net version “of all this”, using the ‘movielens_emb’ tab in our Excel file, the “Mini net” section in ‘lesson5movielens.ipynb’

01:33:15 What is happening inside the “Training Loop”, what the optimizer ‘optim.SGD()’ and ‘momentum=’ do, spreadsheet ‘graddesc.xlsm’ basic tab

01:41:15 “You don’t need to learn how to calculate derivates & integrals, but you need to learn how to think about the spatially”, the ‘chain rule’, ‘jacobian’ & ‘hessian’

01:53:45 Spreadsheet ‘Momentum’ tab

01:59:05 Spreasheet ‘Adam’ tab

02:12:01 Beyond Dropout: ‘Weightdecay’ or L2 regularization