Exploring fastai with Excel and Python

Notes from fastai teaching

You can search all fastai videos with keywords! Thank you very much @fmussari for this wonderful tool!

What Jeremy said about Affine function and non-linear activation function and what does ReLU do for neuralnet in general?

How should we think of deep learning jargons?

Why is non linear activation the power source of neuralnet mathematically?

What does Jeremy say about epochs?

Summary
  • What is an epoch?
  • What could happen if you run 10 epochs with lots of weights and high learning rate?
  • What would happen related to overfitting if you do just 1 epoch?

Are these Jeremy’s experimental knowledge or there is some more blogs or papers can read upon?

Radek on weight decay

Do weight_decay vs l2_reg behave differently under the same but more complex optimization stepping functions?