Lesson 7 - Official topic

Note: This is a wiki post - feel free to edit to add links from the lesson or other useful info.

Resources

Links from lesson

  • fill me with interesting stuff

Other useful links

13 Likes

Second to last lesson :frowning:

7 Likes
  • check here if audio is clear
  • check here if it is not

0 voters

Edit: Thank you all, it seems the audio is clear for most people

seems like both weight decay and learning rate can be used to address overfitting. How do we use them together ? And how do we find the optimal values of both when used in the same equation.

The learning rate doesn’t do any regularization, it’s the size of your updates.

They’re separate hyperparameters so you can specify them together. I don’t know how to find the optimal value for weight decay but learning_rate you can use lr_finder()

1 Like

is LR only to decrease the number of epochs to achieve optimal results ?

Try re-running the learning rate finder with different wd parameters (see code). For example, try anywhere between [1e-5, 1e-4, 1e-3, 1e-2, 1e-1].

1 Like

No. If you don’t have a proper value, you won’t train at all. Look again at the past lessons to refresh your mind, this has been covered in chapter 4 and 5.

How do we set the weight decay’s hyper parameter? Can we do something like a random search or grid search approach or is there a better way to set it?

2 Likes

There is no proper way we have found yet. So trying various values is still the best solution.

5 Likes

What’s the advantage of creating our own embedding layer over the stock PyTorch one? I think I missed that

1 Like

What’s the difference between PyTorch’s nn.Module and fastai2’s Module?

2 Likes

fastai’s Module removes the need to call super().__init__(), which you need to call at each nn.Module init.

8 Likes

This is wrong :frowning: Shawshank redemption should be on top!

5 Likes

It’s ALMOST as good as Lawnmower Man 2 :stuck_out_tongue:

1 Like

Ah, E.T. My favorite romance film :slight_smile:

4 Likes

how does sample size of the ratings affect our learned bias ranking?

1 Like

did we cover how n_factors is selected?

1 Like