Lesson 5 In-Class Discussion ✅


(Jeremy Howard (Admin)) #1

Use this thread for questions/discussion of today’s lesson. Please do not use this for questions/comments about topics we haven’t covered yet - use the Lesson 5 further discussion thread for that. Also remember to watch the official updates thread.


FAQ, resources, and official course updates ✅
📝 Deep Learning Lesson 5 Notes
Fast.ai v3 2019课程中文版笔记
(Jeremy Howard (Admin)) pinned #2

(Aidan Davis) #8

What is the name of the article Jeremy is referencing?


(Francisco Ingham) #9

(Ilia) #11

Small heads-up questions probably. However, could you please clarify if this plot shows learning rate changing during a single epoch, or during the whole training process? I am expecting that it is the former.


(Charlie Harrington) #12

Can Jeremy please write out that formula he keeps saying?


#13

Jeremy is going to explain that plot later during the lesson. Please wait for him going over it :wink:


#14

parameter = parameter - learning_rate * parameter.grad


(Raj Mapu) #16

question to the folks here,I just forgot from previous lesson, what are the blue box’d activations and purple box’d activation? Why there was two boxes drawn for activation layers initially.


#17

Blue boxes are before the activation function, purple box is after the activation function.


(Charlie Harrington) #20

When we load a pre-trained model, can we explore the activation grids to see see what they might be good at recognizing? How can we generate those images?


(Kunal Seth) #21

What would we do if we have a very high number of images to classify say in the order of 100000 classes?


(YJ Park) #22

Is data.classes different from data.c?


(jyoti prakash) #23

How we initialize random weights?


#25

data.c is the length of data.classes in general.


#26

There are several articles about that. The basic initialization is to use a normal distribution with a standard deviation depending on the number of input channels.


(Cedric Chee) #27

Discriminative layer training: https://docs.fast.ai/basic_train.html#Discriminative-layer-training


(Sasha) #28

So when we first use transfer learning and train - we are only training the random layers we placed on top of the model, or are we also training top few layers of the resnet?


#29

No, only the new layers initialized randomly. As long as you haven’t typed learn.unfreeze().


#30

Maybe a dumb question but why divided by 3?