What is a good learning rate range?


#1

Hi All,

Could you tell me what would be good learning rate parameters (upper and lower bounds) considering the plot below?

Thanks in advance!


(Dominik Engel) #2

You’re looking for the steepest downward slope before that large upward spike, which is this nearly linear part between 1e-4 and 1e-3. I think going with 5e-4 or so is a good start. To be honest, you might want to rewatch the according lesson tho if you cannot answer this yourself. Jeremy explains it pretty nicely.


(Swagato Chatterjee) #3

The upper bound should be around 3e-04 and for the lower bound it completely depends on your data. Check lesson 3 and lesson 4


#4

Thanks for your feedback. I had rewatched that part several times and it was in an attempt to deepen my understanding that I sought help.


#5

Thanks for your feedback. I’m still on lesson 1 but will be sure to be on the look out for what you mentioned. Thanks again @swagman