How to decide number of hidden layers

Part 2 (2019)
1

Jeremey in video1(lecture 8) picking nh(number of hidden layers for CNN) as 50. In second video he picked number of hidden layers as 32[sqrt(5) notebook]

How is he picking these magical number and can someone show me a diagram(if possible) what does he mean by hidden layers?

Does every layer(50) + relu + layer(50) + relu contain 50 layers each or total number of hidden layers(layers) in the network(entire CNN) are 50…so maybe like 25 and 25 or something else

Please help me understand this. Thanks a lot in advance

2

Hi @curious_sperm
In lesson-8, model consists of
Linear Layer(n,nh) -> ReLU(nh,nh) -> Linear Layer(nh,c)

Where n: Total number of inputs
nh: 50
c: outputs (in this case 10)

nh generally comes from trial and error and there is no definite trick to find this number. It is usually somewhere close to c.

1 Like