Hi Malcolm,
Thank you for correcting my calculation! And; yes, I follow that there is a reduction of width; height and channels through the model.
I’m still not clear on how to understand the paragraph in Chap 13 https://github.com/fastai/fastbook/blob/master/13_convolutions.ipynb
A Simple Baseline
The model defined earlier in the chapter is
simple_cnn = sequential(
conv(1 ,4), #14x14
conv(4 ,8), #7x7
conv(8 ,16), #4x4
conv(16,32), #2x2
conv(32,2, act=False), #1x1
Flatten(),
)
So also has a reduction of width; height and channels through the model. However, when discussing increasing the number of output channels; the paragraph still states That means it isn't really learning much at all: the output size is almost the same as the input size. Neural networks will only create useful features if they're forced to do so—that is, if the number of outputs from an operation is significantly smaller than the number of inputs.
- and then redefines the model to have a larger convolution height,width:
return sequential(
conv(1 ,8, ks=5), #14x14
conv(8 ,16), #7x7
conv(16,32), #4x4
conv(32,64), #2x2
conv(64,10, act=False), #1x1
Flatten(),
)
As I understand from the much deeper ResNet architecture; the rule if the number of outputs from an operation is significantly smaller than the number of inputs
is only applicable in certain scenarios; so I’m wondering what those are. Or … which bits of the above I’m mis-understanding; which is possibly more likely.
All thoughts most appreciated