Lesson 14 (2020) discussion

Part 1 (2020) Non-beginner
1

Link between ResNet class parameters and layers

I’ve been trying to wrap my head around how to link ResNet(dls.c, [2,2,2,2]) with ResNet-18 and ResNet(dls.c, [3,4,6,3], 4) with ResNet-50 in the ResNets chapter of the fastbook.

I’ve had a look at the resulting sequential model, as well as the output of learn.summary(). But I am not sure what to “count” as an layer and what not. I came up with a formula that would count every ResBlock as two layers. layer_amount = stem_layer + sum(layers_array*2) + fully_connected_layer.

It works for ResNet(dls.c, [2,2,2,2]) being layer_amount=1+[4,4,4,4]+1 resulting in ResNet-18. But then ResNet(dls.c, [3,4,6,3], 4) would be ResNet-34 based on my formula, but it should be ResNet-50 according to the book.

Can anybody help me out?

2

This picture in the resnet paper can help you out ? ResNet | PyTorch

I believe the original resnet model use 2 layers for the stem (7x7, 64, stride2 and 3x3 maxpool stride 2) which is different from the stem of fastai (which have modified based on maybe some advanced research) - 3 conv2d layers and 1 maxpool

From the original model we have:
Resnet 18: 2 + 2x2x4 = 18
Resnet 50: 2 + 3x3 + 4x3 + 6x3 + 3x3 = 50

Hope it helps

3

Hi @dhoa thanks for your reply. Now I get it :muscle:

In the Resnet-18 the ResBlocks are just two convolutions and in the bigger Resnets the Bottleneck ones are used. Thats why it is

Resnet-50 Input: [3,4,6,3] formula 2 + 3x3 + 4x3 + 6x3 + 3x3 = 50
Resnet-18 Input: [2,2,2,2] forumula 2 + 2x2 + 2x2 + 2x2 + 2x2 = 18

1022×446 21.8 KB

Thanks a lot !

1 Like