How to use custom_head to predict continuous data

Part 2 & Alumni (2018)
1

I try to fine tune pretrained model, says Resnext50, to use with regression problem.

regression_head = nn.Sequential(Flatten(), nn.Linear(2048,1))
learn = ConvLearner.pretrained(arch, data, custom_head=regression_head)
learn.opt_fn = optim.Adam
learn.crit = nn.L1Loss()

Input size is 2048, from last Conv and output 1 , but the error shown size mismatch. What I did wrong?

2

head_reg4 = nn.Sequential(Flatten(), nn.Linear(100352,1)) - for resnext50

3

It was still same error after applied nn.Linear(100352,1).
I have to mention that image input size is 109
I’ve seen from somewhere that the images size matter for FC inputs, how to calculate it?

Here is the final part of learn.summary

(1): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (2): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True)
        )
        (1): Lambda(
        )
      )
      (1): LambdaReduce(
      )
      (2): ReLU()
    )
  )
  (8): Sequential(
    (0): Flatten(
    )
    (1): Linear(in_features=100352, out_features=1, bias=True)
  )
)>

and the error was RuntimeError: size mismatch at /opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/generic/THCTensorMathBlas.cu:247

1 Like
4

for sz = 109 try nn.Linear(32768,1)
Works for me.
To calculate the input size for linear layer, i just added a Flatten() layer on top

regression = nn.Sequential(Flatten())
learn = ConvLearner.pretrained(f_model, md, custom_head=regression)

Then ran

learn.summary()

and took the output_shape.

('Flatten-157',
          OrderedDict([('input_shape', [-1, 2048, 4, 4]),
                       ('output_shape', [-1, 32768]),
                       ('nb_params', 0)]))])

Maybe there is a better way to do that.

1 Like
5

Thank you, that’s worked!