I have a convolutional image segmentation model that starts with a 100x100 image and ends with 100x100 by several classes. A fully connected head layer (Linear) reduces down to 2 classes. Suppose now I want to ‘warm up’ the model, fast training on half size 50x50 images. After that, I want to use what the FC layer has learned on the smaller images as a head start on the full-sized images.
Everything in the convolutional body scales to the smaller image size, but the last Linear layer does not. It takes 100^2 down to 2, but does not understand 50^2, much less how to warm start with weights trained with a smaller input. What would be a good way to handle this situation?
I have thought to use PyTorch nn.interpolate to upsample 50x50 to 100x100 before the FC layer. Is this a reasonable approach? Is interpolate() even differentiable? Alternatively, can an adaptive pooling make an image larger?
Thanks for any advice, even hints.