Finetuning ResNet50 using satellite images with 13 channels

Hi all,

I have been able to finetune the ResNet50 architecture with a set of 27000 satellite RGB images using just the fastai library for land cover classification. Those images have been extracted from Sentinel-2 products that by default come with 13 spectral bands, so more than the usual 3 RGB optical bands. Now since ResNet50 has been trained on RGB images and the channels in ResNet50 are fixed I wonder if there is any simple way to still use it to finetune a model using images with 13 spectral bands. Thanks in advance for any suggestion.

Luigi

Greetings Luigi,

This question has been asked and thoroughly answered on these forums. There is even a parameter that specifies the number of input channels when creating the CNN Learner. You may want to design your own initialization of the first layer though, one that takes account of the contents of the additional channels.

Good luck with your project! It sounds fascinating and useful.
:slightly_smiling_face:

1 Like

Hi,

Another option, provided you have enough resources, is to pretrain the network with BigEarthNet, or just use the weights available there (though you have to convert them from TensorFlow to PyTorch first). Comparing results ImageNet-pretrained and BigEarthNet-pretrained networks could also be interesting.

Hi Janne,
thank you for your suggestion. It doesn’t seem very easy to implement. Anyway I was using the EuroSAT dataset for finetuning that contains only 27000 64x64 pixels images while BigEarthNet contains more than 500k images. Maybe I can already get better results with the larger dataset still using only the RGB channels.