How to do transfer learning with different inputs

[hwasiti (Haider Alwasiti)]

@maral

Maybe a silly idea but you could train 4 resnet models independently.

Let A, B, C, D be your 4 channels.

Model 1 - Train on A, B, C
Model 2 - Train on B, C, D
Model 3 - Train on C, D, A
Model 4 - Train on D, A, B

For inference run your prediction through all 4 models. Add up class probabilities then normalise to produce list of probabilities.

or even better remove the heads of these 4 resnets and combine them by either CNN or FC NN.

See here for a discussion on how to do such type of stacking of NN models.