Parallel / Fused CNNs

Sometimes we may want to classify more than one image at the same time without losing the image resolution. Or we may want to use some multiple features (like 2D wavelet transforms or 2D-ffts) of an image as a input. is a great tool. I wonder if it is possible to build architectures like these with using benefits of transfer learning;

  1. Making parallel CNN networks and connecting the fully connecting layers together.
  2. Or fusing some early convolutional layers together and then continue convolving with the fused features.

It’s very easy using fastai2 and datablock api. You just to need to do:

Datablock(blocks=(ImageBlock, ImageBlock, MultiCategoryBlock), ....)

See or muellerzr vision course

1 Like