Convnets Ensemble

Hey Guys,

I have 3 models I trained and I want to ensemble them together in hope of getting better results.

What is the best way of doing that:

  1. Taking the softmaxes of each model and do averege
  2. Taking the softmaxes and concatenating them and feeding it into Xgboost
  3. Taking the last Linear output (not softmaxes) and feeding it into Xgboost
  4. Other ways I don’t know about
    Thanks in advance!


The short answer is any of those and more, you test what works with a little effort then do more of it. There are many more ways than plain average: weighted average, voting, rank averaging, etc.