I was wondering if anyone has some feedback on the steps required to train ensembles with fastai. Say we have 5 models already trained on 5 folds of the data, what are the next steps to build the final ensembles using fast ai.
I assume we freeze all the 5 models till the last conv layers. Make a prediction using each, which gives us 5 outputs of the last conv layers, then use those to feed them into a new set of FC layers. Not sure how to go about doing that withe fastai.
The second part of the question is once you’ve extracted the weights and built the final model, how do you go about fine tuning it, do you set aside yet another fold for that or still run it through 4/5 of the original folds and use 1/5 as a val set, then run through the whole set once as in more general fine tuning scenario?