I am wondering whether it makes sense to use a different kind of ensembles. So far we used different runs of one model and one training/validation set to build an ensemble.
What about using different training/validation sets to build the ensemble? When we use a single set, we loose the information in the validation set for training. So, my thought is to mitigate this by using different sets every time and average over the different sets!?