How to build an ensemble in tensorflow?

Hi together,

how would you build an ensemble in tensorflow? Let’s say I’d want to combine 5 CNNs for MNIST.
My idea was to build a graph and train it five times and save the weights after every time so that I end up with 5 models.

Then build a new graph for inference where the input goes into five parallel models. I’d then restore my weights into the new graph, all in the same session.

Is that the right track or is there a better way? I couldn’t get it to run this way, not sure why. I assume it has something to do with variable naming.

I find that in practice, its better to just save prediction to file from running the CNNs one by one in a script, and then do ensembling at the end. Big complicated pipes are prone to bugs and cpu/gpu limits.

Hi Jerry,
but this doesn’t help in production. Especially not in time critical applications.
So for production you have to write complicated stuff. Otherwise you might as well only use keras.

1 Like