Running batch jobs on the cloud

Hello everyone!

I’m working on a template for ML projects based on cookiecutter-data-science with a focus on reproducible research.

Right now, I’ve got a script to run jobs on a Docker container and the generated artifacts and metrics are logged on a Mlfow server. These jobs could be training a model or pre-processing some raw data for example.

What I want to accomplish is a way to run those jobs on the cloud: packaging a Docker image, upload it to a cluster and run the job.
Do you know any tool/platform I can use to do that?

I’ve checked out Cloud ML Engine, but I’m looking for a more generic docker-based way.

Thank you so much!