Google Cloud Run seems the best/free production service for your MVP projects

Part 1 (2019)
1

Hi,

TRDR: If you look for free production hosting for your training model, try Google Cloud Run.

I am a web developer and new to ML. I try to deploy the bear classifier to production. I’ve tried

  • AWS Elasticbeanstalk
  • Google App Engine
  • Google Cloud Run

Cloud Run seems the best option if you look for a provider for MVP because

  • Google’s $300 free credits
  • It’s serverless so you only get charged if it has traffic
  • The pricing seems cheap and has a free tier.

The only thing you will need to do is adjust the memory allocation from 256MB(default) to 512MB otherwise fastai will run out of memory.

Let me know what you think:)

2 Likes
2

Agree, the only disadvantage is the cold start of the Containers. After some time of inactivity the container needs a couple of seconds to respond again. Thats why i use digitalocean for small Projects. I‘ve created a docker Image to easily Host fastai2 Models (also works with Google Cloud Run). Hosting starts at 5$ per Month. Here is my Blog and the repo:

https://floleuerer.github.io/2020/04/26/deploy-digitalocean.html

Let me Know what you think or if you have any questions.
Florian

4 Likes
3

@florianl, yup, you are right. Serverless has the issue of cold start. DigitalOccean $5 per month is perfect for starters.

Thank you for sharing your repo. Wow, it has a very helpful README!

1 Like
4

Hi Florian,

that’s awesome, thank you! I’m trying to set up a web app on Digital Ocean. To make things easy, I wanted to test a finished app that’s based on the Render fastai template.

The setup worked flawlessly, but I can’t access it via the web. I suppose something is wrong with the port setup? I don’t use nginx, so the docker container should be accessed directly, right?
I’m a bit overwhelmed. I see the ports defined four times:

  1. In the Dockerfile (EXPOSE 2375) (5000 in the Render example)
  2. In the server.py file uvicorn.run(app=app, host='0.0.0.0', port=int(os.environ.get('PORT', 8080)), log_level="info")
  3. Building the container $ PORT=8080 && docker build -f Dockerfile -t <repo:tag> .
  4. Running the container $ docker run -d -p 8080:8080 --name=<name> <repo:tag>

The docs say that the firewall only allows access through ports 2375/2376. How can I use these?

This is what is running right now:

digitalocean-ports
digitalocean-ports1161×178 9.21 KB

Maybe you or anyone else can help, thank you so much.

Johannes

5

Well, that went better than I hoped, I figured it out. For reference, and maybe it will be helpful for others:

  1. Dockerfile: EXPOSE 2375
  2. server.py uvicorn.run(app=app, host='0.0.0.0', port=2375, log_level="info")
  3. Building: DON’T use PORT=8080, just use docker build -f Dockerfile -t <repo:tag> .
  4. Running the container: docker run -d -p 80:2375 --name <name> <repo:tag>

Running configuration:

digitalocean-ports-correct
digitalocean-ports-correct1083×177 9.28 KB

The app runs smooth and waaaay faster than on Render :slight_smile:

6

Hi Johannes,

usually docker should take care of the firewall rules - but I actually haven’t tried to deploy just the fastai-rest part on DigitalOcean. I guess with $PORT in the Dockerfile and PORT=8080 in the build command it should be working. (Image running on port 8080 and exposed on Port 8080).

EXPOSE $PORT

Using the PORT variable (in the Dockerfile, uvicorn.run and hence the build command) is required for Google Cloud Run to work.

But anyway glad that your app is working and that my Blog was useful. Awesome app :slight_smile:
Florian

1 Like