Productionizing models thread

Hi everyone,

I played a little more with the solutions discussed in this thread and have a webapp up and running: yourcityfrom.space; see also the repo.

Some observations:

  • I didn’t spend a lot of time optimizing my DL docker image (ended up being close to 1G) - that said even if it were smaller, I found having to deploy an image that includes pytorch to be super cumbersome; had to rebuild it a bunch of times and wish it had been smaller and more nimble; even for “exploration” havine to wait 10 minutes for it to build was not fun. I’m going to try and deploy something C++ based and see how that goes.
  • The serverless solutions discussed here (like now) are really cool, but somehow unsuited to the deployment of large docker images. Even if there’s a workaround to deploy a 700MB to the free tier right now, looking at the pricing page doesn’t make me feel comfortable that this is a “safe” solution. I ended up going docker-compose with three containers on a DO droplet.
  • The deployment experience is pretty rough generally, and it feels like it shouldn’t be. Even though now looks super easy, if you want to deploy a real webapp there’s still a lot of stuff to take care of (CORS, local VS deploy envs, port configs, non-standard APIs, docker caches, . . . )

Will continue working on this and report back!

16 Likes