Poll: How do you deploy your models in production?

Part 1 (2019)
1

How do you deploy your models in production?

0 voters

If you have a minute, add a comment below for why you picked your platform of choice. Thanks!

2

Google kubernetes engine

1 Like
3

Iā€™d like to deploy my model as part of an executable for Linux and Windows. I know this is old-fashioned, but it needs to work on hospitalsā€¦

Iā€™ve run into a small problem, though. It seems like fastai has a hard dependency on tk, so Iā€™m having trouble deploying my models in a Python environment without tk. I get ā€œNo module named tkinterā€ when trying to import fastai.vision. I even get this error if I do ā€œimport fastai.basic_trainā€ and then do load_learner(path).

Is it possible to get around this? I tried just loading the model using torch: torch.load(src, strict=False) and then running it as a pure torch model. But this way, my input data doesnā€™t get transformed, and thus my model doesnā€™t perform as it should. Iā€™ve also tried to replicate what is going on in basic_train.load_learner but Iā€™ve had no luck so far.

Can you please point me in the right direction?

Kind regards,
Andreas

4

Hosting on my own dedicated server.

5

Deploy fastai model with GKE on GCP
GKE = google kubernetes engine
GCP = google compute platform

flask (starlette application)
web server code -> https://github.com/render-examples/fastai-v3
save model to google drive -> https://course.fast.ai/deployment_render.html
move server.py (app.py) to the same dir as dockerfile

dockerfile -> (1.8 GB)
I used the docker file from https://github.com/render-examples/fastai-v3
but changed the app/server.py to server.py because I moved the file into the dockerfile directory

build docker image with google cloud build
google deployment tools -> https://www.jhanley.com/google-cloud-run-getting-started-with-python-and-flask/
John Hanleyā€™s page works except I had to use GKE instead of cloud run to deploy model (bc of memory?)
cloud build can build the docker image much faster than docker desktop great for big images (fastai)
cloud build will build and publish the image to google container registry
I didnā€™t use docker desktop to build the docker image

deploy the image to GKE
cloud run didnā€™t work (memory?) so I used GKE
you can use the google console or command line to deploy the image to GKE
ignore some warning messages about pytorch version incompatibility
Exampleā€¦ SourceChangeWarning: source code of class ā€˜fastai.text.models.awd_lstm.LinearDecoderā€™ has changed.
expose application to internet --port 80 --target-port 8080
have to use both port and target-port for public web page
view web page ā€¦ GKE -> services & Ingress (menu on the left)-> overview (menu on top) -> External endpoints

6

I see that 52% of the people that voted used Google App Engine, why is that?? even though Jeremy uses render (13% of people that voted) if iā€™m not mistakenā€¦

7

I deploy everything on SeeMe.ai but I might be biased since I created itā€¦ :slight_smile:

Itā€™s easy, you can share models and we have native iOS (with Core ML support) and Android appsā€¦

8

Heroku/voila because it was the first one I came across which was free and easy and worked properly.

9

Heroku