Puting the Model Into Production: Web Apps

Part 1 (2019)
64

Hi all.
I extended a little the Zeit blueprint described here (I did not fork it as I could not find the original repo).

I tried do make it a little more general in order to avoid to hard code in the server.py file the informations on the model. My repo with the implementation is GitHub - francescogianferraripini/qzeit-vision: A Blueprint for Dockerized Computer Vision Fastai model serving. It works on the zeit paid plans
Therefore I created a model metadata json like this one (the file is modelDefinitionTemplate.json):

[
    {
        "name":"isDamaged",
        "classes":["00-damage", "01-whole"],
        "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data1a-frozen10epochs.pth",
        "modelFileName":"data1a-frozen10epochs",
        "modelType":"resnet50",
        "imageSize":299
    },    
    {
        "name":"damageLocation",
        "classes":["00-front", "01-rear", "02-side"],
        "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data2a-frozen10epochs-unfrozen10epochs.pth",
        "modelFileName":"data2a-frozen10epochs-unfrozen10epochs",
        "modelType":"resnet50",
        "imageSize":299
    },    
    {
        "name":"damageSeverity",
        "classes":["01-minor", "02-moderate", "03-severe"],
        "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data3a-frozen10epochs-unfrozen10epochs.pth",
        "modelFileName":"data3a-frozen10epochs-unfrozen10epochs",
        "modelType":"resnet50",
        "imageSize":299
    }
]

When server.py starts for the first time, it looks for a file called modelDefinition.json in the root, parses it and downloads the relevant .pth files, like the original one.
When an image is passed to the web service, it evaluates that against the many models defined in the file.

In this way we can manage and maintain a unique server.py that is independent on the models.

Moreover, I changed a little the dockerfile. It now expects a parameter, which is the url where to download modelDefinition.json, that is passed at deployment time. In this way, we can also maintain a single dockerfile and leverage the docker layers in a more simple way.
Therefore, in order to deploy it on zeit, you have now to type:

now --build-env MODEL_URL="https://raw.githubusercontent.com/francescogianferraripini/qcarcrash/master/qcarcrashModelDefinition.json"

MODEL_URL is where the modelDefinition.json can be downloaded.
I believe this approach can be useful to separate, training, developement of the serving code, and deployment/production operations and be the base for a model repository and deployment portal.

There are many todos left:

  • Extend the model definition file in order to include more metadata, either useful for the prediction (like the transformations or the normalization) or for documentation about the training part, accuracy etc, and expose it also as a web service
  • Generate automatically and publish on a repo the modelDefinition and the pth files from learners in the training notebook
  • Create similar server.py for other scenarios like tabular.

Let me know your thoughts!

7 Likes