Large model deployment on google cloud function

Part 1 (2019)
1

Hi everyone.
I have a model that classifies different cars.
Its size is 80 MB after I export it (I’m using resnet34).
In addition to the the size of the model, all of the libraries of fastai have a size of about 700MB.
All of that combined is over 500MB which is the limit in google cloud for deployment of cloud functions.

My question is, is there any way to reduce the size of the model and libraries significantly enough to go under 500MB?

Thanks!

Edit: for different reasons, the model has to be deployed on a cloud function. On each activation I just load the model and make the prediction.

1 Like
2

Hello @hyaxia are your sure that you forced the installation to use CPU version ?
If you just did pip install fastai, fastai will pull the CUDA version of Pytorch (arround 700MB I think).

It takes less than 1sec for my Cloud Functions, which runs a 90MB model, to download model from bucket, load_learner, predict and return the result. I picked 1GB of RAM. Doesn’t work with 256MB.

edit: better version with requests

main.py

from fastai.learner import load_learner
from fastai.vision.core import PILImage
from google.cloud import storage
import requests as r


storage_client = storage.Client()
bucket = storage_client.get_bucket("model-chef-oeuvre")
blob = bucket.blob("model.pkl")
blob.download_to_filename("/tmp/model.pkl")
learner = load_learner("/tmp/model.pkl")


def run(request):

    response = r.get(
        request.args["image"],
        headers={
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"
        }
    )
    
    # PILImage.create(BytesIO(response.content)

    label, label_idx, _ = learner.predict(PILImage.create(response.content))

    return str(label)

requirements.txt

https://files.pythonhosted.org/packages/e0/c7/1c91a71b413c82cd4c49fb8b6676f6135650cd2cca2745a96bd84a56166c/ipython-7.19.0-py3-none-any.whl
torch==1.7.0+cpu
torchvision==0.8.1+cpu
-f https://download.pytorch.org/whl/torch_stable.html
fastai
google-cloud-storage
requests
1 Like
3

Hey @aquadzn, thanks for the reply, I did try to change my requirements.txt file to match yours, but it fails with the message {"ResourceType":"gcp-types/cloudfunctions-v1:projects.locations.functions","ResourceErrorCode":"400","ResourceErrorMessage":"Build failed: pip_install_from_wheelshad stderr output:\n/opt/python3.7/bin/python3.7: No module named pip\n\nerror:pip_install_from_wheels returned code: 1; Error ID: ECB5F712"}.

From what I’ve red, it is related to something internal in one of the libraries.
Is your deployment process unique by any means?

Also, when I try to install torch==1.7.0+cpu locally, it tell me it did not found that version.
I’m using mac, but still its suppose to find it.

Any idea what the problems might be?
Thanks!

4

try with Python 3.8 runtime and remove pip from your requirements.txt if it’s inside

and locally it is because there are specific versions of torch for macOS, search inside the link in requirements.txt

5

thanks a lot, changing to python3.8 worked (wondering why, but sadly I don’t have the time to delve deep into the reasons).

About the local version, looks like there is no cpu version for macos, I guess I’ll just create mac-requirements.txt file specifically for macos.

Thanks again :slight_smile: