Large model deployment on google cloud function

Part 1 (2019)
2

Hello @hyaxia are your sure that you forced the installation to use CPU version ?
If you just did pip install fastai, fastai will pull the CUDA version of Pytorch (arround 700MB I think).

It takes less than 1sec for my Cloud Functions, which runs a 90MB model, to download model from bucket, load_learner, predict and return the result. I picked 1GB of RAM. Doesn’t work with 256MB.

edit: better version with requests

main.py

from fastai.learner import load_learner
from fastai.vision.core import PILImage
from google.cloud import storage
import requests as r


storage_client = storage.Client()
bucket = storage_client.get_bucket("model-chef-oeuvre")
blob = bucket.blob("model.pkl")
blob.download_to_filename("/tmp/model.pkl")
learner = load_learner("/tmp/model.pkl")


def run(request):

    response = r.get(
        request.args["image"],
        headers={
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"
        }
    )
    
    # PILImage.create(BytesIO(response.content)

    label, label_idx, _ = learner.predict(PILImage.create(response.content))

    return str(label)

requirements.txt

https://files.pythonhosted.org/packages/e0/c7/1c91a71b413c82cd4c49fb8b6676f6135650cd2cca2745a96bd84a56166c/ipython-7.19.0-py3-none-any.whl
torch==1.7.0+cpu
torchvision==0.8.1+cpu
-f https://download.pytorch.org/whl/torch_stable.html
fastai
google-cloud-storage
requests
1 Like