Hi, how de we found out about the layer URLS? I am looking everywhere and there is not listing from AWS ??
The only layers urls are the 2 listed here ?? arn:aws:lambda:AWS_REGION:934676248949:layer:pytorchv1-py36:1 PyTorch 1.0.1
Concerning the trace_input parameter, what value would we set when using text classification?
The text length varies depending on the text, so a fixed value would not be the right way, or am I missing something?
You are almost there.
I think this is becuase you are "POST"ing to port 80, the default, you need to send your request to port 3000. Pay attention to the :3000 at the end of your endpoint. E.g. you need:
Iāve just followed the instructions and deployed my code using that guide, thank you @matt.mcclean
Love the fact that there is no running cost as this is serverless, and that for most of us, it is also free as the per-run cost is going to be well below the free tier (except for the very small cost to host the model on S3).
I found that local development is VERY slow since every POST takes up to a minute to answer. Somehow, when deployed, the lambda also takes some time to answer the first time, but then answers basically with no delay. I think this is called ācold startā, and to avoid it I make a dummy call from the front-end as soon as the page loads, so the API is ready when the first real call is done.
TL;DR - once you have custom transformations and models, try finding the cause through debugging by testing interactions between different library versions! + unzip_requirements.py strategy rocks
Hi everybody! Thanks matt for the guidleines, it helped a lot.
However Iām stuck at this point, when running locally
sam local invoke PyTorchFunction -n env.json -e event.json
Which throw this error:
File "/home/michal/.local/lib/python3.7/site-packages/botocore/client.py", line 586, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidParameterValueException: An error occurred (InvalidParameterValueException) when calling the GetLayerVersion operation: Invalid Layer name: arn:aws:lambda:eu-west-1:934676248949:layer:pytorchv1-py36
I am using your yaml template, just changed the region: Default: "arn:aws:lambda:eu-west-1:934676248949:layer:pytorchv1-py36:1"
My S3 bucket is in this same region. I tried many things but I cannot find what is the problem. Has anyone an idea?
Update:
I have created my own pytorch layer. But still have the same error message.
Update2:
My awscli was configured on the wrong region
Iām super-excited about being able to deploy with lamdba but running into this problem when testing locally or after deploying to lambda:
/tmp/sls-py-req/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.7) or chardet (3.0.4) doesnāt match a supported version!
Interested to know if anyone has deployed a fastai text classifier (v1 pretrained AWD_LSTM + fine tuning) to lambda+api gateway using the torchscript method from the course docs.
Iāve trained a product categorization model which Iād really like to be able to deploy for an internal POC.
Iām currently hitting issues with the torch.jit.trace part. The course docs give the below example which I understand the shape (being used for images). When trying to modify this for a text classifier Iām not sure what to set these values to.
# example from the docs
trace_input = torch.ones(1,3,299,299).cuda()
jit_model = torch.jit.trace(learn.model.float(), trace_input)
The model just takes in a single string and outputs predictions across 817 classes.
Iāv tried changing this to the below and got the below error.
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)
Iāve tried other combinations of this but have largely gotten the same error.
Iāve searched these forums and google but cant find an that really helps.
Any help would be greatly appreciated (even it is simple, no this method doesnāt work for text models because of xyz)
My learner doesnāt have anything special, it mostly the example text classification in the docs.
I suspected it didnāt as I saw some discussions on the Pytorch forums but I didnāt really understand the reasons they were giving so wasnāt sure if it was an issue for me or not. I ended up starting with the Render starter repo then scraping the UI part and writing a small API with FastAPI.
For anyone else who primarily needs a backend web-service rather than a fancy UI, I can recommend this approach as FastAPI gives nice Swagger interface with little effort. Makes testing the API endpoints easy.
Interesting to hear youāre using FastAPI - thatās what we (@aychang) opted for when we created AdaptNLP, which is our fastai-inspired wrapper for Hugging Face and Flair.
Many thanks to @matt.mcclean for the guide, it helped a lot.
However iām stucked. When testing locally I have an import error for ālibtorch.soā.
Iām using a custom lambda layers as the links provided do not work for me.
In the ācreate_layer_zipfile.shā i saw that ālibtorch.soā is removed
rm ./torch/lib/libtorch.so
Not removing it creates a zip file that exceed 250Mb (AWS lambda limit), removing it gives the error mentioned above.
The only difference with mattās code, is that I have changed the download link in the requirements to torch 1.2.0.
Hi, been trying to get this to work for me but having a problem when lambda tries to open the model.
Here is the log file grabbed from AWS CloudFunction and it looks like the PyTorch canāt read it properly? I trained this model using a Pytorch 1.4 and Python 3.7.
Would it make sense to downgrade to PyTorch 1.1 and reconstruct the model from there?
[INFO] 2020-05-07T17:04:57.431Z Loading model from S3
ļæ
17:04:59
Model file is : res50_stage1_v4.pth
Model file is : res50_stage1_v4.pth
ļæ
17:04:59
Loading PyTorch model
Loading PyTorch model
ļ
17:05:00
module initialization error: version_number <= kMaxSupportedFileFormatVersion ASSERT FAILED at /pytorch/caffe2/serialize/inline_container.cc:131, please report a bug to PyTorch. Attempted to read a PyTorch file with version 2, but the maximum supported version for reading is 1. Your PyTorch installation may be too old. (init at /pytorch/caffe2/serialize/inline_container.cc:131) frame #0: std::func
ļ
17:05:00
END RequestId: 8cafea90-765c-4a19-8ce0-8525956ad0ce
ļ
17:05:00
REPORT RequestId: 8cafea90-765c-4a19-8ce0-8525956ad0ce Duration: 149.70 ms Billed Duration: 200 ms Memory Size: 3008 MB Max Memory Used: 446 MB
ļæ
17:05:00
module initialization error version_number <= kMaxSupportedFileFormatVersion ASSERT FAILED at /pytorch/caffe2/serialize/inline_container.cc:131, please report a bug to PyTorch. Attempted to read a PyTorch file with version 2, but the maximum supported version for reading is 1. Your PyTorch installation may be too old. (init at /pytorch/caffe2/serialize/inline_container.cc:131) frame #0: std::funct
module initialization error
version_number <= kMaxSupportedFileFormatVersion ASSERT FAILED at /pytorch/caffe2/serialize/inline_container.cc:131, please report a bug to PyTorch. Attempted to read a PyTorch file with version 2, but the maximum supported version for reading is 1. Your PyTorch installation may be too old. (init at /pytorch/caffe2/serialize/inline_container.cc:131)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7f6d223d9441 in /tmp/sls-py-req/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7f6d223d8d7a in /tmp/sls-py-req/torch/lib/libc10.so)
frame #2: caffe2::serialize::PyTorchStreamReader::init() + 0xed1 (0x7f6d23fc8431 in /tmp/sls-py-req/torch/lib/libcaffe2.so)
frame #3: caffe2::serialize::PyTorchStreamReader::PyTorchStreamReader(std::istream*) + 0x48 (0x7f6d23fc90f8 in /tmp/sls-py-req/torch/lib/libcaffe2.so)
frame #4: torch::jit::import_ir_module(std::function<std::shared_ptrtorch::jit::script::Module (std::vector<std::string, std::allocator<std
ļæ
17:05:00
/tmp/sls-py-req/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.8) or chardet (3.0.4) doesnāt match a supported version!
/tmp/sls-py-req/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.25.8) or chardet (3.0.4) doesnāt match a supported version!
ļæ
17:05:00
RequestsDependencyWarning)
RequestsDependencyWarning)
I then imported the old model in jupyter notebook with:
from fastai.vision import *
classes = [āAngerā, āDisgustā, āSurpriseā, āSadnessā, āHappinessā, āNeutralā, āContemptā, āFearā]
data = ImageDataBunch.single_from_classes(āā, classes, ds_tfms=None)
learner = cnn_learner(data, models.resnet34)
learner.load(āgokul-sentiment-stage-5nā)
Thanks so much!
Iāll give that a shot. Currently to overcome this problem, I just went with Google Cloud Functions. It seems slow (800ms per inference on a resnet50) so I wanna give AWS lambda another try.