Platform: SageMaker ✅


(Matt McClean) #104

I have uploaded another example here. It is a notebook that allows you to train your fast.ai model using SageMaker notebook instance then it shows how to upload the model to S3 and deploy as a SageMaker endpoint. You can deploy the endpoint locally to test as well.

There is no support for Elastic Inference with PyTorch models as far as I know. In any case you can launch an endpoint with a CPU based instance (instead of GPU). We recently announced support for ONNX models and the Elastic Inference. An example is shown here.


(GENNAN CHEN) #105

Thanks @matt.mcclean. Hope EI will support fastai soon. BTW, I cannot see the example in the link you provided

github keep complaining that Something went wrong and need reload. Tried a few times but no luck. Wondering if just my mac.

Gen


(Matt McClean) #106

Try it again as it worked for me


(GENNAN CHEN) #107

@matt.mcclean Thanks for providing this info. After reviewing it, it seems we jump through a lot of hoops to use SageMaker’s predefined interface and I am not so sure it can scale since it did not support EI. Do you think it is better off to package them (fastai/model/data) in docker and deployed with lambda/API gateway? I might be wrong here. Please feel free to correct me

Gen


(Matt McClean) #108

Not sure what you mean by it won’t scale as it doesn’t support EI. It will depend on the type of fast.ai model and your latency and cost constraints. You can run fast.ai models with non-GPU based instances (e.g m4, m5, c5 etc.) but you may have increased latency on your inference calls compared to a GPU or EI based instance.
For sure you can scale out the SageMaker endpoints horizontally using the Autoscaling feature.

I would recommend you testing out your model with different instance types to find the optimum one taking into consideration both performance/latency and costs


#109

I figured out the issue, other people in my company also use sagemaker and they’re trashing my envs. The “Python3” environment already existed and so the Lifecycle scripts did not configure that environment correctly.

Is there anyone who can tell me how to modify my start up scripts/etc to provide a clean environment and hopefully name it something unique so other people will not trample it? I did quite a bit of googling on conda/source/activate/etc but I honestly can’t make it all work on my own, I’m not super familiar with nix environments and this is the first I’m looking at python, which google is telling me has a pretty rough time with these exact problems. :frowning:

Any help would be appreciated


(GENNAN CHEN) #110

@sublimemm if you check the life cycle scripts provided by @matt.mcclean. I think you wants to modify line 57 for https://course-v3.fast.ai/setup/sagemaker-create to have different display name


(Matt McClean) #111

I have just published a new Production guide for deploying your fast.ai model on SageMaker here: https://course.fast.ai/deployment_amzn_sagemaker.html


(Matt McClean) #112

The instructions to setup a SageMaker notebook are now even easier and faster. We can now provision all the resources in a CloudFormation script avoiding manual steps.

The setup guide has been updated here: https://course.fast.ai/start_sagemaker.html


#113

I setup sagemaker, I was very careful to start and stop instance. I did not even use it much as I got working with something else. Yet I just got a 300$ bill. I am not even sure why. There is no detail of what exactly caused it. I thought I was being very careful: how could I have run in such costs without my even noticing?
Anyway, one piece of advice: set a budget limit alarm!
I think personally when you first start using sagemaker you should be alerted by default. Up to you to raise your budget alarm…


#114

go to your aws billing dashboard, they have every cent detailed