PR to fix: https://github.com/fastai/course20/pull/28
2 Likes
I selected the Frankfurt, Germany link since I live in Northern Europe, and got this response when trying to create the stack.
CREATE_FAILED The requested resource notebook-instance/ml.p2.xlarge is not available in this region (Service: AmazonSageMaker; Status Code: 400; Error Code: ResourceLimitExceeded; Request ID: 12ba24b0-cf28-4e37-aec0-e04a199ef168)
Hmm, that instance type is definitely supposed to be available in the Frankfurt region. Check out the pricing page under the “On-Demand Notebook Instances” tab and Frankfurt region.
You might need to request a quota increase.
Yes, AWS is insane.
I tried Ireland instead, but ran into having to request a quota increase. It took hours to get a response, which only repeated my request back to me. By that time I was up and running with Paperspace, so I cancelled AWS. Not going back to AWS anytime soon.
Has anyone seen this error while trying to deploy the model?
sagemaker_containers._errors.ClientError: name ‘load_learner’ is not defined
The error is coming from:
File “/usr/local/lib/python3.6/dist-packages/inference.py”, line 15, in model_fn
learn = load_learner(model_dir, fname=‘export.pkl’)
The file inference.py contains the 4 methods: model_fn , input_fn , predict_fn & output_fn
and it imports:
import logging, requests, os, io, glob, time
from fastai.vision import *
I followed the instructions in this link https://github.com/fastai/course-v3/blob/master/docs/deployment_amzn_sagemaker.md
Could it be the instance where the model is deployed does not have the right version of fastai?
This is my deploy instruction (from the link above):
predictor = model.deploy(initial_instance_count=1, instance_type=‘ml.t2.medium’)
@sujatha - The same question seems to be addressed in the previous version of course category.
1 Like
If you are looking for a low-cost way to deploy your fastai models into production, AWS Lambda just announced support for Container images to package your code and dependencies such as PyTorch & fastai libraries and your exported fastai model. I have setup an example project using the SAM CLI here: https://github.com/mattmcclean/fastai-container-sam-app.
Would love to hear your feedback!
This is great. I will try it out. How can this be used when the training data keeps changing on a daily or regular basis like in recommendation engine or forecasting?
Hi, I’m Sal and am trying to start the course. I have been following the Sagemaker instructions but AWS has told me that ml.p3.2xlarge is not available in the US-West, which seems odd to me. Is there any way to use the Notebook with other available instances?
Thanks
I was eventually able to get this working by using the template found at: https://course.fast.ai/start_sagemaker
and then adding the nohup prefix to the pip install commands.
Thanks @AlisonDavey and @ganesh.bhat
You do need use Cloud Formation: create-stack and enter the template manually in designer or upload.
Perhaps someone could add this to the official button-click launches. It would save some confusion for people not experienced with Cloud Formation
For reference here is my working version:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
InstanceType:
Type: String
Default: ml.p2.xlarge
AllowedValues:
- ml.p3.2xlarge
- ml.p2.xlarge
Description: Enter the SageMaker Notebook instance type
VolumeSize:
Type: Number
Default: 50
Description: Enter the size of the EBS volume attached to the notebook instance
MaxValue: 17592
MinValue: 5
Resources:
Fastai2SagemakerNotebookfastaiv4NotebookRoleA75B4C74:
Type: AWS::IAM::Role
DeletionPolicy: Delete
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: sagemaker.amazonaws.com
Version: “2012-10-17”
ManagedPolicyArns:
- Fn::Join:
- “”
- - “arn:”
- Ref: AWS::Partition
- :iam::aws:policy/AmazonSageMakerFullAccess
Metadata:
aws:cdk:path: CdkFastaiv2SagemakerNbStack/Fastai2SagemakerNotebook/fastai-v4NotebookRole/Resource
Fastai2SagemakerNotebookfastaiv4LifecycleConfigD72E2247:
Type: AWS::SageMaker::NotebookInstanceLifecycleConfig
DeletionPolicy: Delete
Properties:
NotebookInstanceLifecycleConfigName: fastai-v4LifecycleConfig
OnCreate:
- Content:
Fn::Base64: >-
#!/bin/bash
set -e
echo "Starting on Create script"
sudo -i -u ec2-user bash <<EOF
touch /home/ec2-user/SageMaker/.create-notebook
EOF
cat > /home/ec2-user/SageMaker/.fastai-install.sh <<\EOF
#!/bin/bash
set -e
echo "Creating dirs and symlinks"
mkdir -p /home/ec2-user/SageMaker/.cache
mkdir -p /home/ec2-user/SageMaker/.fastai
[ ! -L "/home/ec2-user/.cache" ] && ln -s /home/ec2-user/SageMaker/.cache /home/ec2-user/.cache
[ ! -L "/home/ec2-user/.fastai" ] && ln -s /home/ec2-user/SageMaker/.fastai /home/ec2-user/.fastai
echo "Updating conda"
conda update -n base -c defaults conda -y
conda update --all -y
echo "Starting conda create command for fastai env"
conda create -mqyp /home/ec2-user/SageMaker/.env/fastai python=3.6
echo "Activate fastai conda env"
conda init bash
source ~/.bashrc
conda activate /home/ec2-user/SageMaker/.env/fastai
echo "Install ipython kernel and widgets"
conda install ipywidgets ipykernel -y
echo "Installing fastai lib"
nohup pip install -r /home/ec2-user/SageMaker/fastbook/requirements.txt
nohup pip install fastbook sagemaker
echo "Installing Jupyter kernel for fastai"
python -m ipykernel install --name 'fastai' --user
echo "Finished installing fastai conda env"
echo "Install Jupyter nbextensions"
conda activate JupyterSystemEnv
nohup pip install jupyter_contrib_nbextensions
jupyter contrib nbextensions install --user
echo "Restarting jupyter notebook server"
pkill -f jupyter-notebook
rm /home/ec2-user/SageMaker/.create-notebook
echo "Exiting install script"
EOF
chown ec2-user:ec2-user /home/ec2-user/SageMaker/.fastai-install.sh
chmod 755 /home/ec2-user/SageMaker/.fastai-install.sh
sudo -i -u ec2-user bash <<EOF
nohup /home/ec2-user/SageMaker/.fastai-install.sh &
EOF
echo "Finishing on Create script"
OnStart:
- Content:
Fn::Base64: >-
#!/bin/bash
set -e
echo "Starting on Start script"
sudo -i -u ec2-user bash << EOF
if [[ -f /home/ec2-user/SageMaker/.create-notebook ]]; then
echo "Skipping as currently installing conda env"
else
# create symlinks to EBS volume
echo "Creating symlinks"
ln -s /home/ec2-user/SageMaker/.fastai /home/ec2-user/.fastai
echo "Updating conda"
conda update -n base -c defaults conda -y
echo "Activate fastai conda env"
conda init bash
source ~/.bashrc
conda activate /home/ec2-user/SageMaker/.env/fastai
echo "Updating fastai packages"
nohup pip install fastai fastcore sagemaker --upgrade
echo "Installing Jupyter kernel"
python -m ipykernel install --name 'fastai' --user
echo "Install Jupyter nbextensions"
conda activate JupyterSystemEnv
nohup pip install jupyter_contrib_nbextensions
jupyter contrib nbextensions install --user
echo "Restarting jupyter notebook server"
pkill -f jupyter-notebook
echo "Finished setting up Jupyter kernel"
fi
EOF
echo "Finishing on Start script"
Metadata:
aws:cdk:path: CdkFastaiv2SagemakerNbStack/Fastai2SagemakerNotebook/fastai-v4LifecycleConfig
Fastai2SagemakerNotebookfastaiv4NotebookInstance7C46E7E0:
Type: AWS::SageMaker::NotebookInstance
DeletionPolicy: Retain
Properties:
InstanceType:
Ref: InstanceType
RoleArn:
Fn::GetAtt:
- Fastai2SagemakerNotebookfastaiv4NotebookRoleA75B4C74
- Arn
DefaultCodeRepository: https://github.com/fastai/fastbook
LifecycleConfigName: fastai-v4LifecycleConfig
NotebookInstanceName: fastai-v4
VolumeSizeInGB:
Ref: VolumeSize
Metadata:
aws:cdk:path: CdkFastaiv2SagemakerNbStack/Fastai2SagemakerNotebook/fastai-v4NotebookInstance
CDKMetadata:
Type: AWS::CDK::Metadata
Properties:
Modules: aws-cdk=1.60.0,@aws-cdk/aws-iam=1.60.0,@aws-cdk/aws-sagemaker=1.60.0,@aws-cdk/cloud-assembly-schema=1.60.0,@aws-cdk/core=1.60.0,@aws-cdk/cx-api=1.60.0,@aws-cdk/region-info=1.60.0,jsii-runtime=node.js/v14.8.0
Condition: CDKMetadataAvailable
Conditions:
CDKMetadataAvailable:
Fn::Or:
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- ap-east-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-2
- Fn::Equals:
- Ref: AWS::Region
- ap-south-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-2
- Fn::Equals:
- Ref: AWS::Region
- ca-central-1
- Fn::Equals:
- Ref: AWS::Region
- cn-north-1
- Fn::Equals:
- Ref: AWS::Region
- cn-northwest-1
- Fn::Equals:
- Ref: AWS::Region
- eu-central-1
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- eu-north-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-2
- Fn::Equals:
- Ref: AWS::Region
- eu-west-3
- Fn::Equals:
- Ref: AWS::Region
- me-south-1
- Fn::Equals:
- Ref: AWS::Region
- sa-east-1
- Fn::Equals:
- Ref: AWS::Region
- us-east-1
- Fn::Equals:
- Ref: AWS::Region
- us-east-2
- Fn::Equals:
- Ref: AWS::Region
- us-west-1
- Fn::Equals:
- Ref: AWS::Region
- us-west-2
1 Like
Here’s a guide on how to deploy a fastai v2 model to a sagemaker endpoint using torchserve (pytorch >= 1.6). Largely based on great prior work by @matt.mcclean
Feel free to use this as a template for deploying your own models. I suffered through a lot of issues getting this working so hopefully I can save you some of the pain.
3 Likes
Thank you for this! I was struggling for a few weeks to make it work with Sagemaker v2 and finally found your template.
I was able to deploy the model to an endpoint and successfully test it. The only thing I’m still fighting is making this work with batch transform. I am trying to run a batch transform on images saved on S3, but I guess I need to modify the serve.py script to make it work, as it expects a URL as a JSON and not the image itself (which is what I think batch transform is doing).
With the original serve.py I would get this error:
2021-03-23T18:01:49.949:[sagemaker logs]: pet-classifier-test/pets-input/himalayan-cats-and-kittens-1.jpg: Requested unsupported ContentType in content_type: None
I tried modifying the input_fn function in serve.py like this:
def input_fn(request_body, content_type=JPEG_CONTENT_TYPE): logger.info('Deserializing the input data.') # process an image uploaded to the endpoint if content_type == JPEG_CONTENT_TYPE: return open_image(io.BytesIO(request_body)) # process a URL submitted to the endpoint if content_type == JSON_CONTENT_TYPE: logger.debug(f'Request body is: {request_body}') request = json.loads(request_body) logger.debug(f'Loaded JSON object: {request}') url = request['url'] logger.info(f'Downloading image from URL: {url}') img_content = requests.get(url).content logger.info(f'Returning image bytes') return io.BytesIO(img_content).read() raise Exception('Requested unsupported ContentType in content_type: {}'.format(content_type))
But I get the same error. Any hints would be appreciated. I will let you know if I’m successful at solving this.
im getting the following error when trying to spin up the predictor endpoint
raise VersionConflict(dist, req).with_context(dependent_req)
cw28jf5snh-algo-1-aihs9 | pkg_resources.VersionConflict: (fastprogress 0.1.21 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('fastprogress>=0.2.1'))
Here’s the full output
[2021-04-12 15:42:18 +0000] [31] [ERROR] Error handling request /ping
ro7acc7tbr-algo-1-bv3eb | Traceback (most recent call last):
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/sagemaker_containers/_modules.py", line 246, in import_module
ro7acc7tbr-algo-1-bv3eb | module = importlib.import_module(name)
ro7acc7tbr-algo-1-bv3eb | File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
ro7acc7tbr-algo-1-bv3eb | return _bootstrap._gcd_import(name[level:], package, level)
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 994, in _gcd_import
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 971, in _find_and_load
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap_external>", line 678, in exec_module
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/app.py", line 14, in <module>
ro7acc7tbr-algo-1-bv3eb | from fastai.text import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/text/__init__.py", line 1, in <module>
ro7acc7tbr-algo-1-bv3eb | from .. import basics
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/basics.py", line 1, in <module>
ro7acc7tbr-algo-1-bv3eb | from .basic_train import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py", line 2, in <module>
ro7acc7tbr-algo-1-bv3eb | from .torch_core import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/torch_core.py", line 2, in <module>
ro7acc7tbr-algo-1-bv3eb | from .imports.torch import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/imports/__init__.py", line 1, in <module>
ro7acc7tbr-algo-1-bv3eb | from .core import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/imports/core.py", line 30, in <module>
ro7acc7tbr-algo-1-bv3eb | pkg_resources.require("fastprogress>=0.2.1")
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 900, in require
ro7acc7tbr-algo-1-bv3eb | needed = self.resolve(parse_requirements(requirements))
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 791, in resolve
ro7acc7tbr-algo-1-bv3eb | raise VersionConflict(dist, req).with_context(dependent_req)
ro7acc7tbr-algo-1-bv3eb | pkg_resources.VersionConflict: (fastprogress 0.1.21 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('fastprogress>=0.2.1'))
ro7acc7tbr-algo-1-bv3eb |
ro7acc7tbr-algo-1-bv3eb | During handling of the above exception, another exception occurred:
ro7acc7tbr-algo-1-bv3eb |
ro7acc7tbr-algo-1-bv3eb | Traceback (most recent call last):
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/base_async.py", line 56, in handle
ro7acc7tbr-algo-1-bv3eb | self.handle_request(listener_name, req, client, addr)
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/ggevent.py", line 160, in handle_request
ro7acc7tbr-algo-1-bv3eb | addr)
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/gunicorn/workers/base_async.py", line 107, in handle_request
ro7acc7tbr-algo-1-bv3eb | respiter = self.wsgi(environ, resp.start_response)
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/sagemaker_pytorch_container/serving.py", line 103, in main
ro7acc7tbr-algo-1-bv3eb | user_module = modules.import_module(serving_env.module_dir, serving_env.module_name)
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/sagemaker_containers/_modules.py", line 251, in import_module
ro7acc7tbr-algo-1-bv3eb | six.reraise(_errors.ImportModuleError, _errors.ImportModuleError(e), sys.exc_info()[2])
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/six.py", line 692, in reraise
ro7acc7tbr-algo-1-bv3eb | raise value.with_traceback(tb)
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/sagemaker_containers/_modules.py", line 246, in import_module
ro7acc7tbr-algo-1-bv3eb | module = importlib.import_module(name)
ro7acc7tbr-algo-1-bv3eb | File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
ro7acc7tbr-algo-1-bv3eb | return _bootstrap._gcd_import(name[level:], package, level)
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 994, in _gcd_import
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 971, in _find_and_load
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap_external>", line 678, in exec_module
ro7acc7tbr-algo-1-bv3eb | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/app.py", line 14, in <module>
ro7acc7tbr-algo-1-bv3eb | from fastai.text import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/text/__init__.py", line 1, in <module>
ro7acc7tbr-algo-1-bv3eb | from .. import basics
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/basics.py", line 1, in <module>
ro7acc7tbr-algo-1-bv3eb | from .basic_train import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/basic_train.py", line 2, in <module>
ro7acc7tbr-algo-1-bv3eb | from .torch_core import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/torch_core.py", line 2, in <module>
ro7acc7tbr-algo-1-bv3eb | from .imports.torch import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/imports/__init__.py", line 1, in <module>
ro7acc7tbr-algo-1-bv3eb | from .core import *
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/fastai/imports/core.py", line 30, in <module>
ro7acc7tbr-algo-1-bv3eb | pkg_resources.require("fastprogress>=0.2.1")
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 900, in require
ro7acc7tbr-algo-1-bv3eb | needed = self.resolve(parse_requirements(requirements))
ro7acc7tbr-algo-1-bv3eb | File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 791, in resolve
ro7acc7tbr-algo-1-bv3eb | raise VersionConflict(dist, req).with_context(dependent_req)
ro7acc7tbr-algo-1-bv3eb | sagemaker_containers._errors.ImportModuleError: (fastprogress 0.1.21 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('fastprogress>=0.2.1'))
my requirements file contains
fastai==1.0.61
fastprogress==0.2.1
So i should be using fastprogress 0.2.1
Running the Pytorch model function works fine
from sagemaker.pytorch.estimator import PyTorch
estimator = PyTorch(entry_point='app.py',
source_dir='source',
base_job_name='***',
role=****',
framework_version='1.0.0',
py_version='py3',
instance_count=1,
instance_type="local")
estimator.fit(data_location)
However, this error occurs when i running the following
predictor = estimator.deploy(initial_instance_count=1, instance_type="local")
I’ve look at this stackoverflow post with no success
Hi guys,
I’m was not able to deploy to Sagemaker today or yesterday. Around week ago it worked.
The problem is that fastai -kernel never shows up to notebook pages (or within 30 minutes) .
I guess this command is not executed successfully in the scrips:
“conda update -n base -c defaults conda -y”
In “LifecycleConfigOnCreate” logs this shows up and nothing comes after it:
2021-05-03T09:05:06.346+03:00 Updating conda
2021-05-03T09:05:25.352+03:00 Collecting package metadata (current_repodata.json): …working… done
2021-05-03T09:06:42.368+03:00 Solving environment: …working… done
2021-05-03T09:07:05.373+03:00 ## Package Plan ## environment location: /home/ec2-user/anaconda3 added / updated specs: - conda
2021-05-03T09:07:05.373+03:00 The following packages will be downloaded: package | build ---------------------------|----------------- anyio-2.2.0 | py37h06a4308_1 123 KB ca-certificates-2021.4.13 | h06a4308_1 114 KB certifi-2020.12.5 | py37h06a4308_0 141 KB cryptography-3.4.7 | py37hd23ed53_0 904 KB docutils-0.17 | py37h06a4308_1 686 KB jupyter-packaging-0.7.12 | pyhd3eb1b0_0 14 KB jupyter_server-1.4.1 | py37h06a4308_0 311 KB jupyterlab-3.0.14 | pyhd3eb1b0_1 3.5 MB jupyterlab_server-2.4.0 | pyhd3eb1b0_0 37 KB lxml-4.6.3 | py37h9120a33_0 1.2 MB nbclassic-0.2.6 | pyhd3eb1b0_0 19 KB openssl-1.1.1k | h27cfd23_0 2.5 MB pip-21.0.1 | py37h06a4308_0 1.8 MB scipy-1.6.2 | py37h91f5cce_0 15.4 MB sniffio-1.2.0 | py37h06a4308_1 15 KB urllib3-1.26.4 | pyhd3eb1b0_0 105 KB ------------------------------------------------------------ Total: 26.8 MB
2021-05-03T09:07:05.374+03:00 The following NEW packages will be INSTALLED: anyio pkgs/main/linux-64::anyio-2.2.0-py37h06a4308_1 colorama pkgs/main/noarch::colorama-0.4.4-pyhd3eb1b0_0 cryptography pkgs/main/linux-64::cryptography-3.4.7-py37hd23ed53_0 docutils pkgs/main/linux-64::docutils-0.17-py37h06a4308_1 jupyter-packaging pkgs/main/noarch::jupyter-packaging-0.7.12-pyhd3eb1b0_0 jupyter_server pkgs/main/linux-64::jupyter_server-1.4.1-py37h06a4308_0 jupyterlab pkgs/main/noarch::jupyterlab-3.0.14-pyhd3eb1b0_1 jupyterlab_server pkgs/main/noarch::jupyterlab_server-2.4.0-pyhd3eb1b0_0 lxml pkgs/main/linux-64::lxml-4.6.3-py37h9120a33_0 nbclassic pkgs/main/noarch::nbclassic-0.2.6-pyhd3eb1b0_0 pip pkgs/main/linux-64::pip-21.0.1-py37h06a4308_0 requests pkgs/main/noarch::requests-2.25.1-pyhd3eb1b0_0 sniffio pkgs/main/linux-64::sniffio-1.2.0-py37h06a4308_1 urllib3 pkgs/main/noarch::urllib3-1.26.4-pyhd3eb1b0_0
2021-05-03T09:07:05.374+03:00 The following packages will be UPDATED: ca-certificates conda-forge::ca-certificates-2020.12.~ --> pkgs/main::ca-certificates-2021.4.13-h06a4308_1 openssl conda-forge::openssl-1.1.1j-h7f98852_0 --> pkgs/main::openssl-1.1.1k-h27cfd23_0 scipy 1.6.1-py37h91f5cce_0 --> 1.6.2-py37h91f5cce_0
2021-05-03T09:07:05.374+03:00 The following packages will be SUPERSEDED by a higher-priority channel: certifi conda-forge::certifi-2020.12.5-py37h8~ --> pkgs/main::certifi-2020.12.5-py37h06a4308_0
2021-05-03T09:07:05.374+03:00 Downloading and Extracting Packages
2021-05-03T09:07:06.374+03:00 openssl-1.1.1k | 2.5 MB | | 0% openssl-1.1.1k | 2.5 MB | ########## | 100% openssl-1.1.1k | 2.5 MB | ########## | 100% lxml-4.6.3 | 1.2 MB | | 0% lxml-4.6.3 | 1.2 MB | ########## | 100% lxml-4.6.3 | 1.2 MB | ########## | 100% sniffio-1.2.0 | 15 KB | | 0% sniffio-1.2.0 | 15 KB | ########## | 100% sniffio-1.2.0 | 15 KB | ########## | 100% cryptography-3.4.7 | 904 KB | | 0% cryptography-3.4.7 | 904 KB | ########## | 100% cryptography-3.4.7 | 904 KB | ########## | 100% certifi-2020.12.5 | 141 KB | | 0% certifi-2020.12.5 | 141 KB | ########## | 100% certifi-2020.12.5 | 141 KB | ########## | 100%
2021-05-03T09:07:07.375+03:00 jupyterlab-3.0.14 | 3.5 MB | | 0% jupyterlab-3.0.14 | 3.5 MB | ########## | 100% jupyterlab-3.0.14 | 3.5 MB | ########## | 100% urllib3-1.26.4 | 105 KB | | 0% urllib3-1.26.4 | 105 KB | ########## | 100% urllib3-1.26.4 | 105 KB | ########## | 100% jupyterlab_server-2. | 37 KB | | 0% jupyterlab_server-2. | 37 KB | ########## | 100% jupyterlab_server-2. | 37 KB | ########## | 100% pip-21.0.1 | 1.8 MB | | 0% pip-21.0.1 | 1.8 MB | ########## | 100% pip-21.0.1 | 1.8 MB | ########## | 100% nbclassic-0.2.6 | 19 KB | | 0% nbclassic-0.2.6 | 19 KB | ########## | 100% nbclassic-0.2.6 | 19 KB | ########## | 100% jupyter_server-1.4.1 | 311 KB | | 0% jupyter_server-1.4.1 | 311 KB | ########## | 100% jupyter_server-1.4.1 | 311 KB | ########## | 100%
2021-05-03T09:07:08.375+03:00 scipy-1.6.2 | 15.4 MB | | 0% scipy-1.6.2 | 15.4 MB | ####### | 71% scipy-1.6.2 | 15.4 MB | ########## | 100% scipy-1.6.2 | 15.4 MB | ########## | 100% anyio-2.2.0 | 123 KB | | 0% anyio-2.2.0 | 123 KB | ########## | 100% anyio-2.2.0 | 123 KB | ########## | 100% jupyter-packaging-0. | 14 KB | | 0% jupyter-packaging-0. | 14 KB | ########## | 100% jupyter-packaging-0. | 14 KB | ########## | 100% ca-certificates-2021 | 114 KB | | 0% ca-certificates-2021 | 114 KB | ########## | 100% ca-certificates-2021 | 114 KB | ########## | 100%
2021-05-03T09:07:09.376+03:00 docutils-0.17 | 686 KB | | 0% docutils-0.17 | 686 KB | ########## | 100% docutils-0.17 | 686 KB | ########## | 100%
2021-05-03T09:07:09.376+03:00 Preparing transaction: …working… done
2021-05-03T09:07:09.376+03:00 Verifying transaction: …working… failed
2021-05-03T09:07:09.376+03:00 The environment is inconsistent, please check the package plan carefully
2021-05-03T09:07:09.376+03:00 The following packages are causing the inconsistency: - defaults/noarch::sphinx==3.5.1=pyhd3eb1b0_0 - defaults/linux-64::scikit-image==0.17.2=py37hdf5156a_0 - defaults/noarch::numpydoc==1.1.0=pyhd3eb1b0_1 - defaults/noarch::seaborn==0.11.1=pyhd3eb1b0_0 - defaults/linux-64::secretstorage==3.3.1=py37h06a4308_0 - defaults/linux-64::conda==4.8.4=py37_0 - defaults/linux-64::scikit-learn==0.23.2=py37h0573a6f_0 - defaults/linux-64::spyder==4.2.1=py37h06a4308_1 - defaults/linux-64::_anaconda_depends==2020.07=py37_0 - defaults/linux-64::anaconda-client==1.7.2=py37_0 - defaults/noarch::anaconda-project==0.9.1=pyhd3eb1b0_1 - defaults/linux-64::scipy==1.6.1=py37h91f5cce_0 - defaults/linux-64::statsmodels==0.12.2=py37h27cfd23_0 - defaults/linux-64::keyring==22.0.1=py37h06a4308_0 - defaults/linux-64::patsy==0.5.1=py37_0 - defaults/linux-64::pyopenssl==19.1.0=py37_0
2021-05-03T09:07:09.376+03:00 ==> WARNING: A newer version of conda exists. <== current version: 4.8.4 latest version: 4.10.1
2021-05-03T09:07:09.376+03:00 Please update conda by running $ conda update -n base -c defaults conda
2021-05-03T09:07:09.376+03:00 RemoveError: ‘requests’ is a dependency of conda and cannot be removed from
2021-05-03T09:07:09.376+03:00 conda’s operating environment.
I’m using this deploy command (from fastai’s website):
https://eu-west-1.console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?filter=active&templateURL=https://fastai-cfn.s3.amazonaws.com/sagemaker-cfn-course-v4.yml&stackName=FastaiSageMakerStack
Does anybody know how to fix this, or am I missing something?
I guess packages just got into a state where they won’t work together.
I tried tuning scripts in the Jupyter console, and as result I managed to get the fastai kernel activated.
I’m not sure exactly what were the needed actions, but adding these commands (before the
“conda update -n base -c defaults conda -y” -command )
might have had an effect:
conda install conda=4.10.1 -y
conda update --all -y
Hi,
I hope this is relevant to post here. I’ve created a guide for getting the fastai-v4 course setup on an AWS SageMaker instance in the EU London region. I came up against quite a few issues in terms of the ml.p2 instance types not being available in my region and the fastai kernel being missing. So I thought I’d collate all my findings and solutions in one place. I’m also very new to all of this therefore my broken down guide might be helpful to real newbies like myself 
Thanks,
Chelsea
Hello, Arizona_Sky!
Thanks for the note! I faced a similar issue later on and wound up modifying the stack template as documented here: FastaiSageMakerStack template (https://fastai-cfn.s3.amazonaws.com/sagemaker-cfn-course-v4.yml) fails updating conda due to conda env update dependencies · Issue #60 · fastai/course20 · GitHub
Agree with you that the cloud formation template should be modified on the course site to prevent repeated fails on new entrants to fastai on AWS SageMaker.
Best
PO
Me too
Hello! I am new to this course and AWS Sagemaker. I’ve been trying to launch the AWS CloudFormation specifically for Tokyo Region. I already increased my limit to 1 as instructed here: Amazon SageMaker | Practical Deep Learning for Coders
However, I am getting the error:
May I ask if there’s a link on how to solve this? Thanks!
Same here, I’m getting the same error when trying to launch from US West 1. I’ve been using my own Sagemaker instance but have run into constant issues with conflicting packages and having to re-install my packages every time I want to study. That’s costing me 20-30 minutes each session so I figured I’d try the cloudformation path. No luck yet but I’ll post here if I make any progress.
The underlying S3 file it’s trying to access is at the following location and I wonder if there is a new version or if its no longer public: https://s3-eu-west-1.amazonaws.com/mmcclean-public-files/sagemaker-fastai-notebook/sagemaker-cfn-course-v4.yaml