Setup problems: AWS

Navid · January 2, 2017, 10:08pm

If anyone is having problems when using the aws-setup.sh script but not when typing the same commands directly into the terminal, it probably has to do with copy / paste and Windows extra characters (that are invisible).

For example, the aws-get-p2 would work but not the aws-ssh and aws-stop command.

I was able to cleanup the script with dos2unix program which can be installed from the Cygwin installer. (just dos2unix aws-alias.sh and it will be fixed)

rachel · January 3, 2017, 12:10am

Hi @jhanna, make sure you have the updated version of our setup script from Github. We’ve updated to unix line endings (running dos2unix as @mcclap suggested should also work)

rachel · January 3, 2017, 12:14am

@kinsvater Several of us have gotten this error before and it’s been fixed by shutting down the jupyter notebook and restarting it (I’m not sure what the error cause is).

rachel · January 3, 2017, 12:19am

EDIT: I created a t2.large for Europe and have updated the setup scripts to automatically get your region from your AWS config. The new t2 setup script is here

@ga63zuh @biermann We have not created a t2.large for Europe yet. I will update this post once I do. In the meantime, you can use the us-west-2 instance (although the latency will be higher).

Also, you need to use a t2.large and not a t2.micro We changed this after recording Lesson 1 (since it turned out the t2.micro ran out of memory too quickly for our uses).

rachel · January 3, 2017, 12:22am

@krish @maz You need to reboot your computer.

EDIT: And by reboot, I mean restart your instance (you could do aws-stop and aws-start if you are using are aliases).

maz · January 3, 2017, 2:09am

Thanks. @rachel . Yah, I rebooted the instance to make it work. @krish did a reboot of the instance solve the problem ?

kinsvater · January 3, 2017, 8:10am

Thank you. Now it works. Did not try the most obvious thing: restarting. Sorry for that.

anaik2 · January 3, 2017, 10:52am

Thanks @vinvinvin, glad it helped!

gator05156 · January 3, 2017, 9:21pm

I am having a problem with running my bash file. I followed the instructions, but when I run the bash file I get the following errors. Can you point me in the right direction to solve my problem?

$ bash setup_p2.sh
rtbassoc-3fd75a59

"n error occurred (InvalidRouteTableId.Malformed) when calling the CreateRoute operation: Invalid id: "rtb-e7206680

"n error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityGroupIngress operation: Invalid id: "sg-0ed71576
setup_p2.sh: line 39: syntax error: unexpected end of file

jeremy · January 4, 2017, 2:33am

Looks like you have windows line endings in your file. Run dos2unix, or grab the latest fixed script from github

contractorwolf · January 4, 2017, 3:19am

Hey, great videos @jeremy and @rachel but I am having an issue running “bash setup_p2.sh” command, I get “setup_p2b.sh: line 119: syntax error: unexpected end of file”. I am looking at the file and it only has 118 rows, and I googled the error and it says that it maybe an unclosed quote, but the syntax looks right in vs code. I am unsure of what to try next. I used cURL to download it and even tried copy and paste the code from github to create the file, same issue.

I am on a macbook pro. thx

stevejburr · January 4, 2017, 11:17am

I’ve currently got the same error using Windows/Cygwin, have tried both downloading and copying the code from github without any luck

EDIT:
Tried using an older script from the github history and now get “invalid endpoint” errors instead:
“Invalid endpoint: https://ec2.eu-west-1.amazonaws.com”.

Not sure what changed in the last commit to cause the end of line/syntax errors.

EDIT 2:
This issue seems to be fixed by the pull request here:

github.com

ericschwarzkopf/courses/blob/dc06ce745a30850e7937858fb26a67df2aff329d/setup/setup_p2.sh

#!/bin/bash
# set -x # Use for debug mode

# settings
export name="fast-ai"
export cidr="0.0.0.0/0"

# get the correct ami
export region=`aws configure get region`
if [ $region = "us-west-2" ]; then 
   export ami="ami-bc508adc" # Oregon
elif [ $region = "eu-west-1" ]; then 
   export ami="ami-b43d1ec7" # Ireland
elif [ $region = "us-east-1" ]; then
  export ami="ami-31ecfb26" # Virginia
else
  echo "Only eu-west-1 (Ireland) and us-east-1 (Virginia) are currently supported"
  exit 1
fi

This file has been truncated. show original

contractorwolf · January 4, 2017, 2:32pm

@stevejburr thanks for the quick response. So that linked shell script solves the truncation issue? I am at work and cant test yet. I will test it when I get home and report what I find. Thanks!

marcvanderpeet · January 4, 2017, 4:20pm

I downloaded Cygwin and Python version 2.7. Now I am about to set up a deep learning computer at aws (following this tutorial: https://www.youtube.com/watch?v=8rjRfW4JM2I)

If i run pip install awscli I get this (which is good)

$ pip install awscli
Requirement already satisfied: awscli in c:\users\marc\anaconda2 \lib\site-packages
Requirement already satisfied: s3transfer<0.2.0,>=0.1.9 in c:\users\marc\anaconda2\lib\site-packages (from awscli)
Requirement already satisfied: rsa<=3.5.0,>=3.1.2 in c:\users\marc\anaconda2\lib\site-packages (from awscli)
Requirement already satisfied: PyYAML<=3.12,>=3.10 in c:\users\marc\anaconda2\lib\site-packages (from awscli)
Requirement already satisfied: docutils>=0.10 in c:\users\marc\anaconda2\lib\site-packages (from awscli)
Requirement already satisfied: botocore==1.4.92 in c:\users\marc\anaconda2\lib\site-packages (from awscli)
Requirement already satisfied: colorama<=0.3.7,>=0.2.5 in c:\users\marc\anaconda2\lib\site-packages (from awscli)
Requirement already satisfied: futures<4.0.0,>=2.2.0; python_version == “2.6” or python_version == “2.7” in c:\users\marc\anaconda2\lib\site- packages (from s3transfer<0.2.0,>=0.1.9->awscli)
Requirement already satisfied: pyasn1>=0.1.3 in c:\users\marc\anaconda2\lib\site-packages (from rsa<=3.5.0,>=3.1.2->awscli)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in c:\users\marc\anaconda2\lib\site-packages (from botocore==1.4.92->awscli)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in c:\users\marc\anaconda2\lib\site-packages (from botocore==1.4.92->awscli)
Requirement already satisfied: six>=1.5 in c:\users\marc\anaconda2\lib\site-packages (from python-dateutil<3.0.0,>=2.1->botocore==1.4.92->awscli)

However when I want to configure everything and type aws I get:

$ aws
C:\Users\Marc\Anaconda2\python.exe: can’t open file ‘/cygdrive/c/Users/Marc/Anaconda2/Scripts/aws’: [Errno 2] No such file or directory

Any thoughts on what goes wrong here?

stevejburr · January 4, 2017, 5:10pm

I managed to get my instance up and running with the script I linked, hopefully it works for you too.

Bourhis.bzh · January 4, 2017, 6:28pm

Hi ! What I should change to run a t2.micro instance instead of a p2.xlarge one ?
Many thanks !

gator05156 · January 4, 2017, 7:54pm

I am using wget https://raw.githubusercontent.com/fastai/courses/master/setup/setup_p2.sh to get the raw file. when I run it I get " line 119: syntax error: unexpected end of file" I have tried wget using local encoding but still the same error

gator05156 · January 4, 2017, 8:54pm

When the recent change was made in the github setup_p2.sh file I believe it created a syntax error. I have been having problems with the file. When I went into history and loaded the prior version it worked fine.

gator05156 · January 4, 2017, 9:33pm

this was a huge help

jeremy · January 5, 2017, 4:42am

For those that are having with the new version of the script, can you tell us what OS and python version you’re using?