Setup problems: AWS

Girija · March 21, 2017, 6:24am

Hi Zarak,

Tried… git the following error.

$ export region=$(aws configure get region) if[$region=“us-west-2”];then export ami=“ami-bc508adc”#Oregon elif[$region="eu-west-1];then export ami=“ami-b43d1ec7”#Ireland elif[$region=“us-east-1”];then export ami=“ami-31ecfb26”#Virginia else echo “Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported” exit 1 fi export instanceType=“p2.xlarge”
-bash: syntax error near unexpected token `then’

kolobok · March 22, 2017, 1:50am

Thanks a lot Zarak, it worked.

a_Liz3 · March 22, 2017, 12:15pm

Hi,

This is A.Liz from Shanghai. I had setup AWS account before and used a g2 instance.

So following the AWS setup video (using mac), I met a problem when create a server with setup_p2.sh

when I get setup_p2.sh from git,
wget https://github.com/fastai/courses/blob/master/setup/setup_p2.sh
the file doesn’t seem right.

When I run
$ bash setup_p2.sh
it gets following error:
setup_p2.sh: line 7: syntax error near unexpected token newline' setup_p2.sh: line 7:’

Then I copy the raw from git which is 21 lines, and save to setup_p2.sh
and run $ bash setup_p2.sh
it says
setup_p2.sh: line 20: ./setup_instance.sh: No such file or directory

I think it’s the last line:
. $(dirname “$0”)/setup_instance.sh

Is that because the p2 is not yet approved by AWS?
I try t2 as well, and got the same error.
setup_t2.sh: line 20: ./setup_instance.sh: No such file or directory

Please kindly advice. Thanks!

A_TF57 · March 22, 2017, 2:50pm

Hello,

I was able to follow the AWS Install steps and setup a p2 instance, and ssh into it. But I was doing this on one system. And after using the instance, I stopped it.

Can I redo this process and access the same instance from a different PC?

On a different PC, I modified the aws-alias.sh to point to the correct $instanceId (which is the ID of the already created instance in the step above), but I can’t ssh into it. I get the following error:

Warning: Identity file /home/username/.ssh/aws-key-fast-ai.pem not accessible: No such file or directory.
Permission denied (publickey).

How can I access the same instance from a different system?

z0k · March 22, 2017, 5:24pm

Hi @a_Liz3,

You also need to download the setup_instance.sh script as it is called by setup_p2.sh.

z0k · March 22, 2017, 5:35pm

Hi @A_TF57,

You can create a key-pair on your new system. Then, when you ssh from this system, use the new .pem file instead.

A_TF57 · March 22, 2017, 5:53pm

Hi @z0k ! Thanks for the reply.

I created a new key-pair as you mentioned (used the cli tool) and saved a .pem file in a directory. When I ssh now, using this .pem file, I still get this:

Permission denied (publickey).

a_Liz3 · March 22, 2017, 6:21pm

Thanks! it works now!

z0k · March 22, 2017, 7:32pm

Did you set the permissions on your new key?

chmod 400 <name_of_key>.pem

A_TF57 · March 22, 2017, 7:39pm

I did that and it was again throwing an error.

I actually ended up saving my previously running instance as an AMI, and then launching an instance from this AMI with the new key-pair I had created.

Had to mess around a bit with the Elastic IP and the VPC/subnet, but it’s working finally!

Thanks!

Dan195 · March 22, 2017, 9:46pm

I restarted from scratch today, and the previous issues are fixed, but now I am getting this.

An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max r                                  etries: 4): We currently do not have sufficient p2.xlarge capacity in the Availability Zone you request                                  ed (us-west-2b). Our system will be working on provisioning additional capacity. You can currently get                                   p2.xlarge capacity by not specifying an Availability Zone in your request or choosing us-west-2c.

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain th                                  e parameter resourceIdSet
Waiting for instance start...

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --instance-id: expected one argument

An error occurred (MissingParameter) when calling the RebootInstances operation: The request must contain the parameter                  InstancesSet

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /home/Peter/.ssh/aws-key-fast-ai.pem ubuntu@None

z0k · March 23, 2017, 8:14am

Glad you got the previous issues sorted!

This error seems to be a problem on AWS’ end. I don’t think an AZ is specified in the setup script, so I’m not sure why it claims that you can choose us-west-2c. You could try again at a later time, or you could change your region altogether.

aws configure set region us-east-1

If you change your region, you may want to do a quick check just to make sure you don’t have any leftover resources, such as unused instances, volumes, or elastic IPs. If you run this for loop you should get no output:

for resource in {instances,addresses,volumes}; do aws ec2 describe-$resource; done

edit: This is an aside, but just remembered that the fast.ai script sets up only one subnet which is associated with a single AZ. So while the error message says you can switch your AZ, you actually can’t because the fast-ai subnet CIDR block encompasses that of the entire fast-ai VPC. You could create a new VPC with a subnet for each AZ, similar to how the default one is set up.

guyos · March 23, 2017, 4:03pm

Hi, I got this message when typed in “bash setup/setup_p2.sh”

True

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair ‘aws-key-fast-ai’ does not exist

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start…

Waiter InstanceRunning failed: Max attempts exceeded
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --instance-id: expected one argument

An error occurred (MissingParameter) when calling the RebootInstances operation: The request must contain the parameter InstancesSet

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /home/guyos/.ssh/aws-key-fast-ai.pem ubuntu@None

What does this mean?

Girija · March 24, 2017, 6:48am

Hi,

Please help…As suggested tried. but some error.
Tried… git the following error.

$ export region=$(aws configure get region) if[$region=“us-west-2”];then export ami=“ami-bc508adc”#Oregon elif[$region="eu-west-1];then export ami=“ami-b43d1ec7”#Ireland elif[$region=“us-east-1”];then export ami=“ami-31ecfb26”#Virginia else echo “Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported” exit 1 fi export instanceType=“p2.xlarge”
-bash: syntax error near unexpected token `then’
rgds,

gudvred · March 24, 2017, 5:52pm

One thing that worked for me it to use the aws console to create a P2.Xlarge, then go back to the EC2 dashboard and terminate it and then try to run the script again.

Tubs · March 25, 2017, 7:40am

I’m having the same issue as @chenwk89. After amazon set up a p2 server for me, I made it to the point of running bash setup_p2.sh and now I’m getting the following error:

“An error occurred (OptInRequired) when calling the CreateTags operation: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe”.

I’m an American working in India, so I’m assuming I’m getting this error because I’m outside of the states right now. @Zok recommended to @chenwk891to use spot instances. Is that what I should do as well? I was just a little confused by the back and forth on the forum at that point.

z0k · March 25, 2017, 8:07am

Hi @Girija,

Could you try downloading the updated files from GitHub?

z0k · March 25, 2017, 8:17am

Hi @Tubs,

The spot instances are not required; they are a lower cost option to on-demand instances, but require some additional setup. I would recommend starting out with the on-demand instances, which is what the provided scripts set up.

To resolve your error message I’d check to make sure that your access key ID and secret access key are correctly assigned in the .aws/credentials file. If you’re certain they are correct then I’d try contacting AWS support.

Tubs · March 25, 2017, 8:56am

@z0k thanks for your quick reply! I’ll look into that and see if that is the error

Girija · March 25, 2017, 11:29am

Hi Zarak,

I have downloaded all files in the setup folder.
No luck…same error …

$ bash setup_p2.sh
setup_p2.sh: line 7: syntax error near unexpected token newline' setup_p2.sh: line 7:’

$ wget https://github.com/fastai/courses/tree/master/setup/aws-alias.sh
–2017-03-25 16:35:35-- https://github.com/fastai/courses/tree/master/setup/aws-alias.sh
Resolving github.com… 192.30.253.112, 192.30.253.113
Connecting to github.com|192.30.253.112|:443… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://github.com/fastai/courses/blob/master/setup/aws-alias.sh [following]
–2017-03-25 16:35:36-- https://github.com/fastai/courses/blob/master/setup/aws-alias.sh
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
Saving to: ‘aws-alias.sh’

aws-alias.sh [ <=> ] 40.68K 90.5KB/s in 0.4s

2017-03-25 16:35:37 (90.5 KB/s) - ‘aws-alias.sh’ saved [41655]

$ wget https://github.com/fastai/courses/tree/master/setup/install-gpu-azure.sh
–2017-03-25 16:36:26-- https://github.com/fastai/courses/tree/master/setup/install-gpu-azure.sh
Resolving github.com… 192.30.253.112, 192.30.253.113
Connecting to github.com|192.30.253.112|:443… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://github.com/fastai/courses/blob/master/setup/install-gpu-azure.sh [following]
–2017-03-25 16:36:27-- https://github.com/fastai/courses/blob/master/setup/install-gpu-azure.sh
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
Saving to: ‘install-gpu-azure.sh’

install-gpu-azure.s [ <=> ] 43.02K 85.6KB/s in 0.5s

2017-03-25 16:36:28 (85.6 KB/s) - ‘install-gpu-azure.sh’ saved [44050]

$ wget https://github.com/fastai/courses/tree/master/setup/install-gpu.sh
–2017-03-25 16:37:00-- https://github.com/fastai/courses/tree/master/setup/install-gpu.sh
Resolving github.com… 192.30.253.113, 192.30.253.112
Connecting to github.com|192.30.253.113|:443… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://github.com/fastai/courses/blob/master/setup/install-gpu.sh [following]
–2017-03-25 16:37:01-- https://github.com/fastai/courses/blob/master/setup/install-gpu.sh
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
Saving to: ‘install-gpu.sh’

install-gpu.sh [ <=> ] 49.00K 110KB/s in 0.4s

2017-03-25 16:37:02 (110 KB/s) - ‘install-gpu.sh’ saved [50173]

$ wget https://github.com/fastai/courses/tree/master/setup/setup_instance.sh
–2017-03-25 16:37:37-- https://github.com/fastai/courses/tree/master/setup/setup_instance.sh
Resolving github.com… 192.30.253.113, 192.30.253.112
Connecting to github.com|192.30.253.113|:443… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://github.com/fastai/courses/blob/master/setup/setup_instance.sh [following]
–2017-03-25 16:37:38-- https://github.com/fastai/courses/blob/master/setup/setup_instance.sh
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
Saving to: ‘setup_instance.sh’

setup_instance.sh [ <=> ] 71.97K 164KB/s in 0.4s

2017-03-25 16:37:39 (164 KB/s) - ‘setup_instance.sh’ saved [73701]

$ wget https://github.com/fastai/courses/tree/master/setup/setup_p2.sh
–2017-03-25 16:38:02-- https://github.com/fastai/courses/tree/master/setup/setup_p2.sh
Resolving github.com… 192.30.253.112, 192.30.253.113
Connecting to github.com|192.30.253.112|:443… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://github.com/fastai/courses/blob/master/setup/setup_p2.sh [following]
–2017-03-25 16:38:03-- https://github.com/fastai/courses/blob/master/setup/setup_p2.sh
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
Saving to: ‘setup_p2.sh’

setup_p2.sh [ <=> ] 37.97K 165KB/s in 0.2s

2017-03-25 16:38:04 (165 KB/s) - ‘setup_p2.sh’ saved [38879]

$ wget https://github.com/fastai/courses/tree/master/setup/setup_t2.sh
–2017-03-25 16:38:26-- https://github.com/fastai/courses/tree/master/setup/setup_t2.sh
Resolving github.com… 192.30.253.112, 192.30.253.113
Connecting to github.com|192.30.253.112|:443… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://github.com/fastai/courses/blob/master/setup/setup_t2.sh [following]
–2017-03-25 16:38:27-- https://github.com/fastai/courses/blob/master/setup/setup_t2.sh
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
Saving to: ‘setup_t2.sh’

setup_t2.sh [ <=> ] 37.43K 154KB/s in 0.2s

2017-03-25 16:38:28 (154 KB/s) - ‘setup_t2.sh’ saved [38329]

$ bash setup_p2.sh
setup_p2.sh: line 7: syntax error near unexpected token newline' setup_p2.sh: line 7:’