Setup problems: AWS

ecase · June 2, 2017, 1:28pm

When you run aws-configure, are you setting your region to eu-west-1? That should ensure that when you run setup_p2, you linked to the correct ami. & the ami # for irelend referenced in setup_p2 looks different than the one you mentioned, so maybe it was updated since the last files upload?

#!/bin/bash
#
# Configure a p2.xlarge instance

# get the correct ami
export region=$(aws configure get region)
if [ $region = "us-west-2" ]; then
   export ami="ami-bc508adc" # Oregon
elif [ $region = "eu-west-1" ]; then
   export ami="ami-b43d1ec7" # Ireland <----- 
elif [ $region = "us-east-1" ]; then
  export ami="ami-31ecfb26" # Virginia
else
  echo "Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported"
  exit 1
fi

export instanceType="p2.xlarge"

. $(dirname "$0")/setup_instance.sh

dtung · June 2, 2017, 9:01pm

Why the setup_p2.sh script is about p2.xlarge, but we kept asked to raise the limit on g2.2xlarge?

Besides, do we all setup into the same machine? In such case, if too many people use it, isn’t it going to be very slow? When I type nvidia-smi into terminal, it shows Volatile GPU-Util : 99%.

ekund · June 6, 2017, 4:14pm

I am having issues with my setup as well.

I create the instances using setup_p2.sh:

    $ bash setup_t2.sh 
    True
    Waiting for instance start...

    All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
    Connect to your instance: ssh -i /Users/eiserovich1/.ssh/aws-key-fast-ai.pem ubuntu@ec2-52-37-192-166.us-west-2.compute.amazonaws.com

But when I ssh:

$ ssh -i /Users/eiserovich1/.ssh/aws-key-fast-ai.pem ubuntu@ec2-52-37-192-166.us-west-2.compute.amazonaws.com
ssh: connect to host ec2-52-37-192-166.us-west-2.compute.amazonaws.com port 22: Operation timed out

If I try to run aws-ssh it asks me for a password for some reason, which I’m pretty sure it shouldn’t:

$ aws-ssh
Warning: Permanently added '' (ECDSA) to the list of known hosts.
Password:
Password:
Password:
ubuntu@'s password: 
Permission denied, please try again.
ubuntu@'s password: 
Received disconnect from ::1 port 22:2: Too many authentication failures
Authentication failed.

I have tried tearing down the instances, removing the .pem file, and removing the keyparir on the AWS screen, and it’s still not working for me.
Any ideas?

martinpella · June 6, 2017, 4:43pm

Thanks!, your solution worked for me.

karmicjay · June 11, 2017, 1:54am

Sorry newbie here, so I apologize if I am not posting this correctly.
I did not see any posts with the below error. I requested a p2.xlarge instance be created in us- east (N.Virginia). It was completed. After getting past known set up issues, I got this far and am stumped.
My setup_p2 file has been modified for just the eastern region
#!/bin/bash

Configure a p2.xlarge instance

# get the correct ami
export region=$(aws configure get region)
_ export ami=“ami-31ecfb26” # Virginia_
_ _
export instanceType=“p2.xlarge”

. $(dirname “$0”)/setup_instance.sh

The errors I get are below.
I appreciate any help that I can get. Thanks.

$ bash setup_p2.sh

Invalid endpoint: https://ec2.us-east-1.amazonaws.com

Invalid endpoint: https://ec2.us-east-1.amazonaws.com
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument

corbin · June 11, 2017, 12:56pm

In the setup video, they say that the ami they have set up only works in the Oregon region within the US. You’ll need to switch regions and request access to a new instance (it will not carry over to oregon) if you want to utilize the preconfigured AMI

karmicjay · June 11, 2017, 1:01pm

Thanks Corbin. I did request a new instance in Oregon now. Hopefully that works when set up. I will update this thread once we know.

corbin · June 11, 2017, 1:19pm

Also, since you have now run the script unsuccessfully, there will be a number of things you need to delete on Virginia to avoid being charged unnecessarily.

Go to VPC from the main AWS dashboard
If there are ANY VPCs, delete them. Make sure you click the down arrow and release the IP as well.
Everything on the VPC management console should be 0. Feel free to delete everything.
Go to EC2.
Everything here should also be 0. Delete all things.
On your local machine, rm fast-ai-commands and fast-ai-remove.sh
Lastly, rm ~/.ssh/aws-key-fast-ai.pem

Now, once you have been approved (and make sure you give it a couple of hours after approval), you can run the script again and it should work fine. If not, you’ll need to delete everything again as above and start from scratch. You shouldn’t run into any problems though as long as you wait a bit after approval.

karmicjay · June 11, 2017, 1:35pm

Thanks Corbin. Did all the steps up to rm fast-ai-commands. Was I to rm fast-ai-remove.sh and not run it right?
I ran it and got the below. I feel kinda stupid requesting the non Oregon region. Ugh a chance to learn something I guess.
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
_ aws help_
_ aws help_
aws: error: argument --association-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --allocation-id: expected one argument

Invalid endpoint: https://ec2.us-east-1.amazonaws.com

Invalid endpoint: https://ec2.us-east-1.amazonaws.com
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --group-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --association-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --route-table-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --subnet-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --vpc-id: expected one argument
If you want to delete the key-pair, please do it manually.

corbin · June 11, 2017, 1:39pm

The rm script attempts to automate everything I told you to do manually at the aws console. But because there was no instance to terminate, etc., I thought it best to just be safe and ensure everything was deleted without relying on the script. Would have gotten errors anyways.

So you can remove the remove script as well. A new one will be created when you run the setup_p2.sh later.

karmicjay · June 11, 2017, 1:44pm

Thanks again Corbin.

aadimator · June 11, 2017, 7:44pm

I am having a strange problem with my instance. I ran it in the day and it was working fine, I was able to connect through ssh and I even ran “Hello World” on the notebook, to test it out.
But now, I can’t seem to connect to the notebook through the browser. I can still connect to the instance using the ssh from the command line, but when I issue the aws-nb command, I can’t access the notebook using the browser. The browser can’t connect to the server. It gives a connection refused error.
Any idea why this might be happening?

aadimator · June 11, 2017, 7:55pm

Oops… my bad. I thought aws-nb command starts the jupyter notebook server as well. Nevermind, rookie mistake.

Bish · June 13, 2017, 3:27am

Hey Guys,

Struggling to get through the bash setup_p2.sh step. Some of the errors I’m receiving overlap with those mentioned previously, but none of the proposed solutions have worked.I’m currently running Windows 8.1 Here are the error outputs I get:

$ bash setup_p2.sh
rtbassoc-0e1c6b77

An error occurred (InvalidRouteTableId.Malformed) when calling the CreateRoute o"eration: Invalid id: "rtb-ea45278c

An error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityG"oupIngress operation: Invalid id: "sg-201a175b

An error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityG"oupIngress operation: Invalid id: "sg-201a175b
setup_p2.sh: line 13: /home/Bishoy Ghobryal/.ssh/aws-key.pem: No such file or directory
chmod: cannot access ‘/home/Bishoy Ghobryal/.ssh/aws-key.pem’: No such file or directory

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair ‘aws-key’ does not exist

Waiting for instance start…

Thanks for your help!

benshiundu · June 15, 2017, 9:05pm

is there away to access aws instance out of the regions the services are offered?

gunsonj · June 16, 2017, 4:29am

Hi,

Trying to install AWS on Ubuntu Linux 14.04.
Everything is fine till

sudo bash setup_p2.sh
An error occurred (UnauthorizedOperation) when calling the CreateVpc operation: You are not authorized to perform this operation.
This is same error that janardhanp22 had (which is the only occurence of this error in the forum)

I restarted the whole procedure starting from
sudo pip install awscli
but still get the above error message.
Any help greatly appreciated,

Jim

benshiundu · June 16, 2017, 2:12pm

This saved me a lot of time and frustrations. Thanks @jrheard

mehdi · June 23, 2017, 8:48am

I am trying to execute: bash setup_p2.sh and i have this error :

    An error occurred (VpcLimitExceeded) when calling the CreateVpc operation: The maximum number of VPCs has been reached.
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument

An error occurred (InternetGatewayLimitExceeded) when calling the CreateInternetGateway operation: The maximum number of internet gateways has been reached.
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --route-table-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --route-table-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --group-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --group-id: expected one argument

An error occurred (InvalidKeyPair.Duplicate) when calling the CreateKeyPair operation: The keypair 'aws-key' already exists.
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --subnet-id: expected one argument

An error occurred (AddressLimitExceeded) when calling the AllocateAddress operation: The maximum number of addresses has been reached.
Waiting for instance start...

Waiter InstanceRunning failed: Max attempts exceeded
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --instance-id: expected one argument
securityGroupId=
subnetId=
instanceId=
instanceUrl=None
Connect: ssh -i /home/medhi/.ssh/aws-key.pem ubuntu@None

BenStevenson · June 23, 2017, 3:51pm

Hi, I’m also having trouble with the bash setup_p2.sh stage.

I had an issue that has been mentioned above, the wget was downloading in html format.

git clone helped to get the file but when I try to load the file I get an error message saying that there’s no such file or directory.

Does anyone have any suggestions?

Any help would be much appreciated,
Ben

guy · June 23, 2017, 5:17pm

Hi @mehdi. I had the same issue. AWS limits the number of VPC, subnets, gateways, security groups, etc. If you started the process several times, you may have a few lingering around.

Easiest fix is to go to the AWS console and delete every VPC, security group, internet gateway and subnet you don’t use for anything else. AWS won’t let you delete stuff you really need.

Another fix is to change setup_instance.sh to use some of your existing VPC etc. for your new p2 instance. In this case you can comment out all the creation parts and only need to assign the securityGroupId and subnetId that are used to call ‘aws ec2 run-instances’