Setup problems: AWS

ecase · May 31, 2017, 5:58pm

It’s a problem with teh way wget is downloading the setup_p2.sh files – copy & paste the text on github into the setup file and you should be good to go

andrewtuplin · June 1, 2017, 4:43am

Well, I’m happy to report progress of sorts. I’m not sure I can make sense of the results though.

the script is executing “aws configure get region” (just as I did) and testing for a result of “us-west-2” – the result returned to me. How can this test be failing??

z0k · June 1, 2017, 2:50pm

I don’t have Windows so I can’t test this myself, but it’s been suggested to use Ubuntu bash:

Amin · June 2, 2017, 9:19am

Hi, I am trying to connect from Europe. It seems that I can’t find the “setup_p2_ireland.sh” from the following link:
http://files.fast.ai/files/. Could you please provide the link to the “setup_p2_ireland.sh”?

I tried using the “setup_p2.sh” instead but I got the following error message:
An error occurred (InvalidAMIID.NotFound) when calling the RunInstances operation: The image id ‘[ami-bc508adc]’ does not exist

ecase · June 2, 2017, 12:52pm

Try the github repo instead https://github.com/fastai/courses

Amin · June 2, 2017, 1:22pm

Thanks Elizabeth, I already checked the github repo but didn’t manege to fond the “setup_p2_ireland.sh” in there either.

ecase · June 2, 2017, 1:28pm

When you run aws-configure, are you setting your region to eu-west-1? That should ensure that when you run setup_p2, you linked to the correct ami. & the ami # for irelend referenced in setup_p2 looks different than the one you mentioned, so maybe it was updated since the last files upload?

#!/bin/bash
#
# Configure a p2.xlarge instance

# get the correct ami
export region=$(aws configure get region)
if [ $region = "us-west-2" ]; then
   export ami="ami-bc508adc" # Oregon
elif [ $region = "eu-west-1" ]; then
   export ami="ami-b43d1ec7" # Ireland <----- 
elif [ $region = "us-east-1" ]; then
  export ami="ami-31ecfb26" # Virginia
else
  echo "Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported"
  exit 1
fi

export instanceType="p2.xlarge"

. $(dirname "$0")/setup_instance.sh

dtung · June 2, 2017, 9:01pm

Why the setup_p2.sh script is about p2.xlarge, but we kept asked to raise the limit on g2.2xlarge?

Besides, do we all setup into the same machine? In such case, if too many people use it, isn’t it going to be very slow? When I type nvidia-smi into terminal, it shows Volatile GPU-Util : 99%.

ekund · June 6, 2017, 4:14pm

I am having issues with my setup as well.

I create the instances using setup_p2.sh:

    $ bash setup_t2.sh 
    True
    Waiting for instance start...

    All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
    Connect to your instance: ssh -i /Users/eiserovich1/.ssh/aws-key-fast-ai.pem ubuntu@ec2-52-37-192-166.us-west-2.compute.amazonaws.com

But when I ssh:

$ ssh -i /Users/eiserovich1/.ssh/aws-key-fast-ai.pem ubuntu@ec2-52-37-192-166.us-west-2.compute.amazonaws.com
ssh: connect to host ec2-52-37-192-166.us-west-2.compute.amazonaws.com port 22: Operation timed out

If I try to run aws-ssh it asks me for a password for some reason, which I’m pretty sure it shouldn’t:

$ aws-ssh
Warning: Permanently added '' (ECDSA) to the list of known hosts.
Password:
Password:
Password:
ubuntu@'s password: 
Permission denied, please try again.
ubuntu@'s password: 
Received disconnect from ::1 port 22:2: Too many authentication failures
Authentication failed.

I have tried tearing down the instances, removing the .pem file, and removing the keyparir on the AWS screen, and it’s still not working for me.
Any ideas?

martinpella · June 6, 2017, 4:43pm

Thanks!, your solution worked for me.

karmicjay · June 11, 2017, 1:54am

Sorry newbie here, so I apologize if I am not posting this correctly.
I did not see any posts with the below error. I requested a p2.xlarge instance be created in us- east (N.Virginia). It was completed. After getting past known set up issues, I got this far and am stumped.
My setup_p2 file has been modified for just the eastern region
#!/bin/bash

Configure a p2.xlarge instance

# get the correct ami
export region=$(aws configure get region)
_ export ami=“ami-31ecfb26” # Virginia_
_ _
export instanceType=“p2.xlarge”

. $(dirname “$0”)/setup_instance.sh

The errors I get are below.
I appreciate any help that I can get. Thanks.

$ bash setup_p2.sh

Invalid endpoint: https://ec2.us-east-1.amazonaws.com

Invalid endpoint: https://ec2.us-east-1.amazonaws.com
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument

corbin · June 11, 2017, 12:56pm

In the setup video, they say that the ami they have set up only works in the Oregon region within the US. You’ll need to switch regions and request access to a new instance (it will not carry over to oregon) if you want to utilize the preconfigured AMI

karmicjay · June 11, 2017, 1:01pm

Thanks Corbin. I did request a new instance in Oregon now. Hopefully that works when set up. I will update this thread once we know.

corbin · June 11, 2017, 1:19pm

Also, since you have now run the script unsuccessfully, there will be a number of things you need to delete on Virginia to avoid being charged unnecessarily.

Go to VPC from the main AWS dashboard
If there are ANY VPCs, delete them. Make sure you click the down arrow and release the IP as well.
Everything on the VPC management console should be 0. Feel free to delete everything.
Go to EC2.
Everything here should also be 0. Delete all things.
On your local machine, rm fast-ai-commands and fast-ai-remove.sh
Lastly, rm ~/.ssh/aws-key-fast-ai.pem

Now, once you have been approved (and make sure you give it a couple of hours after approval), you can run the script again and it should work fine. If not, you’ll need to delete everything again as above and start from scratch. You shouldn’t run into any problems though as long as you wait a bit after approval.

karmicjay · June 11, 2017, 1:35pm

Thanks Corbin. Did all the steps up to rm fast-ai-commands. Was I to rm fast-ai-remove.sh and not run it right?
I ran it and got the below. I feel kinda stupid requesting the non Oregon region. Ugh a chance to learn something I guess.
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
_ aws help_
_ aws help_
aws: error: argument --association-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --allocation-id: expected one argument

Invalid endpoint: https://ec2.us-east-1.amazonaws.com

Invalid endpoint: https://ec2.us-east-1.amazonaws.com
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --group-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --association-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --route-table-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --subnet-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --vpc-id: expected one argument
If you want to delete the key-pair, please do it manually.

corbin · June 11, 2017, 1:39pm

The rm script attempts to automate everything I told you to do manually at the aws console. But because there was no instance to terminate, etc., I thought it best to just be safe and ensure everything was deleted without relying on the script. Would have gotten errors anyways.

So you can remove the remove script as well. A new one will be created when you run the setup_p2.sh later.

karmicjay · June 11, 2017, 1:44pm

Thanks again Corbin.

aadimator · June 11, 2017, 7:44pm

I am having a strange problem with my instance. I ran it in the day and it was working fine, I was able to connect through ssh and I even ran “Hello World” on the notebook, to test it out.
But now, I can’t seem to connect to the notebook through the browser. I can still connect to the instance using the ssh from the command line, but when I issue the aws-nb command, I can’t access the notebook using the browser. The browser can’t connect to the server. It gives a connection refused error.
Any idea why this might be happening?

aadimator · June 11, 2017, 7:55pm

Oops… my bad. I thought aws-nb command starts the jupyter notebook server as well. Nevermind, rookie mistake.

Bish · June 13, 2017, 3:27am

Hey Guys,

Struggling to get through the bash setup_p2.sh step. Some of the errors I’m receiving overlap with those mentioned previously, but none of the proposed solutions have worked.I’m currently running Windows 8.1 Here are the error outputs I get:

$ bash setup_p2.sh
rtbassoc-0e1c6b77

An error occurred (InvalidRouteTableId.Malformed) when calling the CreateRoute o"eration: Invalid id: "rtb-ea45278c

An error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityG"oupIngress operation: Invalid id: "sg-201a175b

An error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityG"oupIngress operation: Invalid id: "sg-201a175b
setup_p2.sh: line 13: /home/Bishoy Ghobryal/.ssh/aws-key.pem: No such file or directory
chmod: cannot access ‘/home/Bishoy Ghobryal/.ssh/aws-key.pem’: No such file or directory

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair ‘aws-key’ does not exist

Waiting for instance start…

Thanks for your help!