Setup problems: AWS


(Elizabeth) #618

It’s a problem with teh way wget is downloading the setup_p2.sh files – copy & paste the text on github into the setup file and you should be good to go


(Andrew Tuplin) #619

Well, I’m happy to report progress of sorts. I’m not sure I can make sense of the results though.

the script is executing “aws configure get region” (just as I did) and testing for a result of “us-west-2” – the result returned to me. How can this test be failing??


(Zarak) #620

I don’t have Windows so I can’t test this myself, but it’s been suggested to use Ubuntu bash:


#621

Hi, I am trying to connect from Europe. It seems that I can’t find the “setup_p2_ireland.sh” from the following link:
http://files.fast.ai/files/. Could you please provide the link to the “setup_p2_ireland.sh”? :slight_smile:

I tried using the “setup_p2.sh” instead but I got the following error message:
An error occurred (InvalidAMIID.NotFound) when calling the RunInstances operation: The image id ‘[ami-bc508adc]’ does not exist


(Elizabeth) #622

Try the github repo instead https://github.com/fastai/courses


#623

Thanks Elizabeth, I already checked the github repo but didn’t manege to fond the “setup_p2_ireland.sh” in there either.


(Elizabeth) #624

When you run aws-configure, are you setting your region to eu-west-1? That should ensure that when you run setup_p2, you linked to the correct ami. & the ami # for irelend referenced in setup_p2 looks different than the one you mentioned, so maybe it was updated since the last files upload?

#!/bin/bash
#
# Configure a p2.xlarge instance

# get the correct ami
export region=$(aws configure get region)
if [ $region = "us-west-2" ]; then
   export ami="ami-bc508adc" # Oregon
elif [ $region = "eu-west-1" ]; then
   export ami="ami-b43d1ec7" # Ireland <----- 
elif [ $region = "us-east-1" ]; then
  export ami="ami-31ecfb26" # Virginia
else
  echo "Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported"
  exit 1
fi

export instanceType="p2.xlarge"

. $(dirname "$0")/setup_instance.sh

(David Tung) #626

Why the setup_p2.sh script is about p2.xlarge, but we kept asked to raise the limit on g2.2xlarge?

Besides, do we all setup into the same machine? In such case, if too many people use it, isn’t it going to be very slow? When I type nvidia-smi into terminal, it shows Volatile GPU-Util : 99%.


(Esther) #627

I am having issues with my setup as well.

I create the instances using setup_p2.sh:

    $ bash setup_t2.sh 
    True
    Waiting for instance start...

    All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
    Connect to your instance: ssh -i /Users/eiserovich1/.ssh/aws-key-fast-ai.pem ubuntu@ec2-52-37-192-166.us-west-2.compute.amazonaws.com

But when I ssh:

$ ssh -i /Users/eiserovich1/.ssh/aws-key-fast-ai.pem ubuntu@ec2-52-37-192-166.us-west-2.compute.amazonaws.com
ssh: connect to host ec2-52-37-192-166.us-west-2.compute.amazonaws.com port 22: Operation timed out

If I try to run aws-ssh it asks me for a password for some reason, which I’m pretty sure it shouldn’t:

$ aws-ssh
Warning: Permanently added '' (ECDSA) to the list of known hosts.
Password:
Password:
Password:
ubuntu@'s password: 
Permission denied, please try again.
ubuntu@'s password: 
Received disconnect from ::1 port 22:2: Too many authentication failures
Authentication failed.

I have tried tearing down the instances, removing the .pem file, and removing the keyparir on the AWS screen, and it’s still not working for me.
Any ideas?


#628

Thanks!, your solution worked for me.


(karmic jay) #630

Sorry newbie here, so I apologize if I am not posting this correctly.
I did not see any posts with the below error. I requested a p2.xlarge instance be created in us- east (N.Virginia). It was completed. After getting past known set up issues, I got this far and am stumped.
My setup_p2 file has been modified for just the eastern region
#!/bin/bash

Configure a p2.xlarge instance

# get the correct ami
export region=$(aws configure get region)
_ export ami=“ami-31ecfb26” # Virginia_
_ _
export instanceType=“p2.xlarge”

. $(dirname “$0”)/setup_instance.sh

The errors I get are below.
I appreciate any help that I can get. Thanks.

$ bash setup_p2.sh

Invalid endpoint: https://ec2.us-east-1.amazonaws.com

Invalid endpoint: https://ec2.us-east-1.amazonaws.com
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument


(Corbin Albert) #631

In the setup video, they say that the ami they have set up only works in the Oregon region within the US. You’ll need to switch regions and request access to a new instance (it will not carry over to oregon) if you want to utilize the preconfigured AMI


(karmic jay) #632

Thanks Corbin. I did request a new instance in Oregon now. Hopefully that works when set up. I will update this thread once we know.


(Corbin Albert) #633

Also, since you have now run the script unsuccessfully, there will be a number of things you need to delete on Virginia to avoid being charged unnecessarily.

  1. Go to VPC from the main AWS dashboard
  2. If there are ANY VPCs, delete them. Make sure you click the down arrow and release the IP as well.
  3. Everything on the VPC management console should be 0. Feel free to delete everything.
  4. Go to EC2.
  5. Everything here should also be 0. Delete all things.
  6. On your local machine, rm fast-ai-commands and fast-ai-remove.sh
  7. Lastly, rm ~/.ssh/aws-key-fast-ai.pem

Now, once you have been approved (and make sure you give it a couple of hours after approval), you can run the script again and it should work fine. If not, you’ll need to delete everything again as above and start from scratch. You shouldn’t run into any problems though as long as you wait a bit after approval.


(karmic jay) #634

Thanks Corbin. Did all the steps up to rm fast-ai-commands. Was I to rm fast-ai-remove.sh and not run it right?
I ran it and got the below. I feel kinda stupid requesting the non Oregon region. Ugh a chance to learn something I guess.
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
_ aws help_
_ aws help_
aws: error: argument --association-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --allocation-id: expected one argument

Invalid endpoint: https://ec2.us-east-1.amazonaws.com

Invalid endpoint: https://ec2.us-east-1.amazonaws.com
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --group-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --association-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --route-table-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --subnet-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

_ aws help_
_ aws help_
_ aws help_
aws: error: argument --vpc-id: expected one argument
If you want to delete the key-pair, please do it manually.


(Corbin Albert) #635

The rm script attempts to automate everything I told you to do manually at the aws console. But because there was no instance to terminate, etc., I thought it best to just be safe and ensure everything was deleted without relying on the script. Would have gotten errors anyways.

So you can remove the remove script as well. A new one will be created when you run the setup_p2.sh later.


(karmic jay) #636

Thanks again Corbin.


(Aadam ‍) #637

I am having a strange problem with my instance. I ran it in the day and it was working fine, I was able to connect through ssh and I even ran “Hello World” on the notebook, to test it out.
But now, I can’t seem to connect to the notebook through the browser. I can still connect to the instance using the ssh from the command line, but when I issue the aws-nb command, I can’t access the notebook using the browser. The browser can’t connect to the server. It gives a connection refused error.
Any idea why this might be happening?


(Aadam ‍) #638

Oops… my bad. I thought aws-nb command starts the jupyter notebook server as well. Nevermind, rookie mistake.


(Bishoy Ghobryal) #639

Hey Guys,

Struggling to get through the bash setup_p2.sh step. Some of the errors I’m receiving overlap with those mentioned previously, but none of the proposed solutions have worked.I’m currently running Windows 8.1 Here are the error outputs I get:

$ bash setup_p2.sh
rtbassoc-0e1c6b77

An error occurred (InvalidRouteTableId.Malformed) when calling the CreateRoute o"eration: Invalid id: "rtb-ea45278c

An error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityG"oupIngress operation: Invalid id: "sg-201a175b

An error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityG"oupIngress operation: Invalid id: "sg-201a175b
setup_p2.sh: line 13: /home/Bishoy Ghobryal/.ssh/aws-key.pem: No such file or directory
chmod: cannot access ‘/home/Bishoy Ghobryal/.ssh/aws-key.pem’: No such file or directory

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair ‘aws-key’ does not exist

Waiting for instance start…

Thanks for your help!