Setup problems: AWS


(Benson Shiundu) #640

is there away to access aws instance out of the regions the services are offered?


(Jim Gunson) #642

Hi,

Trying to install AWS on Ubuntu Linux 14.04.
Everything is fine till

sudo bash setup_p2.sh
An error occurred (UnauthorizedOperation) when calling the CreateVpc operation: You are not authorized to perform this operation.
This is same error that janardhanp22 had (which is the only occurence of this error in the forum)

I restarted the whole procedure starting from
sudo pip install awscli
but still get the above error message.
Any help greatly appreciated,

Jim


(Benson Shiundu) #643

This saved me a lot of time and frustrations. Thanks @jrheard


(mehdi) #645

I am trying to execute: bash setup_p2.sh and i have this error :

    An error occurred (VpcLimitExceeded) when calling the CreateVpc operation: The maximum number of VPCs has been reached.
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument

An error occurred (InternetGatewayLimitExceeded) when calling the CreateInternetGateway operation: The maximum number of internet gateways has been reached.
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --route-table-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --route-table-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --group-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --group-id: expected one argument

An error occurred (InvalidKeyPair.Duplicate) when calling the CreateKeyPair operation: The keypair 'aws-key' already exists.
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --subnet-id: expected one argument

An error occurred (AddressLimitExceeded) when calling the AllocateAddress operation: The maximum number of addresses has been reached.
Waiting for instance start...

Waiter InstanceRunning failed: Max attempts exceeded
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --instance-id: expected one argument
securityGroupId=
subnetId=
instanceId=
instanceUrl=None
Connect: ssh -i /home/medhi/.ssh/aws-key.pem ubuntu@None

(Benjamin Stevenson) #646

Hi, I’m also having trouble with the bash setup_p2.sh stage.

I had an issue that has been mentioned above, the wget was downloading in html format.

git clone helped to get the file but when I try to load the file I get an error message saying that there’s no such file or directory.

Does anyone have any suggestions?

Any help would be much appreciated,
Ben


(Guy) #647

Hi @mehdi. I had the same issue. AWS limits the number of VPC, subnets, gateways, security groups, etc. If you started the process several times, you may have a few lingering around.

Easiest fix is to go to the AWS console and delete every VPC, security group, internet gateway and subnet you don’t use for anything else. AWS won’t let you delete stuff you really need.

Another fix is to change setup_instance.sh to use some of your existing VPC etc. for your new p2 instance. In this case you can comment out all the creation parts and only need to assign the securityGroupId and subnetId that are used to call ‘aws ec2 run-instances’


(Guy) #648

Did you clone the entire directory? Looks like you are missing setup_instance.sh (as opposed to setup_p2.sh which just calls setup_instance.sh


(Johnny Chan) #649

I bumped into similar issue and solved it recently. i.e. the following error persists despite having the correct IAM and secrete key stored at ~/.aws/credentials, and region ~/.aws/config

i.e. An error occurred (AuthFailure) when calling the AllocateAddress operation: AWS was not able to validate the provided access credentials

I solved this by realising these AWS specific environmental variables AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEYandAWS_DEFAULT_REGIONoverrides the ones specified in the credentials and config files (created byaws configure`).

Check if you have these environmental variables. If you do, make sure the values match the ones in credentials and config files.

Reference: AWS CLI Setup


(Johnny Chan) #650

This is how I did mine:

Pre-requisite: have git and Anaconda installed. Created a conda environment with a suitable version of Python and anaconda package, and awscli). Activate that conda environment.

Step 1: Download the GitHub repository:

git clone https://github.com/fastai/courses

Step 2: navigate to setup.

Step 3: bash setup_p2.sh (which involves the setup_instance.sh file that sits within the same directory).


(Pawel Godula) #651

Hello

I have installed aws cli, but since my user name has a space in it (my user name is ‘godula pawel’), the catalogue where I installed everything also has a space in the filepath.

Cygwin is not able to read filepath with space in it. Interestingly, i was able to install aws cli, but i am not able to run it. (see attached screenshot).

Is the only solution to install everything (anaconda, cygwin, etc.) into some other catalogue than default, with no spaces in it?

Thanks so much!


(Elizabeth) #652

to really make sure you’re getting everything from the github repo – git clone -r _URL_

also check and see if maybe the setup_instance file’s name was changed


(Elizabeth) #653

Hey! At least in the most recent setup, the name of the key that was setup wasn’t aws-key but something like fast-ai-aws-key – check the name of the key by cd ~/.ssh (or wherever you keep your keys) and ls to list the name of the keys in the directory to check and see if you are referencing the correct key


(Shravan Kumar) #654

I am getting the following error: ssh -i /home/Shravan Kumar/.ssh/aws-key.pem ubuntu@ec2-52-24-7-137.us-west-2.compute.amazonaws.com
Warning: Identity file /home/Shravan not accessible: No such file or directory.
ssh: Could not resolve hostname Kumar/.ssh/aws-key.pem: no address associated with name
Can someone help me how to tackle this error?


(Jose Alberto Soler) #655

I am getting frustrated with the setup of the course environment. Since I own a powerful desktop PC (decent quad core + gtx1060), I decided it would be easier to skip AWS and run everything local.
I installed all the dependecies myself but had some problems because I was running Keras 2 instead of Keras 1. I updated some of the code and got through most of lesson 1, but got bored of trying to modify so many lines of code and tried to run a VM and use the scripts of another user who modified everything for python 3 and keras 2. This didn’t work because as it turns out VMware does not support GPU passthrough.
I could go on but you get the point.
At this point I am left considering, if it was unwise to not use AWS from the beginning? Is it really easier to set up than a local config? I know it’s relatively cheap but it seems pointless to rent a computer in the cloud having a more powerful computer idling in the office. Any thoughts/advice on this? Thank you


#656

AWS is a pain to use; expensive for the on demand instances if you use it a lot; and very slow compared to latest GPUs you can buy. I ended up writing a package to use spot instances (https://github.com/simonm3/xdrive) which are about 25c/hour rather than 90c. However it always seems to cost more than you expect; and is very slow compared to the course notebooks that were originally run on titan x.

I recommend either buying a used desktop (or use the one you have) and a good GPU such as the 1080ti which I understand is 2-3 times faster than AWS. Ideally just use the host OS - you don’t say why you need a VM. Or if you really want a VM then use docker which definitely works with GPUs - with a bit of tweaking.


(Jose Alberto Soler) #657

So, apparently all is good now. I installed Ubuntu on a second SSD and used the script provided by Abhik Mitra here
All dependencies were installed and configured, awesome work! Only problem was the datasets were moved, otherwise it worked great.
If you are in the same situation as me, I highly recommend this route. Thanks a lot to Abkik for sharing the script.


(Jose Alberto Soler) #658

Thank you simoneva for your input, that was my intuition too. I finally got it working on my local computer (see response to my original post)


(Benjamin Stevenson) #659

@guy @Atlas7 @ecase

Ahah! I hadn’t realised that it called on another file in the directory. It’s working now.

Thanks so much for your help!


(Stuart Hallows) #660

Perhaps the Windows line endings have been reintroduced. I just had to do dos2unix *.sh before I could run the scripts (using Bash on Ubuntu on Windows). I was seeing very similar issues to you.


(Justin ) #661

Hello!
I am trying to set this up on cloud9 since I am working on a chromebook. I have tried a couple of variations of using wget and uplaoding the file and trying to change directories but I get this message or a message about directories. Any suggestions?

sirjustin:~/workspace $ bash setup_p2.sh.1
setup_p2.sh.1: line 7: syntax error near unexpected token newline' setup_p2.sh.1: line 7:'
sirjustin:~/workspace $

Thanks!