Setup problems: AWS

h.khanpour · October 1, 2017, 9:36pm

I have the same problem…is there anybody to help? my time set is correct…

jayfree · October 2, 2017, 6:03am

Make sure that you ran this line of code

export PATH="$HOME/anaconda2/bin:$PATH"

I was setting up my home deep learning machine and I was manually typing the commands from the “install-gpu.sh” script. I missed to type this line and was getting you same error. After updating the PATH variable, this error was gone.

sabliao · October 2, 2017, 8:02pm

I’m following the above video to set up my environment. I tried pip install awscli like it’s done in the video but I get:

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/awscli'

Usually I would rerun the pip install w/ sudo in these cases, but I assumed Anaconda would take care of permission issues like this. Why is the user/instructor in the video able to run w/o sudo?

prav · October 2, 2017, 9:09pm

I did the install as described but was not able to launch the aws instance. Then I deleted everything and did the whole process again. Now I am getting this error:
$ bash setup_p2.sh
rtbassoc-e4d9c49e
True

An error occurred (InvalidAMIID.NotFound) when calling the RunInstances operation: The image id ‘[ami-bc508adc]’ does not exist

I downloaded the setup_p2.sh from location : http://files.fast.ai/files/setup_p2.sh" is that correct?
Also do I have to wait for Amazon to approve before I go ahead and create an instance.

Please help.

Thanks

sabliao · October 3, 2017, 5:33am

To add more context for what I’ve done since posting this question: I have the anaconda path added to my ~/.zshrc file (/Users/sabrinaliao/anaconda2/bin) and sourced it (but I’m not sure my default python is the Anaconda installation b/c I didn’t see anything that says ‘anaconda’ when I run python unlike what’s shown in the example here). The pip install of awscli still failed, but then I thought to create a a virtual environment using conda. I ran conda create -n fast_ai and then activated the virtual environment. However, I still got a permission denied error when I tried pip install. I then tried conda install awscli but got:

Fetching package metadata ...........

PackageNotFoundError: Packages missing in current channels:

  - awscli

We have searched for the packages in the following channels:

  - https://repo.continuum.io/pkgs/main/osx-64
  - https://repo.continuum.io/pkgs/main/noarch
  - https://repo.continuum.io/pkgs/free/osx-64
  - https://repo.continuum.io/pkgs/free/noarch
  - https://repo.continuum.io/pkgs/r/osx-64
  - https://repo.continuum.io/pkgs/r/noarch
  - https://repo.continuum.io/pkgs/pro/osx-64
  - https://repo.continuum.io/pkgs/pro/noarch

Searching for awscli for my fast_ai venv via the anaconda navigator brought up 0 results too (was just double checking). I decided to run the command given here: conda install -c conda-forge awscli, and that didn’t error out, but I don’t know why even after that in this venv awscli isn’t recognized, because during the install with conda-forge, I saw

Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment /Users/sabrinaliao/anaconda2/envs/fast_ai:

The following NEW packages will be INSTALLED:

    awscli:          1.11.120-py27_0    conda-forge
...

which implies to me that awscli was installed. How am I supposed to get awscli installed the right way?

Manishankar · October 3, 2017, 10:22am

Hi,
You should be getting your scripts from https://github.com/fastai/courses/tree/master/setup .
and yes, you need to wait till your p2 instance is approved (believe you’ve already raised the request as mentioned in the video)

Regards,
Manishankar

prav · October 3, 2017, 2:54pm

Thanks Mani,

But if I get it from git and run, I get error for newline and doctype etc.( I used wget and use the web-address of the file. Did I get the html page??)

Thanks

Manishankar · October 3, 2017, 3:13pm

May be. I’m not sure. for example, if you’re getting setup_p2.sh do not wget from the url https://github.com/fastai/courses/blob/master/setup/setup_p2.sh
click on that ‘Raw’ button and you’ll get just the script, the url for setup_p2.sh will be
https://raw.githubusercontent.com/fastai/courses/master/setup/setup_p2.sh - this is the url you should wget from.

prav · October 3, 2017, 5:10pm

Thanks Mani.

I created request for Amazon yesterday but they said that it still require some time. While the request is pending can we do the same setup on t2 instance. How did you progressed while waiting for approval.

Thanks

Vijay · October 4, 2017, 8:57am

I am facing AWS Setup issue.
Earlier i used with windows machine and able to setup AWS instance running perfectly I am able to run the examples without any issue.Now my hardisk got crashed…and i moved to macbook

Now I moved to Macbook and unable to setup AWS enviorment
Here are the details of the issue
While running the
bash setup_p2.sh

An error occurred (InvalidKeyPair.Duplicate) when calling the CreateKeyPair operation: The keypair ‘aws-key-fast-ai’ already exists.
Waiting for instance start…

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i .ssh/aws-key-fast-ai.pem ubuntu@ec2-35-160-100-145.us-west-2.compute.amazonaws.com

Instance started in AWS console

$ssh -i /.ssh/aws-key-fast-ai.pem ubuntu@ec2-35-160-100-145.us-west-2.compute.amazonaws.com
The authenticity of host ‘ec2-35-160-100-145.us-west-2.compute.amazonaws.com (35.160.100.145)’ can’t be established.
ECDSA key fingerprint is SHA256:2ZofqZB12g1u5IQtTOAjXTTKArQ7vTPqHiJW+T95n+8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘ec2-35-160-100-145.us-west-2.compute.amazonaws.com,35.160.100.145’ (ECDSA) to the list of known hosts.
Enter passphrase for key ‘/Users/vijaykumar/.ssh/aws-key-fast-ai.pem’:
Permission denied (publickey).

I have set the persmission 600 for .pem files

I have tried by creating new keypair but I am getting the same error.
please help… i can’t proceed got stuck in this

Mnemic · October 4, 2017, 8:59am

I was trying to install the nvidia drivers in my ubuntu partition, everything was going alright until I tried to use the command sudo modprobe nvidia and got this message “modprobe: ERROR: could not insert ‘nvidia_384’: No such device” how can I fix this?

iaming · October 5, 2017, 1:46am

the setup_ts.sh:

if [ $region=“us-west-2” ]; then
export ami=“ami-f8fd5998”;
elif [ $region=“eu-west-1” ]; then
export ami=“ami-9e1a35ed”;
elif [ $region=“us-east-1” ]; then
export ami=“ami-9c5b438b”;
else
echo “Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported”;
exit 1
fi

If you dont have us-east-1, why do you have these elif???

Manishankar · October 5, 2017, 6:22am

I’m not sure if a t2 x large is available to you already. give it a try, or if not try using t2 micro. I’m stuck with aws set up issue myself where I’m unable to ssh to the server.

sabliao · October 7, 2017, 4:29am

In case this helps anyone else, a friend helped me realize my n00b mistake: I need to use aws, not awscli. doh.

LarryL · October 12, 2017, 9:33pm

I am trying to do the setup. Nothing is working so far despite i’m a seasoned AWS user…shame on myself
I tried with my company account and my personal account…

I am trying to follow your video.
There’re several issues
1.
PengyuL@TTL001593 ~
$ aws configure
AWS Access Key ID [****************A]:
AWS Secret Access Key [****************ERib]:
Default region name [eu-west-1]:
Default output format [text]:

PengyuL@TTL001593 ~
$ bash setup_p2.sh
Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported

I tried to change the region name to us-west-2(Oregon), eu-west-1(Ireland), us-west-2
none of them is working. The errors are pop up saying
"Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported"

I have no access to platform.ai/files to download the files.

You don’t have access

Troubleshooting Info

User:xxxx@gmail.com
URL:https://www.platform.ai/files
If you should have access, please contact arshak@startup.ml and provide the troubleshooting info above.

If you’re signed in with multiple Google accounts, try a different account.

I manually setup the ami and instance and call the setup_instance.sh
but…

An error occurred (InvalidID) when calling the CreateTags operation: The ID ‘vpc’ is not valid

An error occurred (InvalidID) when calling the CreateTags operation: The ID ‘igw’ is not valid

An error occurred (InvalidID) when calling the CreateTags operation: The ID ‘sub’ is not valid

An error occurred (InvalidID) when calling the CreateTags operation: The ID ‘rtb’ is not valid

An error occurred (InvalidID) when calling the CreateRoute operation: The ID ‘rt’ is not valid

An error occurred (InvalidGroupId.Malformed) when calling the AuthorizeSecurityG"oupIngress operation: Invalid id: "sg-b33d25cb

An error occurred (InvalidGroupId.Malformed) when calling the RunInstances operation: Invalid id: "sg-b33d25cb
"

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start…

Waiter InstanceRunning failed: Waiter encountered a terminal failure state
usage: aws [options] [ …] [parameters]

could you please help?? many thanks.

heavy02011 · October 15, 2017, 6:50pm

Following the intro video I did

create user with administrator rights
aws configure

and got the following message / error

$ bash setup_p2.sh 
True

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair     'aws-key-fast-ai' does not exist

An error occurred (MissingParameter) when calling the CreateTags operation: The request must      contain the parameter resourceIdSet
 Waiting for instance start...

Any idea what I’m missing?

minimum · October 16, 2017, 6:24am

Hi, I’m having some trouble in the setup stage.
I’ve gotten my instance limit request approved, but had some issues with pip and cygwin so I switched over to bash on ubuntu on windows. Using that, pip install awscli worked fine, as did everything up until getting the setup_p2.sh script.

I’ve tried a couple different suggested url’s, but they all leave me with the same messages:

root@DESKTOP-V036MPQ:/mnt/c/Windows/System32# wget https://github.com/fastai/courses/blob/master/setup/setup_p2.sh
–2017-10-15 23:24:10-- https://github.com/fastai/courses/blob/master/setup/setup_p2.sh
Resolving github.com (github.com)… 192.30.255.113, 192.30.255.112
Connecting to github.com (github.com)|192.30.255.113|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
setup_p2.sh: Permission denied

Cannot write to ‘setup_p2.sh’ (Success).

Any assistance would be appreciated!

rishman · October 24, 2017, 1:24am

Hello,

I am have been struggling with setting up my AWS instance. I am getting the following error when running setup_p2.sh:

An error occurred (OptInRequired) when calling the CreateVpc operation: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.

Unlike the previous folks hitting this error, I am attempting this in the Unites States.

Thanks!

ctozzi · October 25, 2017, 8:26pm

Still getting an error with Theano:

WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available (error: Unable to get the number of gpus available: no CUDA-capable device is detected)

Tried clearing theano-cache then restarting jupyter, but no luck. Any ideas?

romulomadu · October 27, 2017, 7:01pm

Hi, I’m having this Error when I try to run the setup_p2.sh from ubuntu machine, anyone can help me?

n error occurred (VpcLimitExceeded) when calling the CreateVpc operation: The maximum number of VPCs has been reached.
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument

An error occurred (InternetGatewayLimitExceeded) when calling the CreateInternetGateway operation: The maximum number of internet gateways has been reached.
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --route-table-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --group-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --group-id: expected one argument
setup_p2.sh: linha 13: /home/romulomadu/.ssh/aws-key.pem: Permissão negada
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --subnet-id: expected one argument

An error occurred (AddressLimitExceeded) when calling the AllocateAddress operation: The maximum number of addresses has been reached.
Waiting for instance start…

Waiter InstanceRunning failed: Waiter encountered a terminal failure state
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --instance-id: expected one argument
securityGroupId=
subnetId=
instanceId=
instanceUrl=
Connect: ssh -i /home/romulomadu/.ssh/aws-key.pem ubuntu@