Setup problems: AWS

Part 1 (2017)
296

Hey there, 2 things:

1: Starting over article, step 1 may be a typo?

In the “Starting Over” article, (which is super helpful, appreciate it!)
http://wiki.fast.ai/index.php/Starting_Over_with_AWS

the first instruction is to run rm ~/.ssh/aws-key.pem

The key that is generated for me is actually aws-key-fast-ai.pem

joshua@LAPTOP-C6H0BQOF ~/.ssh
$ ls
aws-key-fast-ai.pem  known_hosts

Not sure if this is a typo, or the script got updated to make a different key file name, but my computer can’t find the file ~/.ssh/aws-key.pem when trying to remove. This confused me for a while so just putting it here as an FYI, it was fixed by running

rm ~/.ssh/aws-key-fast-ai.pem

2. I have an issue with the key pair when running the script with my IAM user. I did give him Administrator access and programmer access (that was on the setup screen in AWS but not in the video)

This was fixed by configuring aws with my root access credentials, then running the script.

:confused: not sure why AWS doesn’t complain when done with IAM credentials in the video, and I would love to know why.

Key Pair Does Not Exist Issue

The script claims the key pair aws-key-fast-ai doesn’t exist, but running ls on the .ssh directory shows that it is indeed there. I’m stuck on the errors generated from running the script.

This problem only occurs when I try to run the script with the IAM user I created.

joshua@LAPTOP-C6H0BQOF ~
$ bash setup_p2.sh.1
True

Waiter InstanceRunning failed: Max attempts exceededling the RunInstances operation: The key pair 'aws-key-fast-ai' does not exist
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:meter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start...
  aws help
  aws <command> help
  aws <command> <subcommand> help
aws: error: argument --instance-id: expected one argument

An error occurred (MissingParameter) when calling the RebootInstances operation: The request must contain the parameter InstancesSet

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /home/joshua/.ssh/aws-key-fast-ai.pem ubuntu@None

The key pair shows up in ssh, so why is the script saying key pair doesn’t exist?

joshua@LAPTOP-C6H0BQOF ~
$ cd .ssh

joshua@LAPTOP-C6H0BQOF ~/.ssh
$ ls
aws-key-fast-ai.pem  known_hosts
2 Likes
298

Hi Josh,
Thank you for the suggestions! It has been solved and it seems the credentials were distorted after copying and pasting. I solved this by typing it letter by letter.

Now looking forward to starting working with it!

Thanks,
Cangxiong

299

Hi,

Did your problem solved?
If yes, were you able to ssh to the instance and have a terminal to it within Cygwin?

For me i was only able to see the instance terminal when establish ssh using Putty…

300

I installed the T2 AWS just fine and got it working. I got my P2 instance approved and went through the same steps to set it up as the T2 but now when I go through “aws-ssh”, I get this error:
Permission denied (publickey).

“aws-get-p2” and “aws-start” seem to work just fine.

Any ideas?

301

Hi
On Windows 7 with Cygwin, I have the same error (in french):

image.png741×244 15.6 KB

By the link to the setup for Europe (setup_p2_ireland.sh) leads to nowhere (404)

Thanks

302

FIXED:
I was able to solve my issue.
Turns out it is not a good idea to have cygwin and MINGW to be installed together.
As soon as i removed MINGW and reboot my computer i was able to successfully establish a ssh session with my AWS instance and to have a prompt from that remote instance.

How I found out it is the problem?
I just run the ssh command prefixed with “strace” command and i saw a lot of calls to MINGW libraries.

$ **strace** ssh -i "~/.ssh/aws-key-fast-ai-t2.pem" ubuntu@x.x.x.x

Hope it will be useful for some of you out there, cause i spent A LOT of hours figuring that out.

Good Luck!

303

Hi @jeremy and @rachel
I have a p2 instance on AWS set up by a DevOps person of my office .
He sent me a pem file along with a static IP .
I ssh to the instance , using the pem file and ran the script install_gpu.sh on my instance.

Later I started my jupyter notebook , by the command jupyter notebook , it started .

But when I tried running the notebook on my web browser , using the static IP i.e
wrote this to my browser ,
:8888/

it didnt start the notebook

I searched internet and also tried
:8888/
that didn’t work either

Please help so that my jupyter notebook gets started :slight_smile:

304

More info:

I can sign into the service just fine when I use “aws-get-p2” with the command it suggests:
ssh -i /Users/victor/.ssh/aws-key-fast-ai.pem ubuntu@ec2-xxxxxx.us-west-2.compute.amazonaws.com

However when I try to use aws-ssh, this uses `ssh -i ~/.ssh/aws-key.pem ubuntu@$instanceIp’

The difference is the aws-key.pem vs aws-key-fast-ai.pem. But if I try to use the fast-ai.pem, the aws-stop commands don’t work.

Any ideas why setup-p2 works but not the aws-ssh? Why are there two different .pem files?

305

Hi All,

I’ve setup the p2.xlarge instance, and it’s working like a charm.

However, I’d like to setup a t2.micro server to do testing on, as Jeremy mentioned.

Can I run the setup_t2.sh script, replacing t2.xlarge with t2.micro, or will I have to make a new instance myself and manually install anaconda/theano/keras/jupyter notebooks/git? Thanks!

1 Like
306

Hi,
As a Windows user, I also ran into the passphrase problem, and I see it is unresolved here. I managed to connect to my p2.xlarge instance using Putty: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/putty.html

307

Found the answer to these problems.

I was using a mix of the new setup_p2.sh and old setup_t2.sh files. The old one uses aws-key.pem while the new use the aws-key-fast-ai.pem.

308

@rachel While I was waiting for permission limit to be increased on p2 instances, I stepped through the setup_p2.sh script.

I found a small bug. In line 42 (creating a subnet). It looks like you copied the previous section and forgot to change internetGatewayId to subnetId.

Here is part of the git diff on that file after my change.
-aws ec2 create-tags --resources $internetGatewayId --tags --tags Key=Name,Value=$name-subnet
+aws ec2 create-tags --resources $subnetId --tags --tags Key=Name,Value=$name-subnet

It’s pretty minor, but I thought you would like to know.

I really appreciate your writing these scripts. I am brand new to AWS and going through your code has really helped me find out how to make it work.

I hope to get my p2 limit raised and start the course soon.

Thank you,
George

309

I use Cygwin on windows and my python is 2.7.

Whenever I use bash setup_p2.sh

I get the following messages.

setup_p2.sh: line 3: $’\r’: command not found
setup_p2.sh: line 7: $’\r’: command not found
setup_p2.sh: line 130: syntax error: unexpected end of file

311

Hi,
I was able to log in to the aws instance according to the aws setup video (around 9ish minutes in) but when I type the ‘nvidia-smi’ command at the prompt I’m told that nvidia-smi command not found. I have seen another instance of this on the forum but I couldn’t figure out what the solution was based on the reply, any thoughts as to what I need to install/run/do to correct this issue? Thanks.

312

Hey! When I run “bash courses/setup/setup_p2.sh”, I get the following output:

Thus far, I’ve tried to rerun the script on a new user, “rm ~/.ssh/aws-key-fast-ai.pem”, looking in the common errors wiki, and searching this forum for more advice. I haven’t yet tried a complete restart, or deleting the old instances. What do you recommend? Thanks!

313

Might have to do with the IAM user for your instance.

Try:

deleting your users and creating a new one (new access key and all)
from command line: run ‘aws configure’

then run the setup script. Also make sure you are using the most up to date .sh script from github. I would get the raw format (go to the github page for the setup file, click on the raw button and copy web address) and pull it in with wget

Not sure if that’ll fix things but good luck.

Josh

314

I had an issue with import theano giving this error:

WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available  (error: Unable to get the number of gpus available: no CUDA-capable device is detected)

However I fixed it by closing down jupyter and running theano-cache clear in the instance terminal.

Hope this helps anyone else who runs into this issue!

2 Likes
315

That looks wrong … where is the IP address? It should be a dotted set of numbers like this 34.102.44.12 so the URL bar should show http://24.291.33.18:8888/ – also note that earlier versions of jupyter would run on port 8080 and jupyter will tell you which port it is listening to when it starts (the port is the part after the colon so :8080)

316

This message is not related to the installation procedure, but I thought it’s important to share again.

For anyone if this is your first time using AWS pay service, make sure to check to see if an instance is running and if it does stop the process otherwise 0.90cent/hour charge rate will be accumulated.

317

This is a good advice. I would also like to point out that Amazon will charge you by rounding up to the nearest hour. I tried to set up some spot instances and when I could not connect, I shut them down after less than 5 minutes of running time. Amazon still charged for a full hour for each of my unsuccessful attempts.