Setup problems: AWS

Hi @heteroskedastic,

Running the setup script again would create a new server, so it’s not necessary. Did you happen to back up the ssh key-pair to your instance? If so, then you can simply ssh into your instance as before. If you don’t remember your elastic-IP, you can run this command:

$ aws ec2 describe-addresses

1 Like

Hi @z0k

Unfortunately, I didn’t back up my ssh keys. I’ve run the describe-address command, but I will still need to generate new ssh keys for AWS, presumably. Do you know how I would go about doing that?

I can see what looks like the relevant line in the setup_instance.sh script:

aws ec2 create-key-pair --key-name aws-key-$name --query 'KeyMaterial' --output text > ~/.ssh/aws-key-$name.pem
	chmod 400 ~/.ssh/aws-key-$name.pem

If I ran these lines, would it “just work”, or is there a bunch of other stuff that I’d need to do as well?

Unfortunately, recovering access to your instance may be a bit of work. You can find instructions on how to regain access in the AWS docs.

Alternatively, if you don’t mind losing all your work, you could start over and run the setup script again for a fresh server.

Looks like starting over is the only practical thing to do. Thanks, @z0k.

1 Like

Hi fast.ai Community,

I’m using the setup_t2.sh script while I wait for AWS to allow me to provision a p2 instance. I’m having some trouble installing Git on my t2 server. I’m trying to install Git so I can clone the repository and have access to the data and the notebooks on my instance. When I run sudo apt-get install git after SSH’ing into my instance successfully, I get the following error message:

Err http://us-west-2.ec2.archive.ubuntu.com/ubuntu/ trusty-updates/main git-man all 1:1.9.1-1ubuntu0.3
  404  Not Found [IP: 54.212.89.51 80]
Err http://security.ubuntu.com/ubuntu/ trusty-security/main git-man all 1:1.9.1-1ubuntu0.3
  404  Not Found [IP: 91.189.88.162 80]
Err http://security.ubuntu.com/ubuntu/ trusty-security/main git amd64 1:1.9.1-1ubuntu0.3
  404  Not Found [IP: 91.189.88.162 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/g/git/git-man_1.9.1-1ubuntu0.3_all.deb  404  Not Found [IP: 91.189.88.162 80]

E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/g/git/git_1.9.1-1ubuntu0.3_amd64.deb  404  Not Found [IP: 91.189.88.162 80]

Has anyone else been able to install Git on their instance? If so, can you let me know the steps you took or what I’m doing wrong?

Thanks!

I’m answering my own question in case anyone experiences the same issue. The setup_t2.sh script does not install Git, but the setup_p2.sh script does. This must’ve been omitted in the AMI that was provisioned for the t2 instance. So, if you’re like me and you want to try to get started with a t2 instance while you wait for amazon to allow you to provision a p2 instance, you have to download the notebooks yourself. The way I was able to do this was by running…

wget https://raw.githubusercontent.com/fastai/courses/master/deeplearning1/nbs/lesson1.ipynb

…after SSH’ing into the instance.

If you’re in the same state as me, I hope this helps.

Cheers,
Patrick

2 Likes

when configuring aws and I run the ‘setup_p2.sh’, I met this problem"

$ bash setup_p2.sh
True

An error occurred (InstanceLimitExceeded) when calling the RunInstances operatio                                                                                                                           n: You have requested more instances (1) than your current instance limit of 0 a                                                                                                                           llows for the specified instance type. Please visit http://aws.amazon.com/contac                                                                                                                           t-us/ec2-request to request an adjustment to this limit.

An error occurred (MissingParameter) when calling the CreateTags operation: The                                                                                                                            request must contain the parameter resourceIdSet
Waiting for instance start...

And I don’t know how to figure it out

Hi @libin_china,

Please see here for help with some common problems.

I set up my p2.xlarge in November and had been using it on and off until Mar 31 with no issue.

Since then, I have been leaving big CNNs running for fisheries, and when I return, jupyter is no longer running.
These are jobs that the machine has previously accomplished without jupyter dying.

Have tried rebooting instance.

  1. Has anyone else seen this.
  2. Does anyone know if there is a log where I can get an error message as to why jupyter crashed?

The only thing that has changed over time is that I have saved more files, so maybe something memory related is happening?

I have “pip install awscli”, but when I type aws or aws configre, there is always a wrong message like this, what does it mean “No such file or directory”?

I have tried the step in “Awscli in cygwin”, but it still doesn’t work, I really don’t know how to figure it out.

I have also met this syntax problem, what’s wrong with it ?

I found solutions in this link:http://wiki.fast.ai/index.php/Awscli_in_cygwin, I tried several times, and it works.

Hello, I’m having trouble executing setup_p2.sh. I made sure to also download setup_instance.sh and I have not seen my issue mentioned anywhere else in these forums or online. Any help is appreciated. Here is a screenshot of what happens when I execute setup_p2.sh. There are three screenshots worth but I am a new user and can only paste one image. Thanks in advance.

Randall, did you get AWS Access key and Secret Access key and input them using the AWS CLI?

Hi Patrick,

Yes I entered both keys.

A couple ideas to try to troubleshoot:

1.) Run aws configure again and re-copy in the Access Keys
2.) Make sure that the user of whomever is associated with those Access Keys has Admin policy permissions
3.) Try to run the setup_t2.sh instead to see if you get a different result
4.) Try to spin up an instance from the AWS console instead of using the shell scripts.

Thanks, Patrick. I already tried 1 and confirmed I gave the user admin access, will try 3 and 4 in about two hours when I’m home. Will report back. Thanks again

Edit:
So I got the same result when running setup_t2.sh, but when I tried to start an instant from the AWS console, I saw my account is not fully activated yet. I emailed AWS support about this but this is obviously the issue. Thanks again, Patrick.

Hello! I’m having the same “Connection timed out” problem. I tried starting over, and when that didn’t work, I changed the “Inbound” and “Outbound rules”. I also tried sourcing the aws-alias.sh file, but got the following error message:

-bash: aws-alias.sh: line 25: unexpected EOF while looking for matching `’’
-bash: aws-alias.sh: line 30: syntax error: unexpected end of file

So, I added an “’” to the end of line 7, which seemed to do the trick. Now running aws-get-p2 and aws-start work, but I’m still getting the “Operation timed out” error when I try to ssh in.
Any advice would be appreciated. Thanks in advance!

I am attempting to connect to my instance and get this error:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0755 for ‘/home/USER/.ssh/aws-key-fast-ai.pem’ are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key “/home/USER/.ssh/aws-key-fast-ai.pem”: bad permissions
Permission denied (publickey).

I have the AdministratorAccess policy setup for my testuser. How do I update the permissions?