Setup problems: AWS

@ethan I am no expert here but, the things what @jeremy suggested about redoing the setup worked for me. Originally I had problems with the ~/.ssh and @rachel 's suggestion below worked (after deleting the local key file)

cd ~
mkdir .ssh
chmod 700 ~/.ssh
Then try running the setup script again

Just in case this is helpful to you :slight_smile:

2 Likes

My g2.xlarge instance was just approved today. This is not the p2 instance, right?
The study group yesterday was really helpful and helped me with an initial set up on an m4.large instance so I could start reviewing lesson 1.

Can I migrate everything I started on the M4.large instance to the g2.xlarge? Or do I need to do all the installs again. ?

Thank you! I was able to get everything working, and am now trying to run Cats & Dogs!!! :smiley:

1 Like

Melissa even when they say that g2 is approved, they normally seem to approve p2 as well. So I’d try that first, since g2 is far from ideal for deep learning.

All the awscli stuff you installed on your own machine will still be fine, so you just need to run setup_p2.sh to create your p2 instance. Then you can copy over anything you want from your m4 instance - or better still, commit it to github then check it out on the p2.

1 Like

Hi @rachel! Thank you for answering me and @janardhanp22. Yes, I already set up an instance and started it. I sourced aws-alias.sh and ran all commands that you provided. I got an error message as seen from the screenshot; You have requested more instances (2) than your current instance limit of 1 allows for the specified instance type. Should I increase instance limit?

@analytics20132333 1 instance is enough. You are getting an error with aws-start because your instance is already running (as you can see in the browser console) and it thinks you are trying to launch an additional instance.

The ssh error is because you have the incorrect Public DNS (the number after ec2- doesn’t match what you see in the browser console). That is probably the DNS for an older instance that you deleted.

You could just type in the current number, or a better solution is to use:
aws-ip
ssh -i /home/rachel/.ssh/aws-key.pem ubuntu@$instanceIp
which sets the bash variable $instanceIp and then uses that to ssh

@janardhanp22 If you are starting over, re-watch the beginning of Jeremy’s AWS setup video. There were a few steps such as making yourself an admin (through the browser console), running aws-configure, etc. to make sure you have everything set up correctly.

Thank you, for the answer. Now, aws-start works and I can do ssh to the correct DNS! Yay!

I am still trying your better solution. I do not think I have aws-ip. How to install it? I think it is a basic question, but I am not sure how to. Thanks.

@analytics20132333 I’m glad it’s working! Sorry, I forgot to add, we added aws-ip to a more recent version of the aws-alias.sh file. You can get it by running the below (again):
wget http://www.platform.ai/files/aws-alias.sh
source aws-alias.sh

Cool! Thanks a lot for your consistent help! :slight_smile:

Hi, I have run the setup file and it worked well 3 days ago. Then I terminated the instance. Today I ran it again (bash setup_p2.sh), but after I tried to connect, it asked me to provide a passphrase. I don’t know what I should enter here. Thanks. The following is the script:

$ ssh -i /home/Xuemei/.ssh/aws-key.pem ubuntu@ec2-35-161-246-33.us-west-2.compute.amazonaws.com
The authenticity of host ‘ec2-35-161-246-33.us-west-2.compute.amazonaws.com (35.161.246.33)’ can’t be established.
ECDSA key fingerprint is SHA256:+HtFcttJwyog5T9R6IkHcNS3h+KYwNn7NiRkpP6ku+Y.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘ec2-35-161-246-33.us-west-2.compute.amazonaws.com,35.161.246.33’ (ECDSA) to the list of known hosts.
Enter passphrase for key ‘/home/Xuemei/.ssh/aws-key.pem’:

1 Like

@xchen somehow you have created a passphrase for your ssh key. The script we provided doesn’t add one, so you must have gone “off script” somewhere :slight_smile:

I suggest you remove everything and start afresh, if you can’t remember your passphrase for you ssh key

Once you successfully create a p2 instance, you shouldn’t need to run setup_p2.sh again.

I’m going to create a wiki on how to delete everything and start again, but I probably won’t have time for it until tomorrow.

Hello, while running the setup_p2.sh file I get the InstanceLimitExceeded error. I checked through the common problems and it says that, probably dont have any approved instances yet, but I checked and do have a p2.xlarge instance running. It says that I’m asking for more instances(2) than my limit(1).
I do receive an instanceUrl, but not an instanceId. My laptop went kaput, so I’m a bit lagging, but i hope to catch up soon. :slight_smile:

Thanks @jeremy and @rachel
Since I terminated the instance (rather than stop), I thought I would run setup_p2 again to have an instance. I can’t imagine why running it twice means I created a passphrase. I would like to start fresh and know exactly how.

@cmattey you already have an instance created. Go into the AWS console web page and terminate it. Then try again.

Thank you @jeremy , I was stopping the instance rather than terminating it, but now I run into the same problem of this prompt asking for a passphrase. Maybe I should start over as well.

Here is the problem I’m having…I had the initial Cygwin install, followed the instructions for fixing the problem – and ended with this. I’m really not sure what to do.

nicho@DESKTOP-3549AKI ~
$ pip install awscli
Collecting awscli
Using cached awscli-1.11.10-py2.py3-none-any.whl
Collecting s3transfer<0.2.0,>=0.1.9 (from awscli)
Using cached s3transfer-0.1.9-py2.py3-none-any.whl
Collecting docutils>=0.10 (from awscli)
Using cached docutils-0.12.tar.gz
0 [main] python2.7 7996 child_info_fork::abort: unable to remap termios.dl l to same address as parent (0x700000) - try running rebaseall
Error [Errno 11] Resource temporarily unavailable while executing command py thon setup.py egg_info
Exception:
Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/pip/basecommand.py”, line 215, in main
status = self.run(options, args)
File “/usr/lib/python2.7/site-packages/pip/commands/install.py”, line 310, in run
wb.build(autobuilding=True)
File “/usr/lib/python2.7/site-packages/pip/wheel.py”, line 750, in build
self.requirement_set.prepare_files(self.finder)
File “/usr/lib/python2.7/site-packages/pip/req/req_set.py”, line 370, in prepa re_files
ignore_dependencies=self.ignore_dependencies))
File “/usr/lib/python2.7/site-packages/pip/req/req_set.py”, line 601, in prep are_file
abstract_dist.prep_for_dist()
File “/usr/lib/python2.7/site-packages/pip/req/req_set.py”, line 127, in prep
for_dist
self.req_to_install.run_egg_info()
File “/usr/lib/python2.7/site-packages/pip/req/req_install.py”, line 425, in r un_egg_info
command_desc=‘python setup.py egg_info’)
File “/usr/lib/python2.7/site-packages/pip/utils/init.py”, line 678, in ca ll_subprocess
cwd=cwd, env=env)
File “/usr/lib/python2.7/subprocess.py”, line 711, in init
errread, errwrite)
File “/usr/lib/python2.7/subprocess.py”, line 1235, in _execute_child
self.pid = os.fork()
OSError: [Errno 11] Resource temporarily unavailable

I’m presently trying to uninstall everything and start completely over.

So I unistalled, reinstalled, and ended at the same problem which was this…
nicho@DESKTOP-3549AKI ~
$ aws
C:\Users\nicho\Anaconda2\python.exe: can’t open file ‘/cygdrive/c/Users/nicho/Anaconda2/Scripts/aws’: [Errno 2] No such file or directory

Then I used the problem solution from before - which is a long run of this…

awscli\help.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\paramfile.py’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\paramfile.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\plugin.py’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\plugin.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\schema.py’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\schema.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\shorthand.py’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\shorthand.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\table.py’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\table.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\testutils.py’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\testutils.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\text.py’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\text.pyc’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\topics\config-vars.rst’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\topics\return-codes.rst’) was not one of the expected responses: y, n
Proceed (y/n)? Your response (‘c:\users\nicho\anaconda2\lib\site-packages\awscli\topics\s3-config.rst’) was not one of the e

Which does not end with a prompt to do the next line. So I opened another window. went through the whole thing.

And ended back here again.

nicho@DESKTOP-3549AKI ~
$ pip install awscli
Collecting awscli
Downloading awscli-1.11.10-py2.py3-none-any.whl (1.0MB)
100% |████████████████████████████████| 1.0MB 848kB/s
Collecting s3transfer<0.2.0,>=0.1.9 (from awscli)
Downloading s3transfer-0.1.9-py2.py3-none-any.whl (53kB)
100% |████████████████████████████████| 61kB 3.1MB/s
Collecting docutils>=0.10 (from awscli)
Downloading docutils-0.12.tar.gz (1.6MB)
100% |████████████████████████████████| 1.6MB 574kB/s
0 [main] python2.7 6784 child_info_fork::abort: address space needed by ‘unicodedata.dll’ (0x1120000) is already occupied
Error [Errno 11] Resource temporarily unavailable while executing command python setup.py egg_info
Exception:
Traceback (most recent call last):
File “/usr/lib/python2.7/site-packages/pip/basecommand.py”, line 215, in main
status = self.run(options, args)
File “/usr/lib/python2.7/site-packages/pip/commands/install.py”, line 310, in run
wb.build(autobuilding=True)
File “/usr/lib/python2.7/site-packages/pip/wheel.py”, line 750, in build
self.requirement_set.prepare_files(self.finder)
File “/usr/lib/python2.7/site-packages/pip/req/req_set.py”, line 370, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File “/usr/lib/python2.7/site-packages/pip/req/req_set.py”, line 601, in _prepare_file
abstract_dist.prep_for_dist()
File “/usr/lib/python2.7/site-packages/pip/req/req_set.py”, line 127, in prep_for_dist
self.req_to_install.run_egg_info()
File “/usr/lib/python2.7/site-packages/pip/req/req_install.py”, line 425, in run_egg_info
command_desc=‘python setup.py egg_info’)
File “/usr/lib/python2.7/site-packages/pip/utils/init.py”, line 678, in call_subprocess
cwd=cwd, env=env)
File “/usr/lib/python2.7/subprocess.py”, line 711, in init
errread, errwrite)
File “/usr/lib/python2.7/subprocess.py”, line 1235, in _execute_child
self.pid = os.fork()
OSError: [Errno 11] Resource temporarily unavailable

@xchen Here is a wiki on how to start over with your AWS install. It is mostly screenshots right now: http://wiki.fast.ai/index.php/Starting_Over_with_AWS

2 Likes