Setup problems: AWS


(Rob Harrand) #707

Hi all,

I’ve been trying to create the P2 instance using Windows 10 and Cygwin for 2 days. I also tried installing Ubuntu bash to no avail.

After almost losing the will to live and having experienced just about every setup_p2.sh error under the sun, I installed a Linux virtual machine, copied the entire setup folder from the git repository to my (virtual) Ubuntu desktop, installed aws, then ran the aws configuration followed by bash setup_p2.sh from the linux command line. Everything worked like a dream, and now the P2 instance is up and running. If you’re in a similar situation, this approach may help.


Can't Run setup_p2.sh; Help With Bash
(Derrick Schultz) #708

Yep, this was really helpful. Thanks!


#709

samsonpaturi,

The error is suggestive of an error in the way ssh is invoked. In other words, one of the parameters is wrong.

Wild guess: there’s supposed to be a space after the ‘i’ in “-i/home/ss…”

If that’s not it, then closely review the ssh command line.

HTH,
JP


(SAMSON PATURI) #710

Hi JP, thanks for your suggestion. The issue is not the space, I tried that already. I get the same output if I simply type ssh command. I am basically stuck without any progress.


#711

Oh, that’s a bummer. Let’s keep trying:

Can you ssh successfully anywhere else with any set of parameters? Back at school or in some corporate setup, I had access to many servers to practice on. Nowadays, I’d Google a bit and/or would setup my own SSH server. This thread for example, looks like a promising source of test locations:

You need to isolate the fault. Is it with the specific AWS parameters? Is it a ssh client install issue? Funky interaction with your shell?

Also, are you able to determine exactly which ssh client is being called? My memory of which unix commands are available on Cygwin is fuzzy, but perhaps “which ssh” would do it.


(Ryan J Chase) #712

After a LOT of time spent on this, I finally found a solution that works.

The primary issue is that the cygwin didn’t come with python installed, and doesn’t know where to find the existing Windows Anaconda version on your machine. This can be verified by running which python from within cygwin - it couldn’t find where python is saved. Note that this can be confusing because running pip install awscli likely doesn’t throw an error message. Cygwin actually installs awscli in the Window’s Anaconda installation of Python (I find this odd since we didn’t run conda install awscli).

HOWEVER, rather than try to point cygwin to the already installed version of Anaconda python on your machine it will save you a ton of headache to just install a cygwin-specific instance of python.

The initial steps to run in cygwin are (documented here: http://wiki.fast.ai/index.php/Awscli_in_cygwin)

  1. pip uninstall awscli
  2. wget rawgit.com/transcode-open/apt-cyg/master/apt-cyg
  3. install apt-cyg /bin
  4. apt-cyg install python
  5. wget https://bootstrap.pypa.io/get-pip.py python get-pip.py pip install awscli

…Note, however, that the first command pip uninstall awscli “hung up” for me. So just escape out of it using quit() and continue with the others in order.

You can check that everything worked if you run which python in cygwin and it points to the cygin version (i.e. /usr/bin/python , as opposed to: /users/…/Anaconda2/).

The next CRITICAL step is that when you download all the shell scripts from Github setup folder (https://github.com/fastai/courses/tree/master/setup), Windows automatically CRLF line terminators! Therefore, in cygwin, run the following commands:

  1. apt-cyg install dos2unix
  2. dos2unix setup_p2.sh
  3. dos2unix setup_instance.sh
  4. then finally, bash setup_p2.sh

This should do the trick.


(Susan Li) #713

Hi, mchineLRN, I am having the same issue setting up, I ran wget https://github.com/fastai/courses/blob/master/setup/setup_p2.sh and wget https://raw.githubusercontent.com/fastai/courses/master/setup/setup_instance.sh but I am still getting
$ bash setup_p2.sh
setup_p2.sh: line 2: syntax error near unexpected token newline' setup_p2.sh: line 2:

Can you please let me know how did you solve this?

Thank you
Susan


(Susan Li) #714

Hi, I am having the same error,
$ bash setup_p2.sh
setup_p2.sh: line 2: syntax error near unexpected token newline' setup_p2.sh: line 2:

and I tried wget https://raw.githubusercontent.com/fastai/courses/master/setup/aws-alias.sh
wget https://raw.githubusercontent.com/fastai/courses/master/setup/setup_instance.sh
wget https://raw.githubusercontent.com/fastai/courses/master/setup/setup_p2.sh
with no luck. You said copy paste text on github, which text?
Thanks
Susan


#715

setup_instance.sh:
wget https://raw.githubusercontent.com/fastai/courses/master/setup/setup_instance.sh

should fix your problem if I remember correctly. Not sure the logic behind it but I installed it just playing around it worked for me.


(Susan Li) #716

Hi Josh, do you still remember how you solved this problem? my CAT command is full of HTML tags. What scripts did you use to make it right?

Thanks
Susan


(Zarak) #717

Hi @susanli,

Did you delete the original setup_instance.sh file? If not, each new file you download with wget is appended with a number.


(Susan Li) #718

Thank you, now it works like a dream!


(Susan Li) #719

Thanks for your help, now it works!


(George McIntire) #720

When I try to ssh into the AWS instance I keep getting this:

connect to host ec2-XX-XXX-XXX-XX.us-west-2.compute.amazonaws.com port 22: Operation timed out

How do I solve this?


(Zarak) #721

Hi @georgemcintire,

A few things come to mind:

  1. Is your instance running?
  2. Can you confirm that the IP address is correct?’
  3. Do the inbound rules in your security group have your local IP in the source list?

(Max0924) #722

Hi, I am having the same error,
$ bash setup_p2.sh
setup_p2.sh: line 2: syntax error near unexpected token newline’
setup_p2.sh: line 2:'
but
how to delete the original setup_instance.sh file?


(Zarak) #723

Hi @Max0924,

It should just be the first file you downloaded. You can find all of them by typing

ls | grep setup_instance.sh 

in the directory you downloaded the scripts. Once you remove the original file, you can use wget to get the proper version from the raw url.


#724

I am using the Ubuntu Bash on Win10 because for some reasons Cygwin does not recognize commands like “conda” or “pip” on my computer ( I tried to add Anaconda to .bashrc but it still didn’t work).

Everything went fine in Ubuntu Bash until I set up the instance and tried to connect to it.

So I SSH, and it said,

I removed everything and started over but still kept receiving “Permission denied (publickey).”

What can I do?

Help much appreciated.


(Zeinab Sadeghipour) #725

Hi,

I ran “bash setup_p2.sh” and I get the following error:

True

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair ‘aws-key-fast-ai’ does not exist

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start…

Waiter InstanceRunning failed: Max attempts exceeded
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --instance-id: expected one argument

An error occurred (MissingParameter) when calling the RebootInstances operation: The request must contain the parameter InstancesSet
All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /home/Zeinab/.ssh/aws-key-fast-ai.pem ubuntu@None

I checked and I know that the file “aws-key-fast-ai.pem” does not exist, but I don’t know how to create it. Any help would be appreciated.


(Elian) #726

Many thanks Max, worked out fine for me !