Setup problems: AWS


(Anurag Goel) #484

Try this:

chmod 400 /home/USER/.ssh/aws-key-fast-ai.pem

(Zarak) #487

Hi @cghosh,

You also need to download the setup_instance.sh file.


(LaurenM) #493

I’m getting the same error. I tried Ubuntu Bash like Jeremy recommended and still get the error. I confirmed that both filepaths referenced exist. The only thing I can think is I’m using Python 3.5 on my Anaconda instead of 2.7 like suggested? I installed awscli with pip.


#494

I am trying to install on Windows with Cygwin. configuring AWS didn’t work so I used the fix here (http://wiki.fast.ai/index.php/Awscli_in_cygwin) but when I run “install apt-cyg /bin” I get:

install: cannot create regular file ‘/bin/apt-cyg’: Permission denied

Any help is appreciated.


(LaurenM) #495

Unfortunately, this did not work for me. I do think what they are trying to solve is the issue though. Unfortunately I don’t know enough programming to understand how to get it to work myself. Both in cygwin and ubuntu bash for Windows, I believe the path for the version of Python is incorrect. I need it to use the anaconda, but it’s defaulting to python in windows I think. This has happened to me before, but fortunately reinstalling Anaconda worked in that it asks you if you want it to trace the right path for you. I might try that next and see if it helps, but I’m not sure if it will


(LaurenM) #496

Getting same error as a few others (using both cygwin and ubuntu bash for Win10) – only when asking cygwin about aws, seems to be pointing to the same directory it’s telling me doesn’t exist:


(Jason Benn) #497

tl;dr: my CLI and my AWS dashboard appear to disagree about the state of my account, which is preventing me from running the setup_t2 startup script (I’m out of Elastic IPs and VPCs).

More detail:

awscli says I have two VPCs on my default profile, and 5 on my other profile (which was before I figured out how to set my default AWS profile):


I deleted the only VPC on my dashboard (which had a non-matching ID anyway) to see if it would affect the output of the CLI, and it did not:


So I figured I must be signed is as the wrong user, right? But on the dashboard I’m signed in to an account with one IAM user:


And that user has the same access key (and secret) that I’m using in my AWS CLI configuration.



I am thoroughly confused. Where are all of the VPCs that my CLI thinks exists but that I can’t see in my dashboard? Do I have another user with these credentials? Am I misunderstanding IAM? I followed the setup instructions in the YouTube video up until the setup_t2 step and my region is correct.


(Jason Benn) #498

Discovered the answer - my VPCs and EIPs did indeed exist under that account, I had just somehow created them in the wrong availability zones! It’s almost like each AZ is its own little AWS world.


(Evan) #499

There’s an issue in the aws_alias.sh file, it’s missing a single quote at the end of line 7.
Found, added a ' at the end, worked for me - but others might run into this too
Looks like someone already submitted a PR to fix: https://github.com/fastai/courses/pull/80/files


(Peter Blackledge) #500

For what it’s worth - after a couple of days fiddling with cygwin, anaconda & Bash on Ubuntu on Windows, getting the same errors reported by you and others in this thread - trying the solutions proposed in Stack Overflow as well as here, without resolving the problem -

I gave up the effort on Windows 10, instead installed Linux Mint on VirtualBox on my Windows PC, and now have a working setup. Would it be worth you trying that approach?


Update: having satisfied myself that I could get a p2 instance running via VirtualBox, I tackled the native Windows installation again.

As noted elsewhere in this thread, the key instructions for enabling Awscli in Cygwin are in the course wiki at http://wiki.fast.ai/index.php/Awscli_in_cygwin. I’d tried them before without success. This time I deleted all AWS files in my Anaconda installation (in \Lib\site-packages and \Scripts) before running the awscli_in_cygwin instructions - success!

‘ssh’ was not recognised by Cygwin and had to be installed manually.

Finally, running setup_p2.sh was throwing up lots of error messages, so I followed the ‘Starting Over with AWS’ in the wiki (http://wiki.fast.ai/index.php/Starting_Over_with_AWS) to clear the crud from previous failures, and setup_p2.sh ran successfully. I can now start, connect & stop instances via Cygwin - hooray! Time to tackle Lesson 1…


(Jake) #501

Thanks for this great advice! You’d mentioned editing the volume size to 30, and I am just starting the course. I made the change, deleted my original setup according to the wiki ‘start over’ instructions, and went back through the walkthrough I ran into this error:

$ bash setup_p2.sh
True

An error occurred (InvalidBlockDeviceMapping) when calling the RunInstances operation: Volume of size 30GB is smaller than snapshot ‘snap-2b6c1a18’, expect size >= 128GB

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start…

Waiter InstanceRunning failed: Waiter encountered a terminal failure state
usage: aws [options] [ …] [parameters]

Any ideas?


(Kyler Connelly) #502

Hello,
I am having an issue where I have installed AWS on my windows 10 machine using pip but when I use the aws command in cygwin it says it cannot find it, even though the path it says it is looking in contains what it is looking for.

I have uninstalled any version I python I had before and re-installed anaconda with python 2.7. Nothing seems to work, have tried all the usual things like restarting and checking environmental variables.

The command prompt and windows powershell both recognize the aws command and can give me the version after installing it.
Thank you!
-Kyler


(Rothrock) #503

Sorry to have mislead you @jeh0753. I didn’t actually try that code and I guess that the setup.sh is only part of the process. I didn’t take into account that the AMI ‘snapshot’ was taken with a 128GB volume. Unfortunately I’m not expert with AMI and/or creating an image which starts at 30GB.

I’m going to go back and edit my post. Sorry that you had to be on the receiving end of my lack of knowledge.


(Zarak) #504

Hi @jeh0753,

Try following the steps in this post to use a 30GB AMI volume.


(Zarak) #505

Hi @lhclack @connelly,

I don’t have Windows so I’m not sure if I can help, but it seems that aws is still pointing to the Anaconda installed Python rather than Cygwin. What is the output you get when you type aws --version and which python pip?

@lhclack What is the output when you execute type aws? Maybe all you need to do (after following the steps you described in the wiki) is to clear the cache: hash -d aws.


(Charles Fried) #506

awscli installed fine however when I run aws configure I get the following error:

$ aws
C:\users\charles.fried\appdata\local\programs\python\python35\python.exe: can’t open file ‘/cygdrive/c/Users/charles.fried/AppData/Local/Programs/Python/Python3 5/Scripts/aws’: [Errno 2] No such file or directory

Any ideas?

C:\Users\charles.fried\AppData\Local\Programs\Python\Python35\Scripts is referenced in Path and aws.cmd is in the file.


(Charles Fried) #507

Tried this solution with no luck: http://wiki.fast.ai/index.php/Awscli_in_cygwin

The error persists:

$ aws
C:\users\charles.fried\appdata\local\programs\python\python35\python.exe: can’t open file ‘/cygdrive/c/Users/charles.fried/AppData/Local/Programs/Python/Python35/Scripts/aws’: [Errno 2] No such file or directory

Edit: Got it working using the step in the wiki linked above, it’s important to manually delete the AWS file in the python directory whether it be Anaconda or Python in my case.


(Kyler Connelly) #508

Thank you for the reply. I ended up doing the sequence described here:
http://wiki.fast.ai/index.php/Awscli_in_cygwin

I had to do it twice and manually delete some aws files in the anaconda install location, the uninstall command did not seem to be working properly on aws.

I am able to proceed now.


(LaurenM) #509

This is very helpful and settles my mind that I’m not crazy! I could definitely put linux on one of my computers. I used to dual boot and be primarily a linux user in the past but my work has brought me back to Windows and it’s been convenient enough that I just haven’t used linux in a few years. So that shouldn’t be a problem and might actually be more comforting to do this course on!


(LaurenM) #510

I’ll check again, but I believe the “which” brought me to the correct python, however I will definitely apply the cache clear and see if there’s anything else like manually deleting the aws directory like others mentioned. Worst case is I think I’ll try this setup from a linux machine. Thank you for your help!!