Setup problems: AWS

There’s an issue in the aws_alias.sh file, it’s missing a single quote at the end of line 7.
Found, added a ' at the end, worked for me - but others might run into this too
Looks like someone already submitted a PR to fix: https://github.com/fastai/courses/pull/80/files

For what it’s worth - after a couple of days fiddling with cygwin, anaconda & Bash on Ubuntu on Windows, getting the same errors reported by you and others in this thread - trying the solutions proposed in Stack Overflow as well as here, without resolving the problem -

I gave up the effort on Windows 10, instead installed Linux Mint on VirtualBox on my Windows PC, and now have a working setup. Would it be worth you trying that approach?


Update: having satisfied myself that I could get a p2 instance running via VirtualBox, I tackled the native Windows installation again.

As noted elsewhere in this thread, the key instructions for enabling Awscli in Cygwin are in the course wiki at http://wiki.fast.ai/index.php/Awscli_in_cygwin. I’d tried them before without success. This time I deleted all AWS files in my Anaconda installation (in \Lib\site-packages and \Scripts) before running the awscli_in_cygwin instructions - success!

‘ssh’ was not recognised by Cygwin and had to be installed manually.

Finally, running setup_p2.sh was throwing up lots of error messages, so I followed the ‘Starting Over with AWS’ in the wiki (http://wiki.fast.ai/index.php/Starting_Over_with_AWS) to clear the crud from previous failures, and setup_p2.sh ran successfully. I can now start, connect & stop instances via Cygwin - hooray! Time to tackle Lesson 1…

1 Like

Thanks for this great advice! You’d mentioned editing the volume size to 30, and I am just starting the course. I made the change, deleted my original setup according to the wiki ‘start over’ instructions, and went back through the walkthrough I ran into this error:

$ bash setup_p2.sh
True

An error occurred (InvalidBlockDeviceMapping) when calling the RunInstances operation: Volume of size 30GB is smaller than snapshot ‘snap-2b6c1a18’, expect size >= 128GB

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start…

Waiter InstanceRunning failed: Waiter encountered a terminal failure state
usage: aws [options] [ …] [parameters]

Any ideas?

Hello,
I am having an issue where I have installed AWS on my windows 10 machine using pip but when I use the aws command in cygwin it says it cannot find it, even though the path it says it is looking in contains what it is looking for.

I have uninstalled any version I python I had before and re-installed anaconda with python 2.7. Nothing seems to work, have tried all the usual things like restarting and checking environmental variables.

The command prompt and windows powershell both recognize the aws command and can give me the version after installing it.
Thank you!
-Kyler

Sorry to have mislead you @jeh0753. I didn’t actually try that code and I guess that the setup.sh is only part of the process. I didn’t take into account that the AMI ‘snapshot’ was taken with a 128GB volume. Unfortunately I’m not expert with AMI and/or creating an image which starts at 30GB.

I’m going to go back and edit my post. Sorry that you had to be on the receiving end of my lack of knowledge.

Hi @jeh0753,

Try following the steps in this post to use a 30GB AMI volume.

1 Like

Hi @lhclack @connelly,

I don’t have Windows so I’m not sure if I can help, but it seems that aws is still pointing to the Anaconda installed Python rather than Cygwin. What is the output you get when you type aws --version and which python pip?

@lhclack What is the output when you execute type aws? Maybe all you need to do (after following the steps you described in the wiki) is to clear the cache: hash -d aws.

awscli installed fine however when I run aws configure I get the following error:

$ aws
C:\users\charles.fried\appdata\local\programs\python\python35\python.exe: can’t open file ‘/cygdrive/c/Users/charles.fried/AppData/Local/Programs/Python/Python3 5/Scripts/aws’: [Errno 2] No such file or directory

Any ideas?

C:\Users\charles.fried\AppData\Local\Programs\Python\Python35\Scripts is referenced in Path and aws.cmd is in the file.

Tried this solution with no luck: http://wiki.fast.ai/index.php/Awscli_in_cygwin

The error persists:

$ aws
C:\users\charles.fried\appdata\local\programs\python\python35\python.exe: can’t open file ‘/cygdrive/c/Users/charles.fried/AppData/Local/Programs/Python/Python35/Scripts/aws’: [Errno 2] No such file or directory

Edit: Got it working using the step in the wiki linked above, it’s important to manually delete the AWS file in the python directory whether it be Anaconda or Python in my case.

1 Like

Thank you for the reply. I ended up doing the sequence described here:
http://wiki.fast.ai/index.php/Awscli_in_cygwin

I had to do it twice and manually delete some aws files in the anaconda install location, the uninstall command did not seem to be working properly on aws.

I am able to proceed now.

1 Like

This is very helpful and settles my mind that I’m not crazy! I could definitely put linux on one of my computers. I used to dual boot and be primarily a linux user in the past but my work has brought me back to Windows and it’s been convenient enough that I just haven’t used linux in a few years. So that shouldn’t be a problem and might actually be more comforting to do this course on!

I’ll check again, but I believe the “which” brought me to the correct python, however I will definitely apply the cache clear and see if there’s anything else like manually deleting the aws directory like others mentioned. Worst case is I think I’ll try this setup from a linux machine. Thank you for your help!!

1 Like

For some reason when I run “jupyter notebook” in my Instance and go to the correct URL for my instance and port, I get a “This site can’t be reached” error. Any help is appreciated.

Hi @randy912,

Could you paste the output from the console from where you launched the notebook?

[I 20:11:52.399 NotebookApp] [nb_conda_kernels] enabled, 2 kernels found
[W 20:11:52.762 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 20:11:52.795 NotebookApp] ✓ nbpresent HTML export ENABLED
[W 20:11:52.795 NotebookApp] ✗ nbpresent PDF export DISABLED: No module named nbbrowserpdf.exporters.pdf
[I 20:11:52.798 NotebookApp] [nb_conda] enabled
[I 20:11:52.828 NotebookApp] [nb_anacondacloud] enabled
[I 20:11:52.830 NotebookApp] Serving notebooks from local directory: /home/ubuntu
[I 20:11:52.830 NotebookApp] 0 active kernels
[I 20:11:52.830 NotebookApp] The Jupyter Notebook is running at: http://[all ip addresses on your system]:8888/
[I 20:11:52.830 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Do you see any messages here when you try to connect? Do you have any other notebooks running?

No notebooks running, nothing in the JS console, just this

Can you try getting the IP by running this in your terminal

aws ec2 describe-instances --filters "Name=instance-id,Values=$instanceId" --query "Reservations[0].Instances[0].PublicIpAddress"

then copy and paste it into your browser with :8888 appended.

When I copy-paste and run that it says “None”, yet I can connect to my Instance just fine and run nvidia-smi, etc. I can also start jupyter notebook but it still doesn’t show up in the browser.

Did you use the setup scripts to set up the instance? Could you run the following and paste the output here:

aws ec2 describe-instances