Thanks for the advice!! I ended up having to use your suggestion, but changed my ami to be ami=“ami-bc508adc”. I also had to follow the starting over instructions (http://wiki.fast.ai/index.php/Starting_Over_with_AWS) but it worked out!
I have my GPU server now, but I set up my instance in Oregon despite being based in the UK. Will this be a problem?
So I tend to do all of my development through an online coding environment (Codeanywhere) and I was having trouble using it because the additional requirements of Anaconda including various libraries wouldn’t fit on my tiny container with a 2GB disk. I ended up with a convoluted process:
My PC -> Codeanywhere -> DigitalOcean Droplet -> AWS instance
But now I’ve got rid of the DigitalOcean Droplet because I see no reason why we need Anaconda or Cygwin on our local machines, surely we just need the aws CLI and that is enough to stop/start instances and SSH in. All the python-y behaviour is on the AWS instance. New structure:
My PC -> Codeanywhere -> AWS instance
Have I missed something I am going to trip up on later with this?
Hi @heteroskedastic as long as you aren’t going through any other servers elsewhere (which I expect not) the latency is probably acceptable. If you find it takes too long to get a response back from the GPU server command line or notebook I would consider moving it over to Ireland, but really I think you should be fine.
Thanks, Benjamin. How would you go about moving over to Ireland?
Hi! I’ve been fighting with the same problem for 2 days. That what helped me:
- I uninstalled cygwin and Anaconda and reloaded computer
- Installed cygwin and followed steps from here: http://thecoatlessprofessor.com/programming/installing-amazon-web-services-command-line-interface-aws-cli-for-windows-os-x-and-linux-2/
- Checked aws comand at cygwin and everything was Ok
- Installed Anaconda
- Rechecked aws comand at cygwin. Ok - again.
1 Like
Hi Tim,
To get going with Ireland:
- Save off any important data from your old instance
- Delete the instance and any Elastic IPs.
- Reconfigure your AWS CLI for Ireland by running
aws configureand changing your region toeu-west-1 - Rerun
setup_p2.shand this should set your instance up in Ireland, check on the AWS dashboard to be sure. - Copy your data onto the new instance
Hi @heteroskedastic, if you just got started, it’s probably easiest to destroy your Oregon instance and start over creating an Ireland instanct.
@Cherie I haven’t seen this error before. I’m not sure what would be causing it. I’m glad you found a workaround
To anyone having trouble with line endings when running the setup scripts: The problem may be in your Github autocrlf config (Github will change the style of line endings for you depending on how you have this set, which could cause an error). The line endings are set to unix format in the current github versions, so you may want to try:
git config --global core.autocrlf false
Read this page for more info.
Has anyone run into an AWS authentication error when trying to process the setup_t2.sh script?
This might have to do with my wifi switching to a secure login page in the middle of running the script…
I went to run the script setup_t2.sh The terminal ended up giving me an authentication error that I can’t seem to get around. I re-entered my credentials (which were copied into a notepad), and double checked them against my amazon account (still active)
I’m configured my AWS account to the [us-west-2] region because the script won’t run if other regions are specified (even though there is an AMI in Tokyo), from what I read it just means I would suffer greater latency but I believe that shouldn’t cause any issues running the script… perhaps I’m wrong on that.
This is what I ran (setup_t2.sh.1 was generated from wget to the raw github file)
bash setup_t2.sh.1
The result was an authentication error
joshua@LAPTOP-C6H0BQOF ~
$ bash setup_t2.sh.1
hostname 'ec2.us-west-2.amazonaws.com' doesn't match 'securelogin.arubanetworks.com'
hostname 'ec2.us-west-2.amazonaws.com' doesn't match 'securelogin.arubanetworks.com'
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
Could not connect to the endpoint URL: "https://ec2.us-west-2.amazonaws.com/"
hostname 'ec2.us-west-2.amazonaws.com' doesn't match 'securelogin.arubanetworks.com'
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
An error occurred (AuthFailure) when calling the AllocateAddress operation: AWS was not able to validate the provided access credentials
Waiting for instance start...
Waiter InstanceRunning failed: AWS was not able to validate the provided access credentials
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --instance-id: expected one argument
An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials
All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /home/joshua/.ssh/aws-key-fast-ai.pem ubuntu@
re-rerunning the script gave me a different authentication error, where AWS couldn’t validate my access credentials…
joshua@LAPTOP-C6H0BQOF ~
$ bash setup_t2.sh.1
An error occurred (AuthFailure) when calling the CreateVpc operation: AWS was not able to validate the provided access credentials
An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --vpc-id: expected one argument
An error occurred (AuthFailure) when calling the CreateInternetGateway operation: AWS was not able to validate the provided access credentials
An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
Anyone seen this issue before?
Thanks for the detailed advice @benjaminramsden. I’m wondering if I’d need to contact AWS to have them increase my limit for Ireland, because at the moment the increase I secured seems to be specifically for Oregon.
Could anyone describe the cost of running one of these AWS servers and the expected length of the course? I had assumed that they were available for free, but that doesn’t seem to be the case. Also, are there are any suggested ways to minimise the cost?
Hi,
has anyone come across the below error
bash setup_p2.sh
setup_p2.sh: line 3: $’\r’: command not found
setup_p2.sh: line 7: $’\r’: command not found
setup_p2.sh: /cygdrive/c/Program Files/Anaconda2/Scripts/aws: C:\Program: bad interpreter: No such file or directory
setup_p2.sh: line 119: syntax error: unexpected end of file
i got this error too, but when i compared the setup_p2.sh file in my local machine the one in the github, i quickly realized my local file download the file with html syntax too, so i copied the setup_p2.sh in the github and replace it and it works. so open both files and compare…
Hi everyone, the setup worked just nicely! I also want to gain acces to the AWS environment from another device so that I show it to my colleagues about this course.
I ran the bash setup_t2.sh on the new device and ssh -i /Users/xxx/.ssh/aws-key-fast-ai.pem ubuntu@ec2-xx-xx-xxx-xxx.eu-west-1.compute.amazonaws.com to start the instance, but I got the following error message:
The authenticity of host ‘ec2-xx-xx-xxx-xxx.eu-west-1.compute.amazonaws.com(xx-xx-xxx-xxx)’ can’t be established.
ECDSA key fingerprint is .
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘ec2-xx-xx-xxx-xxx.eu-west-1.compute.amazonaws.com’ (ECDSA) to the list of known hosts.
Enter passphrase for key ‘/Users/xxxxxx/.ssh/aws-key-fast-ai.pem’:
Permission denied (publickey).
Do you have any idea what I need to do in order to make it work?
Cheers, Pieter