Setup problems: AWS


(grovalmitch) #817

More on…python


(mahesh) #818

When I using this method I encountered that error so you should have to follow this guidance.thevideo/pair


#819

Sorry I’m just now getting back - I just contacted AWS and they got everything set up for me - something on the backend


(Renato Wagner) #820

Can anyone help me, please. I am new here and tried to read here and solve but I failed. Thank you for your time:


(Renato Wagner) #822

Could someone help me, please?


#823

Hello, I am from near London, so was looking for the “setup_p2_ireland.sh” on the GitHub repo, but unfortunately cannot find it anywhere. Has it been removed?


#824

The ~/.ssh/aws-fast-ai.pem file is empty, and I am unable to find or don’t know where to look in the aws console for the key either. I have been Googling around, and getting this page on the top most of the time.


(Alron) #825

Hello, I am also getting the same error. Have you found a fix to this? Can anyone help? :slight_smile:

P.S. I’ve also requested AWS to increase service limit to allow me to use P2, and they’ve already approved my request.


#826

You can access the keys on the EC2 dashboard.

You are using Linux, right? And are checking for the .ssh folder locally on your computer and not on the remote machine?

I am sorry I cannot be of more help but I use a different setup and have some AMIs / volumes configured and thus don’t want to tear down my env which I would need to do to troubleshoot this. Maybe someone following the canonical way of doing this for the course could chime in.

I follow this setup as described here but only look to it as the last resolve - hopefully someone who uses the default scripts will be able to help you troubleshoot further.


#827

Hi all, does anyone use crestle https://www.crestle.com/faq ? I’m interested how it works in compare to AWS and are there any issues with docker? Thanks u very much!


(Fernando Santos) #828

I guess the AWS agreement is still on, I got this limit increase not so long ago, but there are some other alternatives to get a running GPU machine in a fast way, check https://www.floydhub.com/ or https://www.paperspace.com/


#829

I’m going to use crestle.com, there is a docker repo designed to courset(thanks to mr. Anurag Goel): https://hub.docker.com/r/deeprig/fastai-course-1/


(Thomas Wallace) #830

To save some money, I’ve always used aws spot instances. Here in the eastern US, specifically in zone 1c, prices rarely go above $0.25/hr. The downside is I sometimes get kicked off if prices rise. That hasn’t happened yet with a p2 instance (~20 hrs usage).

Below is the command I run:

aws ec2 request-spot-instances --spot-price 0.30 --instance-count 1 --type “one-time” --launch-specification file://./p2.json

json (sorry couldn’t upload):

{
“ImageId”: “ami-31ecfb26”,
“KeyName”: “xxxxxxxxxx”,
“InstanceType”: “p2.xlarge”,
“NetworkInterfaces”: [
{ “DeviceIndex”: 0, “SubnetId”: “subnet-xxxxxxx”, “Groups”: [ “sg-xxxxxxxxx” ],
“AssociatePublicIpAddress”: true
}
],
“BlockDeviceMappings”: [
{
“DeviceName”: “/dev/sda1”,
“Ebs”: {
“Encrypted”: false,
“DeleteOnTermination”: true,
“VolumeSize”: 128,
“VolumeType”: “gp2”
}
}
],
“Placement”: { “AvailabilityZone”: “us-east-1c” }
}

I also hacked install_gpu. I ran into an issue I think because cuda 9.0 is the current version as of a couple of days ago. So I eliminated ‘sudo apt-get upgrade’ as well as the cuda driver and anaconda installs, since those two items are already installed on the ami.

For now, I keep the server in an ‘old’ state so I can concentrate on the deep learning.


Setup problems: Azure
(Sandeep Reddy) #831

Hi, I have started this course today and am very excited. Following exactly what was told in the video walkthrough, I am getting the following error while trying to setup a t2 server. My user credentials are correct and I’ve tried with two users. My default location is set to us-west-2 (I’m in India but I’m ok with the extra latency). Can someone help me proceed forward?

An error occurred (AuthFailure) when calling the CreateVpc operation: AWS was not able to validate the provided access credentials

An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument

An error occurred (AuthFailure) when calling the CreateInternetGateway operation: AWS was not able to validate the provided access credentials

An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --internet-gateway-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument

An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument

An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --route-table-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --route-table-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --vpc-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --group-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --group-id: expected one argument
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --subnet-id: expected one argument

An error occurred (AuthFailure) when calling the CreateTags operation: AWS was not able to validate the provided access credentials

An error occurred (AuthFailure) when calling the AllocateAddress operation: AWS was not able to validate the provided access credentials
Waiting for instance start…

Waiter InstanceRunning failed: AWS was not able to validate the provided access credentials
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --instance-id: expected one argument

An error occurred (AuthFailure) when calling the DescribeInstances operation: AWS was not able to validate the provided access credentials

An error occurred (AuthFailure) when calling the RebootInstances operation: AWS was not able to validate the provided access credentials

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /Users/sandeep.reddy/.ssh/aws-key-fast-ai.pem ubuntu@

Thanks in advance.


(Sandeep Reddy) #832

I was able to resolve this issue by correcting the time set on my computer (which was off by 8min). Hope this helps someone. :slight_smile:


#833

For eu-west-1 (Ireland), in line 16 in the setup_t2.sh file, change to ami-9e1a35ed (from ami-f8fd5998 for us-west-2). That worked for me.

Best, Jasper


#834

For eu-west-1 (Ireland), in line 16 in the setup_t2.sh file, change to ami-9e1a35ed (from ami-f8fd5998 for us-west-2). That worked for me. Best, Jasper


#837

Hi - I have been having the same problem. Dont see a solution posted for it. Please if someone can let me know.
I get the following error:
’ is not validred (InvalidID) when calling the CreateRoute operation: The ID 'rtb-8ac560f3

Thanks


(Deepak) #838

Hi Jeremy/Rachel,

I am trying to do the setup for t2 instance but i am facing issue, while connecting to ssh command. I am getting resource temporarily unavailable message. Sorry, i was not able to find any relevant post regarding this.

root@01HW1102594:~# ssh -i /root/.ssh/aws-key.pem ubuntu@ec2-52-11-56-142.us-west-2.compute.amazonaws.com
ssh: connect to host ec2-52-11-56-142.us-west-2.compute.amazonaws.com port 22: Resource temporarily unavailable

Thanks!!
Deepak


(Manishankar) #839

Hello,
I’m unable to connect to the p2 instance through ssh. I’ve made sure the instance is running, went through the forum for possible solutions, tried terminating and starting properly a couple of times (released all resources as mentioned n wiki). there are a few posts on top where @rachel has advised on similar issues, but I’m not getting any errors here, simply connection timing out! I’m using the scripts from git (https://github.com/fastai/courses/tree/master/setup) . Any help is greatly appreciated. TIA.



Edit : tried the below using aws-alias.sh. tried with a t2 xlarge instance as restarts are cheaper.