Setup problems: AWS

I successfully set up a t2 and a p2 instance on AWS, but I was wondering if I can point them both to the same storage volume (so I don’t keep copying files around). Is that possible and if so how would I set this up?

It is possible but not super straightforward I am afraid.

The easiest route is to create a separate volume and be attaching + mounting it as needed. Problem is, when you create an EBS volume you also need to partition it and make a filesystem on it…

You could google for the above actions or try to work out what needs to be done following my tutorial from step 11: https://becominghuman.ai/automated-aws-spot-instance-provisioning-with-persisting-of-data-ce2b32bdc102.

Actually, if you want to be attaching it manually, I think this howto by @slavivanov might be even better: https://blog.slavv.com/learning-machine-learning-on-the-cheap-persistent-aws-spot-instances-668e7294b6d8

Good luck! :slight_smile:

thanks @radek

Hi. I’d like some help with my setup.

Firstly, I have a setup with nvidia gtx 1060 (zotac) with 6 gb vram, 8gb ram and a core i3-6100 processor. Is it possible to create a server using my own pc? Or will be more of a hassle than to have an AWS account?

Also, being from India, should I still opt for US (West)?

Any guidance is appreciated.

If your nvidia card supports CUDA (quite sure it does) you should be good to go.

Are you already running linux? If yes, then getting set up for the course will be relatively easy.

Picking the AWS region will only impact prices (they vary by region) and latency to the server - I would pick the location closest to you that has GPU units. For me, I have to go across the entire Europe to get to my AWS region, but the latency for typing etc is unnoticeable.

Good luck with the setup!

Yes, I am running Linux and I just found a guide here to setup:http://forums.fast.ai/t/making-your-own-server/174/195

Just wanted to clarify: Does having AWS account make the course easier to follow than rolling your own setup?

If you have the hardware available (like you do) probably running this on local machine is actually easier.

1 Like

I have downloaded the setup files from github and tried bash setup_p2.sh. I get errors that I don’t fully understand why they are happening. I’m new to AWS and will appreciate any help. I am pasting the error below:

###Begin paste
bash setup_p2.sh
True

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair ‘aws-key-fast-ai’ does not exist

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start…

Waiter InstanceRunning failed: Max attempts exceeded
usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --instance-id: expected one argument

An error occurred (MissingParameter) when calling the RebootInstances operation: The request must contain the parameter InstancesSet

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /home/venkatesh/.ssh/aws-key-fast-ai.pem ubuntu@None

###End paste

Hi jeeyung,
How did you solve this problem? I’m having the same issue.
Thanks.

Where are you pasting this? This looks like a file that should be run from console at some point during setup (after aws-cli configuration).

Please refer to the Getting Started instructions on the course page. Here you can find a video with a walkthrough

Thank you for the reply.

I am not pasting this – this is the error message I get. I get this message on running bash setup_p2.sh that I downloaded from GitHub page given in here –http://course.fast.ai/start.html after following the instructions given in this introduction video.

Ok, sorry, I misread your post.

The first error that you are getting is this:

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair 'aws-key-fast-ai' does not exi

I think it is raised by this part of the code:

if [ ! -d ~/.ssh ]
then
	mkdir ~/.ssh
fi

if [ ! -f ~/.ssh/aws-key-$name.pem ]
then
	aws ec2 create-key-pair --key-name aws-key-$name --query 'KeyMaterial' --output text > ~/.ssh/aws-key-$name.pem
	chmod 400 ~/.ssh/aws-key-$name.pem
fi

export instanceId=$(aws ec2 run-instances --image-id $ami --count 1 --instance-type $instanceType --key-name aws-key-$name --security-group-ids $securityGroupId --subnet-id $subnetId --associate-public-ip-address --block-device-mapping "[ { \"DeviceName\": \"/dev/sda1\", \"Ebs\": { \"VolumeSize\": 128, \"VolumeType\": \"gp2\" } } ]" --query 'Instances[0].InstanceId' --output text)
aws ec2 create-tags --resources $instanceId --tags --tags Key=Name,Value=$name-gpu-machine

This is in setup_instance.sh which is called by the setup_p2.sh that you are running.

Can you check what keys you have in ~/.ssh? Could you also log into console on AWS and check the key pair named aws-key-fast-ai exists?

@jnastaskin - are you still experiencing this problem or did you find a solution?

I am using a different setup on AWS so unfortunately can’t run this myself to verify what’s happening there…

BTW this is the description of the error code from here:

The specified key pair name does not exist. Ensure that you specify the region in which the key pair is located, if it's not in the default region.

One guess just came to my mind now that I think of it!!! Maybe you have the key already in ~/.ssh and it is not being uploaded to AWS?

From your terminal please execute rm -f ~/.ssh/aws-key-fast-ai* and give it another go :wink:

Hi, i want to use my own PC as a server, but i don’t really want to install linux in dual boot for now, are there any guides how to set up a server on Windows 10? Thanks

More on…python

When I using this method I encountered that error so you should have to follow this guidance.thevideo/pair

Sorry I’m just now getting back - I just contacted AWS and they got everything set up for me - something on the backend

Can anyone help me, please. I am new here and tried to read here and solve but I failed. Thank you for your time:

Could someone help me, please?

Hello, I am from near London, so was looking for the “setup_p2_ireland.sh” on the GitHub repo, but unfortunately cannot find it anywhere. Has it been removed?

The ~/.ssh/aws-fast-ai.pem file is empty, and I am unable to find or don’t know where to look in the aws console for the key either. I have been Googling around, and getting this page on the top most of the time.