Setup problems: AWS


(Girija) #359

Please help. I am getting the same error

$ ./setup_p2.sh
./setup_p2.sh: line 7: syntax error near unexpected token newline' ./setup_p2.sh: line 7:

I did exactly as taught in the video.

rgds,
girija


(Girija) #360

Hi,

Copied all the scripts in the folder from git and still getting the same error

$ export ami=“ami-bc508adc”
$ export instanceType=“p2.xlarge”
$ bash setup_instance.sh
setup_instance.sh: line 7: syntax error near unexpected token newline' setup_instance.sh: line 7:

rgds,
girija


(Anirudh) #361

I get the following error when I try connecting to my instance:
$ ssh -i /home/User/.ssh/aws-key-fast-ai.pem ubuntu@ec2-52-25-3-109.us-west-2.compute.amazonaws.com
-bash: ssh: command not found

Can someone please tell me what I’m doing wrong? Thank you in advance!


(Ximeng Yu) #362

I can’t find solution in Common Problems.Here’s my error.

$ rm ~/.ssh/aws-key.pem
rm: cannot remove ‘/home/admin/.ssh/aws-key.pem’: No such file or directory

$ bash setup_p2.sh
setup_p2.sh: line 7: syntax error near unexpected token newline' setup_p2.sh: line 7:


(aartika) #363

Hi i am getting the following error while setting up aws:

i tried pip install aws…but no help

Please help !!

Thanks


(Zarak) #364

Hi,

Can you try running this script:

#!/bin/bash
#
# Configure a p2.xlarge instance

# get the correct ami
export region=$(aws configure get region)
if [ $region = "us-west-2" ]; then
   export ami="ami-bc508adc" # Oregon
elif [ $region = "eu-west-1" ]; then
   export ami="ami-b43d1ec7" # Ireland
elif [ $region = "us-east-1" ]; then
  export ami="ami-31ecfb26" # Virginia
else
  echo "Only us-west-2 (Oregon), eu-west-1 (Ireland), and us-east-1 (Virginia) are currently supported"
  exit 1
fi

export instanceType="p2.xlarge"

If it works, then you should replace all the backticks with dollar parens the way I have done it here.


(Ximeng Yu) #365

Yes, thank you.It worked. But I get another error.
$ bash setup_p2.sh
True

An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair ‘aws-key-fast-ai’ does not exist

An error occurred (MissingParameter) when calling the CreateTags operation: The request must contain the parameter resourceIdSet
Waiting for instance start…

usage: aws [options] [ …] [parameters]
To see help text, you can run:

aws help
aws help
aws help
aws: error: argument --instance-id: expected one argument

An error occurred (MissingParameter) when calling the RebootInstances operation: The request must contain the parameter InstancesSet

All done. Find all you need to connect in the fast-ai-commands.txt file and to remove the stack call fast-ai-remove.sh
Connect to your instance: ssh -i /home/admin/.ssh/aws-key-fast-ai.pem ubuntu@None


(Zarak) #366

Is there already an aws-key-fast-ai.pem file in your .ssh directory?

$ ls ~/.ssh | grep pem

Can you also check the key-pairs you have on AWS.

$ aws ec2 describe-key-pairs


(segovia) #367

I am still struggling with setting up the AWS instance for my lesson 1. My problem is that Python can not find the utils module to import.

No module named utils

The ‘~.nbs’ directory is just empty.

And I suspect this is caused by incorrect AWS setup. Here’s what I have done:

  1. I put two .sh files (setup_p2.sh and setup_instance.sh) into my home directory first
  2. then I ran the bash setup_p2.sh to create the instance.

Am I doing anything wrong here? Maybe I should use the aws-alias.sh file too?


(Zarak) #368

If you’re able to SSH into your EC2 instance, then your setup was probably successful.

I think the nbs directory is empty by default. You can find the notebook files in the deeplearning1/nbs folder.


(segovia) #369

Thank you @z0k! I can ssh into the instance, but only found these four folders in the home directory:

anaconda2
downloads (which has a cuda folder and a .deb file for cuda installation)
git (which is empty)
nbs (which is also empty)

I do not see the deeplearning1 folder…


(Zarak) #370

Sorry, what I should have said is that you can clone the repo to get the files

git clone https://github.com/fastai/courses.git

Note that there are no data files - you have to download those separately from Kaggle (or here for the processed version of dogs versus cats).


(Des) #371

Hey guys!
I dont know, if it is a right topic, but how much money do you spent on servertime in order to complete the course?
I have only around 100~200$ avaiable, and woundering is it even worth to strat the whole thing…


(Rothrock) #372

My first month of the course was about $42 (U.S.), but I’ve only done the first 3 lessons. The most expensive part (unless you forget to shut off your p2!) is the EBS storage volumes which are created by default at 128GB. That was about half the expense.

So if cost is a concern (and when isn’t it, I mean really!) you can edit the t2 and p2 setup files to create smaller volumes to start with. There is almost no way to make them smaller, but they are easy to make bigger, so if you really get into it and need more space you can increase the size with a couple clicks.

EDIT 4/24/17
Evidently the Amazon Image Snapshot expects a partition of 128GB so you can’t just edit the setup.sh. Sorry for misleading folks. I would still recommend finding a way to start with less disk space if you can figure it out. ~~Look in the setup_instance.sh file and find the line ...{ \"VolumeSize\": 128... and edit that to a smaller number. You get 30 with the free tier total – so if you setup both t2 and p2 at 15 you won’t go over. Of course I don’t know if you can fit everything in 15 GB. ~~
End EDIT

One other trick is that the instances are only billed in full hour increments each time you start and stop. So if you start an instance run for 10 minutes, stop it, start it again, run for another 10 minutes, stop it again, you will be charged for two hours. So don’t do that!

Here are my recommendations based and things I wish I had known given my experience and my budget.

  • Edit volume size to 30, increase it if needed later.
  • Only set up a p2.xlarge. I ran t2 for 14 hours and p2 for 16 hours. To me it probably would have been worth $11.20 to just have the one instance to manage and it would have saved on other costs.
  • The elastic IP will add about $6 to your bill
  • Don’t start and stop your instance unnecessarily (but don’t forget to turn it off when you aren’t using it)
  • Some of the simpler examples like mnist are pretty reasonable on a laptop or desktop CPU, play with that to explore the layers and different model features if you can.

(Rothrock) #373

Oh and yes, IT IS WORTH IT! @jeremy and @rachel are brilliant and sharing practical real world experience and knowledge. They community here is awesome and so helpful. If you want to learn about this field which is shaping our world and future, then this is the place to do it!


(Zarak) #374

If you really want to slash your costs on AWS, consider spinning up spot instances as shown in this thread. I’ve been paying $0.15-0.20 per hour instead of $0.90 for on-demand instances. You can also dispense with the elastic IP if you find you don’t need it and want to pay less.

Also note that you’ll need to use a different AMI when setting up your EC2 instance with a smaller EBS size. If you do a search in this forum you should be able to find some more info on the topic.


(Jeremy Howard) #375

Thanks for the nice words! :slight_smile:


(Jason Carter) #376

Awesome tips! I was wondering the same (how to reduce server cost). I’ve just started the Lesson 1 homework and keeping a close eye on my server time. Only $7 CAD so far.

Still would have been nice to know these settings before setting things up, perhaps these tips can be added to the wiki to new comers.


(Wilson Mar) #377

At the bottom of the page http://course.fast.ai/lessons/aws.html
the URL of “this link” is
https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=us-west-2#Limits:

I would suggest that a note be added that, since we have an international audience,
that each select the appropriate AWS region at the upper right corner of the AWS screen.


#378

Hi I am not able to connect to jupyter notebook using the instanceURL:8888.

getting ERR_CONNECTION_TIMED_OUT.

Jupyter notebook is running successfully on the instance.

[I 16:30:10.813 NotebookApp] [nb_conda_kernels] enabled, 2 kernels found
[W 16:30:11.168 NotebookApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 16:30:11.202 NotebookApp] ✓ nbpresent HTML export ENABLED
[W 16:30:11.202 NotebookApp] ✗ nbpresent PDF export DISABLED: No module named nbbrowserpdf.exporters.pdf
[I 16:30:11.204 NotebookApp] [nb_conda] enabled
[I 16:30:11.234 NotebookApp] [nb_anacondacloud] enabled
[I 16:30:11.237 NotebookApp] Serving notebooks from local directory: /home/ubuntu
[I 16:30:11.237 NotebookApp] 0 active kernels
[I 16:30:11.237 NotebookApp] The Jupyter Notebook is running at: http://[all ip addresses on your system]:8888/
[I 16:30:11.237 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Help?