Secondary AWS instance guidelines?

Looks like AWS is going to be a part of our toolbox in future time…

Until now I’ve been using exclusively p2 instance but I think time has come to follow one of Jeremys advice on this: to create a secondary instance, cheaper, I’m thinking of m4.large but could also be t2. micro, I think. So…

-What AMI should we choose fot this m4? Same as for p2, fastai AMI? Not having a GPU not sure if fastai environment will work. Maybe just a plain ubuntu AMI?

-What should be the prefered way of exchanging files between this two instances?

Apart of this doubts (main one is about the AMI), any other hint by those more experienced on how to best setup this “tandem” of instances will be welcome!

2 Likes

I experienced AWS instance error, the us-west-2a region is not available for p2 instance. There is no option for me to change the Availability Zone. Due to availability issue of p2, I used p3 instead. I could launch a p3 instance which cost 3x more. However, there was a connectivity problem. I could not run the notebook nor see my files in AWS.

What can I do now?

The time has come to switch to Paperspace…
Or Spot Instances…

@Moody, Im a noob with AWS but by what I would do is, first, backup everything if you havent done already as soon as you can connect to that p3, and second… maybe another region for p2? I dont think there should be much downside of choosing another US, region or not specifying region, maybe a bit more lag?

Paperspace is not working for fastai library, at least their P5000 machines. A pity cause Im too looking forward to using them but just wanted to warn that that switching in this moment to a Paperspace P5000 is not going to work. :frowning:

2 Likes

@spisakjo Any suggestion from AWS prospective? I got the below response from the AWS account enquires who was not technical. If I switch to another zone, am I not going to have the similar problem?

"The reason you receive this error is that the availability zone (into which you are attempting to launch an instance) is out of capacity. For example, if you attempt to launch an ‘m1.xlarge’ instance in the ‘us-east-1a’ availability zone, but there are no more ‘m1.xlarge’ instances available for provisioning, you will receive an “insufficient capacity” error message.

It’s important to note that availability zones are not common to all accounts. i.e. The ‘us-east-1a’ availability zone for my account may be a different actual zone than the ‘us-east-1a’ availability zone for your account.

You can attempt to wait for some time before attempting to launch the instance again to see if capacity has since been made available.

To move to a new availability zone, you will need to create an AMI of your instance to launch into a new availability zone.

Here you can find useful information about this case:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-capacity.html

Also, we have the AWS Forums, where you can have support from our Engineers for free:

https://forums.aws.amazon.com

And if you would like to sing up for Premium Support, here is the info:

@Moody note that @spisakjo isn’t able to provide tech support here - but I’m happy to help!

As @miguel_perez said, you should go ahead and try another region. The capacity issue is specific to the region you’re using.

1 Like