Hey everyone,
Started this course this week, I’ve been having trouble setting up AWS in general. But I’ve finally managed to get an instance running! However, I still have a few problems and questions:
*When I access the instance (t2.xlarge) it is extremely slow, everything from simple ls or cd commands to running the initial jupiter notebook command. Is there a reason for this?
thanks!
*Its also been mentioned that I would have to apply for a p2 instance? (running the t2 script currently) How do I go about getting the p2?
*Finally I’m wondering how to efficiently transfer data to an instance. For example the dog-cat data needed for lesson one. Will I have to copy it to the instance every time I want to work with that data? And also will I get charged for transferring data to the instance?
Thanks!
Did you watch the video linked from the wiki? There’s an overlay that mentions how you request access to a p2.xlarge instance
Regarding data persistence, I think it depends on what kind of instance you setup. You can read about the differences between a normal instance and a spot instance here. There are instructions for making data persist across spot instance usages. That’s not a feature from Amazon and is something you need to setup on your own.
There have been some threads about persisting data across usage periods of spot instances. Here’s one thread from Feb. 25 and one from March 17. I’m not sure if these are the same methods discussed in the Wiki or not.
Regarding slowness of t2 there could be numerous reasons, from you being “lucky” to be placed on coincidentally problematic AWS host (which is rare) to exhausting CPU credits. t2 is burstable type of instance, so when you utilize CPU on t2 for enough period of time - it will start to throttle CPU. For t2.xlarge it is around 5h24m of 100% CPU utilization.
Another possible reason if you started instance from custom AMI, EBS snapshots are stored on S3 and blocks are not restored back to EBS until you first read them (mean read the files on your EC2 box), therefore at first there is a noticeable slowness until all blocks will be read at least once. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html
Thanks for the reply. That makes sense. I’ve just followed Robs advice and sent off an application for the P2 instance. Will I have the same issues? I guess I probably will as it is a custom AMI. Is an EBS what I need to effectively store my data. Or will I just have to secure copy to the instance every time? Also will I be charged for copying to an instance even if I terminate after I’ve finished?
@blair The effectiveness would really depend on how much data you will be storing and usage pattern of it.
You can checkout EFS option and this thread as well.
If I understand your correctly - you want to snapshot and terminate instance every time, in this case - yes you have to pre-warm EBS each time you start instance.
But you can just stop it while not using it, change instance type and start it when needed. EBS volume in this case remains on the same place and no prewarming needed. Yes you will be charged if you go beyond free tier on EBS Gb/months.
P2 does not have any CPU usage limitations, but EBS behavior still applies.
Ok i think I understand… maybe. I really don’t want to secure copy the datasets and notebooks every time I want to work on the course. You mentioned that I can stop the p2 instance when not using it, but surely that will still cost me money? Do I set up an EBS and run it constantly and then connect it to my p2 instance when ever I spin it up? Sorry, as you can tell I really have no clue what I’m doing here.
@blair when you stop instance, you don’t get charged for actual instance run time, but yes, you will still get charged for EBS volumes.
However, there is a free tier, which may allow you to avoid EBS charges completely. You can read about free tier here https://aws.amazon.com/free/faqs/?ft=nf
Monthly Calculator