Reduce size of volume


(Swathi Shyam Sunder) #41

I actually deleted my previous instance and created this once freshly, just today.
Maybe I will redo the whole process again soon and update here !!


(vedshetty) #42

cool … also happy to walk you through doing it via aws console - less hassle IMO.


(lin.crampton) #43

Jeremy,

I’m working on setting up a different Ubuntu machine with a different Nvidia card. When I tried to use your install-gpu.sh on this new setup, it died on the first line of the script.
sudo apt-get update && apt-get --assume-yes upgrade

My install required that I sudo twice,
sudo apt-get update && sudo apt-get --assume-yes upgrade

In my setup, sudo only holds for the first command and I have to re-sudo the second, even if it is chained through a &&. For example:
sudo touch /iamsam && rm /iamsam
rm: remove write-protected regular empty file ‘/iamsam’? y
rm: cannot remove ‘/iamsam’: Permission denied
but the command works if I put a sudo before both commands. sudo touch /iamsam && sudo rm /iamsam works, but it requires a second sudo.

Is this due to a difference in our setups … do you really only have to sudo once?

lin


(Jeremy Howard) #44

Heh - curious. I’m pretty sure it worked for me as it’s shown there; but perhaps I just saved the commands wrongly. @vshets did you have to add the extra sudo there?


(vedshetty) #45

Nope … I dont remember having to modify this file … maybe @skh knows a thing about this behavior?


#46

Hi @vshets,

Thanks a lot for your effort in creating AMI with reduced size. However, for some reason I can’t access slack (I only tried today though) through the slack link (updated one) provided on forum. I wonder whether you could paste your solution steps on forum as well, since forum is the first place we are recommended to go.

Thank you for your hard work on this issue.

Daniel


(Jeremy Howard) #47

@vshets has an ‘s’ at the end of his name, so just fixing it here so he gets a notification…


#48

thanks Jeremy


(vedshetty) #49

Hi @Daniel … sorry for the late reply. I will have to rejig my memory to recall the exact steps but this thread from the start is an attempt to create an ami from scratch. Pls let us know if you are running into specific issues.


#50

Thanks @vshets


(NB) #51

Hi Daniel,

I was able to use a smaller EBS by following the following steps:

  1. Stop and terminate all EC2 instances. Then go to the EBS section and delete/detach all volumes.
  2. Get the latest scripts from the fastai github directory onto your local machine with git. This depends on how exactly you got the scripts from the fastai github in the first place. I use mobaterm to ssh into a permanently running linux VM, and used git fetch to update the repo. You could just re-download the zip from github if you are using windows.
  3. Copy either setup_p2.sh or setup_t2.sh to setup.sh (I did cp setup_t2.sh setup.sh on linux)
  4. Open setup.sh and change two lines – change ami to the new ami that @vshets created, and change the VolumeSize to 30.

See below for the diff (the lines starting with < are from the new setup file I created):

ubuntu@ip-172-31-13-157:~/courses/setup$ diff setup.sh setup_t2.sh
12d11
<    export ami="ami-9c54f4fc" # Oregon
56c55
< export instanceId=`aws ec2 run-instances --image-id $ami --count 1 --instance-type t2.xlarge --key-name aws-key-$name --security-group-ids $securityGroupId --subnet-id $subnetId --associate-public-ip-address --block-device-mapping "[ { \"DeviceName\": \"/dev/sda1\", \"Ebs\": { \"VolumeSize\": 30, \"VolumeType\": \"gp2\" } } ]" --query 'Instances[0].InstanceId' --output text`
---
> export instanceId=`aws ec2 run-instances --image-id $ami --count 1 --instance-type t2.xlarge --key-name aws-key-$name --security-group-ids $securityGroupId --subnet-id $subnetId --associate-public-ip-address --block-device-mapping "[ { \"DeviceName\": \"/dev/sda1\", \"Ebs\": { \"VolumeSize\": 128, \"VolumeType\": \"gp2\" } } ]" --query 'Instances[0].InstanceId' --output text`
ubuntu@ip-172-31-13-157:~/courses/setup$

The final step is to run bash setup.sh

Hope this helps you. The VM needs a minimum EBS of 30GB. I am in the us-west-2 region and it costs $3 USD a month.


Setup problems: AWS
Which steps would change if I use Papersace or Floydhub?
#52

Thank you very much @nunb, it works on my Mac as well.


(carlos roberto) #53

Hi @jeremy,
I am trying to access the mentioned AMI created to build a reduced volume but I cannot access the Slack. I tried to use the link mentioned by @rachel to receive an invitation but I never receive the link to slack in my email. So, I am struggling to try to create a new AMI. Any help on this?
Regards.


#54

It would be great if the AMI and smaller volume were part of the setup scripts in the repo


#55

@nunb I tried your steps and got the instance running. After ssh’ing in, I ran install-gpu.sh. However, running nvidia-smi fails. I get the message “NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.”

@jeremy any ideas?


(NB) #56

what type of instance are you using @stickperson?


#57

@nunb t2. Are p2 instances the only ones with GPUs?


(NB) #58

yes – you need a P2 instance to use the nvidia-smi command.

If you are using a T2 the nvidia command won’t work. you will have to configure keras to use cpu (it’s a one line change) and in general everything will run, but just slower. there is a procedure to request p2 instances … you have to open a case with AWS support.


#59

Ah, easy enough. Might give the t2 instance a try. If it’s less than 5x slower price wise it’s worth it.


(David Gutman) #60

For training it will be a lot slower than 5x. There is also the older G2 instance, but not that much cheaper and a much weaker GPU.