Reduce size of volume


(vedshetty) #1

Currently by default, the AMI sets the volume to 128 GB and I am looking to downsize it to something smaller like 64 GB. I have looked online and tried a couple of them but in the end I was unable to boot from the newly created volume. Since we all started using the same AMI to set this up, I was wondering if anyone has found an easy way to create a new volume synced from the old volume … and boot from the newly minted smaller volume.
I have looked here, here and here but I could not find any of them to work. Any pointers.


My experience keeping it on AWS free tier
(Jeremy Howard) #2

Maybe easiest just to create a new instance using the AWS Console (GUI), and select your preferred volume size there. Then copy across any files you need.


(vedshetty) #3

Great! There are several Linux AMI distros … do you have any particular recommendations or can I choose “Amazon Linux AMI 2016.09.0 (HVM), SSD Volume Type - ami-b04e92d0”


(Jeremy Howard) #4

@vshets you should use our class AMI, since they’re all set up for you. See the setup scripts to get the correct AMI id for each of the t2 and p2 instance types.


(vedshetty) #5

I tried that at first, but it complained that since the AMI (used for the class) is of size 128 GB, I cannot choose a size lower than that. My target size is 30 GB.


(Jeremy Howard) #6

Oh I didn’t know about that limitation. Can you copy and paste the exact error message?


(vedshetty) #7

Sure …


(vedshetty) #8

Hmm … now I think about it is because of the volume already available. May be I should create a new volume? The old one is called ‘Root’ and new volume will be called ‘EBS’.
Edit:I am unable to create an instance without having the root as well… only if there was a way to edit the root storage as well?


(Jeremy Howard) #9

@vshets one obvious solution is to just launch a standard Ubuntu AMI, and then run our script that sets it up from scratch: http://www.platform.ai/files/install-gpu.sh . Although I suspect there’s an easier way…


Problems with the install-gpu.sh script
(vedshetty) #10

Cool … i shall try that option. I am not very familiar with the nuances from about the distros: any chance you meant this:
Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-a9d276c9# or this
Ubuntu Server 14.04 LTS (HVM), SSD Volume Type - ami-01f05461

as the above two are the only ones available under Ubuntu. or does it not really matter?


(Jeremy Howard) #11

Yup one of them. I’d suggest 16.04 - no point using an outdated version! :slight_smile:


(vedshetty) #12

I am running into this error when i run the install gpu script:

tar (child): cudnn.tgz\r: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
: No such file or directory: cuda
cp: cannot stat 'lib64/*': No such file or directory
cp: cannot stat 'include/*': No such file or directory
install-gpu.sh: line 40: $'\r': command not found
install-gpu.sh: line 41: jupyter: command not found
install-gpu.sh: line 42: python: command not found
: No such file or directoryupyter/jupyter_notebook_config.py
: No such file or directoryupyter/jupyter_notebook_config.py
mkdir: cannot create directory ‘nbs\r’: File exists
install-gpu.sh: line 47: $'\r': command not found

Ran this command and got this:

dos2unix cudnn.tgz
dos2unix: Binary symbol 0x1F found at line 1
dos2unix: Skipping binary file cudnn.tgz

Now fixed thanks to @skh
dos2unix install_gpu.sh

and re run the bash install_gpu.sh command


(vedshetty) #13
Writing default config to: /home/ubuntu/.jupyter/jupyter_notebook_config.py
Enter password:
Verify password:
install-gpu.sh: line 43: .jupyter/jupyter_notebook_config.py: No such file or directory
install-gpu.sh: line 45: .jupyter/jupyter_notebook_config.py: No such file or directory

Which password is this referring to? Left it blank. I got the above error. Any pointers?


(Jeremy Howard) #14

@vshets this is the password that will be set for your jupyter notebook. I’ve set it to ‘dl_course’ for our default setups.


(vedshetty) #15

ok … thanks.
Any thoughts why I get the error at:

echo "c.NotebookApp.password = u'"$jupass"'" >> .jupyter/jupyter_notebook_config.py

c.NotebookApp.open_browser = False" >> .jupyter/jupyter_notebook_config.py

(Jeremy Howard) #16

It sounds like the line just above didn’t run for some reason:

jupyter notebook --generate-config

That should create the directory that’s missing. Oh I know - perhaps you’re not currently in your home directory? You run be running from within your home dir. So:

cd ~

(vedshetty) #17

When I do this:

ls /home/ubuntu/.jupyter/
jupyter_notebook_config.py

Thoughts?


(Jeremy Howard) #18

Right, but if you run the script when you’re not in the home directory, it won’t work, since the script has ‘.jupyter/jupyter_notebook_config.py’, instead of ‘~/.jupyter/jupyter_notebook_config.py’. So either cd to your home dir, or update the script to add ‘~/’ in both places.


(vedshetty) #19

Yes! It works. Thanks Jeremy!


(Jeremy Howard) #20

@vshets I think it’s awesome that you’re trying to get this working. Very bold! :slight_smile: