Script to provision AMI from base AMIs


#1

Is it possible to publish the script that provisioned the AMI’s the install script uses?

why is an AMI required? As far as I can see in the setup scripts, they only require Ubuntu 16.04 with Git installed, is this correct?

Thanks for the great resources / course / help! :pray:


(Tait Larson) #2

Did you look at the following?


#3

Hi, thanks for your answer. I did l look at these resources, my quesion is in regard to the base AMI used in the setup scripts.

I’m wondering why a pre-made AMI is required (e.g. ami-bc508adc in https://github.com/fastai/courses/blob/master/setup/setup_p2.sh#L8 ), and why can’t we just provision the entire server from one of the vanilla AMI’s Amazon offers.

Thanks again!


(Tait Larson) #4

Oh got it. My bad.

I’m sure you could setup your own AMI. I haven’t done this so I can’t speak to how hard it would be. You’re using Python and Keras as a layer on top of Theano and Tensorflow which is a layer on top of CUDA / GPU programming. I’m not sure if a few sudo apt-get commands and a large EBS volume get you all the way there but it seems like it would.

I think the AMI makes it easy to ensure everyone is using the same libraries on top of hardware that’s been tested and works with the iPython notebooks. Sure, you could do this with Chef, bash or whatever else you’d for configuring a box but I think the AMI approach was simpler and less error prone.

FWIW, part one of the course uses Keras 1.x and Python 2.7. Keras has since shipped version 2.0 and part two of the course uses Python 3.


#5

Great, thanks for your reply.

The install-gpu.sh script (https://github.com/fastai/courses/blob/master/setup/install-gpu.sh) takes care of installing nvidia gpu drivers, Anaconda (includes python), theano and keras.

That’s why I’m wondering what is the need for the AMI :thinking: … AFAICT it’s only Ubuntu with Git…

I will give the setup scripts a try on a vanilla Amazon Ubuntu AMI. If I find anything interesing, I’ll update this post.

Thanks again for your reply! :smile:


(Tait Larson) #6

Ah, I see. Does another script tie the large EBS volume to the AMI or is that handled when the AMI is created? I can’t remember how AWS does this. If its added after the fact via another script I’m all out of reasons why a custom AMI is needed :slight_smile:


#7

Yes, the infrastructure-setup script ties a generic EBS to the instance: https://github.com/fastai/courses/blob/master/setup/setup_instance.sh#L64

I’m all out of reasons too, thats why I started the thread :slight_smile: thanks for your help! :smile:


(Pierre Guillou) #8

Hi @guatebus, I published a post that solves the question “how to setup fastai in an Amazon AWS region without fastai AMI like in Europe” .