View source | Discuss this page | Page history | Printable version   

Release Management/Managing Amazon EC2

Contents

Introduction

The Release Management Team is in charge of managing various servers and developer services on Amazon EC2. This document explains the basics behind this service and some management instructions.

Terminology


Introduction to Amazon EC2

Amazon EC2 is an easy to use and powerful service to have and manage your server infrastructures. You can install any Linux distribution, create an image AMI) out of it, boot instances from that AMI, manage individual firewall groups and much more, all on your own and without any constant assistance from the hosting provider.

There are mainly two management methods:

  1. Command line.
  2. Using a web console.

The web console is relatively new and doesn't have everything that the command line has. However, some specific operations that require a big money payment are now only available through the web console.

Installation of command line tools

These instructions assume you use A UNIX based OS, like GNU/Linux. First of all, install Ruby and Curl. Then, download and install the ec2-api-tools, used to stop/start/manage the EC2 resources:

$ cd /usr/local
$ wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
$ unzip ec2-api-tools.zip && rm -f ec2-api-tools.zip
$ ln -sf ec2-api-tools-* ec2-api-tools

And do the same with the ec2-ami-tools, used to create custom AMIs:

$ cd /usr/local
$ wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip
$ unzip ec2-ami-tools.zip && rm -f ec2-ami-tools.zip
$ ln -sf ec2-ami-tools-* ec2-ami-tools

Now add them to your user's PATH, as well as some additional required variables to your shell's init. For example, if you use Bash:

$ echo 'export EC2_AMITOOL_HOME=/usr/local/ec2-ami-tools' >> $HOME/.bashrc
$ echo 'export EC2_HOME=/usr/local/ec2-api-tools' >> $HOME/.bashrc
$ echo 'export PATH=$PATH:/usr/local/ec2-ami-tools/bin:/usr/local/ec2-api-tools/bin' >> $HOME/.bashrc
$ source $HOME/.bashrc

Configuration

In order to identify who you are, you need to specify a private key and a certificate. We add them as variables too:

$ mkdir $HOME/.ec2
$ echo 'export EC2_PRIVATE_KEY=/home/johndoe/.ec2/pk-52a7d7892f31482984054b6222196bd0.pem' >> $HOME/.bashrc
$ echo 'export EC2_CERT=/home/johndoe/.ec2/cert-52a7d7892f31482984054b6222196bd0.pem' >> $HOME/.bashrc
$ source $HOME/.bashrc

As the RM staff for these 2 files if you think you should have access to them.

And as we primarily use the European service:

$ echo "export EC2_URL='https://eu-west-1.ec2.amazonaws.com'" >> $HOME/.bashrc
$ source $HOME/.bashrc

Also you have to add to your ec2 user id to the bashrc. As a summary, your $HOME/.bashrc part of ec2 variables should appear like this:

export EC2_AMITOOL_HOME=/usr/local/ec2-ami-tools
export EC2_HOME=/usr/local/ec2-api-tools
export PATH=$PATH:/usr/local/ec2-ami-tools/bin:/usr/local/ec2-api-tools/bin
export EC2_URL='https://eu-west-1.ec2.amazonaws.com'

export EC2_PRIVATE_KEY=/home/johndoe/.ec2/pk-xxxxxxxxxxxxxx.pem
export EC2_CERT=/home/johndoe/.ec2/cert-xxxxxxxxxxxxx.pem
export EC2_USER_ID=xxxxxxxxxxxxx
export EC2_ACCESS_KEY=xxxxxxxxxxxxxxxxx
export EC2_SECRET_KEY=xxxxxxxxxxxxxxxxxxx

Finally, and the first time you start using EC2 you must create a keypair. This is used to access the started AMIs using SSH. Example:

$ ec2-add-keypair johndoe-eu

This will output a private key, which you can save in $HOME/.ec2/johndoe-eu.

Listing instances

To list the available instances:

$ ec2-describe-instances

To list a specific instance:

$ ec2-describe-instances i-xxxxxx

Starting a new instance

To start a new instance from a known AMI:

$ ec2-run-instances ami-xxxxxx

Frequent used arguments:

Terminating an instance

To terminate a running instance:

$ ec2-terminate-instances i-xxxxxx

WARNING: when you terminate an instance you loose all the information in that machine. Don't do it unless you know what you are doing.

Security groups

Amazon EC2 allows having an external firewall, other than the one that the operating system may include. This firewall rules are applied per groups, and one must select to which group and instance belongs to in the moment of starting it.

By default, there is a group called default and every instance is started there. Example of creating a new group, entering an instance in that group and managing the rules:

$ ec2-add-group myservice.openbravo.com -d "Security group for myservice"
$ ec2-start-instances i-xxxxxx -g myservice.openbravo.com
$ ec2-start-instances i-yyyyyy -g myservice.openbravo.com
$ ec2-authorize -p 80 myservice.openbravo.com
$ ec2-authorize -p 443 myservice.openbravo.com
$ ec2-authorize -p 22 -s 1.2.3.4/32 myservice.openbravo.com

We have create the group and started two instances inside that group. Then we have opened HTTP (80) and HTTPS (443) in that group, as well as SSH (22) for a specific IP address (1.2.3.4).


Elastic IPs

One of the nicest feature of EC2 regarding DNS management is called Elastic IP. This allows to allocate an IP to an EC2 account, and then assign it to one instance or to another on the fly. Example:

$ ec2-allocate-address
ADDRESS	1.2.3.4
$ ec2-associate-adddress 1.2.3.4 -i i-xxxxxx

This is useful to restore a machine very rapidly. If 1.2.3.4 is registered to a specific (sub)domain name, there is no need to change the DNS every time there is a disaster and the machine must be recovered in another instance. We can just reassociate the IP to another instance.

Regions: US/EU

EC2 offers two possible regions, the United States (US) and Europe (EU). By default the command line tools will take for granted you want to use them in the US. To use them in EU, export the EC2_URL variable:

$ export EC2_URL=https://eu-west-1.ec2.amazonaws.com

We mostly use it in EU.

Amazon Elastic Block Storage (EBS)

Amazon provides block level storage volumes for use with Amazon EC2 instances. This storage is used for backing up current instance.

We can create EBS volume ranging from 1GB - 1TB and it costs $0.10/GB per month.

To create a new EBS volume of 20GB in Europe:

$ ec2-create-volume -s 20 -z eu-west-1b

Attach it on a device (like /dev/sdj) on an instance:

$ ec2-attach-volume vol-yyyyyy -i i-xxxxxx -d /dev/sdj

Format it:

$ mkfs.ext3 /dev/sdj

Mount it inside the instance and copy the required data to it.

Documentation

For more information about Amazon EC2:

Retrieved from "http://wiki.openbravo.com/wiki/Release_Management/Managing_Amazon_EC2"

This page has been accessed 12,286 times. This page was last modified on 9 November 2009, at 10:27. Content is available under Creative Commons Attribution-ShareAlike 2.5 Spain License.