Release Management/Managing Amazon EC2
Contents |
Introduction
The Release Management Team is in charge of managing various servers and developer services on Amazon EC2. This document explains the basics behind this service and some management instructions.
Terminology
- EC2: short name for Amazon Elastic Compute Cloud, a service that provides resizable compute capacity in the cloud. More. Machines are used in this service.
- S3: a storage service in the cloud.
- AMI: stands for Amazon Machine Image and it's an operating system image stored in Amazon S3. It contains all the information necessary to boot instances of your software.
- 'Instance: once an AMI has been started, the resultant running system is called instance. There can be many instances launched from the same AMI ID.
- more
Introduction to Amazon EC2
Amazon EC2 is an easy to use and powerful service to have and manage your server infrastructures. You can install any Linux distribution, create an image AMI) out of it, boot instances from that AMI, manage individual firewall groups and much more, all on your own and without any constant assistance from the hosting provider.
There are mainly two management methods:
- Command line.
- Using a web console.
The web console is relatively new and doesn't have everything that the command line has. However, some specific operations that require a big money payment are now only available through the web console.
Installation of command line tools
These instructions assume you use A UNIX based OS, like GNU/Linux. First of all, install Ruby and Curl. Then, download and install the ec2-api-tools, used to stop/start/manage the EC2 resources:
$ cd /usr/local $ wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip $ unzip ec2-api-tools.zip && rm -f ec2-api-tools.zip $ ln -sf ec2-api-tools-* ec2-api-tools
And do the same with the ec2-ami-tools, used to create custom AMIs:
$ cd /usr/local $ wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip $ unzip ec2-ami-tools.zip && rm -f ec2-ami-tools.zip $ ln -sf ec2-ami-tools-* ec2-ami-tools
Now add them to your user's PATH, as well as some additional required variables to your shell's init. For example, if you use Bash:
$ echo 'export EC2_AMITOOL_HOME=/usr/local/ec2-ami-tools' >> $HOME/.bashrc $ echo 'export EC2_HOME=/usr/local/ec2-api-tools' >> $HOME/.bashrc $ echo 'export PATH=$PATH:/usr/local/ec2-ami-tools/bin:/usr/local/ec2-api-tools/bin' >> $HOME/.bashrc $ source $HOME/.bashrc
Configuration
In order to identify who you are, you need to specify a private key and a certificate. We add them as variables too:
$ mkdir $HOME/.ec2 $ echo 'export EC2_PRIVATE_KEY=/home/johndoe/.ec2/pk-52a7d7892f31482984054b6222196bd0.pem' >> $HOME/.bashrc $ echo 'export EC2_CERT=/home/johndoe/.ec2/cert-52a7d7892f31482984054b6222196bd0.pem' >> $HOME/.bashrc $ source $HOME/.bashrc
As the RM staff for these 2 files if you think you should have access to them.
And as we primarily use the European service:
$ echo "export EC2_URL='https://eu-west-1.ec2.amazonaws.com'" >> $HOME/.bashrc $ source $HOME/.bashrc
Also you have to add to your ec2 user id to the bashrc. As a summary, your $HOME/.bashrc part of ec2 variables should appear like this:
export EC2_AMITOOL_HOME=/usr/local/ec2-ami-tools export EC2_HOME=/usr/local/ec2-api-tools export PATH=$PATH:/usr/local/ec2-ami-tools/bin:/usr/local/ec2-api-tools/bin export EC2_URL='https://eu-west-1.ec2.amazonaws.com' export EC2_PRIVATE_KEY=/home/johndoe/.ec2/pk-xxxxxxxxxxxxxx.pem export EC2_CERT=/home/johndoe/.ec2/cert-xxxxxxxxxxxxx.pem export EC2_USER_ID=xxxxxxxxxxxxx export EC2_ACCESS_KEY=xxxxxxxxxxxxxxxxx export EC2_SECRET_KEY=xxxxxxxxxxxxxxxxxxx
Finally, and the first time you start using EC2 you must create a keypair. This is used to access the started AMIs using SSH. Example:
$ ec2-add-keypair johndoe-eu
This will output a private key, which you can save in $HOME/.ec2/johndoe-eu.
Listing instances
To list the available instances:
$ ec2-describe-instances
To list a specific instance:
$ ec2-describe-instances i-xxxxxx
Starting a new instance
To start a new instance from a known AMI:
$ ec2-run-instances ami-xxxxxx
Frequent used arguments:
- -t type: instance type. There are several to choose from. To sum it up:
- m1.small: 1.7GB RAM, 1 EC2 Compute Unit, 160 GB storage, 32-bit platform, moderate I/O Performance, $0.10/hour.
- c1.medium: 1.7GB RAM, 5 EC2 Compute Unit, 350 GB storage, 32-bit platform, moderate I/O Performance, $0.20/hour.
- m1.large: 7GB RAM, 4 EC2 Compute Unit, 850 GB storage, 64-bit platform, high I/O Performance, $0.40/hour.
- m1.xlarge: 15GB RAM, 8 EC2 Compute Unit, 1690 GB storage, 64-bit platform, high I/O Performance, $0.80/hour.
- c1.xlarge: 7GB RAM, 20 EC2 Compute Unit, 1690 GB storage, 64-bit platform, high I/O Performance, $0.80/hour.
- -g groupname: used to add an instance to a security group.
- -k keypair: SSH public key to add to the /root/.ssh/authorized key of the AMI (if the AMI is prepared to do that).
Terminating an instance
To terminate a running instance:
$ ec2-terminate-instances i-xxxxxx
WARNING: when you terminate an instance you loose all the information in that machine. Don't do it unless you know what you are doing. |
Security groups
Amazon EC2 allows having an external firewall, other than the one that the operating system may include. This firewall rules are applied per groups, and one must select to which group and instance belongs to in the moment of starting it.
By default, there is a group called default and every instance is started there. Example of creating a new group, entering an instance in that group and managing the rules:
$ ec2-add-group myservice.openbravo.com -d "Security group for myservice" $ ec2-start-instances i-xxxxxx -g myservice.openbravo.com $ ec2-start-instances i-yyyyyy -g myservice.openbravo.com $ ec2-authorize -p 80 myservice.openbravo.com $ ec2-authorize -p 443 myservice.openbravo.com $ ec2-authorize -p 22 -s 1.2.3.4/32 myservice.openbravo.com
We have create the group and started two instances inside that group. Then we have opened HTTP (80) and HTTPS (443) in that group, as well as SSH (22) for a specific IP address (1.2.3.4).
Elastic IPs
One of the nicest feature of EC2 regarding DNS management is called Elastic IP. This allows to allocate an IP to an EC2 account, and then assign it to one instance or to another on the fly. Example:
$ ec2-allocate-address ADDRESS 1.2.3.4 $ ec2-associate-adddress 1.2.3.4 -i i-xxxxxx
This is useful to restore a machine very rapidly. If 1.2.3.4 is registered to a specific (sub)domain name, there is no need to change the DNS every time there is a disaster and the machine must be recovered in another instance. We can just reassociate the IP to another instance.
Regions: US/EU
EC2 offers two possible regions, the United States (US) and Europe (EU). By default the command line tools will take for granted you want to use them in the US. To use them in EU, export the EC2_URL variable:
$ export EC2_URL=https://eu-west-1.ec2.amazonaws.com
We mostly use it in EU.
Amazon Elastic Block Storage (EBS)
Amazon provides block level storage volumes for use with Amazon EC2 instances. This storage is used for backing up current instance.
We can create EBS volume ranging from 1GB - 1TB and it costs $0.10/GB per month.
To create a new EBS volume of 20GB in Europe:
$ ec2-create-volume -s 20 -z eu-west-1b
Attach it on a device (like /dev/sdj) on an instance:
$ ec2-attach-volume vol-yyyyyy -i i-xxxxxx -d /dev/sdj
Format it:
$ mkfs.ext3 /dev/sdj
Mount it inside the instance and copy the required data to it.
Documentation
For more information about Amazon EC2:
- Official command line reference.
- Glossary.