How to Handle AWS Through the Command Line

Cloud Computing Amazon Web Services
#1

What are Amazon Web Services (AWS)?

Amazon AWS is a collection of services based on Amazon’s infrastructure that Amazon provides to the general public. These services include computing resources, file storage, databases, and even crowd-sourced manual labor. See http://aws.amazon.com/products/ for an overview of the provided services.

What is the Amazon AWS Command Line Interface (CLI)?

The AWS CLI tool enables to control all operational aspects of AWS from the command line. This is a great advantage for automating processes and for people (like me) with a preference for textual user interfaces.

How to create an AWS account

Head over to https://aws.amazon.com/ and sign up for an account. This process will require you to have a credit card and a mobile phone at hand. Then come back and read on.

How to generate access keys

Before the AWS CLI can be configured for use, you need to create a user with the required permissions and download his access keys (AWS Access Key ID and AWS Secret Access Key) for use in the AWS CLI.

In the AWS Console (https://console.aws.amazon.com/), open your account menu in the upper right corner and click on “Security Credentials”:

Security Credentials

If a dialog pops up for you, just dismiss it for now by clicking “Continue to Security Credentials”.

Then, in the sidebar on the left, click on “Groups”:

Groups

Create a group (e.g. “Developers”) and attach the policy “AmazonEC2FullAccess” to it.

Then, in the sidebar on the left, click on “Users”:

Users

Create a new user, and then copy or download the security credentials to a safe place. You will need them soon.

Click on the new user, then “Add User to Groups” to add the user to the group you’ve just created before. This gives the user (and the keys) the required capabilities to manipulate EC2

Install AWS CLI via Homebrew (OS X)

(Linux users can skip to the next section.)

On OS X, Homebrew provides a simple way to install other software from the command line and is widely used. Even though the AWS CLI documentation recommends installation via pip (the Python package manager), I chose to install AWS CLI via Homebrew as it is more common. AWS CLI on Homebrew might lag behind a version compared to pip, though.

Open the Terminal application and install Homebrew by running:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

The installation script will guide you through the necessary steps to get Homebrew set up. Once finished, install AWS CLI using:

$ brew install awscli
$ aws --version

If the last command successfully shows you the version of the AWS CLI, you can continue on to the section about configuring AWS CLI.

Install AWS CLI via pip (Linux)

On Debian or Ubuntu Linux, run:

$ sudo apt-get install python-pip
$ sudo pip install awscli
$ aws --version

On Fedora, run:

$ sudo yum install python-pip
$ sudo pip install awscli
$ aws --version

If the last command successfully shows you the version of the AWS CLI, you can continue on with the next section.

Configure AWS CLI and Run Your First Virtual Server

Run aws configure and paste in the credentials you’ve received earlier:

$ aws configure
1. AWS Access Key ID [None]: AKIAJXIXMECPZBXKMK7A
2. AWS Secret Access Key [None]: XUEZaXQ32K+awu3W+I/qPyf6+PIbFFORNM4/3Wdd
3. Default region name [None]: us-west-1
4. Default output format [None]: json

Here you paste the credentials you’ve copied or downloaded above, for “AWS Access Key ID” and “AWS Secret Access Key”. (Don’t bother trying the values given in the example, as I’ve already changed the keys.)

We’ll use the region “us-west-1” here. If you want to use another one, you will have to find an equivalent AMI (HD image, “Ubuntu Server 14.04 LTS (HVM), SSD Volume Type”, ID ami-df6a8b9b in region “us-west-1”) with a different ID for your region.

The output formats available are “json”, “table” and “text”, and can be changed for each individual AWS CLI command by appending the --output option.

  • “json” is the default and produces pretty-printed (though not key-sorted) JSON output.
  • “table” produces a human-readable presentation.
  • “text” is a tab-delimited format that is easy to parse in shell scripts.

Help on AWS CLI

The AWS CLI is well documented on http://aws.amazon.com/documentation/cli/, and man pages for all commands are available by appending help to the end of the command line:

1. aws help
2. aws ec2 help
3. aws ec2 run-instances help

Amazon EC2

Amazon EC2 is the central piece of AWS. The EC2 websites says: “Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud.” In other words, EC2 provides the capability to run virtual machines connected to the Internet.

That’s what will make our Nginx run, so let’s make use of it. To do that, we need to enable SSH networking and generate an SSH key for logging in.

Setting Up the Security Group (Firewall)

Security Groups are virtual firewalls. To make a virtual machine accessible, it is associated with one (or more) security group. A security group defines which ports are open and to what IP ranges.

Without further ado:

1. aws ec2 create-security-group --group-name tutorial-sg --description "Tutorial security group"
2. aws ec2 authorize-security-group-ingress --group-name tutorial-sg --protocol tcp --port 22 --cidr 0.0.0.0/0
3. aws ec2 authorize-security-group-ingress --group-name tutorial-sg --protocol tcp --port 80 --cidr 0.0.0.0/0

This creates a security group called “tutorial-sg” with ports 22 and 80 open to the world.

To confirm that you have set it up correctly, you can then run:

1. aws ec2 describe-security-groups --group-name tutorial-sg \

2. --query 'SecurityGroups[0].{name:GroupName,description:Description,ports:IpPermissions[*].{from:FromPort,to:ToPort,cidr:IpRanges[0].CidrIp,protocol:IpProtocol}}'

The --query option is a great way to filter through AWS CLI JSON output. You can safely remove the --query option from the aws ec2 describe-security-groups command to see the JSON output in full.

The AWS CLI documentation has more information about AWS CLI output manipulation and the --query option.

Generate an SSH Key

To actually log in via SSH, we need an SSH key:

1. aws ec2 create-key-pair --key-name tutorial-key --query 'KeyMaterial' --output text > tutorial-key.pem
2. chmod 0400 tutorial-key.pem

Run Your First Instance (Virtual Machine) on EC2

Finally, we can run our first instance on AWS! Remember that the image ID “ami-df6a8b9b” is specific to the region “us-west-1”. In case you wonder about the size of the disk, this command will also create a new 8 GB disk volume based on the size of the specified disk image:

1. instance=$(aws ec2 run-instances --image-id ami-df6a8b9b --count 1 --instance-type t2.micro \
2.    --security-groups tutorial-sg --key-name tutorial-key \
3.    --query 'Instances[0].InstanceId' --output text) ; echo Instance: $instance

This shows you the IP address and the state of the new instance in a nice table:

1. aws ec2 describe-instances --instance-ids $instance \
2.    --query 'Reservations[*].Instances[*].[InstanceId,PublicIpAddress,State.Name]' --output table

Install Nginx and Open Your Shiny New Website

And now we can log in to the new instance to install Nginx:

1. ipaddr=$(aws ec2 describe-instances --instance-ids $instance \
2.    --query 'Reservations[0].Instances[0].PublicIpAddress' --output text) ; echo IP: $ipaddr
3. ssh -i tutorial-key.pem ubuntu@$ipaddr sudo apt-get update && \
4. ssh -i tutorial-key.pem ubuntu@$ipaddr sudo apt-get install -y nginx

If you now open the website at $ipaddr in your browser (OS X: open http://$ipaddr, Ubuntu:xdg-open http://$ipaddr), you should be greeted with the “Welcome to nginx!” message.

Cleaning Up

In case you might want to stop your instance again:

1. aws ec2 stop-instances --instance-ids $instance

To start the instance again, substitute start for stop (caution: the IP address will probably change), and to completely remove the instance (including the volume), substitute terminate for stop.

Resources