EDITOR’S NOTE AWS GETS REACTIVE WITH AUTO SCALING

WITH AWS CLOUDFRONT, IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD SCALING METHOD? DO YOUR HOMEWORK

When AWS Takes the Wheel Amazon Web Services offers capabilities that automate processes and functions of cloud applications—promising to save time and money. But first, organizations must establish a firm understanding of their workloads and patterns.

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL2

EDITOR’SNOTE

Up Your Apps With AWS Automation

From automated bill payments to mobile applications that remind you to feed your cat, automation has come a long way—saving peo-ple time and money.

It can do the same for those in the IT world. Amazon Web Services offers automation ser-vices like CloudWatch and Auto Scaling to minimize user responsibility and maximize efficiency of cloud applications. In this three-part guide, we review some of the benefits of these services. Among them, automatically adding or removing instances, avoiding scaling inaccuracies and producing better applications.

Software engineer Brian Tarbox kicks things off with a look at reactive scaling—which enables a program to automatically react to load changes in its environment—and how AWS’s Auto Scaling service is putting the concept to work.

Next, technology journalist George Lawton

interviews businesses about their experiences with AWS CloudFront. It’s designed to improve dynamic content and optimize delivery of static content quickly and cost-effectively. It does a better job than Amazon Simple Storage Services, Lawton finds, but he warns, “It is not a silver bullet for every source of delay in Web performance.”

To close, Tarbox delves deeper into public cloud scaling and reviews the pros and cons of using predictive versus automated scaling. Auto scaling workloads according to demand is great for business, but there are other options to consider. “Choosing the right one for your AWS cloud, however, depends on how much you know about your workloads and their pat-terns,” Tarbox said. n

Katherine WileySite Editor, SearchAWS

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL3

PROGRAMMING

AWS Gets Reactive With Auto Scaling

Reactive programming, which enables a program to automatically react to load changes in its environment and continue run-ning, is one of many IT buzzwords thrown around lately. But the concept isn’t new. Many programs that are labeled “reactive” were always supposed to be resilient in the face of change. Amazon Web Services offers an Auto Scaling service that allows the system to react to such changes.

Reactivity is really a systems concept. Each program should be resilient to change, and the overall system should add, remove and modify programs and resources as needed.

Auto Scaling enables a system to add re-sources, typically AWS Elastic Compute Cloud (EC2) instances, when the load increas-es—and remove them when the load dimin-ishes. There are four types of Auto Scaling: fixed, manual, scheduled and dynamic.

Fixed Auto Scaling ensures that AWS

replaces instances that fail, while manual Auto Scaling lets you add or remove instances by hand. Scheduled Auto Scaling is used when your load varies only by date or time, and dynamic Auto Scaling is for when your load changes unpredictably.

You may think the term load is predefined to mean something like CPU percentage or Ubuntu load average or Web-response time, but cloud administrators have full control over what constitutes load. You can use CloudWatch metrics to define these conditions and then group them into one or more policies, which can be predefined metrics or ones you create for specific system needs.

LOADED DEMANDS

To understand AWS Auto Scaling, you need to understand two important concepts: hysteresis and EC2 pricing.

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL4

PROGRAMMING

Hysteresis means adding a lag time to the response a system has to an event. Take your home thermostat, for example. If you set it to 68 degrees in the winter, the furnace won’t turn on until the temperature is slightly below 68, and then it will run until it reaches slightly above 68. That’s so the furnace won’t run for a minute, shut off and then turn back on again.

With EC2 pricing, AWS charges for instances by the hour. If you start an instance and then kill it after 10 minutes, you’re charged for a full hour. If, within a single hour, you terminate and start a new instance five times, you are charged for five hours.

Combine these concepts with Auto Scaling and it makes sense: You want the system to react to changes in load. If that load is highly dynamic, however, you should moderate

your response to it. This refers more often to removing instances than to adding them. When your load increases, you often have no option but to add capacity by adding instances. When the load decreases, you have some flexibility in deciding when to remove instances. If an instance has been running for only a fraction of the current hour, it makes sense to leave it run-ning for a while in case the load picks up again.

You can configure AWS CloudWatch alarms to fire only after a condition has lasted for a specified amount of time. For example, you could create an alarm that triggers the removal of instances after some load metric dips below a threshold for more than 30 minutes. That way, instances won’t be removed during brief dips in load only to have to be re-created within the same charging hour.

SPECIFIED INSTANCES

An Auto Scaling Policy determines the speed at which instances ramp up and down. You can have policies that add or remove one instance at a time and others that add or remove a set percentage of instances all at once.

To understand Amazon Web Services’ Auto Scaling, you need to understand two impor- tant concepts: hysteresis and EC2 pricing.

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL5

PROGRAMMING

The Auto Scaling group specifies the avail-ability zone of the instances as well as the minimum, maximum and desired number of instances. You can also think of this as your safety net; no matter how badly you miscon-figure things, you can’t accidently create more instances than the group maximum.

In the Auto Scaling launch configuration, you describe what type of instances you want, specifying, among other things, the Amazon

Machine Image and instance type. There are several options for creating instances, and you can specify any or all of these in the launch configuration.

Overall, the process would require you to put data into a metric (if using a custom metric) and then trigger an alarm. The alarm will trig-ger a policy that creates or removes instances according to the configuration within the group. —Brian Tarbox

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL6

DELIVERY

With AWS CloudFront, It’s a Matter of Content

The primary use case for Amazon Web Services CloudFront lies in optimizing the delivery of static content to users. But Amazon has raised the bar for the content distribution network with features for accelerating dynamic content as well. This makes the service more useful for non-video-related applications, such as speeding the distribution of user-gener-ated and personalized content. Also, Amazon charges the same for distributing dynamic con-tent as it does for distributing static content. These costs start on par with AWS’s Simple Storage Service (S3) but can drop as volume grows.

For example, TwitPic was one of the early adopters of CloudFront for improving the per-formance of its image-sharing application. The service allows users to post images to Twitter and other social media platforms. “We used CloudFront to deliver user-uploaded images to our 50 million registered users. We served

over 50 billion monthly API calls using Cloud-Front,” said Steve Corona, formerly the CTO of TwitPic and currently a principal architect at Bigcommerce.

Leveraging CloudFront allowed TwitPic to scale without having to build a large IT depart-ment. The company would have otherwise faced a much greater challenge in growing cost-effectively. Corona explained, “We were deliv-ering petabytes of traffic per month. Originally we did this using Amazon S3 because, well, we didn’t know what we were doing. It turned out CloudFront was actually cheaper for the amount of traffic that we were serving. To top that, it was much faster than serving data straight out of S3.”

UNDER THE HOOD

A typical website includes a mix of static and dynamic content. Static content can include

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL7

DELIVERY

images and style sheets that are the same across all users and are best cached at the edges of the content distribution network (CDN). Dynamic content includes information that changes frequently or is personalized based on user preferences, behavior, location or other factors. Consequently, even so-called dynamic content can be cached for varying periods.

Dynamically generating parts of a website causes an increase in network traffic latency, multiple connections to the origin server and increased CPU load. To address this situation, CloudFront uses a framework for processing query string parameters and cookies for deliv-ering the entire website without having to sep-arate out static and dynamic content or manage multiple domains.

CloudFront also allows application develop-ers to configure time-to-live (TTL) values for each file by setting the cache control headers on files saved on the origin server. CloudFront uses these to determine how frequently the edge location retrieves an updated file from the origin server. When the files change frequently, the developer needs to set a short TTL, which can be as little as zero seconds.

CACHE BEHAVIOR BASICS

It is prudent to look at the configurations for the cache settings used to ensure that users get fresh data. CloudFront uses a variant of the Least Recently Used, or LRU, caching algo-rithm. After content has been replicated in the CloudFront cache, users can pull data from the cache rather than directly from the origin server. If the TTL setting for the content is too short, users will effectively pull content directly from the origin server. On the other hand, if the TTL is set for a longer duration, it’s important to ensure that the application is configured to send an invalidation request.

For example, the Star Media Group, a divi-sion of Toronto Star Newspapers, uses Cloud-Front to deliver new online content of the Toronto Star, Canada’s largest daily newspaper. With more than 3.3 million monthly active users, scale and performance are two important parts of the design. At the same time, different types of content need to be set to expire with different frequencies.

To address this challenge, the Star Media IT team set up 19 different cache behaviors for different content types. It also established

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL8

DELIVERY

design guidelines to limit the number of query strings and to use client-side cookies rather than server-side cookies. This makes the con-tent more cacheable while reducing the number of variations of content to cache.

This approach also makes it possible to cache what is traditionally considered dynamic content. For example, popular search results are cached for two minutes, which increases the speed of delivery for popular searches and reduces the load on the origin server by elimi-nating the need to respond to popular queries each time. This approach balanced caching with the need to keep search results current and ultimately led to a 50% improvement in response time.

KEEPING IT FRESH

Like election results, sports scores and stock quotes, data can be cached for short periods, ranging from a second to a minute, to get the benefit of caching without having to send out stale data, said Nihar Bihani, product manager for Amazon CloudFront. For example, in 2012, National Public Radio delivered the election

results using CloudFront and a TTL setting of 30 seconds.

In other cases, the content can be geographi-cally generated, which makes it easier to cache content for short periods in a given region. For example, Earth Networks, which has developed

the popular Weather Bug application, uses CloudFront to cache weather data. Since users in a geographic area tend to hit the same edge server, this makes it possible for them to pull in weather updates without having to go back to the origin server for each request.

Bihani said this kind of personalization based on location is possible using specially crafted cookies and query strings, in which data about the user is recorded and becomes part of the cache key. He noted, “It is dynamic in the

CloudFront can cache data such as election results and sports scores for short periods to get the benefit of caching without having to send out stale data, Amazon says.

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL9

DELIVERY

sense that it is personalized for each user, but at the same time the data can be cached, and so everyone in San Francisco can get a cached copy rather than going to the Weather Bug ori-gin server.”

THE VALUE OF NO CACHE

The main benefits from a CDN lie in replicat-ing content closer to the users. But even when the TTL is set at zero, website performance can still improve if CloudFront is used as a proxy for improving the connection to the origin server rather than as a cache, Bihani said. In these instances, the edge servers can leverage a single, optimized connection with the back-end server that can share a single Transmission Control Protocol (TCP) connection across mul-tiple users and reduce the number of hops that a packet must take to the origin server.

When the first user requests content through the edge, there is some overhead and delay in setting up the connection with the origin server, but subsequent requests through the edge server can use the TCP connection that

has already been established between the ori-gin server and CloudFront edge servers, thus reducing delays in retrieving content.

Bihani explained, “The further away from the origin server, the more packet drops that happen on the Internet. Every time a packet drops, we have to retransmit. With CloudFront, not only does it use the persistent connection, but those are also optimized for performance.” On a long connection—such as a user in Singapore retrieving content from an origin server in Virginia—using CloudFront as a proxy can cut latency in half. A user in New York would see some improvement, but not as much.

While CloudFront can play a role in improv-ing the performance of dynamic content, it is not a silver bullet for every source of delay in Web performance. As it stands now, the focus of the CloudFront service has been on optimiz-ing the network path for moving content. It does not currently support other techniques, such as front-end optimization, JavaScript and CSS minification or image resizing for improv-ing Web performance. —George Lawton

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL10

MANAGEMENT

Deciding on a Cloud Scaling Method? Do Your Homework

Is Amazon Web Services’ Auto Scaling the best method for your enterprise workloads, or is predictive scaling better?

The ability to scale workloads according to demand is one of the biggest benefits of the public cloud. And there are three public cloud scaling methods most enterprises use: reactive, predictive and analytic.

Choosing the right one for your AWS cloud, however, depends on how much you know about your workloads and their patterns.

n Reactive scaling. AWS Auto Scaling is a reac-tive service that adds new servers in response to increased load on the system. A lagging indicator is a time delay between the new load being placed on the system and the system’s response. Depending on how long it takes for new resources such as Elastic Compute Cloud (EC2) instances to come online, the lag can be measured in minutes or tens of minutes.

Lag time depends on a number of factors. If you create on-demand instances, they spin up within tens of seconds. If, on the other hand, you are using spot instances, it takes on aver-age five minutes to start. Once your instance is running, it still may not be ready to perform a task—depending on whether you are using cooked Amazon Machine Images (AMIs) or not.

A cooked instance strategy means your AMI includes all the tools you need, so as soon as the machine starts, you can start working. Non-cooked images are bare-bones and require scripts such as Puppet or Chef to download and install packages before the system is available.

n Predictive scaling. Some companies take a different approach. Netflix, for example, uses predictive scaling to anticipate load and pre-emptively launch additional instances. This approach is based on the notion that load patterns are often repeatable and therefore

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL11

MANAGEMENT

predictable. For example, you don’t have to go too far out on a limb to predict that Friday night will see a higher load on the Netflix serv-ers than Friday afternoon will.

Using this approach, Netflix provisions extra resources ahead of the anticipated additional load. It may incur a small increase in cost, but it also may avoid a temporary degrada-tion in user experience as the system avoids a slowdown.

n Analytic scaling. Some systems generate their own load rather than have it created externally. A streaming video system, for example, might download new movies that need to be trans-coded. A financial system might download new stock data every night. In both cases, the “load” on the system is a known value because the system itself creates the load.

In these examples, the system can add or remove instances based on its own understand-ing of the load.

Combining all of these cloud scaling methods may be the best approach. If you use AWS Auto Scaling, the component doing the scaling is the AWS system. You don’t have to worry about

monitoring it, because Auto Scaling is part of the AWS platform. But if your system does its own scaling, then the component that owns it has to be made resilient in case your auto-scal-ing service dies.

For this reason—and because your predic-tive or analytic scaling algorithms might have errors—many systems use AWS Auto Scaling to establish a minimum number of instances. Set up an Auto Scaling group with a rule such as “Keep at least one EC2 instance running” and let your custom scaling rules handle the upper limits. The asymmetry of the AWS Auto Scal-ing rule mirrors the asymmetry of having too many instances (extra cost but good response time) versus having too few instances (lower cost but potentially poor response time). —Brian Tarbox

Choosing the right public cloud scaling method for your AWS cloud depends on how much you know about your workloads and their patterns.

HOME

EDITOR’S NOTE

AWS GETS REACTIVE

WITH AUTO SCALING

WITH AWS CLOUDFRONT,

IT’S A MATTER OF CONTENT

DECIDING ON A CLOUD

SCALING METHOD?

DO YOUR HOMEWORK

WHEN AWS TAKES THE WHEEL12

ABOUT THE

AUTHORS

BRIAN TARBOX has been doing programming since he created a timing-and-scoring program for the Head of the Connecticut Regatta in 1981. Though primarily an Amazon Java programmer, he is a firm believer that engi-neers should be multilingual and use the best language for the problem. Email him at briantarbox@gmail.com.

GEORGE LAWTON is a technology journalist who has worked in systems engineering and database and software development for the Biosphere II project in Oracle, Ariz., and elsewhere. Over the last 20 years, he has written about topics such as software development, DevOps, sys-tems integration, Java and cloud computing. Email him at glawton@glawton.com.

When AWS Takes the Wheel is a SearchAWS.com e-publication.

Scot Petersen | Editorial Director

Jason Sparapani | Managing Editor, E-Publications

Joe Hebert | Associate Managing Editor, E-Publications

Jan Stafford | Executive Editor

Brein Matturro | Managing Editor

Katherine Wiley | Site Editor

Linda Koury | Director of Online Design

Neva Maniscalco | Graphic Designer

Rebecca Kitchens | Publisher | rkitchens@techtarget.com

Amalie Keerl | Director of Product Management akeerl@techtarget.com

TechTarget 275 Grove Street, Newton, MA 02466

www.techtarget.com

© 2014 TechTarget Inc. No part of this publication may be transmitted or re-produced in any form or by any means without written permission from the publisher. TechTarget reprints are available through The YGS Group.

About TechTarget: TechTarget publishes media for information technology professionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and pro-cesses crucial to your job. Our live and virtual events give you direct access to independent expert commentary and advice. At IT Knowledge Exchange, our social community, you can get advice and share solutions with peers and experts.

COVER ART: THINKSTOCK

STAY CONNECTED!

Follow @SearchAWS today