© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Scott WardSolutions Architect

Amazon Aurora for Enterprise Database Applications

Enterprise database requirements ..

Database engine with enterprise class availability, performance, scalability, and security.

Managed services – instant provisioning, push button scaling, automated backups, patching, monitoring, migration.

1

2

Goal: Provide fully managed enterprise class database service without the cost and complexity of commercial database software.

Traditional relational databases

Gradual improvements on decades old design

Accommodate different server and storage hardware

Too complex to tune to achieve optimal performance

Layers of software to mitigate potential points of failure

‘Cloudification’ by virtue of additional layers

High cost, complex and punitive licensing terms Multiple layers of functionality in a monolithic stack.

SQL

Transactions

Caching

Logging

Relational database re:Imagined

We started with a blank sheet of paper and reimagined relational database for the cloud

Amazon Aurora is purpose built for the cloud

Designed from the ground up using AWS technology

Distributed component architecture with built-in redundancy

High-Availability and scale out are part of database core design

Self-healing components designed for resilience

Architected for security and performance

Security

Isolate your data and control access

Virtual Private Cloud (VPC)

1. Define your own private address space in the AWS cloud.

2. Create public and private subnets.

3. Define security groups for your database.

4. Connect via VPN and Direct Connect.

Encryption at rest using Keys you create an manage using KMS

Data, automated backups, snapshots, and replicas in the same cluster all automatically encrypted.

Seamless encryption and decryption, requiring no changes to your application.

SSL encryption in transit

Encryption at rest and transit

Enterprise-class performance

Write performance (console screenshot)

MySQL Sysbench

R3.8XL with 32 cores and 244 GB RAM

4 client machines with 1,000 threads each

MySQL Sysbench

R3.8XL with 32 cores and 244 GB RAM

Single client with 1,000 threads

Read performance (console screenshot)

Writes scale with table count

Tables Amazon Aurora

MySQLI2.8XL

local SSD

MySQLI2.8XL

RAM disk

RDS MySQL30K IOPS

(single AZ)

10 60,000 18,000 22,000 25,000

100 66,000 19,000 24,000 23,000

1,000 64,000 7,000 18,000 8,000

10,000 54,000 4,000 8,000 5,000

Write-only workload1,000 connectionsQuery cache (default on for Amazon Aurora, off for MySQL)

Write scales with number of connections

Connections Amazon Aurora

RDS MySQL30K IOPS

(single AZ)

50 40,000 10,000

500 71,000 21,000

5,000 110,000 13,000

OLTP WorkloadVariable connection count250 tablesQuery cache (default on for Amazon Aurora, off for MySQL)

Less IOs to backend

Effective query caching

Replica managementDo less work

Do it efficiently

Latch free lock management

Adaptive thread pools

Asynchronous commits

Consistent, low-latency writes

AZ 1 AZ 2

PrimaryInstance

StandbyInstance

Amazon Elastic Block Store (EBS)

Amazon S3

EBSmirror

EBS

EBSmirror

AZ 1 AZ 3

PrimaryInstance

Amazon S3

AZ 2

ReplicaInstance

ImprovementsConsistency—tolerance to outliers

Latency—synchronous vs. asynchronous replication

Efficiency—significantly more efficient use of network I/O

Log records

Binlog

Data

Double-write buffer

FRM files, metadata

Type of writes

MySQL with standby Amazon Aurora

async4/6 quorum

PiTR

Sequentialwrite

Sequentialwrite Distributed

writes

Designed for high-availability

Aurora storageHighly available by default

6-way replication across 3 AZs 4 of 6 write quorum

Automatic fallback to 3 of 4 if an Availability Zone (AZ) is unavailable

3 of 6 read quorum

SSD, scale-out, multi-tenant storage Seamless storage scalability Up to 64 TB database size Only pay for what you use

Log-structured storage Many small segments, each with their own redo

logs Log pages used to generate data pages Eliminates chatter between database and storage

SQL

Transactions

AZ 1 AZ 2 AZ 3

Caching

Amazon S3

Lose two copies or an AZ failure without read or write availability impact

Lose three copies without read availability impact

Automatic detection, replication, and repair

SQL

Transaction

AZ 1 AZ 2 AZ 3

Caching

SQL

Transaction

AZ 1 AZ 2 AZ 3

Caching

Read and write availability Read availability

Self-healing, fault-tolerant

Continuous backupSegment snapshot Log records

Recovery Point

Segment 1

Segment 2

Segment 3

Time

Take periodic snapshot of each segments in parallel. Stream the redo logs to S3. Backup happens continuously without performance or availability impact At restore retrieve the appropriate segment snapshots and log streams to storage nodes. Apply log streams to segment snapshots in parallel and asynchronously.

Traditional databasesHave to replay logs since the last checkpoint

Single-threaded in MySQL; requires a large number of disk accesses

Amazon AuroraUnderlying storage replays redo records on demand as part of a disk read

Parallel, distributed, asynchronous

Checkpointed Data Redo Log

Crash at T0 requiresa re-application of theSQL in the redo log sincelast checkpoint

T0 T0

Crash at T0 will result in redologs being applied to each segmenton demand, in parallel, asynchronously

Instant crash recovery

Faster, more predictable failover

AppRunningFailure Detection DNS Propagation

Recovery Recovery

DBFailure

MYSQL

AppRunning

Failure Detection DNS Propagation

Recovery

DBFailure

AURORA WITH MARIADB DRIVER

1 5 - 2 0 s e c

3 - 2 0 s e c

To cause the failure of a component at the database node:ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}] 

To simulate the failure of disks:ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN [DISK index | NODE index] FOR INTERVAL interval

To simulate the failure of networking:ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type [TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval

To simulate the failure of an Aurora Replica:ALTER SYSTEM SIMULATE percentage_of_failure PERCENTREAD REPLICA FAILURE [TO ALL | TO "replica name"] FOR INTERVAL interval

Simulate failures using SQL

Delivered as a managed service

Databases are hard to manage

RDS platform: managing databases made easy

Schema design Query construction Query optimization

Backup & recoveryIsolation & security Industry compliancePush-button scalingAutomated patchingAdvanced monitoringRoutine maintenance

Amazon RDS takes care of your time-consuming database management tasks, freeing you to focus on your applications and business

You

RDS

Advanced monitoringSingle page dashboard for OS and process diagnostics in AWS console

Customize dashboard with choice of metrics and layout

Add alarms on specific metrics

Metrics egress via Cloud Watch Logs into 3rd party monitoring tools like Graphite etc.

Support for metrics crossover into CloudWatch

Metrics such as load average, detailed CPU utilization, detailed disk IO, and per process provides at a fixed range of granularities ranging from 60 to 1 seconds.

Integrations with tools and dashboards

Migrating to Aurora

MySQL Backup to Aurora

Source MySQL Database Target Aurora

Database

Amazon S3

Move data to the same or different database engine

Keep your apps running during the migration

Start your first migration in 10 minutes or less

Replicate within, to or from AWS EC2 or RDS

AWS Database Migration Service

http://aws.amazon.com/dms

CustomerPremises

Application Users

AWS

InternetVPN

Start a replication instance

Connect to source and target databases

Select tables, schemas or databases

Let the AWS Database Migration Service create tables, load data and keep them in sync

Switch applications over to the target at your convenience

Keep your apps running during the migration

Migrate off Oracle and SQL Server

Move your tables, views, stored procedures and DML to MySQL, MariaDB & Amazon Aurora

Highlight where manual edits are neededAWS Schema Conversion Tool

http://aws.amazon.com/sct

Perfect fit for enterprise 6-way replication across 3

Fail-over in less than 30 secs

Near instant crash recovery

Up to 500K/sec read and 100K/sec write

15 low latency (10ms) Read Replicas

Up to 64 TB DB optimized storage volume

Instant provisioning and deployment

Automated patching and software upgrade

Backup and point-in-time recovery

Compute and storage scaling

Performance and scale

Enterprise class availability

Fully managed service

Many features are unique to Amazon Aurora

Comparing to traditional commercial databases like Oracle• Available only in most expensive database edition (Enterprise Edition)• Failover and Replica – Oracle Active Data guard – Extra $$$ per core• Backup to S3 - Oracle Secure Backup Cloud Module – Extra $$$ per channel• Encryption – Oracle Advanced Security - Extra $$$ per core

Comparing features ..

Don’t be constrained by Licenses, Cost or Capacity

Simple pricingNo licensesNo lock-inPay only for what you use

Discounts44% with a 1-year RI63% with a 3-year RI

vCPU Mem Hourly Price

db.r3.large 2 15.25 $0.29

db.r3.xlarge 4 30.5 $0.58

db.r3.2xlarge 8 61 $1.16

db.r3.4xlarge 16 122 $2.32

db.r3.8xlarge 32 244 $4.64

• Storage consumed, up to 64 TB, is $0.10/GB-month• IOs consumed are billed at $0.20 per million I/O• Prices are for Virginia

Enterprise grade, open source pricing

Cost of Ownership: Aurora vs. Commercial databasesOracle on EC2 Configuration Hourly Cost

Primaryr3.8XL

Standbyr3.8XL

Replicar3.8XL

ReplicaR3.8XL

Storage6TB / 30K PIOP

Storage6TB / 30K PIOP

Storage6TB / 30K PIOP

Storage6TB / 30K PIOP

$2.93/hr

$2.93/hr

$2.93/hr $2.93/hr

$3.75/hr

$3.75/hr $3.75/hr

Instance cost: $11.72/ hrLicense cost: $63.12/hrStorage cost: $15.00 / hr

Total cost: $89.84 / hr

$3.75/hr

Enterprise License

$15.78/hr

Enterprise License

$15.78/hr

Enterprise License

$15.78/hr

Enterprise License

$15.78/hr

Cost of Ownership: Aurora vs. Oracle Aurora Configuration Hourly Cost

Instance cost: $13.92/ hrStorage cost: $5.15 / hr

Total cost: $19.07 / hr

Primaryr3.8XL

Replicar3.8XL

ReplicaR3.8XL

Storage / 6TB

$4.64/ hr $4.64/ hr $4.64/ hr

$5.15 / hr

Storage IOPs assumptions:1. Average IOPs is 50% of Max IOPs2. 50% savings from shipping logs vs. full pages

78.7%Savings

No idle standby instance

Single shared storage volume

No POIPs – pay for use IO

Reduction in overall IOP

Enterprise use cases

Web and mobile

Content management

E-commerce, retail

Internet of Things

Search, advertising

BI and analytics

Games, media

Common Customer Use Cases

Fastest growing service in AWS history1000+ customer after 10 days of launch

Zynga: Online Gaming

Online and Mobile Gaming Company

Challenges running traditional relational database technologies at scale

Instance recovery time Monitor mundane but critical jobs

Latency associated with existing DB storage solution

Aurora benefits:  ~9K selects/second during peak for 150GB data

set Reduced operational overhead Simplified scaling for large scale events

Expedia: On-line travel marketplace Real-time business intelligence and analytics on

a growing corpus of on-line travel market place data.

Current SQL server based architecture is too expensive. Performance degrades as data volume grows.

Cassandra with Solr index requires large memory footprint and hundreds of nodes, adding cost.

Aurora benefits:  Aurora meets scale and performance

requirements with much lower cost. 25,000 inserts/sec with peak up to 70,000. 30ms

average response time for write and 17ms for read, with 1 month of data.

World’s leading online travel company, with a portfolio that includes 150+ travel sites in

70 countries.

Alfresco: Enterprise Content Management

Needed database to scale without any degradation in performance.

Benefits:  Alfresco on Amazon Aurora, scaled to 1 billion

documents with a throughput of 3 million per hour, which is 10 times faster than their MySQL environment.

Provides Enterprise Content Management software built on

open standards. 

Alfresco - Benchmark Results

Document load rate 1000 documents per second (with 10 nodes) Load rate was consistent even passing the 1B document Sub-second login times and good responses for other actions

Open Library: 4.5s Page Results: 1s Navigate to Site: 2.3

Aurora indexes used efficiently at 3.2TB No indications of any size-related bottlenecks with 1.1 Billion Documents CPU loads:

Database: 8-10% Alfresco (each of 10 nodes): 25-30%

Insurance claims processing ICSC Provides fully integrated policy management,

claim and billing solutions for property/casualty insurance organizations

For the last 12 years ISCS has used SQL Server & Oracle commercial databases for operational & warehouse data

The cost and maintenance of traditional commercial database has increasingly become the biggest expenditure and maintenance headache 

Maintaining its customer SLAs requires complex, difficult-to-manage replication and redundancy across multiple geographic locations

As customer data grows, backup/restore times for its largest data sets have progressed to unacceptable levels.       

Aurora benefits

SQL Server backups that once took 5-6 hours daily now happen continuously on Aurora. Snapshots from one customer database (~ 5TB in size) take 5 minutes to make and less than an hour to restore. ISCS can actually test disaster recovery daily if it wanted to.

Data that was once only available “daily, batch” into Redshift can now be migrated continuously using Aurora read-replicas and Change Data Capture (CDC).

Performance at scale is linear since ISCS’s application, like Aurora, is optimized for multiple, concurrent read requests to the database.

Multi-AZ Aurora read-replicas also eliminate the need for additional licenses/deployments of SQL Server.

The cost of a “more capable” deployment on Aurora has proven to be about 70% less than ISCS’s SQL Server deployments.

Amazon Aurora: Earth Networks

Earth Networks process over 25 terabytes of real-time data daily, so need a scalable database that can rapidly grow with expanding data analysis

Benefits:  Aurora performance and scalability works well

with their rapid data growth. Moving from SQL Server to Aurora was very easy

Operates world's largest weather and lightning sensor

networks and technology

Want to learn more?

Amazon Aurora home page:https://aws.amazon.com/rds/aurora/start/

• Product Reviews• Whitepapers• Hands on Labs• Customer References

Registration Open Now

Learn more reinvent.awsevents.com

November 28-December 2, 2016 | The Venetian - Las Vegas, NV

Thank you!

Scott WardSolutions Architect