turbocharge purpose built databases with amazon ... - aws
TRANSCRIPT
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Japan Purpose Built Database Week
Tom KuehleAmazon ElastiCache GTM Specialist
Improve application performance with in-memory data stores
Turbocharge purpose built databases with Amazon ElastiCache
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Modern applications and the need for purpose-
built databases
Agenda
How ElastiCacheenhances purpose-built application performance
Upcoming ElastiCachefeatures that will enhance
performance, security, and reliability
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data modernization drivers
Data grows 10x every 5 years driven by network connected smart devices
Time-to-insights and actions in milliseconds at Internet
scale
Growing need for multiple databases and data models within the same application
Explosion of data Micro-services changing data and application requirements
Applications require global scale and rapid response
Speed Scale
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Professionals use purpose-built tools
Using microservices over monolithic applications
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Our approach
Use Amazon ElastiCache in-memory database to
improve the performance of purpose built databases and
analytics solutions
Architect services ground-up for the
explosion of data, cloud workload migration, and
using micro services
Select from a portfolio of purpose-built
services, optimized to your application
requirements
Innovate faster through managed services
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Purpose-built databasesThe growing AWS database portfolio
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Database caching• Real time data processing• Versatile data models• Workload off-loading
Universal role of ElastiCacheApplication performance
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Request Rate (Velocity)High Low
Latency (Response time)Low High
Dat
a So
urce
sUnstructured/High Variety
Structured/Low Variety
Amazon S3
AmazonDynamoDBand DAX
AmazonElastiCache Amazon
DocumentDB
AmazonCloudSearch and
Elasticsearch ServiceAmazon
RDS & Amazon Redshift
Amazon Keyspaces
Databases and Analytics Service Roles
Amazon Glacier
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The need for speed
ElastiCache + RDS ElastiCache + AuroraElastiCache + RedshiftElastiCache + NeptuneElastiCache + DynamoDBElastiCache + DocumentDBElastiCache + S3/object stores
ElastiCache + …. better together!
“Query response time has become the most important DB metric”ScaleGrid, March 2019
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Turbocharge real-time applicationsCritical component to optimizing user experiences
• Seeing frequent identical queries
• Experiencing high latency on reads
• Requiring sub-millisecond performance to render reads
• Looking for cost efficiencies from high I/O due to heavy reads
• retrieving data from optimized databases often falls short of modern, real-time application requirements.
• add ElastiCache to your end-to-end architecture.
Use ElastiCache when:
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Database Proxy EC2 Tier Amazon RDS,
Aurora, Redshift
AmazonElastiCache
Database Proxy*Applicatio
n
• Eliminate duplicate queries• Reduce relational workloads• Improve end-user response times • No application code changes• Cost neutral or likely cost savings
Modernize legacy SQL applicationsPerformance improvement at a cost savings
Application
* DB proxy available through AWS partners or built with code refactoring
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Duolingo uses AWS databases to serve up over 31 billion items for 80 language courses with high performance and scalability
Primary database: Amazon DynamoDB• 24,000 reads and 3,000 writes per second• Personalize lessons for users taking 6bn exercises per month
In-memory caching: Amazon ElastiCache• Instance access to common words and phrases
Transactional data: Amazon Aurora• Maintain user data
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Business OpportunityGrab is Southeast Asia’s largest ride-hailing service with 45M downloads & 2.5 M daily rides.
ChallengesAverage response time of the API layer is <40ms. Redis was introduced to for real time processing of 4 million daily bookings. Tried running Redis on EC2, but time consuming, error prone, and expensive.
SolutionUsing both RDS and Amazon ElastiCache in multi-AZ allowed them to outsource all the management to AWS. ElastiCache as a caching layer on RDS MySQL provides sub-millisecond latency even in times of peak traffic of hundreds of thousands of cab requests per minute.
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Drive faster time-to-insightsUse ElastiCache within your end-to-end data analytics architectures
Data Warehousing
Big DataProcessing
Interactive Query
Operational Analytics
Real timeAnalytics
PredictiveAnalytics
ElastiCache drives fast, real-time processing of large data analytics volumesRetrospective analysis and reporting
Actions
RecommendationsVisualizations
Time-to-insight performanceData ingestion (e.g., mobile, IoT)
Real-time edge analytics (e.g., consumer services, security, AdTech)
Machine learning (e.g., inferences, pattern detection, recommendations)
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Enhance IoT and analyticsExample: ETL filter and buffer
AmazonKinesis
AmazonManagedStreamingfor Kafka
AmazonElastiCache
for Redis
DataStreams
DataSources
DataTransformation
AWS Lambda
Amazon EMR
Amazon EC2
LoadBuffer fast-moving data in-memory
Amazon RDS
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Expedia uses AWS databases for global inventory and pricing analytics that require consistent performance at scale
Real-time stream processing: Amazon ElastiCache• Multi-stream union and self-join• 24-hour look-back window
Transactional data: Amazon Aurora• Operational queries
Data warehousing: Amazon Redshift• Analytical queries
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1
The Future is Purpose Built
Data volume growth is explosive, and data analysis drives competitive
advantage---
Micro service architectures provide developers the right tools for the
right job---
The era of monolithic applications is over
2
Modern Applications have a Need for Speed
Modern data-driven apps need low latency response times
---Data pipelines need fast ingest,
throughput, and analysis ---
Trends in IoT, ML, edge, 5G and real-time services are accelerating velocity
3
Amazon ElastiCache is a Critical Component
In-memory caching to improve database response times and
workload throughput---
Multitude of key-value data models for real-time processing
---Most any applications can benefit
from faster response times
Important Take-Aways
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What’s new on the horizon with Amazon ElastiCache?
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The features we are sharing are not a commitment to deliver in 2020. Unless already launched, they simply provide general guidance on product direction and priorities.
Please contact your local AWS account team if you desire greater detail. Under a nondisclosure agreement (NDA) we will be able to share more confidential information.
Also note this is just a sample of 2020 features. There are many more product enhancements outside of what is being discussed today.
2020 FeaturesBefore we begin
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2020 Roadmap ThemesElastiCache Product Development Categories
Ø Performance: cluster sizes, available memory, CPU horsepower
Ø Open source: version concurrency, community contributions
Ø Security/authorization: role-based access control, consistent IAM
Ø HA/DR: cross region replication, cluster mode migrations
Ø Data proximity: moving data processing closer to the end consumer
Ø Data persistence: reducing or eliminating in-memory data loss
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Automate geo replication of customer's Redis data.
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
More enhancements coming soon including increased region replications
Primary(active) region
Read/Write
Secondary (Passive) Region
Read
Secondary (Passive) RegionRead
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Enable new high performance instances deploying ARM technology
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Leverage new Redis 6.0 Features
Benefits• Client-side caching OSS code, snapshot,
and throughput improvements.
• Better authentication and authorization for Redis removing an important roadblock for adoption by customers
• Bring the power of Redis 6.0 to a fully managed service that’s well integrated with AWS infrastructure and services.
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Outposts brings the AWS cloud experience to a customer's physical location.
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ElastiCache Local Zones provides closer access points to end users.
Benefits• Data residency in your
city/state/country boundary.
• Reduce latency using close to the user deployments with seamless access to other AWS services.
© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Provides consistent authentication experience across AWS services