amazon wallet: increasing performance with dynamodb
TRANSCRIPT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Wallet: Increasing performance with DynamoDB
A M Z 3 0 2
Jagan Cheboluraghava
Senior Software Dev Manager
Amazon, Payment Services
Artem Pliasunov
Senior Software Dev Engineer
Amazon, Payment Services
Agenda
Overview of Amazon Wallet
Why did we move to Amazon DynamoDB?
How did we implement migration with zero downtime?
How did Wallet benefit by migrating to Amazon DynamoDB?
Q&A
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Wallet enables customers to pay for their orders
Wallet’s functionality
Keeping up with exponential growth
• 5 billion requests daily
• 10 billion records
• Exponential growth year over year
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Wallet
service
Database
Wallet
service
Database
Wallet
service
Shard 2
Shard 1
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
Hot
Warm
Warm
Hot
!
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
DB scaling
Vertically Horizontally
Shard 1A
Shard 1A
Shard 2A
Shard 1A SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
Shard NA SB 1-NA SB 2-NA
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 1
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 2
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 3
Wallet
service
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Region 4
!
!
!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Goals
• Reduce the cost of operations
• Simplify and reduce the cost of scaling
Requirements for migration
• Zero downtime
• Keep data integrity
• Improve performance
• Improve availability
• Allow launch of new features
Database decoupling
❑ Database consumers
❑ Join queries
❑ Triggers
❑ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
Decoupling database consumers
Amazon
Redshift Business
analytics
Wallet
service
Wallet
crawler
Oracle
Amazon
Redshift
Business
analytics
Wallet
service
Wallet
crawler
Amazon
DynamoDB
Oracle
Database decoupling
✓ Database consumers
❑ Join queries
❑ Triggers
❑ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
Amazon
Redshift
Business
analytics
Wallet
service
Wallet
crawler
Amazon
DynamoDB
Oracle
Breaking join query
Payment instruments
Customer ID
Payment instrument ID
Type
Token
…
External destinations
Customer ID
Payment instrument ID
Address ID
…
SELECT FROM external_destinations
JOIN payment_instruments
External destinations
Customer ID
Payment instrument ID
Address ID
…
Payment instruments
Customer ID
Payment instrument ID
…
QUERY external_destinations
QUERY payment_instruments
Database decoupling
✓ Database consumers
✓ Join queries
❑ Triggers
❑ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
External destinations
Customer ID
Payment instrument ID
Address ID
…
Payment instruments
Customer ID
Payment instrument ID
…
QUERY external_destinations
QUERY payment_instruments
Database decoupling
✓ Database consumers
✓ Join queries
✓ Triggers
✓ Stored procedures
❑ Indexes
❑ Unique constraints
❑ Transactions
❑ Views
Stored procedures
• Moved to application side
• Set before create/update
Database decoupling
✓ Database consumers
✓ Join queries
✓ Triggers
✓ Stored procedures
✓ Indexes
✓ Unique constraints
❑ Transactions
❑ Views
Payment instruments
IDX U Payment instrument ID
IDX Customer ID
…
Payment instruments
PK Customer ID
SK Payment instrument ID
…
GSI: PIID to customer ID
PK Payment instrument ID
SK Customer ID
…
Database decoupling
✓ Database consumers
✓ Join queries
✓ Triggers
✓ Stored procedures
✓ Indexes
✓ Unique constraints
✓ Transactions
✓ Views
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
3-step migration
Dual read/write
Backfill Switch
Wallet service
Business
Persistence
API
Oracle
Business
Business object
dao: Dao
…
<<interface>>
DAO
Persistence
Oracle DAO
db: Oracle
…
Business
Business object
dao: Dao
…
<<interface>>
DAO
Persistence
Dynamo DAO
db: DynamoDB
…
Amazon DynamoDBOracle
Business
Business object
dao: Dao
…
<<interface>>
DAO
Persistence
Amazon DynamoDB
Multiple DB DAO
oracleDAO: DAO
dynamoDAO: DAO
…
Oracle DAO
db: Oracle
…
DynamoDB DAO
db: DynamoDB
…
Oracle
Dual read/write
Consistent results
Verification
Verification
Transaction consistency
Execution time
All records migrated
Backfill
Records in time
Dual read/write
Backfill Continuous Backfill
Using Wallet service to migrate itself
Data migration service or
data pipelineWallet service
Backfill setup
Wallet
team
Backfill
control
interface
Wallet
service
Wallet service
clients
Hot
replica
Warm
replica
Wallet
backfill
service
Full Backfill
Continuous Backfill
Migration takes time
• Expected: 1 billion records / 1 day requires ~ 11,600 TPS
• Expected: 5,000 TPS → ~2.3 days?
• Actual: TPS ~600 = ~19 days
Getting ready to switch
✓ Dual read/write
✓ Backfill
✓ Switch
1. MATCH
2. DISCREPANT - no data found
3. INCOMPATIBLE - no data found
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Benefits
• Improved latency by 50%
• Increased request per second per host by 40%
• Reduced operations costs by 90%
Benefits
• Autoscaling (including on-demand)
• Time to live (TTL)
• Point-in-time recovery
• On-demand backup and restore
• Global tables
• Increased number of supported global secondary indexes
From Managing DB to Managed DB
Shard 2A
Shard 2B
Shard 1A
Shard 1B
Availability Zone A
Availability Zone B
SB 1-1A SB 2-1A
SB 1-2A SB 2-2A
SB 1-2B SB 2-2B
SB 1-1B SB 2-1B
Wallet
service
Wallet
service
Amazon
DynamoDB
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.