may, 2019 - percona...© 2018, amazon web services, inc. or its affiliates. all rights reserved. ©...

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

May, 2019


Open source engines Commercial engines

RDS platform

• Automatic fail-over

• Backup & recovery

• X-region replication

• Isolation & security

• Industry compliance

• Automated patching

• Advanced monitoring

• Routine maintenance

• Push-button scaling

Image credit: By Mackphillips - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=55946550

https://commons.wikimedia.org/w/index.php?curid=55946550


CLIENTS

RDS

PostgreSQL

Aurora

PostgreSQL

EBSAurora

Storage

Postgres 9.6/10/11 — same extensions

Backup/Recovery - PITR

High Availability & Durability

Secure – IAM Auth

Read Replicas

Cross Region Snapshots

Scale Compute – Online Scale Storage

Outbound Logical Replication

Cross Region Replication

11 - Soon


Log-Based Storage


Queued Work

Log Buffer

PostgreSQL Aurora PostgreSQL

Storage

A Queued Work

Storage

B C D E

0 0 0 0 0

A B C D E

2 2 1 0 1

A B C D E

4 3 4 2 4

A B C D E

6 5 6 3 5

A B C D E

Durability

Tracking


Aurora

update t set y = 6;

Block in

Memory

t-v1t-v2t-v3

Aurora

Storage

t-v1t-v2t-v3

no

checkpoint

=

no FPW

Block in

Memory

PostgreSQL

t-v1t-v2t-v3

checkpoint

datafile

t-v1t-v2

Full

Block

t-v3

WAL

archive

4K

4K8K

update t set y = 6;


Aurora

RW

Peer

Storage

Nodes

Coalesce


0

5,000

10,000

15,000

20,000

25,000

30,000

1 31 61 91 121 151 181 211 241 271

Inse

rts

Pe

r S

eco

nd

Minutes

Insert Workload—PostgreSQL 9.6

BASE 16GB Max WAL Aurora PostgreSQL


3,729

4,871

17,158

-

5,000

10,000

15,000

20,000T

PS

(2

Up

da

tes

pe

r T

ran

sacti

on

)

Update Workload—PostgreSQL 9.6

BASE 16GB Max WAL Aurora PostgreSQL


3 GiB Redo

Recovered in 19

seconds

10 GiB Redo

Recovered in 50

seconds

30 GiB Redo

Recovered in 123

seconds

0

20

40

60

80

100

120

140

160

0 20,000 40,000 60,000 80,000 100,000 120,000 140,000

Re

co

ve

ry T

ime

in

Se

co

nd

s (l

ess

is

be

tte

r)

Writes / Second (more is better)

R EC O VERY T IME FR O M C R A SH UNDER LO A D

Bubble size represents redo log, which must be recovered

As PostgreSQL

throughput goes up, so

does log size and crash

recovery time

Amazon Aurora has no redo.

Recovered in 3 seconds while

maintaining significantly greater

throughput.


Base Architecture


RO

Application

RW

Application

RO

Application

Async

Invalidation

& Update

Async

Invalidation

& UpdateWrite log

recordsRead

blocks

RW

Automatic

Scalable

to 64TB

Availability zone 1 Availability zone 3Availability zone 2

Au

rora

Sto

rag

e

RORORORO


Durability—4 of 6 Quorum


EBS EBS

Availability zone 1

Availability zone 2 Availability zone 3

EBS

COMMIT

Region


610

21

31

712

28

123

0

20

40

60

80

100

120

140

50 90 99.9 99.99

La

ten

cy (

ms)

Percentile

High Concurrency Sync Write Test

2 Node (4 copy) 3 Node (6 Copy)


Aurora Storage

Availability zone 1

Availability zone 2 Availability zone 3

COMMIT

Region

Aurora Storage

Aurora Storage

Aurora Storage

Aurora Storage

Aurora Storage


0.00

100.00

200.00

300.00

400.00

500.00

600.00

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

resp

on

se t

ime

, m

s

minute

sysbench response time (p95), 30 GiB, 1024 clients

PostgreSQL (Single AZ, No Backup) Amazon Aurora (Three AZs, Continuous Backup)


Replicas and Clones


PostgreSQL

RW

EBS Snapshot

PostgreSQL

RO

EBS

update


Aurora

RW

Aurora

RO

update

Aurora Storage

update in

memory


Async Replication

and applyPostgreSQL

Aurora

PostgreSQL

RW ROaccounts

tellers

branches

history

accounts

tellers

branches

history

Async Replication

and memory update

RW ROaccounts

tellers

branches

history

accounts


pgbench RW 8K tps on Primary – RO 200k tps on replica

backfill

pgbench_history


RO

Application

RW

Application

RW

Reporting

Application

Write log

recordsRead

blocks


Au

rora

Sto

rag

e

Primary Storage

Clone Storage

Clone


0

5000

10000

15000

20000

25000

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78

Tra

nsa

ctio

ns

Pe

r S

eco

nd

(T

PS

)

Minutes

PGBench RW Scale 10K - Target Rate 20K TPS

Main Database

Clone Database


Replication


PostgreSQL

instance


Caching


488GB RAMPG + OS

processes

shared_buffer

s

25%

Linux

Pagecache

50+%

select of data –

check for block

in

shared_buffers

if not in

shared_buffers

load from

pagecache/disk

EBS

duplicate

buffers

PG + OS

processes

shared_buffer

s

75%

PostgreSQL Aurora PostgreSQL

Aurora

Storage

select of data –

check for block

in

shared_buffers

or load from

Aurora storage

Survivable

Cache


689,068

417,496

334,691

682,931

-

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

Tran

sact

ion

s P

er

Seco

nd

(tp

s)

pgbench read only - scale 22,000 - r4.16xlarge

Aurora 75% Cache PostgreSQL 25% Cache

1.6x 2.0x

18K read iops

no reads

heavy double

buffering

no double

buffering

no survivable

cache

Approx 350GB working

set


0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

0 60 120 180 240 300 360 420 480 540 600 660 720 780 840 900 960 1020 1080 1140 1200

Tra

nsa

cti

on

s p

er

Se

co

nd

(T

PS

)

Seconds

PGBench 20X RO / 1X RW 160GB Cached - Failover at 600 Seconds

Baseline

340 seconds

32 seconds


RO

Application

RW

Application

RO

Application

Async

Invalidation

& Update


Au

rora

Sto

rag

e

RORORORO

apg_ccm_enabled=on


0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

0 60 120 180 240 300 360 420 480 540 600 660 720 780 840 900 960 1020 1080 1140 1200

Tra

nsa

cti

on

s p

er

Se

co

nd

(T

PS

)

Seconds

PGBench 20X RO / 1X RW 160GB Cached - Failover at 600 Seconds

Baseline

CCM Enabled

32 seconds

340 seconds


Performance


• enable_bitmapscan=off

• enable_indexscan=off


Use Baseline


Vacuuming


4,000

5,000

6,000

7,000

8,000

9,000

10,000

1 61 121 181 241 301 361 421 481 541 601 661 721 781 841 901 961 1021 1081 1141 1201 1261

TP

S

Minutes

Updates—No Vacuum Running

transaction id wrap

around


PostgreSQL

Aurora PostgreSQLSubmit

Batch I/O

up to

256 blocks

402 seconds

163 seconds


Thank you!

may, 2019 - percona...© 2018, amazon web services, inc. or its affiliates. all rights reserved. ©...

Documents