PostgreSQL ReplicationPostgreSQL Replicationin 2018in 2018

FOSDEM 2018 Brussels, Belgium

Magnus Hagander magnus@hagander.net

Magnus HaganderMagnus HaganderRedpill Linpro

Infrastructure servicesPrincipal database consultant

PostgreSQLCore Team memberCommitterPostgreSQL Europe

ReplicationReplication

"PostgreSQL doesn't have"PostgreSQL doesn't havereplication"replication"

"PostgreSQL doesn't have"PostgreSQL doesn't havereplication"replication"

"So we have to use MySQL"

ReplicationReplicationWasn't true back thenEven less true now!Now there are too many choices?

But you have to pick oneAnd can be hard to use

ReplicationReplicationCan be done at different layersFrom hardware

(ish)To application

Replication layersReplication layers↓  Application↓  App-in-database↓  Database logical↓  Database physical↓  Operating system↓  Hardware

Start from the bottomStart from the bottom

SAN replicationSAN replicationHardware takes care of replicationBlock levelTransparent to OS

And to PostgreSQLCommon enterprise solution

Especially with VMs

SAN replicationSAN replicationFrom single rackTo multi-siteSynchronousGuaranteed to never fail

Riiiiight...

Replication layersReplication layers↓  Application↓  App-in-database↓  Database logical↓  Database physical↓  Operating system↓  Hardware

DRBDDRBDSimilar in style to SANImplementation in OS driverPerformance?

Replication layersReplication layers↓  Application↓  App-in-database↓  Database logical↓  Database physical↓  Operating system↓  Hardware

Database physicalDatabase physicalWAL based replicationFile based from 8.3Streaming since 9.0Synchronous since 9.1

Transaction level mixingQuorum commit since 10(etc)

wal_level = 'replica'wal_level = 'replica'

Synchronous modeSynchronous modeofflocalonremote_apply

Synchronous modeSynchronous modeSingle

First

Quorum

synchronous_standby_names = s1,s2,s3

synchronous_standby_names = 2(s1,s2,s3)

synchronous_standby_names = ANY 2(s1,s2,s3)

Streaming replicationStreaming replicationPrimary choice todayEasy to set upHard to get wrongEfficientBuilt-in

Streaming replicationStreaming replication$ pg_basebackup -D /var/lib/pgsql \ -h primary -U replica \ -R -S replica1 -P $ sudo service postgresql-9.6 start

for pre-10, add -X stream

Streaming replicationStreaming replicationArchitecture/compile flag dependentWhole cluster onlyStandby completely read-onlyPrimary → standby onlyExcellent for availability

Streaming replicationStreaming replicationNo built-in cluster management

Manual or automaticProvides infrastructure

No fail-back(no easy one)

Easy to get started, harder to maintain

Cluster managementCluster managementPatroniPatroni

Designed for automatic managementIncluding automatic failoverUses etcd, zookeeper, or consulIntegrates with haproxy

Cluster managementCluster managementrepmgrrepmgr

Fewer pre-requisitesEasier for manual management

Comes with repmgrd that does automaticDoes not handle connection management

Use e.g. rebouncerOr haproxy

Cluster managementCluster managementPAFPAF

Integrates with pacemaker/corosyncManagement of other servicesManages virtual IP

Replication layersReplication layers↓  Application↓  App-in-database↓  Database logical↓  Database physical↓  Operating system↓  Hardware

Database logicalDatabase logicalLogical decoding since 9.4Logical replication since 10

Built-in, that isPiggy-backs on WALVery low overhead

wal_level = 'logical'wal_level = 'logical'

Logical replicationLogical replicationReconstructs changes by rowReplicates row content

not SQL statementsFully transactional

Logical replicationLogical replicationTable-level partial replicationTable-level bi-directional replication

Logical replicationLogical replicationCREATE TABLE testtable (a int PRIMARY KEY, b text);

CREATE PUBLICATION testpub FOR TABLE testtable;

Logical replicationLogical replicationCREATE TABLE testtable (a int PRIMARY KEY, b text);

CREATE SUBSCRIPTION testsub CONNECTION 'host=/tmp port=5500 dbname=postgres user=mha' PUBLICATION testpub;

Logical replicationLogical replicationData replication only

No schemaNo sequences

Suitable for data distributionBut not for HALacks failover slots!

pglogicalpglogicalExternal version of logical replicationMerged piece by pieceMore capabilities!Not as deeply integrated

pglogicalpglogicalSequence replicationRow based filteringColumn based filteringMerging and conflict resolution...

pglogicalpglogicalSupports PostgreSQL 9.4Zero (or close to zero) downtime upgrades!

Replication layersReplication layers↓  Application↓  App-in-database↓  Database logical↓  Database physical↓  Operating system↓  Hardware

App-in-databaseApp-in-databaseTrigger based systems

SlonyBucardoLondiste...

Trigger basedTrigger basedFor a long time the only choiceNow mostly supersededMuch higher overhead than logicalComplex scenarios

Multimaster?Multimaster?I I needneed it! it!

Do you really need it?Do you really know what it means?No transparent options

Because...

Multimaster?Multimaster?OptionsOptions

BDRFork!

BucardoOr just don't?

Replication layersReplication layers↓  Application↓  App-in-database↓  Database logical↓  Database physical↓  Operating system↓  Hardware

ApplicationApplicationReplication done entirely in applicationVery difficult for transactionalUseful in limited cases

SummarySummary

High AvailabilityHigh AvailabilityUse streaming replicationMix of sync and asyncConsider patroni or repmgr

Read query offloadingRead query offloadingUse streaming replication(see previous slide)

Data distributionData distributionLogical replication in 10pglogical in 9.4+

Or in 10 if built-in is not enoughUpgrade away from your Slony...

Need both?Need both?Use both!

Thank you!Thank you!Magnus Hagander

magnus@hagander.net @magnushagander

http://www.hagander.net/talks/

This material is licensed