dcp - deep dive on the next generation streaming protocol: couchbase connect 2014

Download DCP - Deep Dive on the Next Generation Streaming Protocol: Couchbase Connect 2014

Post on 20-Aug-2015



Data & Analytics

4 download

Embed Size (px)


  1. 1. DCP: Deep Dive on the NextGeneration Streaming ProtocolCihan Biyikoglu | Director of Product Management, CouchbaseMike Wiederhold | Software Engineer, Couchbase
  2. 2. AgendaAgenda Architecture OverviewDCP within Couchbase Architecture DCP BenefitsRebalance, XDCR, Views and more DCP Under the HoodDCP Protocol PropertiesHow DCP recovers from failuresand more2014 Couchbase, Inc. 2
  3. 3. DCP and Couchbase Server Architecture
  4. 4. DCP and Couchbase Server ArchitectureData Sync is the Heart of Any Big Data System!Fundamental piece of the architecture!- Data Sync maintains Data Redundancy for HA&DR- Protect against failures node, rack, region etc.- Data Sync maintains Indexes!- Spatial, Full-text,- Indexing is key to building faster access paths to query data
  5. 5. What is DCP?What is DCP?DCP is an innovative protocol that drive data sync for Couchbase Serverv3.0. Increase data sync efficiency with massive data footprints Remove slower Disk-IO from the data sync path Improve latencies replication for data durability In future, will provide a programmable data sync protocol for externalstores outside Couchbase ServerIn 3.0, DCP powers many critical components Lets take a look!2014 Couchbase, Inc. 5
  6. 6. Couchbase Server Architecture Data Sync with Replicationread/write/updateSERVER 1ActiveSERVER 2ActiveAPP SERVER 2SERVER 3ActiveAPP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPShard 5Shard 2Shard 9ShardShardShardShard 4Shard 7Shard 8ShardShardShardShard 1Shard 3Shard 6ShardShardShardReplica Replica ReplicaShard 4Shard 1Shard 8ShardShardShardShard 6Shard 3Shard 2ShardShardShardShard 7Shard 9Shard 5ShardShardShardDCP Replicates Data Among Nodes
  7. 7. APP SERVER 2SERVER 4 SERVER 5ActiveReplicaActiveReplicaAPP SERVER 1COUCHBASE Client LibraryCLUSTER MAPread/write/updateCOUCHBASE Client LibraryCLUSTER MAPSERVER 1ActiveShard 9 ShardReplicaShard 4Shard 1Shard 8ShardShardShardSERVER 2ActiveShard 8 ShardReplicaShard 6Shard 3Shard 2ShardShardShardSERVER 3ActiveShard 6 ShardReplicaShard 7Shard 9Shard 5ShardShardShardread/write/updateShard 5Shard 2ShardShardShard 4Shard 7ShardShardShard 1Shard 3ShardShardCouchbase Server ArchitectureDCP Drives Rebuilding of Replicas Under Topology Changes
  8. 8. Couchbase Server Architecture - ViewsSERVER 1ActiveShard 5Shard 2ShardShardReplicaShard 4Shard 1ShardShardShard 1APP SERVER 2SERVER 3ActiveShard 5Shard 2ShardShardReplicaShard 4Shard 1ShardShardShard 1SERVER 2ActiveShard 5Shard 2ShardShardReplicaShard 4Shard 1ShardShardShard 1APP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPDCP Powers View Maintenance
  9. 9. Couchbase Server San FranciscoSERVER 1 SERVER 2 SERVER 3Couchbase Server New YorkSERVER 1 SERVER 2 SERVER 3Couchbase Server ArchitectureDCP Powers XDCR Cross Data Center Replication
  10. 10. Tap vs. DCPTAP DCPMajor difference between TAP and DCP:Ordering No ordering guaranteed! Ordered Tap guarantees you will see all mutations at least once, but doesnt guarantee any specific order Tap doesnt have the ability to restart from anywhere De-duplication of items means that we cannot tell when we have a consistent view of the databaseRestart-ability Not really! Granular Restart-abilityConsistency No snapshotting capabilitieshere!Snapshots give a consistentview of the DB.Performance No memory based supportfor Views & XDCRMemory based all datasynchronizationcomponents!2014 Couchbase, Inc. 10
  11. 11. New Features
  12. 12. New Capabilities with DCPDCP allows us to build many features that were not possible with the TAPprotocol Faster indexing Faster XDCR Incremental Backups Delta Recovery for Faster Rebalance Improved Durability with ReplicateTo And many more2014 Couchbase, Inc. 12
  13. 13. XDCR With DCPUp to 4x better on XDCR latency between clusters between 3.0 & 2.5.1*Note: Absolute latency depend on WAN latency and bandwidthFaster XDCR6005004003002001000Time (ms)90th Percentile. Replication Latency2.5.1 3.0Major Architectural Change: In 2.x changes needed to bepersisted before the XDCR modulereplicated them In 3.0 DCP allows changes to bereplicated from memoryThe result is lower latency and higherthroughput for XDCR traffic2014 Couchbase, Inc. 1390th percentile replication lag (ms), 5 -> 5 UniDir, 2 buckets x 500M x 1KB, 10K SETs/sec, LAN
  14. 14. Views with DCPUp to 50x Faster Indexing from 2.5 to 3.0*Note: Absolute latency depend on disk IO speedFaster Views4000035000300002500020000150001000050000Time (ms)Indexing Under Load2.5.1 3.0Major architectural changes: In 2.x, the indexer required mutations to bepersisted before they were eligible for indexing In 3.0 DCP allow mutations to be indexed as soonas the index can consume them through a DCPstream Developers no longer need to observe forpersistence before issuing a stale=falsequery2014 Couchbase, Inc. 14*95th percentile Indexing latency (ms), 1 bucket x 20M x 2KB, non-DGM, 1 view, 250 mutations/sec/node, 400 queries/sec
  15. 15. Incremental BackupCouchbase 3.0 supports incremental backups by taking advantage of theordered property in DCPMajor advantages: Shorter backup times Smaller backup size2014 Couchbase, Inc. 15
  16. 16. Delta Node Recovery with DCPDelta Recovery100sx Reduction in Time to Re-Add Node from 2.5 to 3.0Note: The absolute performance improvement depend on data size and mutation count that needs to be caught upMajor Advantages: In 2.x adding back a server means completely rebuilding it! In 3.x the cluster manager utilizes the ordering property of DCP to resumereplica building from where the added server left offThe net result is shorter time to add failed nodes back into the cluster meaningless time spent rebalancing2014 Couchbase, Inc. 16
  17. 17. Faster Durability with ReplicateToDurability with DCPUp to 150x Improvement on ReplicateTo latency between 3.0 and 2.5.1*Note: Absolute latency depend on network bandwidth and latency6004002000Time (ms)95 Percentile ReplicateTo=1Latency2.5.1 3.0Improved latency on ReplicateTo DCP Improves replication speed and Durability with replication and betterprotects data!*95th percentile ReplicateTo=1 latency (ms), 1 bucket x 200M x 1KB, 250 mutations/sec/node
  18. 18. DCP Deep Dive
  19. 19. Major Architectural ConceptsMajor Architecture Concepts: Ordering Restart-ability Consistency (Snapshots) Performance (Memory-Based)2014 Couchbase, Inc. 19
  20. 20. Ordered MutationsTo build interesting features a streaming protocol needs to have aconcept of when operations happened.Couchbase operation ordering at the vBucket level: Each mutation is assigned a sequence number Sequence numbers increase monotonically Sequence numbers are assigned on a per vBucket basis2014 Couchbase, Inc. 20
  21. 21. Version HistoriesIn a distributed system failure is expected and we must be able to detectthese failures to properly keep ordering in the system.Failover Logs: Each VBucket contains a failover log A failover log contains one or more failover entries Each failover entry contains a VBucket UUID/Sequence Number pair2014 Couchbase, Inc. 21
  22. 22. Version History (Failover Example)VB UUID SeqnoX 0VB UUID SeqnoY 100100 Lastt Seqno:: 1000DCP Replication StreamX 0Last Seqno: 0Active RAecptilvicea2014 Couchbase, Inc. 22
  23. 23. RestartabilityFine grained restartability: Ordering allows Couchbase nodes to resume receiving data from any point If nodes contain different data histories we can reconcile the differencesusing rollbacks2014 Couchbase, Inc. 23
  24. 24. Restartability ExampleVB UUID SeqnoX 0VB UUID SeqnoY 100Y 100Last Seqno: 80 Last Seqno: 1500


View more >