under the hood: couchbase server 4.0 architecture: couchbase connect 2015

UNDER THE HOODCOUCHBASE SERVER 4.0 ARCHITECTURE

Cihan BiyikogluDirector of Product Management

©2015 Couchbase Inc. 2

Agenda

Goals Give you a full tour of the mansion! Zoom into major components and services in Couchbase Server 4.0

Impress your date…


Agenda

Overview Cluster Architecture – 10k ft view

Deep Dive Connectivity Architecture with Client SDKs Replication Architecture with DCP Cluster Management Services

Data Service, Index Service, Query Service Recap Q&A


Disclaimer

Couchbase Server 4.0 and ForestDB are still in development and the final version of the products may not be identical in details discussed on this session.

Overview


Couchbase Server 4.0 - Cluster Architecture

STORAGE

Couchbase Server 1

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache

Cluster ManagerCluster Manager

Managed CacheStorage

Data Service

Index Service

Query Service

STORAGE

Couchbase Server 2

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 3

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 4

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 5

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 6

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service






©2014 Couchbase Inc.

Couchbase Server 4.0 - Cluster Architecture

STORAGE

Couchbase Server 1

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache



Data Service

Index Service

Query Service

STORAGE

Couchbase Server 2

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 3

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 4

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 5

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service

STORAGE

Couchbase Server 6

SHARD7

SHARD9

SHARD5

SHARDSHARDSHARD

Managed Cache


Data Service

Index Service

Query Service






Deep DiveConnectivityReplication

Cluster ManagerData Service

Indexing & Index ServiceQuery Service


Connectivity and Client Libraries

Type Port Endpoint

REST 8091, 18091 Admin Connections Pointed at any node in the cluster

REST 8091, 18092 Query with View Load balanced across node of the cluster that runs data service

REST 8093, 18093 Query with N1QL Load balanced across node of the cluster that runs query service

ONLINE 11210, 11207 Core Data OperationsState-full connections from client app to nodes of the cluster that runs data service


Connectivity and Client Libraries

Connectivity Phases1. Auth2. Discovery

Get cluster map

3. Service ConnectionAuth to

ServiceRun

operationIf

(topology_change) Rerun

#2

10

1 2

…


Discovery and Cluster Map


Discovery and Cluster Map – 2 New Nodes


Database Change Protocol (DCP)

Fast Streaming Replication DCP - An open streaming protocol that conveys the consistent database state to all

consumers Ordering (vbucket based seq.number) Re-startable, Resumable (version histories and rollbacks) Consistent (snapshots) High Performance (memory based with dedup)

Master

Local Replic

a

Index

Map/Reduc

e

Remote

Replica

IndexMap/Reduc

e

Source Cluster

Cross Data Center Cluster

Hadoop

Client/Applicati

on

NotificationIn future

Integration

Backup/Export

Tooling


Cluster Manager

Cluster Manager = Governor of the ClusterManages cluster level operations and coordination among nodes

Cluster Membership & Service Layout Node Status & Failover Data Placement & Rebalance Auth

17


Cluster Manager

Inside Cluster Manager

per-node-&-bucket services

generic distributed facilities

generic local facilities

Logging and Other Services

distributed node discovery

Master Services- cluster level

operations - data placement - rebalancer- auto-failover

Admin Portal – REST API

Global Config (gossip replication)

Local Config Store

Per-node Services - Heartbeats, - Babysitter

Bucket services - dcp init and teardown- stats collectors,

Auth

19©2014 Couchbase Inc.

Adding Nodes to Cluster Online

ACTIVE ACTIVE ACTIVE

REPLICA REPLICA REPLICA

Couchbase Server 1 Couchbase Server 2 Couchbase Server 3

ACTIVE ACTIVE

REPLICA REPLICA

Couchbase Server 4 Couchbase Server 5

SHARD5

SHARD2

SHARD SHARD

SHARD4

SHARD SHARD

SHARD1

SHARD3

SHARD SHARD

SHARD4

SHARD1

SHARD8

SHARD SHARD SHARD

SHARD6

SHARD3

SHARD2

SHARD SHARD SHARD

SHARD7

SHARD9

SHARD5

SHARD SHARD SHARD

SHARD7

SHARD

SHARD6

SHARD

SHARD8

SHARD9

SHARD

READ/WRITE/UPDATE

Cluster Manager receives the new nodes

- Node inherit cluster settings

- Move active and replica vbuckets using DCP

- As vbuckets catch up, Initiate online handoff from “existing node” to “new node”

Clients Receive Topology Change Notification

- Trap not_my_vbucket errors

- Refresh cluster map and retry operation


Deeper Dive into Architecture

THUR @1.00 - Architecture Track

Deep Dive into Cluster Manager in Couchbase Server 4.0

Dave Finlay, Senior Director of Development, Couchbase


Data Service Data Service = GET/SET + Map-Reduce Views*Tackles fast core data operations with efficient caching and disk persistence

Core Database Operations Core GET/SET operations Couchstore Based Storage

Terms:Bucket = database reside within a clustervBucket = hash partition of the database that reside within a node 22


Data Manager Architecture

…

Database Engine (ep-engine)

Listener

vBucket Manager

Item Pager

Expiry PagerCheckpoint Manager

CachePartition Hash

Tables (Active and

Replica)

Partition Hash Tables

(Active and Replica)

Partition Hash Tables

(Active and Replica)

AuthNetwork IO

Flusher

Scheduler

Reader IO

Writer IO

Non IO

Batch Reader

©2014 Couchbase Inc.

APPLICATION SERVER

MANAGED CACHE

DISK

DISK

Data Manager - Update Operation

24

DOC 1

DOC 1

DOC 1

DOC 1DOC 1

Caching based on Memcached: App gets an ACK when update is successfully in RAM Or RAM+Replicated

Or RAM+Persisted

Or RAM+Replicated+Persisted

Or …

DCP base Replication: updates queued to other nodes

Couchstore based Storage: updates queued for storage

DCP


Data Manager - Cache Ejection

APPLICATION SERVER

MANAGED CACHE

DISK

DISK

DOC 1

DOC 2DOC 3DOC 4DOC 5

DOC 1

DOC 2 DOC 3 DOC 4 DOC 5

DCP


APPLICATION SERVER

MANAGED CACHE

DISK

DISK

DCP

DOC 1

Data Manager - Cache Miss



GETDOC 1

DOC 1

DOC 1



WED @4.30 - Architecture Track

Next Generation Storage Engine: ForestDB

Chiyoung Seo, Software Engineer, Couchbase



THUR @2.30 - Architecture Track

Under the Hood ForestDB: Performance on SSDs and File

Systems

Sundar Sridharan, Senior Developer, Couchbase Prof. Sang-Won Lee, Sungkyunkwan University,

Suwon, Korea


Indexing

Indexers Views: Incremental Map/Reduce with customer

JavaScript for complex indexing logic for online reporting and analytics

Spatial Views: Incremental/Reduce with R-tree Indexes for Bounding-Box queries

GSI (Global Secondary Indexes): Efficient indexes for secondary lookups and ad-hoc query processing


Indexing with Views

Incremental Map/Reduce ViewsQueries with incremental map/reduce processing that can execute custom JS

Flexible Indexer Indexer & Query Processor Couchstore Based Storage

31


Views

32

APPLICATION SERVER

VIEWINDEXER

Query Set

1. SET operation

• In Memory SET: queued for DCP Replication

• View Indexer: Executes incremental map/reduce on a batch of updates

• Couchstore based Storage: updates queued for storage

2. View Query Engine: REST Based queries with filters, limit and more executed with scatter-gather


Index Service Global Secondary Indexes (NEW in 4.0)Tackles indexer for fast query execution with efficient index maintenance for N1QL Queries

High Performance Indexing Projector and Router : Coordinate and communicate efficient

index change notifications between data service and index service.

Supervisor – Indexer and scannerIndexer : Maintain large number of indexes as change

notifications arriveScanner: Respond to Query Service index-scan requests with

rich set of consistency dials

Index Storage &CachingForestDB: Brand new storage engine for high performance

index caching and storage

33


Data Service

Projector & Router

Indexing Service

Query ServiceIndex Service

SupervisorIndex maintenance &

Scan coordinator

Index#2

Index#1

Query Processorcbq-engine

Bucket#1

Bucket#2

DCP StreamIndex#4Index#3

...

Bucket#2

Bucket#1

Projector and Router: 1 Projector and Router per node1 stream of changes per buckets per supervisor

ForestDBStorage Engine

Supervisor1 Supervisor per nodeMany indexes per Supervisor




Global Secondary Indexing in Couchbase Server 4.0

Cihan Biyikoglu, Director of Product Management, Couchbase




Deep Dive into Global Secondary Indexing Architecture in Couchbase

Server 4.0

John Liang, Architect, Couchbase


Query Service Query Service = N1QLTackles N1QL Query execution

Query Execution N1QL Parser & Optimizer: tokenize N1QL statement, and

generate an execution plan based utilizing indexes Query Execution Engine: Assigns resources to query and

coordinates query execution. Data Sources: Pluggable “data source driver” layer for

accessing data sources in Couchbase Server (data and index service) and other external data provides

38


Query Service N1QL Query Processing

Query Engine

Query Processor

Listeners

Parser Optimizer

Data Stores

Execution Engine

Couchbase Server

Auth DataIndexers

GSIView

s

Others…

8093/18903

File systemData Service

Index Service

......

Cluster Manager

Bucket#2

Bucket#2

Index#2

Index#1




Deep Dive into N1QL Internals in Couchbase Server 4.0

Keshav Murthy, Director of Development, Couchbase


Recap

Couchbase Server decouples competing workloads into Services

Each Service tunes itself with caching and storage services

Each Service can be independently scale

Q&ACihan Biyikoglu

[email protected]@cihangirb

Get Started Today Couchbase Server 4.0 & N1QL

Couchbase.com/beta

Thank you.

under the hood: couchbase server 4.0 architecture: couchbase connect 2015

Technology

query service recap

hood couchbase server

disclaimer couchbase

cluster rest

query withview load

client libraries

agenda goals

port endpoint rest