introduction to couchbase server 2.0

1

Introduction to Couchbase Server 2.0

Dipti BorkarDirector, Product Management

2

2.0

NoSQL Document Database

Couchbase Server

3

Easy Scalabili

ty

Consistent High

Performance

Always On

24x365

Grow cluster without application changes, without downtime with a single click

Consistent sub-millisecond read and write response times

with consistent high throughput

No downtime for software upgrades, hardware maintenance, etc.

Couchbase Server

JSONJSONJSON

JSONJSON

PERFORMANCE

Flexible Data Model

JSON document model with no fixed schema.

4

Flexible Data Model

• No need to worry about the database when changing your application

• Records can have different structures, there is no fixed schema

• Allows painless data model changes for rapid application development

{ “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA”}

JSONJSON

JSON JSON

5

New in 2.0

JSON support Indexing and Querying

Cross data center replicationIncremental Map Reduce

JSONJSONJSON

JSONJSON

6

Couchbase Server 2.0 Architecture

Hea

rtbe

at

Proc

ess

mon

itor

Glo

bal s

ingl

eton

sup

ervi

sor

Confi

gura

tion

man

ager

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpREST managem

ent A

PI/W

eb UI

HTTP8091

Erlang port mapper4369

Distributed Erlang21100 - 21199

Erlang/OTP

storage interface

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

Memcached

New Persistence Layer

8092Query API

Query Engine

Data Manager Cluster Manager

7

Couchbase Server 2.0 Architecture

Hea

rtbe

at

Proc

ess

mon

itor

Glo

bal s

ingl

eton

sup

ervi

sor

Confi

gura

tion

man

ager

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpREST managem

ent A

PI/W

eb UI

HTTP8091

Erlang port mapper4369

Distributed Erlang21100 - 21199

Erlang/OTP

storage interface

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

Object-level Cache

New Persistence Layer

8092Query API

Query Engine

8

Couchbase deployment

Data Flow

Cluster Management

Web Application

CouchbaseClient Library

9

COUCHBASE OPERATIONS

10

33 2

Single node - Couchbase Write Operation2

Managed Cache

Dis

k Q

ueue

Disk

Replication Queue

App Server

Couchbase Server Node

Doc 1Doc 1

Doc 1

To other node

11

33 2

Single node - Couchbase Update Operation2

Managed Cache

Dis

k Q

ueue

Replication Queue

App Server


Doc 1’

Doc 1

Doc 1’Doc 1

Doc 1’

Disk

To other node

12

GET

Doc 1

33 2

Single node - Couchbase Read Operation2

Dis

k Q

ueue

Replication Queue

App Server


Doc 1

Doc 1Doc 1

Managed Cache

Disk

To other node

13

33 2

Single node - Couchbase Cache Eviction2

Dis

k Q

ueue

Replication Queue

App Server


Doc 1

Doc 6Doc 5Doc 4Doc 3Doc 2

Doc 1

Doc 6 Doc 5 Doc 4 Doc 3 Doc 2

Managed Cache

Disk

To other node

14

33 2

Single node – Couchbase Cache Miss2

Dis

k Q

ueue

Replication Queue

App Server


Doc 1

Doc 3Doc 5 Doc 2Doc 4

Doc 6 Doc 5 Doc 4 Doc 3 Doc 2

Doc 4

GET

Doc 1

Doc 1

Doc 1

Managed Cache

Disk

To other node

15

COUCHBASE SERVER CLUSTER

Cluster wide - Basic Operation

• Docs distributed evenly across servers

• Each server stores both active and replica docsOnly one server active at a time

• Client library provides app with simple interface to database

• Cluster map provides map to which server doc is onApp never needs to know

• App reads, writes, updates docs

• Multiple app servers can access same document at same time

User Configured Replica Count = 1

READ/WRITE/UPDATE

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 1

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc

SERVER 2

Doc 8

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

REPLICA

Doc 6

Doc 3

Doc 2

Doc

Doc

Doc

REPLICA

Doc 7

Doc 9

Doc 5

Doc

Doc

Doc

SERVER 3

Doc 6

APP SERVER 1

COUCHBASE Client Library

CLUSTER MAP


CLUSTER MAP

APP SERVER 2

Doc 9

16

Cluster wide - Add Nodes to Cluster

• Two servers addedOne-click operation

• Docs automatically rebalanced across clusterEven distribution of docsMinimum doc movement

• Cluster map updated

• App database calls now distributed over larger number of servers

REPLICA

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc 4

Doc 1

Doc

Doc

SERVER 1

REPLICA

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc 6

Doc 3

Doc

Doc

SERVER 2

REPLICA

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc 7

Doc 9

Doc

Doc

SERVER 3 SERVER 4 SERVER 5

REPLICA

ACTIVE

REPLICA

ACTIVE

Doc

Doc 8 Doc

Doc 9 Doc

Doc 2 Doc

Doc 8 Doc

Doc 5 Doc

Doc 6

READ/WRITE/UPDATE READ/WRITE/UPDATE

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2



17

Cluster wide - Fail Over Node

REPLICA

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc 4

Doc 1

Doc

Doc

SERVER 1

REPLICA

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc 6

Doc 3

Doc

Doc

SERVER 2

REPLICA

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc 7

Doc 9

Doc

Doc

SERVER 3 SERVER 4 SERVER 5

REPLICA

ACTIVE

REPLICA

ACTIVE

Doc 9

Doc 8

Doc Doc 6 Doc

Doc

Doc 5 Doc

Doc 2

Doc 8 Doc

Doc

• App servers accessing docs

• Requests to Server 3 fail

• Cluster detects server failedPromotes replicas of docs to activeUpdates cluster map

• Requests for docs now go to appropriate server

• Typically rebalance would follow

Doc

Doc 1 Doc 3

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2



18

DEMO TIME

19

Indexing and Querying – The basics

• Define materialized views on JSON documents and then query across the data set • Using views you can define

• Primary indexes • Simple secondary indexes (most common use case)• Complex secondary, tertiary and composite indexes• Aggregations (reduction)

• Indexes are eventually indexed • Queries are eventually consistent with respect to

documents• Built using Map/Reduce technology

• Map and Reduce functions are written in Javascript

20

33 2

Eventually indexed Views – Data flow2

Managed Cache

Dis

k Q

ueueDisk

Replication Queue

App Server


Doc 1Doc 1

Doc 1

To other node

View engine

Doc 1

21


Cluster wide - Indexing and Querying


ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 1

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

APP SERVER 1


CLUSTER MAP


CLUSTER MAP

APP SERVER 2

Doc 9

• Indexing work is distributed amongst nodes

• Large data set possible

• Parallelize the effort

• Each node has index for data stored on it

• Queries combine the results from required nodes

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 2

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

Doc 9

ACTIVE

Doc 5

Doc 2

Doc

Doc

Doc

SERVER 3

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

Doc 9

Query

22

Cross Data Center Replication – The basics

• Replicate your Couchbase data across clusters• Clusters may be spread across geos• Configured on a per-bucket basis• Supports unidirectional and bidirectional operation• Application can read and write from both clusters

(active – active replication)• Replication throughput scales out linearly• Different from intra-cluster replication

23

33 2

Cross data center replication – Data flow2

Managed Cache

Dis

k Q

ueue

Disk

Replication Queue

App Server


Doc 1Doc 1

Doc 1

To other node

XDCR Queue

Doc 1

To other cluster

24

Cluster wide - XDCR

COUCHBASE SERVER CLUSTERNY DATA CENTER

ACTIVE

Doc

Doc 2

SERVER 1

Doc 9

SERVER 2 SERVER 3

RAM

Doc Doc Doc

ACTIVE

Doc

Doc

Doc RAM

ACTIVE

Doc

Doc

DocRAM

DISK

Doc Doc Doc

DISK

Doc Doc Doc

DISK

COUCHBASE SERVER CLUSTERSF DATA CENTER

ACTIVE

Doc

Doc 2

SERVER 1

Doc 9

SERVER 2 SERVER 3

RAM

Doc Doc Doc

ACTIVE

Doc

Doc

Doc RAM

ACTIVE

Doc

Doc

DocRAM

DISK

Doc Doc Doc

DISK

Doc Doc Doc

DISK

25

DEMO TIME

26

Demo: The next big social game

3 Objects (documents) within game:• Players•Monsters• Items

Gameplay:• Players fight monsters•Monsters drop items• Players own items

27

Player Document

{"jsonType": "player","uuid": "35767d02-a958-4b83-8179-616816692de1","name": "Keith4540","hitpoints": 75,"experience": 663,"level": 4,"loggedIn": false

}

Player ID

28

Item Document

{"jsonType": "item","name": "Katana_e5890c94-11c6-65746ce6c560","uuid": "e5890c94-11c6-4856-a7a6-65746ce6c560","ownerId": "Dale9887"

}

Item ID

Player ID

29

Monster Document

{"jsonType": "monster","name": "Bauchan9932","uuid": "d10dfc1b-0412-4140-b4ec-affdbf2aa5ec","hitpoints": 370,"experienceWhenKilled": 52,"itemProbability": 0.5050581341872865

}

Monster ID

30

GAME ON!

31

Full Text Search Integration

• Elastic Search is good for ad-hoc queries and faceted browsing• Couchbase adapter uses XDCR to push mutations to ESDocs are

indexed by Elastic Search• Couchbase ES Adapter is cluster-aware

ElasticSearch

Unidirectional Cross Data Center Replication

32

Full Text Search

33

Couchbase SDKs

Java SDK

.Net SDK

PHP SDK

Ruby SDK

…and many more

Java client API

User Code

Couchbase Server

CouchbaseClient cb = new CouchbaseClient(listURIs,"aBucket", "letmein");

cb.set("hello", 0, "world");cb.get("hello");

http://www.couchbase.com/develop

Couchbase Java Library (spymemcached)

34

QUESTIONS?

34

35

THANK YOU

[email protected]@DBORKAR

35

introduction to couchbase server 2.0

Technology

server doc

doc servers doc

docdoc doc

doc docs

operation doc

doc calls

doc cluster map

node replication doc