launch webinar - introducing couchbase server 2.0
TRANSCRIPT
![Page 1: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/1.jpg)
![Page 2: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/2.jpg)
Introducing Couchbase Server 2.0
Dipti BorkarDirector, Product Management
![Page 3: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/3.jpg)
2.0
NoSQL Document Database
Couchbase Server
![Page 4: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/4.jpg)
• Leading NoSQL database project focused on distributed database technology and surrounding ecosystem
• Supports both key-value and document-oriented use cases
• All components are available under the Apache 2.0 Public License
• Obtained as packaged software in both enterprise and community editions.
Couchbase Open Source Project
Couchbase Open Source Project
![Page 5: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/5.jpg)
Easy Scalabili
ty
Consistent High
Performance
Always On
24x365
Grow cluster without application changes, without downtime with a single click
Consistent sub-millisecond read and write response times
with consistent high throughput
No downtime for software upgrades, hardware maintenance, etc.
JSONJSONJSON
JSONJSON
PERFORMANCE
Flexible Data Model
JSON document model with no fixed schema.
Couchbase Server
![Page 6: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/6.jpg)
Flexible Data Model
• No need to worry about the database when changing your application
• Records can have different structures, there is no fixed schema
• Allows painless data model changes for rapid application development
{ “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA”}
JSONJSON
JSON JSON
![Page 7: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/7.jpg)
New in 2.0
JSON support Indexing and Querying
Cross data center replicationIncremental Map Reduce
JSONJSONJSON
JSONJSON
![Page 8: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/8.jpg)
Additional Couchbase Server Features
Built-in clustering – All nodes equal
Data replication with auto-failover
Zero-downtime maintenance
Built-in managed cached
Append-only storage layer
Online compaction
Monitoring and admin API & UI
SDK for a variety of languages
![Page 9: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/9.jpg)
Couchbase Server 2.0 Architecture
Hea
rtbe
at
Proc
ess
mon
itor
Glo
bal s
ingl
eton
sup
ervi
sor
Confi
gura
tion
man
ager
on each node
Reba
lanc
e or
ches
trat
or
Nod
e he
alth
mon
itor
one per cluster
vBuc
ket s
tate
and
repl
icati
on m
anag
er
http
REST
man
agem
ent A
PI/W
eb U
I
HTTP8091
Erlang port mapper4369
Distributed Erlang21100 - 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached
New Persistence Layer
8092Query API
Que
ry E
ngin
e
Data Manager Cluster Manager
![Page 10: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/10.jpg)
Couchbase Server 2.0 Architecture
Hea
rtbe
at
Proc
ess
mon
itor
Glo
bal s
ingl
eton
sup
ervi
sor
Confi
gura
tion
man
ager
on each node
Reba
lanc
e or
ches
trat
or
Nod
e he
alth
mon
itor
one per cluster
vBuc
ket s
tate
and
repl
icati
on m
anag
er
http
REST
man
agem
ent A
PI/W
eb U
I
HTTP8091
Erlang port mapper4369
Distributed Erlang21100 - 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached
New Persistence Layer
8092Query API
Que
ry E
ngin
e
![Page 11: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/11.jpg)
COUCHBASE OPERATIONS
![Page 12: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/12.jpg)
33 2
Single node - Couchbase Write Operation
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
![Page 13: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/13.jpg)
33 2
Single node - Couchbase Update Operation
Managed Cache
Dis
k Q
ueue
Replication Queue
App Server
Doc 1’
Doc 1
Doc 1’Doc 1
Doc 1’
Disk
To other node
Couchbase Server Node
![Page 14: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/14.jpg)
GET
Doc
1
33 2
Single node - Couchbase Read Operation
Dis
k Q
ueue
Replication Queue
App Server
Doc 1
Doc 1Doc 1
Managed Cache
Disk
To other node
Couchbase Server Node
![Page 15: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/15.jpg)
COUCHBASE SERVER CLUSTER
Basic Operation
• Docs distributed evenly across servers
• Each server stores both active and replica docsOnly one server active at a time
• Client library provides app with simple interface to database
• Cluster map provides map to which server doc is onApp never needs to know
• App reads, writes, updates docs
• Multiple app servers can access same document at same time
User Configured Replica Count = 1
READ/WRITE/UPDATE
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 1
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc
SERVER 2
Doc 8
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
REPLICA
Doc 6
Doc 3
Doc 2
Doc
Doc
Doc
REPLICA
Doc 7
Doc 9
Doc 5
Doc
Doc
Doc
SERVER 3
Doc 6
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
Doc 9
![Page 16: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/16.jpg)
Add Nodes to Cluster
• Two servers addedOne-click operation
• Docs automatically rebalanced across clusterEven distribution of docsMinimum doc movement
• Cluster map updated
• App database calls now distributed over larger number of servers
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3 SERVER 4 SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc
Doc 8 Doc
Doc 9 Doc
Doc 2 Doc
Doc 8 Doc
Doc 5 Doc
Doc 6
READ/WRITE/UPDATE READ/WRITE/UPDATE
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
COUCHBASE SERVER CLUSTER
User Configured Replica Count = 1
![Page 17: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/17.jpg)
Fail Over Node
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3 SERVER 4 SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc 9
Doc 8
Doc Doc 6 Doc
Doc
Doc 5 Doc
Doc 2
Doc 8 Doc
Doc
• App servers accessing docs
• Requests to Server 3 fail
• Cluster detects server failedPromotes replicas of docs to activeUpdates cluster map
• Requests for docs now go to appropriate server
• Typically rebalance would follow
Doc
Doc 1 Doc 3
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
User Configured Replica Count = 1
COUCHBASE SERVER CLUSTER
![Page 18: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/18.jpg)
DEMO TIME
![Page 19: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/19.jpg)
• Define materialized views on JSON documents and then query across the data set
• Using views you can define• Primary indexes • Simple secondary indexes (most common use case)• Complex secondary, tertiary and composite indexes• Aggregations (reduction)
• Indexes are eventually indexed
• Queries are eventually consistent
• Built using Map/Reduce technology • Map and Reduce functions are written in Javascript
Indexing and Querying – The basics
![Page 20: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/20.jpg)
COUCHBASE SERVER CLUSTER
Indexing and Querying
User Configured Replica Count = 1
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 1
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
Doc 9
• Indexing work is distributed amongst nodes
• Large data set possible
• Parallelize the effort
• Each node has index for data stored on it
• Queries combine the results from required nodes
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 2
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Doc 9
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 3
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
Doc 9
Query
![Page 21: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/21.jpg)
• Replicate your Couchbase data across clusters
• Clusters may be spread across geos
• Configured on a per-bucket (per-database) basis
• Supports unidirectional and bidirectional operation
• Application can read and write from both clusters – Active – Active replication
• Replication throughput scales out linearly
• Different from intra-cluster replication
Cross Data Center Replication – The basics
![Page 22: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/22.jpg)
33 2
Cross data center replication – Data flow
2
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
XDCR Queue
Doc 1
To other cluster
![Page 23: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/23.jpg)
Cross Data Center Replication (XDCR)COUCHBASE SERVER CLUSTER
NY DATA CENTERACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2 SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
DocRAM
DISK
Doc Doc Doc
DISK
Doc Doc Doc
DISK
COUCHBASE SERVER CLUSTERSF DATA CENTER
ACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2 SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
DocRAM
DISK
Doc Doc Doc
DISK
Doc Doc Doc
DISK
![Page 24: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/24.jpg)
Couchbase SDKs
Java SDK
.Net SDK
PHP SDK
Ruby SDK
…and many more
Java client API
User Code
Couchbase Server
CouchbaseClient cb = new CouchbaseClient(listURIs,"aBucket", "letmein");
cb.set("hello", 0, "world");cb.get("hello");
http://www.couchbase.com/develop
Couchbase Java Library (spymemcached)
![Page 25: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/25.jpg)
DEMO TIME
![Page 26: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/26.jpg)
Demo: The next big social game
3 Objects (documents) within game:• Players•Monsters• Items
Gameplay:• Players fight monsters•Monsters drop items• Players own items
![Page 27: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/27.jpg)
Player Document
{"jsonType": "player","uuid": "35767d02-a958-4b83-8179-616816692de1","name": "Keith4540","hitpoints": 75,"experience": 663,"level": 4,"loggedIn": false
}
Player ID
![Page 28: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/28.jpg)
Item Document
{"jsonType": "item","name": "Katana_e5890c94-11c6-65746ce6c560","uuid": "e5890c94-11c6-4856-a7a6-65746ce6c560","ownerId": "Dale9887"
}
Item ID
Player ID
![Page 29: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/29.jpg)
Monster Document
{"jsonType": "monster","name": "Bauchan9932","uuid": "d10dfc1b-0412-4140-b4ec-affdbf2aa5ec","hitpoints": 370,"experienceWhenKilled": 52,"itemProbability": 0.5050581341872865
}
Monster ID
![Page 30: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/30.jpg)
GAME ON
![Page 31: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/31.jpg)
www.couchbase.com/download
![Page 32: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/32.jpg)
MCGRAW HILL EDUCATION LABS LEARNING PORTAL
![Page 33: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/33.jpg)
Use Case: Content and metadata store
Building a self-adapting, interactive learning portal with Couchbase
![Page 34: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/34.jpg)
As learning move online in great numbers
Growing need to build interactive learning environments that
Scale!
Scale to millions of learners
Serve MHE as well as third-party content
Including open content
Support learning apps
010100100111010101010101001010101010
Self-adapt via usage data
The Problem
![Page 35: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/35.jpg)
• Allow for elastic scaling under spike periods
• Ability to catalog & deliver content from many sources
• Consistent low-latency for metadata and stats access
• Require full-text search support for content discovery
• Offer tunable content ranking & recommendation functions
Backend is an Interactive Content Delivery Cloud that must:
XML Databases
SQL/MR Engines
In-memory Data Grids
Enterprise Search Servers
Experimented with a combination of:
Hmmm...this looks kinda like:+ Content Caching (Scale)+ Social Gaming (Stats) + Ad Targeting (Smarts)
The Challenge
![Page 36: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/36.jpg)
The Technologies
![Page 37: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/37.jpg)
The Learning Portal
• Designed and built as a collaboration between MHE Labs and Couchbase
• Serves as proof-of-concept and testing harness for Couchbase + ElasticSearch integration
• Available for download and further development as open source code
![Page 38: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/38.jpg)
• Document Modeling
• Metadata & Content Storage
• View Querying to support Content Browsing
• Elastic Search Integration (Full Text Search)
-Content Updated in near Real-Time
-Search Content Summaries
-Relevancy boosted based on User Preferences
• Real-Time Content Updates
• Event Logging for offline analysis
Techniques Used
![Page 39: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/39.jpg)
Couchbase 2.0 + Elasticsearch
Store full-text articles as well as document metadata for image, video and text content in Couchbase
Combine user preferences statistics with custom relevancy scoring to provide personalized search results
Logs user behavior to calculate user preference statistics (e.g. video > text)
1
2 4
Continuously accept updates from Couchbase with new content & stats
3
![Page 40: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/40.jpg)
Data Model
Content Metadata Bucket
User ProfilesBucket
Content StatsBucket
• Stores content metadata for media objects and content for articles
• Includes tags, contributors, type information
• Includes pointer to the media
• Stores user view details per type
• Updated every time a user views a doc with running count
• To be used for customizing ES search results per user preference• Stores content view details
• Updated for every time a document is viewed
• To be used for boosting ES search results based on popularity
![Page 41: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/41.jpg)
Via XDCR
Couchbase ES Transport
Data
Couchbase Server Cluster
MR Views MR Views MR Views MR Views Elastic Search Cluster
RefsTS Query
External Media Store
View Query
App Server
Couchbase Ruby SDK
ES queries over HTTP
Architecture
![Page 42: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/42.jpg)
Market Adoption – CustomersInternet Companies Enterprises
![Page 44: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/44.jpg)
Q & A
![Page 45: Launch Webinar - Introducing Couchbase Server 2.0](https://reader035.vdocuments.mx/reader035/viewer/2022062304/55d53ffbbb61ebd7108b461f/html5/thumbnails/45.jpg)