webinar - einfuehrung in couchbase
Post on 21-May-2015
326 Views
Preview:
DESCRIPTION
TRANSCRIPT
Einführung inCouchbase Server 2.0
• Developer Advocate at Couchbase, Inc.
• Maintainer of the Couchbase Java SDK
• Speaking at Conferences and Meetups
• Living and Working in Vienna, Austria
{“about”: “me”}
1. Introduction to Couchbase Server 2.0
2. Cluster Management
3. Operations & Document Design
4. Introduction to Views
Agenda
Couchbase Server 2.0
Big Release in December 2012
Couchbase Open Source Project
• One of the leading NoSQL database projects focused on distributed database technology and surrounding ecosystem
• Supports both key-value and document-oriented use cases
• All components are available under the Apache 2.0 Public License
• Obtained as packaged software in both enterprise and community editions.
Easy Scalabili
ty
Consistent High
Performance
Always On
24x365
Grow cluster without application changes, without downtime with a single click
Consistent sub-millisecond read and write response times
with consistent high throughput
No downtime for software upgrades, hardware maintenance, etc.
JSONJSONJSON
JSONJSON
PERFORMANCE
Flexible Data Model
JSON document model with no fixed schema.
Core Principles
New in 2.0
JSON support Indexing and Querying
Cross data center replicationIncremental Map Reduce
JSONJSONJSON
JSONJSON
Couchbase Server 2.0 Architecture
Hea
rtbe
at
Proc
ess
mon
itor
Glo
bal s
ingl
eton
sup
ervi
sor
Confi
gura
tion
man
ager
on each node
Reba
lanc
e or
ches
trat
or
Nod
e he
alth
mon
itor
one per cluster
vBuc
ket s
tate
and
repl
icati
on m
anag
er
httpRE
ST m
anag
emen
t API
/Web
UI
HTTP8091
Erlang port mapper4369
Distributed Erlang21100 - 21199
Erlang/OTP
storage interface
Couchbase EP Engine
11210Memcapable 2.0
Moxi
11211Memcapable 1.0
Memcached
New Persistence Layer
8092Query API
Que
ry E
ngin
e
Data Manager Cluster Manager
33 2
Single node - Couchbase Write Operation
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
GET
Doc
1
33 2
Single node - Couchbase Read Operation
Dis
k Q
ueue
Replication Queue
App Server
Doc 1
Doc 1Doc 1
Managed Cache
Disk
To other node
Couchbase Server Node
COUCHBASE SERVER CLUSTER
Basic Operation
• Docs distributed evenly across servers
• Each server stores both active and replica docsOnly one server active at a time
• Client library provides app with simple interface to database
• Cluster map provides map to which server doc is onApp never needs to know
• App reads, writes, updates docs
• Multiple app servers can access same document at same time
User Configured Replica Count = 1
READ/WRITE/UPDATE
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc
SERVER 1
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc
SERVER 2
Doc 8
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc
REPLICA
Doc 4
Doc 1
Doc 8
Doc
Doc
Doc
REPLICA
Doc 6
Doc 3
Doc 2
Doc
Doc
Doc
REPLICA
Doc 7
Doc 9
Doc 5
Doc
Doc
Doc
SERVER 3
Doc 6
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
Doc 9
Add Nodes to Cluster
• Two servers addedOne-click operation
• Docs automatically rebalanced across clusterEven distribution of docsMinimum doc movement
• Cluster map updated
• App database calls now distributed over larger number of servers
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3 SERVER 4 SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc
Doc 8 Doc
Doc 9 Doc
Doc 2 Doc
Doc 8 Doc
Doc 5 Doc
Doc 6
READ/WRITE/UPDATE READ/WRITE/UPDATE
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
COUCHBASE SERVER CLUSTER
User Configured Replica Count = 1
Fail Over Node
REPLICA
ACTIVE
Doc 5
Doc 2
Doc
Doc
Doc 4
Doc 1
Doc
Doc
SERVER 1
REPLICA
ACTIVE
Doc 4
Doc 7
Doc
Doc
Doc 6
Doc 3
Doc
Doc
SERVER 2
REPLICA
ACTIVE
Doc 1
Doc 2
Doc
Doc
Doc 7
Doc 9
Doc
Doc
SERVER 3 SERVER 4 SERVER 5
REPLICA
ACTIVE
REPLICA
ACTIVE
Doc 9
Doc 8
Doc Doc 6 Doc
Doc
Doc 5 Doc
Doc 2
Doc 8 Doc
Doc
• App servers accessing docs
• Requests to Server 3 fail
• Cluster detects server failedPromotes replicas of docs to activeUpdates cluster map
• Requests for docs now go to appropriate server
• Typically rebalance would follow
Doc
Doc 1 Doc 3
APP SERVER 1
COUCHBASE Client Library
CLUSTER MAP
COUCHBASE Client Library
CLUSTER MAP
APP SERVER 2
User Configured Replica Count = 1
COUCHBASE SERVER CLUSTER
Q & A
Operations & Document Design
Fundamentals
• Every Document has a Key assigned to it
• Keys must be max. 255 UTF-8 chars long must be unique in a bucket (“database”) are completely under the control of the application
• Values can be any binary blob (but bonus points when JSON!) can be up to 20MB in size
Basic Store & Retrieve Operations
• get(key)Retrieve a document
• set(key, value)Store a document or replace if it exists
• add(key, value)Store a document and fail if it exists
• replace(key, value)Replace a document and fail if it does not exist
Lots of other Operations
• View operations
• Atomic Counters (increment, decrement)
• Append/Prepend
• CAS (Compare and Set) Optimistic Locking
• “Get with Lock” Write Lock on Objects
• Bulk Operations Saves network overhead
• Stats
Couchbase 2.0 Bonus Points: JSON
• JSON is a lightweight format to represent document structure in a language-independent manner.
• If JSON documents are stored, the View engine can be used.
• Allows to build secondary indexes on your datasets.
• Makes it possible to ask questions like“Give me all user documents by lastname”.
The BIG mental adjustment
• In SQL, we tend to avoid hitting the database as much as possible. Use JOINs and let the DB optimizer figure out what to do
• In Couchbase, get’s and set‘s are so fast they are trivial, not bottlenecks, this is hard for many people to accept at first; Multiple get statements are commonplace, don’t avoid it!
meta{“id”: “u::michael@couchbase.com”,“rev”: “1-0002bce0000000000”,“flags”: 0,“expiration”: 0,“type”: “json”}
document{“uid”: 123456,“firstname”: “Michael”,“lastname”: “Nitschinger”,“age”: 25,“favorite_colors”: [“blue”, “black”],“email”: “michael.nitschinger@...”}
MetaInformationIncluding Key
All Keys Unique and Kept in RAM
DocumentValue
Most Recent In Ram And Persisted To Disk
Document Structure with JSON
Views & Indexes
• Views can cover a few different use cases Simple secondary indexes (the most common) Complex secondary, tertiary and composite indexes Aggregation functions (reduction)
•Example: count the number of North American Ales
Organizing related data
• Built using Map/Reduce Map function creates the index (b-tree) Reduce function summarizes (reduces) information Written using superfast Javascript (V8)
A View in Action
Developing with Couchbase
Aggregate View of Data
http://martinfowler.com/bliki/AggregateOrientedDatabase.html
What is an aggregate?
WhenyouneedtoretrievedatafromRDBMS,youare"aggregating"or"denormalizing"thedataforyourapplicationthroughquerieswithjoins,whereclausesandorderbyclauses.
InDocumentDatabases,insteadofbreakingdataintotablesandforeignkeys,youstoretheaggregatedatatogetherinJSONdocument(s).
Store and Retrieve Aggregates
• Easier to Distribute Data• More Flexibility• Reduced Latency
order::1001{uid: ji22jd,customer: Ann,line_items: [
{ sku: 0321293533, quan: 3, unit_price: 48.0 },{ sku: 0321601912, quan: 1, unit_price: 39.0 },{ sku: 0131495054, quan: 1, unit_price: 51.0 }
],payment: { type: Amex, expiry: 04/2001,
last5: 12345 }}
Relational vs Document Model
31
RelationaldatamodelHighly-structured table organization with rigidly-defined data formats and record structure.
C1 C2 C3 C4
DocumentdatamodelCollection of complex documents witharbitrary, nested data formats andvarying “record” format.
JSONJSON
JSON
{
}
SQL Normalized Tables
32
Addresses
1 DEN 30303CO
2 MV 94040CA
3 CHI 60609IL
Users
KEY First ZIP_IDLast
4 NY 10010NY
1 Jasdeep 2Jaitla
2 Joe 2Smith
3 Ali 2Dodson
4 John 3Doe
ZIP_ID CITY ZIPSTATE
Togetinformationaboutspecificuser,youperformajoinacrosstwotables
foreign key
SELECT*FROMUsersuINNERJOINAddressesaONu.zip_id=a.zip_idWHEREkey=1
Documents are Aggregates
33
+
Addresses
1 DEN 30303CO
2
3 CHI 60609IL
4 NY 10010NY
ZIP_ID CITY ZIPSTATE
Users
KEY First ZIP_IDLast
2
2 Joe 2Smith
3 Ali 2Dodson
4 John 3Doe
Alldatainasingledocument
{“ID”:1,“First”:“Jasdeep”,“Last”:“Jaitla”,“ZIP”:“94103”,“CITY”:“SF”,“STATE”:“CA”}
JSON
=
couchbase.get(“user::1”)
Document Data is an Aggregate
1 Jasdeep Jaitla
94103CASF
1 Jasdeep Jaitla
Document Keys
Keys have to be unique in a Bucket (database) which acts as a namespace.
Keystendtohavethreecategories: human-readable, randomly generated, and hybrid
Humanreadablekeysuseuniquedescriptors:email addresses, social media id’s, phone numbers, blog post titles, usernames, sku’s, etc.
www.couchbase.com/develop
PythonRuby
Go Clojure
Official SDKs
Community & Libraries
Couchbase Views
Indexing Architecture
33 2Managed Cache Disk Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc1Doc1
Doc1
To other node
View Engine
Doc1
Doc Updated in RAM Cache First
Indexer Updates Indexes After On Disk, in Batches
All Documents & Updates Pass Through View Engine
Buckets >> Design Documents >> Views
Couchbase Bucket
Design Document 1 Design Document 2
View ViewViewViewView
Indexers Are Allocated Per Design Doc
All Updated at Same TimeAll Updated at Same TimeAll Updated at Same Time
Can Only Access Data in the Bucket NamespaceCan Only Access Data in the Bucket Namespace
Map() Function => Index
function(doc, meta) {emit(doc.username, doc.email)
}indexed key output value(s)create row
json doc doc metadata
EveryDocumentpassesthroughViewMap()functions
Map
Single Element Keys (Text Key)
function(doc, meta) {emit(doc.email, null)
}text key
Map
doc.email meta.id
abba@couchbase.com u::1
jasdeep@couchbase.com u::2
zorro@couchbase.com u::3
Compound Keys (Array)
function(doc, meta) {emit(dateToArray(doc.timestamp), 1)
} array key
ArrayBasedIndexKeysgetsortedasStrings,butcanbegroupedbyarrayelements
Map
dateToArray(doc.timestamp) value
[2012,10,9,18,45] 1
[2012,9,26,11,15] 1
[2012,8,13,2,12] 1
Reduce Values (doc.abv) with _stats
add _stats built-in reduction
Result Set - Brewery ID’s by Beer
brewery_id
document key (of the beer)
alcohol by volume (abv)
Q & A
Thank you!michael.nitschinger@couchbase.com
@daschl
Get Couchbase Server at http://www.couchbase.com/download
top related