ruby driver explained: datastax webinar may 5th 2015
TRANSCRIPT
© 2015 DataStax, All Rights Reserved.
Datacenter Datacenter
Cassandra Topology
3
Node
NodeNode
Node
Client Client
Node
NodeNode
Node
Client Client
Cluster
© 2015 DataStax, All Rights Reserved.
Datacenter Datacenter
Request Coordinator
4
Node
NodeNode
Node
Client Client
Node
NodeCoordinator
Node
Client Client
Coordinator node:Forwards requeststo corresponding replicas
© 2015 DataStax, All Rights Reserved.
Datacenter
Row Replica
5
Replica
NodeNode
Replica
Client Client
Datacenter
Node
Node
Replica
Client Client
Coordinator
Replica node:Stores a slice of total rows of each keyspace
© 2015 DataStax, All Rights Reserved.
Installation
7
gem 'cassandra-driver', '~> 1.0.0'
gem install cassandra-driver
© 2015 DataStax, All Rights Reserved.
Usage
8
require 'cassandra'
cluster = Cassandra.cluster
cluster.each_host do |h| puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}" end
keyspace = 'system' session = cluster.connect(keyspace)
future = session.execute_async('SELECT * FROM schema_columnfamilies')
future.on_success do |rows| rows.each do |row| puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}" end end
future.join
© 2015 DataStax, All Rights Reserved.
Asynchronous Core
10
Application Thread
Business Logic
Driver
Background Thread
IO Reactor
© 2015 DataStax, All Rights Reserved.
Request Pipelining
11
Client
WithoutRequest Pipelining
Server
Client Server WithRequest Pipelining
1
2
2
3
1
3
1
2
3
1
2
3
© 2015 DataStax, All Rights Reserved.
Future Composition
12
select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id)
future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username'])
future.then do |pages| page = pages.first
User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
13
select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id)
future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username'])
future.then do |pages| page = pages.first
User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
14
select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id)
future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username'])
future.then do |pages| page = pages.first
User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
15
select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id)
future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username'])
future.then do |pages| page = pages.first
User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
16
select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id)
future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username'])
future.then do |pages| page = pages.first
User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
17
select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id)
future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username'])
future.then do |pages| page = pages.first
User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
© 2015 DataStax, All Rights Reserved.
Future Composition
18
[#<User @id=1 @username="avalanche123"; @page=#<Page @slug="avalanche123" ... > ... >, ... ]
© 2015 DataStax, All Rights Reserved.
Application Driver
Load Balancing
20
ApplicationThread
Node
Pool
SessionPool
Pool
Pool
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Load Balancing Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Load Balancing
20
ApplicationThread
Node
Pool
SessionPool
Pool
Pool
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Load Balancing Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Load Balancing
20
ApplicationThread
Node
Pool
SessionPool
Pool
Pool
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Load Balancing Policy
© 2015 DataStax, All Rights Reserved.
Datacenter
Datacenter
DataCenter Aware Balancing
21
Node
Node
NodeClient
Node
Node
Node
Client
Client
Client
Client
Client
Local nodes are queried first, if non are available, the request could be sent to a remote node.
© 2015 DataStax, All Rights Reserved.
Token Aware Balancing
22
Route request directly to Replicas
Node
Node
ReplicaNode
Client
Replica
Replica
Uses prepared statement metadata to get the token
© 2015 DataStax, All Rights Reserved.
Fault Tolerance
24
Coordinator
Node Replica
Replica
ReplicaNode
Business Logic
Driver
Application
© 2015 DataStax, All Rights Reserved. 25
Coordinator
Node Replica
Replica
ReplicaNode
Business Logic
Driver
Application
Invalid Requests
Network Timeouts
Server ErrorsPossible Failures
© 2015 DataStax, All Rights Reserved.
Application Driver
Automatic Retry of Server Errors
26
ApplicationThread
Node
Pool
SessionPool
Pool
Pool
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Load Balancing Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Automatic Retry of Server Errors
26
ApplicationThread
Node
Pool
SessionPool
Pool
Pool
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Load Balancing Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Automatic Retry of Server Errors
26
ApplicationThread
Node
Pool
SessionPool
Pool
Pool
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Load Balancing Policy
© 2015 DataStax, All Rights Reserved. 27
Coordinator
Node Replica
Replica
ReplicaNode
Business Logic
Driver
Application
Unreachable Consistency
© 2015 DataStax, All Rights Reserved.
Coordinator
Node Replica
ReplicaNode
28
Replica
Business Logic
Driver
Application
Read / Write Timeout Error
© 2015 DataStax, All Rights Reserved.
Coordinator
Node Replica
ReplicaNode
28
Replica
Business Logic
Driver
Application
Read / Write Timeout Error
© 2015 DataStax, All Rights Reserved.
Coordinator
Node Replica
ReplicaNode
28
Replica
Business Logic
Driver
Application
Read / Write Timeout Error
read / write timeout
© 2015 DataStax, All Rights Reserved. 29
Coordinator
Node Replica
Replica
ReplicaNode
Business Logic
Driver
Application
Unavailable Error
© 2015 DataStax, All Rights Reserved. 29
Coordinator
Node Replica
Replica
ReplicaNode
Business Logic
Driver
Application
Unavailable Error
unavailable
© 2015 DataStax, All Rights Reserved.
Datacenter Datacenter
Multiple Addresses
32
Node
NodeNode
Node
Client Client
Node
NodeNode
Node
Client Client
Within Datacenter: Private IPs
Across Datacenters: Public IPs
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
ApplicationThread
ApplicationThread
ApplicationThread
Client Cluster
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
ApplicationThread Node
Cluster
ApplicationThread
ApplicationThread
Client Cluster
Address Resolution Policy
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
ApplicationThread Node
Cluster
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Address Resolution Policy
Control Connection
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
ApplicationThread Node
Cluster
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Address Resolution Policy
Control Connection
© 2015 DataStax, All Rights Reserved.
Application Driver
Address Resolution
33
ApplicationThread Node
Pool
Cluster
Pool
Pool
Pool
ApplicationThread
ApplicationThread
Client Cluster
Node
Node
Node
Address Resolution Policy
Control Connection
Session
Questions and Links• Documentation • Code • Example Web-app