database performance with proxy architectures

13
© Continuent 2009 Database Performance Database Performance with Proxy Architectures with Proxy Architectures Tungsten Tungsten SQL Router SQL Router Robert Hodges CTO, Continuent, Inc. Percona Percona Performance Is Everything 2009 Performance Is Everything 2009

Upload: perconaperformance

Post on 16-Jun-2015

1.261 views

Category:

Technology


4 download

TRANSCRIPT

Page 1: Database  Performance With  Proxy  Architectures

© Continuent 2009

Database PerformanceDatabase Performancewith Proxy Architectureswith Proxy Architectures

TungstenTungsten SQL RouterSQL Router

Robert Hodges

CTO, Continuent, Inc.

Percona Percona Performance Is Everything 2009Performance Is Everything 2009

Page 2: Database  Performance With  Proxy  Architectures

© Continuent 2009

The “Proxy Math” Problem

/ Proxy layers solve a number of database problems• Seamless failover• Intelligent load balancing• Intelligent partitioning

/ Big Question: What’s the cost for applications?

/ Let’s run a few tests to show basic limits• Simple queries with data guaranteed to be in buffer cache• Dell SC 1425, Xeon 2 CPU x 2 Cores, 2.8Mhz• No/minimal proxy logic

Page 3: Database  Performance With  Proxy  Architectures

© Continuent 2009

Proxy Overhead Compared to Databases

Page 4: Database  Performance With  Proxy  Architectures

© Continuent 2009

Proxy with Increasing Result Sizes

Page 5: Database  Performance With  Proxy  Architectures

© Continuent 2009

How Do We Avoid Proxy Math?

/ Proxy performance is CPU intensive• Mostly copying and transforming bytes

/ Bad case: 4x CPU resources for same throughput• Could be far worse if proxy does more than copy data• It’s not enough to make proxies faster

/ Alternative 1: Hardware load balancer• If I only had a brain…

/ Alternative 2: VIPs• Handy for availability but load balancing is a problem

/ Alternative 3: An embeddable library• This would work on Amazon as well!

Page 6: Database  Performance With  Proxy  Architectures

© Continuent 2009

Tungsten SQL Router Overview

/ Library-based intelligent failover and load balancing

/ Connection-level routing

/ Implemented as JDBC library wrapper (Java!)• Adds intelligence to vendor drivers

/ DBMS-independent and non-partisan

/ 100% open source• http://sourceforge.net/projects/tungsten

/ Built-in management• Manageable from network• Monitoring integration

/ Routing based on CAP theorem• Trade off consistency vs. availability vs. partition tolerance

Page 7: Database  Performance With  Proxy  Architectures

© Continuent 2009

CAP-Based Routing--“Quality of Service”

Connection for WritesConnection for Writes

jdbcjdbc::t-routert-router://mysvc/db1://mysvc/db1??qos=RW_STRICTqos=RW_STRICT

Connection for ReadsConnection for Reads

jdbcjdbc::t-routert-router://mysvc/db1://mysvc/db1??qos=RO_RELAXEDqos=RO_RELAXED

SessionSessionConsistency*Consistency*

StrictStrictConsistencyConsistency

RelaxedRelaxedConsistencyConsistency

MasterMaster SlaveSlaveSlaveSlave

SlaveSlave

ReplicationReplication

(latency)(latency)

*In progress*In progress

(Data Service)(Data Service)

Page 8: Database  Performance With  Proxy  Architectures

© Continuent 2009

Services and Data Sources

/ SQL Router connects to services, not hosts

/ Services are composed of data sources

/ Configuration stored in CLUSTER_HOME directory

/ Configuration updated by administrative commands

$CLUSTER_HOME/$CLUSTER_HOME/services/services/

mysvc/mysvc/datasource/datasource/

db1.propertiesdb1.propertiesdb2.propertiesdb2.propertiesdb3.propertiesdb3.properties

replicator/replicator/

Page 9: Database  Performance With  Proxy  Architectures

© Continuent 2009

Administration and Monitoring

/ SQL Router library has JMX ports for administration

/ Update status of data sources using rmctl program:

sql-router[mysvc:ONLINE]> db2 offline

SENT NOTIFICATION: DATASTORE db2 offline

sql-router[mysvc:ONLINE]> db1 online

SENT NOTIFICATION: DATASTORE db1 online

sql-router[mysvc:ONLINE]> db3 online

SENT NOTIFICATION: DATASTORE db3 online

/ Listeners for monitoring events• Use UDP or group communications or…• Automatically take data sources online/offline

Page 10: Database  Performance With  Proxy  Architectures

© Continuent 2009

Does It Work with Non-Java Programs?

/ Yes

/ Use Tungsten Connector as front end

/ Full circle back to better SQL proxies

Page 11: Database  Performance With  Proxy  Architectures

© Continuent 2009

Tungsten Drilldown Tungsten Drilldown –– SQL-Router SQL-Router

Page 12: Database  Performance With  Proxy  Architectures

© Continuent 2009

Roadmap

/ Upgraded monitoring• Create/remove data sources automatically from monitoring data• Specify maximum permitted latency

/ Session consistency

/ Partitioning / Sharding• Route databases to different servers

Page 13: Database  Performance With  Proxy  Architectures

© Continuent 2009

HQ and Americas560 S. Winchester Blvd., Suite 500

San Jose, CA 95128

Tel (866) 998-3642

Fax (408) 668-1009

My e-mail: robert.hodges “at” continuent.com

EMEA and APACLars Sonckin kaari 16

02600 Espoo, Finland

Tel +358 50 517 9059

Fax +358 9 863 0060

Contact InformationContact Information

Commercial solutions:http://www.continuent.com

Tungsten open source stack:http://community.continuent.com

http://sourceforge.net/projects/tungsten