continuent tungsten: proxies on steroids for ha and performance!

© Continuent4/14/08

Continuent Tungsten:Proxies on Steroids for HAand Performance!

Technology Preview

Robert Hodges, CTO of Continuent

April 15, 2008

2 © Continuent

What We Will Cover

/ Defining the Scale-Out Problem

/ A Hard Look at MySQL Replication for Scale-Out

/ Can We Do Better?

/ Replication Reloaded: Tungsten Scale-Out Stack

/ How to Get Software and Get Started Today

3 © Continuent

A Word from Our Sponsor: Continuent

/ Company• Leading provider of database availability solutions for open source

databases

/ Products• uni/cluster – multi-master database clustering that replicates data across

multiple databases and load balances reads• Works with off-the-shelf database servers• Uses “database virtualization” to provide a seamless client interface

/ Value• Low-cost open source business critical solutions• Highly available data• Raise performance and hardware utilization through load balancing• Remove chance of data loss

4 © Continuent ConfidentialConfidential4 © Continuent

Defining the Scale-Out Problem

5 © Continuent

Requirement: Availability and Performance

/ Data Availability• Bring up alternate database after local failure• Bring up alternate database after site loss• Keep both sites on-line for immediate failover• Maintain/upgrade/migrate without disruption• Restart/restore following data loss or outage due to human error• Ensure commits to multiple databases and/or sites• Always accept transactions no matter what• Stay available in grid environments

/ Performance• Scale reads transparently• Scale writes transparently• Spread transaction load across multiple active sites• Rapid capacity scaling up/down in grid environments

/ Nirvana: increased data availability andperformance scalability by adding commodityhardware

6 © Continuent

The Dream - Flexible Scale-OutS

cala

bil

ity

Availability

Db

App

Db

Db

App

App


A Hard Look at MySQL Replication

8 © Continuent

MySQL Replication for Scale-Out

/ MySQL database replication has set the high bar forhorizontal scale-out on large websites for many years

/ Replication is “baked-in” to MySQL• Initial set-up takes only a few minutes• Handy SQL commands to configure masters and slaves

/ Flexibly copies all or some databases

/ Offers a choice of statement or row replication• Row replication only in MySQL 5.0 and previous releases

/ Works across WANs, hence can be used to linkmultiple sites

/ In short, it’s a key reason why MySQL is so popular

9 © Continuent

…But Is It Really That Easy?

/ Replication is hard -- no single approach solves allproblems

/ Master is the weak point in the MySQL design• Replacing a crashed master when there are multiple slaves• Sync between binlog and database• Loss of committed transactions

/ Latency due to single threaded update on slaves

/ Administration is harder than it looks• Parameter settings: innodb_flush_log_at_trx_commit=2• Provisioning, resynchronizing, monitoring• Automating failover

/ Read scaling requires application changes

/ Not a write scaling solution

10 © Continuent

A Hard Problem -- Master Election after a Crash

Slave

Master

Slave

Slave

NewNewMasterMaster

WhereWhere was I??was I??

WhereWhere was I??was I??

11 © Continuent

Customer Feedback from Our Clustering Work

/ Use cases for data base clustering driven byHA/scaling/both

/ Analysis of US/Europe sales shows followingpatterns:

• Almost 100% of customers are looking for HA solution• Scalability was a requirement for ~30%

/ What Customers Need• Transparent, easy-to-deploy HA• Protection from data loss/corruption• Higher database performance with fewer application

changes

12 © Continuent

MySQL vs. Customer Requirements

/ MySQL replication is excellent for implementing readscaling on custom applications

• For the do-it-yourself set, it’s the best thing out there

/ BUT--It does not suit everyone:

/ Users who need easy-to-deploy availability

/ Users who cannot afford to lose data

/ Users who cannot afford intrusive applicationchanges

/ Users who are not MySQL experts


Can We Do Better?New Technologies for Scale-Out

14 © Continuent

Continuent Database Clustering History

/ Continuent implemented m/cluster for MySQL -- thefirst successful commercial cluster based on groupcommunications

/ Our current uni/cluster product implements clustersusing middleware and off-the-shelf databases

/ We have supported MySQL and PostgreSQL for years

/ Strong experience in commercial/open sourcesolutions

• Sequoia -- Multi-master state-machine clustering• Myosotis -- Fast native-to-JDBC proxies• Hedera -- Generic group communications adapters• Bristlecone -- Testing tools for scale-out architectures

15 © Continuent

What Have We Learned?

/ Replication works for building clusters - It’s the onlyapproach that really scales

/ Database virtualization using middleware is essentialfor encapsulating location of copies

/ Group communications are hard to configure buthighly effective for controlling clusters

/ Biggest challenges are simple management andsimple application semantics

/ WAN support is the killer application of databaseclustering

/ There is no one-size-fits-all solution

16 © Continuent

What Are the Up-And-Coming Technologies?

/ CAP-aware databases (Consistent, Available,Partition-Tolerant)

• Now appearing in non-SQL databases near you: AmazonDynamo and SimpleDB

/ Pluggable replication -- Different replicationmechanisms for different problems

• GORDA research project has pioneered approach

/ Partitioning -- Dividing data and automatically routingqueries

• MySQL community is driving solutions

/ Open source databases -- This stuff is going to be bigsome day!


Replication Reloaded:The Tungsten Scale-Out Stack

18 © Continuent

Our Goal

Highly reliable, transparent, easy todeploy solutions for data availability

and performance scalability that workwith unaltered, stock RDBMS and

economical, off-the-shelf hardware

TungstenTungsten: An architecture for: An architecture forscale-out that meets the goalscale-out that meets the goal

19 © Continuent

Tungsten Overview

/ Data Services integrate multiple copies of data withinand across sites into a single highly available,performant database

/ Database virtualization with controlled data latencyand consistency

/ Database-neutral replication designed for highavailability

/ Cluster-aware management framework backed byexcellent graphical tools

/ “Stack” organization of technology to allowsubstitution and extensions

20 © Continuent

An Open, Database-Neutral Scale-Out Stack

Tungsten ConnectorTungsten Connector Native Wire Protocol Native Wire Protocol ProxyingProxying;; Connection semanticsConnection semantics

SequoiaSequoiaSynchrononous Synchrononous

Multi-Master ReplicationMulti-Master Replication

Tungsten ReplicatorTungsten ReplicatorAsynchronousAsynchronous

Master/SlaveMaster/Slave ReplicationReplication

Tungsten ManagerTungsten ManagerCluster-awareCluster-aware service management for LAN/WANservice management for LAN/WAN

HederaHederaGroup Group Comm Comm AdaptersAdapters

BristleconeBristleconeScale-Out Scale-Out Perf Perf TestingTesting

Group CommunicationsGroup CommunicationsJGroups, JGroups, AppiaAppia, Spread, Spread

DBMS ServersDBMS ServersMySQLMySQL, , PostreSQLPostreSQL, Oracle, Oracle

Application StacksApplication StacksPHP, PHP, PerlPerl, Java, Ruby, etc., Java, Ruby, etc.

21 © Continuent

Why Are Stacks So Important?

/ No single solution can solve all data availability andscaling use cases

/ Stack approach allows you to reuse components ondifferent scale-out problems

/ The stack is an extensible framework

Tungsten Connector (Tungsten Connector (MyosotisMyosotis))

SequoiaSequoiaTungstenTungsten ReplicatorReplicator

Tungsten ManagerTungsten Manager

HederaHederaBristleconeBristlecone

CachingCaching

PartioningPartioning

Certification-Certification-BasedBased

ReplicationReplication

22 © Continuent

Virtualized SQL Connection Semantics

Application StacksApplication StacksPHP, PHP, PerlPerl, Java, Ruby, etc., Java, Ruby, etc.

NodeNode NodeNode NodeNode NodeNode

All UpdatesVisible

Own UpdatesVisible

UpdatesEventually

Visible

Tungsten Data Service

23 © Continuent

High-Availability Replication

Master-master replication

/ No master failover headaches

/ Automatic load balancing

Master-slave replication

/ Wider range of supported SQL

/ Faster performance

/ Works over WAN

NodeNode

NodeNodeNodeNode

NodeNode

NodeNodeNodeNode

SequoiaSequoia Tungsten ReplicatorTungsten Replicator

24 © Continuent

Managing within and across Sites

HostHost

Service Service ManagerManager

ReplicatorReplicatorNodeNode

HostHost


ReplicatorReplicatorNodeNode

HostHost


ConnectorConnector

ConnectorConnector

HostHost


ConnectorConnector

ConnectorConnector

Site 1Site 1

Administrative Administrative ClientClient

JMXJMX

Group CommunicationGroup Communication

Site 2Site 2

HostHostHostHostHostHost

Event BridgeEvent Bridge

26 © Continuent

Now -- Multi-Master Data Services

/ Ensure constant availability and avoid master failurescenarios

/ Focus: Scalable and easily manageable multi-masterreplication

• High quality management tools• Automatic load balancing of read requests• 3+ master nodes• Seamless connections from native clients and Java• Support for latest MySQL versions and table types• Solaris and Linux platforms

27 © Continuent

Java Clients

Now -- uni/cluster 2008

Client ApplicationsLAMP Stack Clients(PHP, Perl, C, Ruby, etc.)

uni/cluster connector

uni/cluster JDBC Driver

DBServer

DBServer

DBServer

DBServer

Clients

uni/cluster Driver

Controller

JDBC Driver

Controller

JDBC Driver

Controller

JDBC Driver

Controller

JDBC Driver

28 © Continuent

Phase 1 - Master/Slave Data Services

/ Constant availability with high transparency andupdate performance

/ Focus: Seamless upgrade from existing standalonedatabase

• One-step software installation• Rapid data service configuration• Lightweight, multi-platform management tools• No application code changes

/ High availability replication• Automatic and manual promotion of slave to master• Bounded latency between slave and master• Transparent proxying with read distribution to idle slaves

29 © Continuent

Phase 1 - Master/Slave Data Service

ConnectorsFailover, Connection Semantics

Client ClientT

un

gs

ten

Ma

ma

ge

me

nt

To

ols

Service Management

Master DBServer

Slave DBServer

ReplicationNode Mgr

Slave DBServer

ReplicationNode Mgr

ReplicationNode Mgr

30 © Continuent

Phase 2 - Cross-Site Data Services

/ Ensure availability and scaling across sites

/ Focus: Cross-site clustering with efficient failover• Replication of data to secondary site databases• Minimal, understandable application changes• Accept transactions across active systems on multiple sites

/ WAN clustering features• Tools and admin model for cross-site management• Handle WAN quality of service (latency, transient failures)• SQL partitioning

/ Emphasis on WAN solutions that work without cornercases or data inconsistencies

31 © Continuent

Phase 2 - Cross-Site Data Services

Data Service Connectors

Read/WriteApplications

Slave DBServer

Master DBServer

Slave DBServer

Connectors

Site 2Site 2SiteSite 11

Read/WriteApplications

Read-OnlyApplications,

Reports

Read-OnlyApplications,

Reports

DNS Load Balancing


How to Get Software and GetStarted Today

33 © Continuent

Technology Roadmap

/ Uni/cluster 2007.1 is available today!

/ Uni/cluster 2008 available in late Q2• GUI administrative tools• Tungsten management framework• Multiple controllers

/ Tungsten scale-out stack projects will be publishedsoon

/ Tungsten master/slave services will be available earlyQ4

34 © Continuent

Getting Software

/ For access to commercial and open source products,visit us at the following web addresses

www.continuent.com

www.continuent.org

/ Visit us at the Continuent booth in the exhibit hall

/ Watch for more information on Tungsten very soon!

35 © Continuent

For More on Scale-Out Performance…

/ Come hear about scale-out testing using Bristlecone

Portable Scale-OutBenchmarks for MySQL

Wednesday 10:50-11:50am

Ballroom D


Thank you!

Questions?