continuent tungsten proxies on steroids for ha and performance! presentation

8/14/2019 Continuent Tungsten Proxies on Steroids for HA and Performance! Presentation

1/36

Continuent4/14/08

Continuent Tungsten:

Proxies on Steroids for HAand Performance!Technology Preview

Robert Hodges, CTO of Continuent

April 15, 2008


2/36

2 Continuent

What We Will Cover / Defining the Scale-Out Problem

/ A Hard Look at MySQL Replication for Scale-Out

/ Can We Do Better?

/ Replication Reloaded: Tungsten Scale-Out Stack

/ How to Get Software and Get Started Today


3/36

3 Continuent

A Word from Our Sponsor: Continuent

/ Company Leading provider of database availability solutions for open source

databases

/ Products uni/cluster multi-master database clustering that replicates data across

multiple databases and load balances reads Works with off-the-shelf database servers Uses database virtualization to provide a seamless client interface

/ Value Low-cost open source business critical solutions Highly available data Raise performance and hardware utilization through load balancing Remove chance of data loss


4/36

4 Continuent ConfidentialConfidential

4 Continuent

Defining the Scale-Out Problem


5/36

5 Continuent

Requirement: Availability and Performance

/ Data Availability Bring up alternate database after local failure Bring up alternate database after site loss Keep both sites on-line for immediate failover Maintain/upgrade/migrate without disruption Restart/restore following data loss or outage due to human error Ensure commits to multiple databases and/or sites Always accept transactions no matter what Stay available in grid environments

/ Performance Scale reads transparently Scale writes transparently Spread transaction load across multiple active sites Rapid capacity scaling up/down in grid environments

/ Nirvana: increased data availability andperformance scalability by adding commodityhardware


6/36

6 Continuent

The Dream - Flexible Scale-Out

S c a l a

b i l i t y

Availability

Db

App

Db

Db

App

App


7/36

7 Continuent ConfidentialConfidential7 Continuent

A Hard Look at MySQL Replication


8/36

8 Continuent

MySQL Replication for Scale-Out

/ MySQL database replication has set the high bar for horizontal scale-out on large websites for many years

/ Replication is baked-in to MySQL Initial set-up takes only a few minutes Handy SQL commands to configure masters and slaves

/ Flexibly copies all or some databases/ Offers a choice of statement or row replication

Row replication only in MySQL 5.0 and previous releases

/ Works across WANs, hence can be used to link

multiple sites/ In short, its a key reason why MySQL is so popular


9/36

9 Continuent

But Is It Really That Easy?

/ Replication is hard -- no single approach solves allproblems

/ Master is the weak point in the MySQL design Replacing a crashed master when there are multiple slaves Sync between binlog and database

Loss of committed transactions/ Latency due to single threaded update on slaves

/ Administration is harder than it looks Parameter settings: innodb_flush_log_at_trx_commit=2 Provisioning, resynchronizing, monitoring

Automating failover / Read scaling requires application changes

/ Not a write scaling solution


10/36

10 Continuent

A Hard Problem -- Master Election after a Crash

Slave

Master

Slave

Slave

NewNewMaster Master

WhereWhere was I??was I??

WhereWhere was I??was I??


11/36

11 Continuent

Customer Feedback from Our Clustering Work

/ Use cases for data base clustering driven byHA/scaling/both

/ Analysis of US/Europe sales shows followingpatterns:

Almost 100% of customers are looking for HA solution Scalability was a requirement for ~30%

/ What Customers Need Transparent, easy-to-deploy HA Protection from data loss/corruption Higher database performance with fewer application

changes


12/36

12 Continuent

MySQL vs. Customer Requirements

/ MySQL replication is excellent for implementing readscaling on custom applications

For the do-it-yourself set, its the best thing out there

/ BUT--It does not suit everyone:

/ Users who need easy-to-deploy availability

/ Users who cannot afford to lose data

/ Users who cannot afford intrusive applicationchanges

/ Users who are not MySQL experts


13/36


Can We Do Better?

New Technologies for Scale-Out


14/36

14 Continuent

Continuent Database Clustering History

/ Continuent implemented m/cluster for MySQL -- thefirst successful commercial cluster based on groupcommunications

/ Our current uni/cluster product implements clustersusing middleware and off-the-shelf databases

/ We have supported MySQL and PostgreSQL for years

/ Strong experience in commercial/open sourcesolutions

Sequoia -- Multi-master state-machine clustering Myosotis -- Fast native-to-JDBC proxies Hedera -- Generic group communications adapters Bristlecone -- Testing tools for scale-out architectures


15/36


16/36


17/36


Replication Reloaded:

The Tungsten Scale-Out Stack


18/36


19/36

19 Continuent

Tungsten Overview

/ Data Services integrate multiple copies of data withinand across sites into a single highly available,performant database

/ Database virtualization with controlled data latencyand consistency

/ Database-neutral replication designed for highavailability

/ Cluster-aware management framework backed byexcellent graphical tools

/ Stack organization of technology to allowsubstitution and extensions


20/36

20 Continuent

An Open, Database-Neutral Scale-Out Stack

Tungsten Connector Tungsten Connector Native Wire ProtocolNative Wire Protocol ProxyingProxying ;; Connection semanticsConnection semantics

SequoiaSequoiaSynchrononousSynchrononous

Multi-Master ReplicationMulti-Master Replication

Tungsten Replicator Tungsten Replicator AsynchronousAsynchronous

Master/SlaveMaster/Slave ReplicationReplication

Tungsten Manager Tungsten Manager Cluster-awareCluster-aware service management for LAN/WANservice management for LAN/WAN

HederaHederaGroupGroup CommComm AdaptersAdapters

BristleconeBristleconeScale-OutScale-Out Perf Perf TestingTesting

Group CommunicationsGroup CommunicationsJGroups,JGroups, AppiaAppia , Spread, Spread

DBMS ServersDBMS ServersMySQLMySQL ,, PostreSQLPostreSQL , Oracle, Oracle

Application StacksApplication Stacks

PHP,PHP, PerlPerl , Java, Ruby, etc., Java, Ruby, etc.


21/36

21 Continuent

Why Are Stacks So Important?

/ No single solution can solve all data availability andscaling use cases

/ Stack approach allows you to reuse components ondifferent scale-out problems

/ The stack is an extensible framework

Tungsten Connector (Tungsten Connector ( MyosotisMyosotis ))

SequoiaSequoia TungstenTungsten Replicator Replicator

Tungsten Manager Tungsten Manager

HederaHederaBristleconeBristlecone

CachingCaching

PartioningPartioning

Certification-Certification-BasedBased

ReplicationReplication


22/36

22 Continuent

Virtualized SQL Connection Semantics

Application StacksApplication StacksPHP,PHP, PerlPerl , Java, Ruby, etc., Java, Ruby, etc.

NodeNode NodeNode NodeNode NodeNode

All UpdatesVisible

Own UpdatesVisible

UpdatesEventually

Visible

Tungsten Data Service


23/36

23 Continuent

High-Availability Replication

Master-master replication/ No master failover headaches

/ Automatic load balancing

Master-slave replication/ Wider range of supported SQL

/ Faster performance

/ Works over WAN

NodeNode

NodeNodeNodeNode

NodeNode

NodeNodeNodeNode

SequoiaSeq uoia Tungsten Replicator Tung sten Replicator


24/36

24 Continuent

Managing within and across Sites

HostHost

ServiceServiceManager Manager

Replicator Replicator NodeNode

HostHost


Replicator Replicator NodeNode

HostHost


Connector Connector

Connector Connector HostHost


Connector Connector

Connector Connector

Site 1Site 1

AdministrativeAdministrativeClientClient

JMXJMX

Group CommunicationGroup Communication

Site 2Site 2

HostHostHostHostHostHost

Event BridgeEvent Bridge


25/36

25 Continuent

GUI Tools Demo


26/36

26 Continuent

Now -- Multi-Master Data Services

/ Ensure constant availability and avoid master failurescenarios

/ Focus: Scalable and easily manageable multi-master replication

High quality management tools

Automatic load balancing of read requests 3+ master nodes Seamless connections from native clients and Java Support for latest MySQL versions and table types Solaris and Linux platforms


27/36

27 Continuent

Java Clients

Now -- uni/cluster 2008

Client ApplicationsLAMP Stack Clients(PHP, Perl, C, Ruby, etc.)

uni/cluster connector uni/cluster JDBC Driver

DBServer

DBServer

DBServer

DBServer

Clientsuni/cluster Driver

Controller

JDBC Driver

Controller

JDBC Driver

Controller

JDBC Driver

Controller

JDBC Driver


28/36

28 Continuent

Phase 1 - Master/Slave Data Services

/ Constant availability with high transparency andupdate performance/ Focus: Seamless upgrade from existing standalone

database One-step software installation Rapid data service configuration Lightweight, multi-platform management tools No application code changes

/ High availability replication Automatic and manual promotion of slave to master Bounded latency between slave and master Transparent proxying with read distribution to idle slaves


29/36

29 Continuent

Phase 1 - Master/Slave Data Service

ConnectorsFailover, Connection Semantics

Client Client

T u n g s t e n

M a m a g e m e n

t T o o

l s

Service Management

Master DBServer

Slave DBServer

ReplicationNode Mgr

Slave DBServer

ReplicationNode Mgr

ReplicationNode Mgr


30/36

30 Continuent

Phase 2 - Cross-Site Data Services

/ Ensure availability and scaling across sites/ Focus: Cross-site clustering with efficient failover

Replication of data to secondary site databases Minimal, understandable application changes Accept transactions across active systems on multiple sites

/ WAN clustering features Tools and admin model for cross-site management Handle WAN quality of service (latency, transient failures) SQL partitioning

/ Emphasis on WAN solutions that work without corner cases or data inconsistencies


31/36

31 Continuent

Phase 2 - Cross-Site Data Services

Data Service Connectors

Read/WriteApplications

Slave DBServer

Master DBServer

Slave DBServer

Connectors

Site 2Site 2SiteSite 11

Read/WriteApplications

Read-OnlyApplications,

Reports

Read-OnlyApplications,

Reports

DNS Load Balancing


32/36

32 Continuent

ConfidentialConfidential32 Continuent

How to Get Software and Get

Started Today


33/36

33 Continuent

Technology Roadmap

/ Uni/cluster 2007.1 is available today!/ Uni/cluster 2008 available in late Q2

GUI administrative tools Tungsten management framework Multiple controllers

/ Tungsten scale-out stack projects will be publishedsoon

/ Tungsten master/slave services will be available earlyQ4


34/36

34 Continuent

Getting Software

/ For access to commercial and open source products,visit us at the following web addresses

www.continuent.com

www.continuent.org

/ Visit us at the Continuent booth in the exhibit hall

/ Watch for more information on Tungsten very soon!


35/36

35 Continuent

For More on Scale-Out Performance

/ Come hear about scale-out testing using Bristlecone

Portable Scale-OutBenchmarks for MySQL

Wednesday 10:50-11:50am

Ballroom D


36/36

continuent tungsten proxies on steroids for ha and performance! presentation

Documents