mysql cluster carrier grade edition: introduction · pdf filecarrier-grade availability and...
TRANSCRIPT
MySQL Cluster Carrier Grade Edition: Introduction
Oracle Corporation Japan , MySQL Global Business Unit
Master Principal Solutions Specialist | APAC & Japan, Alexander Yu
2011/10/28
Copyright© 2011, Oracle. All rights reserved.
The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any material,
code, or functionality, and should not be relied upon in
making purchasing decisions.
The development, release, and timing of any features
or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
Please Read
Copyright© 2011, Oracle. All rights reserved.
Agenda / Topics
• Intro: MySQL Cluster Real-time Database
• MySQL Cluster 7.2 DMR
• API Flexibility: SQL & NoSQL!
• MySQL Cluster Manager
• Q & A
Copyright© 2011, Oracle. All rights reserved.
40% DATA GROWTH PER YEAR
5.3BN MOBILE SUBS IN 2010
(78% PENETRATION)
13K iPHONE APPS DOWNLOADED PER MINUTE
370K CALL MINUTES EVERY 60 SECONDS
$1TR BY 2014
$700BN IN 2011
2.1BN USERS
8X DATA GROWTH IN 5 YRS
70+ NEW DOMAINS EVERY 60 SECONDS
100K TWEETS PER MINUTE
750M USERS
20M APPS PER DAY
600 NEW VIDEOS
EVERY 60 SECONDS
Copyright© 2011, Oracle. All rights reserved.
Selecting the Right HA Architecture
Copyright© 2011, Oracle. All rights reserved.
HA Solutions
Requirement MySQL Replication Heartbeat + DRBD Oracle VM Template MySQL Cluster
Availability
Platform SupportAll Supported by MySQL
ServerLinux Oracle Linux
All Supported by MySQL
Cluster
Automated IP Failover No Yes YesDepends on Connector and
Configuration
Automated Database
FailoverNo Yes Yes Yes
Automatic Data
ResynchronizationNo Yes N/A - Shared Storage Yes
Typical Failover Time User / Script DependentConfiguration Dependent, 60
seconds and Above
Configuration Dependent, 60
seconds and Above 1 Second and Less
Synchronous ReplicationNo, Asynchronous and Semi-
SynchronousYes N/A - Shared Storage Yes
Geographic Redundancy
SupportYes Yes, via MySQL Replication Yes, via MySQL Replication Yes, via MySQL Replication
Scalability
Number of Nodes One Master, Multiple SlavesOne Active (primary), one
Passive (secondary) Node
One Active (primary), one
Passive (secondary) Node255
Built-in Load BalancingReads, via MySQL
Replication
Reads, via MySQL
Replication
Reads, via MySQL
Replication & During FailoverYes, Reads and Writes
Read-Intensive Workloads Yes Yes Yes Yes
Write-Intensive WorkloadsYes, via Application-Level
Sharding
Yes, via Application-Level
Sharding to Multiple
Active/Passive Pairs
Yes, via Application-Level
Sharding to Multiple
Active/Passive Pairs
Yes, via Auto-Sharding
Scale On-Line (add nodes,
repartition, etc.)No No No Yes
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Real-time Carrier Grade Database
Copyright© 2011, Oracle. All rights reserved.
Customers & ApplicationsHigh Scale, Transactional, Real-Time with HA: Web & Telecoms
• Web
– eCommerce
– User Profile Management
– Session Management & Caching
– Content Management
– On-line Gaming
• Telecoms
– Subscriber Databases (HLR/HSS)
– Service Delivery Platforms
– VoIP, IPTV & VoD
– Mobile Content Delivery
– IP Management
– Payment Gateways
http://www.mysql.com/customers/cluster/
Pre & Post Paid
Copyright© 2011, Oracle. All rights reserved.
HLR / HSS
Billing, AuC, VLR
AuC, Call
Routing, Billing
Location
Updates
MySQL Cluster in Action: http://bit.ly/oRI5tF
• Massive volumes of write traffic
• <3ms database response
• Downtime & lost transactions = lost $
Use-Cases: Cellular Network
Copyright© 2011, Oracle. All rights reserved.
http://mysql.com/customers/view/?id=1080
Use-Cases: eCommerce
• Integrated Service Provider platform
– eCommerce
– Payment processing
– Fulfillment
• Supports 1k+ manufacturers & 18k retail
partners
• Requirements
– Scaling, On-Demand
– HA: failures & on-line upgrades
– High batch & real time loads
– Low TCO: capex and opex
Copyright© 2011, Oracle. All rights reserved.
Use Cases: Flight Control
• US Navy aircraft carriers
• Consolidated flight operations management
system
– Maintenance records
– Fuel loads
– Weather conditions
– Flight deck plans
• Requirements
– No Single Points of Failure
– Complete redundancy
– Small footprint, harsh environment
– 4 x MySQL Cluster nodes, Linux and Windows
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster 7.1 Momentum
Pyro InRoam SDP is an intelligent
application enabling subscribers to
communicate with ease across borders
MySQL Cluster selected as a result of high
transactional throughput, low latency,
carrier-grade availability and low cost
Migrated from a Microsoft platform with
SQL Server
1,000 Downloads per Day
“MySQL Cluster 7.1 gave us the
perfect combination of extreme levels
of transaction throughput, low
latency & carrier-grade availability,
while reducing TCO”
Phani Naik, Pyro Group
MySQL Cluster database
supports 7 million roaming
subscribers per day
Copyright© 2011, Oracle. All rights reserved.
• ACID Compliant Relational Database• SQL & NoSQL interfaces
• Write-Scalable & Real-Time• Distributed, multi-master, auto-sharding
• Optimized in-memory structures & indices
• Near-linear scalability, transparent load balancing
• 99.999% Availability• Shared-nothing, integrated clustering, sub-second recovery
• Local & geographic replication, on-line operations
• Low Barriers to Entry• Open-source, elastic, multiple APIs, management and monitoring tools
• Commodity hardware
MySQL Cluster Carrier Grade: Key Advantages
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Data Nodes
MySQL Cluster Application Nodes
Clients
MGM Node
MySQL Cluster Architecture
•Shared-nothing distributed database with no SPOF:
High Read & Write Performance & 99.999% uptime
NDB API (C++)
ND
B A
PI
ND
B A
PI
MGM Client
MGM Node
MGM API (C)
Copyright© 2011, Oracle. All rights reserved.
• Transparent to the application• No need for application-layer sharding logic
• Partitioning based on hashing the primary key or user defined key
• Each node stores primary fragment for 1 partition and back-up fragment for another
• Transparency maintained during failover, upgrades and scale-out
• No need to limit application to single-shard transactions
Automatic Data Partitioning/Sharding
Copyright© 2011, Oracle. All rights reserved.
Data Node 1
Data Node 2
F1 F3
Primary Fragment
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2Fx
Fx- Node groups are created automatically- # of groups = # of data nodes / # of replicas
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data PartitioningTransparent Load Balancing
Secondary Fragment (fragment replica)
Copyright© 2011, Oracle. All rights reserved.
• 8 Intel servers• Dual-6-core CPUs @2.93 GHz
• 24GB RAM
• 2 Data Nodes per server
• flexAsync benchmark• 16 parallel threads, each issuing 256
simultaneous transactions
• Read / Write 100KB attribute
• Access via NDB API
4.3M
QPS
Scale-Out on Commodity Hardware
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Data Nodes
MySQL Cluster Application Nodes
Clients
MGM Node
MySQL Cluster: Extreme Resilience
•Shared-nothing distributed database with no SPOF:
High Read & Write Performance & 99.999% uptime
NDB API (C++)
ND
B A
PI
ND
B A
PI
MGM Client
MGM Node
MGM API (C)
JDBC (Java)NDB API (C++)ClusterJ (Java)OpenJPA (Java)PHP/P*/ODBCOpenLDAP
ClusterJ
SQL Nodes
Copyright© 2011, Oracle. All rights reserved.
Data Node 1
Data Node 2
F1 F3
Primary Fragment
F3 F1
Data Node 3
Data Node 4
F2 F4
F4 F2
Node Group 1
Node Group 2Fx
Fx
As long as one data node in each node group is running we have a complete
copy of the data
Table T1
P2
P3
P4
Px Partition
4 Partitions * 2 Replicas = 8 Fragments
P1
Automatic Data PartitioningTransparent Load Balancing
Secondary Fragment (fragment replica)
Copyright© 2011, Oracle. All rights reserved.
Online Operations
• Scale the cluster for additional throughput or capacity
–Data and SQL Nodes
–Automatic repartition of tables
• Recover failed nodes
• Rolling
–Upgrade / patch servers & OS
–Upgrade / patch MySQL Cluster
• Backup
• Schema changes
Copyright© 2011, Oracle. All rights reserved.
On-Line Scaling & Maintenance
• Can also update schema on-line
• Upgrade hardware & software with no downtime
• Perform back-ups on-line
1. New node group added
2. Data is re-partitioned
3. Redundant data is deleted
4. Distribution is switched to share load with new node group
Copyright© 2011, Oracle. All rights reserved.
Real-Time, On-Line Schema Changes
• Fully online – transaction response
times unchanged
• Add and remove indexes, add new
columns and tables
• No temporary table creation
• No recreation of data or deletion
required
• Faster and better performing table
maintenance operations
• Less memory and disk requirements
CREATE OFFLINE INDEX b ON t1(b);
Query OK, 1356 rows affected (2.20 sec)
DROP OFFLINE INDEX b ON t1;
Query OK, 1356 rows affected (2.03 sec)
CREATE ONLINE INDEX b ON t1(b);
Query OK, 0 rows affected (0.58 sec)
DROP ONLINE INDEX b ON t1;
Query OK, 0 rows affected (0.46 sec)
ALTER ONLINE TABLE t1 ADD COLUMN d INT;
Query OK, 0 rows affected (0.36 sec)
Copyright© 2011, Oracle. All rights reserved.
Backup & Restore
• Online Backup for the Data only
– Data dump/backup for each Data Node
– Schema dump via mysqldump
• What to configure
– Backup path
• Choose local disk, SAN or NAS
– Backup files can then be picked up by external backup managers
– Compressed backup
• Space optimization/saving
• Backup Initiation
– Schema dump initiated via mysql on a SQL Node
– Backup snapshot of Data at end of backup or start of backup via ndb_mgm
• Backup command
– $ ndb_mgm -e “START BACKUP SNAPSHOTSTART"
• Restore command
– $ ndb_restore -c „192.168.10.10‟ -n 2 -b 1 -p 1 -m (once!)
– $ ndb_restore -c „192.168.10.10‟ -n 2 -b 1 -p 1 -r
– Restore data command issued for each Data Node
– Point-In-Time Recovery
• Need to enable Binary-Logging on MySQL Server
• Replay changes up-to a time using binary logs
Copyright© 2011, Oracle. All rights reserved.
Replication
• Multiple replication topologies available
• Master - Master (Bi-directional)
– Conflict detection and resolution
• Master - Slave(s)
• Circular
• etc
Copyright© 2011, Oracle. All rights reserved.
Cluster 1
Synchronous replication
Cluster 2
MyISAM MyISAM InnoDB
Asynchronous replication
• Synchronous (two phase commit)
replication within a Cluster node
group for HA
• Bi-Direction asynchronous
replication to remote Cluster for
geographic redundancy
• Asynchronous replication to non-
Cluster databases for specialised
activities such as report
generation
• Mix and match replication types
Copyright© 2011, Oracle. All rights reserved.
Geographical Replication
• A SQL Node/MySQL Server is responsible for the replication
– NDB Binlog injector thread
– Subscribes to events in MySQL Cluster
• Injects rows into the Binlog
– All changes in the cluster
– Row-based replication
• Two replication channels
– No single point of failure
Copyright© 2011, Oracle. All rights reserved.
Failure Detection: Majority Rules!
Data Node 1 Data Node 2
Data Node 3 Data Node 4
• Node 2
– One node from each node group? => No
– Shutdown
Network split/shutdown
HB
Copyright© 2011, Oracle. All rights reserved.
Disk Data Tables
• Disk Data Tables– Not enough memory to store all data
– Non-Indexed columns are stored on disk
• Tablespace
• A disk data based table stores its data in a tablespace
• Contains one more data files
• Log File Groups– In order to facilitate rollback
– Undo data in one or more undo log files
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Processes
• Data Nodes
– ndbd/ndbdmt daemon (single threaded/multithreaded)
– Data storage
– Transaction Coordination
• SQL Node
– mysqld daemon
– provides SQL access to dataset
– Handles geographical replication
• Management Nodes
– ndb_mgmd daemon
– cluster configuration, cluster log, default arbitrator
• Management Clients
– ndb_mgm, cluster command line client (interactive/non-interactive)
– start/stop backups, node restarts, filter cluster log.
– mysql, CLI for accesing the SQL Node
• API Nodes/Applications
– Via SQL Node (JDBC/ODBC)
– Direct APIs (C++/Java apps)
– LDAP, Web Services
• MySQL Cluster Manager Agent
– mcmd agent daemon
– mysql, CLI for agent cluster management commands
Copyright© 2011, Oracle. All rights reserved.
• Example 1: Check memory usage/availability
mysql> select * from ndbinfo.memoryusage;
+---------+--------------+--------+------------+-----------+-------------+
| node_id | memory_type | used | used_pages | total | total_pages |
+---------+--------------+--------+------------+-----------+-------------+
| 3 | Data memory | 917504 | 28 | 104857600 | 3200 |
| 3 | Index memory | 221184 | 27 | 11010048 | 1344 |
| 4 | Data memory | 917504 | 28 | 104857600 | 3200 |
| 4 | Index memory | 221184 | 27 | 11010048 | 1344 |
+---------+--------------+--------+------------+-----------+-------------+
• Note that there is a DATA_MEMORY and INDEX_MEMORY row for each data node in the cluster
• If the Cluster is nearing the configured limit then increase the DataMemory and/or IndexMemory
parameters in config.ini and then perform a rolling restart
MySQL Cluster 7.1: ndbinfo
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster 7.1: ndbinfo
• Example 2: Check how many table scans performed on each data node since the last restart
mysql> select node_id as 'data node', val as 'Table Scans' from ndbinfo.counters
where counter_name='TABLE_SCANS';
+-----------+-------------+
| data node | Table Scans |
+-----------+-------------+
| 3 | 3 |
| 4 | 4 |
+-----------+-------------+
• You might check this if your database
performance is lower than anticipated
• If this figure is rising faster than you expected
then examine your application to understand
why there are so many table scans
mysql> show tables;
+-------------------+
| Tables_in_ndbinfo |
+-------------------+
| blocks |
| config_params |
| counters |
| diskpagebuffer |
| logbuffers |
| logspaces |
| memoryusage |
| nodes |
| resources |
| transporters |
+-------------------+10
rows in set (0.10 sec)
Copyright© 2011, Oracle. All rights reserved.
MySQL Enterprise Monitor 2.3
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster: DMR 2
MySQL Cluster 7.2.1 DMR
Copyright© 2011, Oracle. All rights reserved.
Adaptive Query LocalizationScaling Distributed Joins
Copyright© 2011, Oracle. All rights reserved.
A
Q
L
Data Nodes
70xMore
Performance
DON’T COMPROMISE FUNCTIONALITY TO
SCALE-OUT !!
mysqld
mysqld
Data Nodes
• Perform Complex Queries across
Shards
– JOINs pushed down to data nodes
– Executed in parallel
– Returns single result set to MySQL
• Opens Up New Use-Cases
– Real-time analytics
– Recommendations engines
– Analyze click-streams
Copyright© 2011, Oracle. All rights reserved.
Data
Node1
Data
Node2
MySQL
Server
MySQL Cluster 7.2 AQL Test QueryWeb-Based Content Management System
MySQL Cluster 7.2 70x Speedup! (DMR)
Copyright© 2011, Oracle. All rights reserved.
• 70x Speedup for real customer JOIN across
11 tables
• Try Analyze tables to optimize results
• mysql> ANALYZE TABLE <tab-name>;
7.2 = 1.26s
7.1 = 87.23s
7.2
analyzed=
65.3s
Copyright© 2011, Oracle. All rights reserved.
New
NoSQL
Access
Memcached Key-Value API
• Persistent, Scalable, HA Back-End to memcached
– No application changes: re-uses standard
memcached clients & libraries
–Faster Cloud Deployments/Integration
• Consolidate Caching & Database Tiers
–Eliminate cache invalidation
–Simpler re-use of data across services
–Improved service levels
• Flexible Deployment
–Schema or Schema-less storage
Copyright© 2011, Oracle. All rights reserved.
MySQL 5.5 Server Integration
• Exploit Latest Server / Storage Engine
Developments within a Single Application
– InnoDB: Foreign Keys, XA Transactions,
Large Rows
–MySQL Cluster: HA, High Write Rates, Real-
Time
• Reduces Complexity, Simplifies DevOps
• MySQL 5.5
–3x higher performance
– Improved partitioning, diagnostics, availability,
etc.
Copyright© 2011, Oracle. All rights reserved.
Geographic
Replication
Enhanced Cross Data Center ScalabilityGeographic Replication
• Replicating complete clusters across
data centers
– DR & data locality
– No passive resources
• Simplified Active / Active Replication
– Eliminates requirement for application &
schema changes
– Transaction-level rollback
Copyright© 2011, Oracle. All rights reserved.
Data Node 1
Synchronous
ReplicationData Node 3
Data Node 2
Data Node 4
Enhanced Cross Data Center ScalabilityMulti-Site Clustering
• Split data nodes & node groups across
data centers
⁃ Synchronous replication & auto- failover
between sites
⁃ Improved heartbeating to handle network
partitions
• Extends Disaster Recovery Options
• More Resilient Cloud Deployments
• Active/Active with no need for conflict
handling
Copyright© 2011, Oracle. All rights reserved.
•SQL: Complex, relational queries
•HTTP / memcached: Key-Value web services
•Java: Enterprise Apps
•NDB API: Real-time services
Mix
&
Match
Data Nodes
NDB API
Clients
Native memcached HTTP/REST
JDBC / ODBC
PHP / PERL
Python / Ruby
API Flexibility:SQL & NoSQL Combined
Copyright© 2011, Oracle. All rights reserved.
SQL and NoSQL Combined!
Mix & Match!
Same data accessed
simultaneously through
SQL & NoSQL interfaces
•NoSQL – Multiple ways to bypass SQL, and maximize performance:
•NDB API. C++ for highest performance, lowest latency
•Cluster/J for optimized access in Java
•NEW! Memcached. Use all your existing clients/applications
Copyright© 2011, Oracle. All rights reserved.
NoSQL With NDB API
Best possible performance
MySQL Cluster Data Nodes
Applications with embedded NDB API Library
Clients • Application embeds the NDB API C++
interface library
• NDB API make intelligent decision (where
possible) about which data node to send
queries to– With a little planning in the schema design, achieve
linear scalability
• Used by all of the other application nodes
(MySQL, LDAP, ClusterJ,…)
• Best possible performance but requires >
development skill
• Favourite API for real-time network
applications
• Foundation for all interfaces
Copyright© 2011, Oracle. All rights reserved.
NoSQL with memcached
• Memcached is a distributed memory based
hash-key/value store with no persistence to disk
• NoSQL, simple API, popular with developers
• MySQL Cluster already provides scalable, in-
memory performance with NoSQL (hashed)
access as well as persistence• Provide the Memcached API but map to NDB API calls
• Writes-in-place, so no need to invalidate cache
• Simplifies architecture as caching & database
integrated into 1 tier
• Access data from existing relational tables
7.2DM
Memcached protocol
Copyright© 2011, Oracle. All rights reserved.
•Flexible:
• Deployment options
• Multiple Clusters
• Simultaneous SQL Access
• Can still cache in Memcached server
• Flat key-value store or map to multiple
tables/columns
Simple:set maidenhead 0 0 3
SL6
STORED
get maidenhead
VALUE maidenhead 0 3
SL6
END
7.2DMNoSQL with Memcached Pre-GA version available from labs.mysql.com
Copyright© 2011, Oracle. All rights reserved.
Cluster & Memcached – Schema-Free
<town:maidenhead,SL6>
key value
<town:maidenhead,SL6>
key value
generic table
Application view
SQL view
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011
Copyright© 2011, Oracle. All rights reserved.
Cluster & Memcached - Configured Schema
<town:maidenhead,SL6>
prefix key value
<town:maidenhead,SL6>
key value
Config tables map.zip
Application view
SQL view
The existence, content and timing of future releases described here is included for information only and may be changed at Oracles discretion. October 3rd, 2011
Copyright© 2011, Oracle. All rights reserved.
ClusterJ/JPA Plugin
49
• JPA interface built upon this new Java layer:
⁃Java Persistence API compliant
• Implemented as an OpenJPA plugin
⁃Uses ClusterJ where possible, reverts to JDBC for some
operations
⁃Higher performance than JDBC
⁃More natural for most Java designers
⁃Easier Cluster adoption for web applications
• Rich set of existing MySQL connectors
exist but performance worse than NDB API
• New Domain Object Model Persistence API
(ClusterJ) :
⁃Java API
⁃High performance, low latency
⁃Feature rich
Copyright© 2011, Oracle. All rights reserved. 50
ClusterJ
• High Performance, Easy to Use
• In the style of Hibernate / JPA / JDO
• Domain Object Model DataMapper pattern–Data is represented as domain objects
–Domain objects are separate from business logic
–Domain objects are mapped to database tables
• Built on ndbjtie–JNI adapter
–integral part of MySQL Cluster
–Straight mapping of MySQL Cluster API (a.k.a NDB API) to Java
• Does not support relationships–Look at JDO / JPA for these modelling patterns
Copyright© 2011, Oracle. All rights reserved.
ClusterJ Objects
Session
Factory
Configuration
properties
Session &
Transaction
Session &
Transaction
Session &
Transaction
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Domain
Object
Annotated
interface
Domain
Object
• SessionFactory
–One per Cluster per JVM
• Session
–Instance per user; represents a
Cluster connection
• Transaction
–Instance per Session
• Query
–Multiple instances per Session
Copyright© 2011, Oracle. All rights reserved.
// Create a session (connection to the database)
SessionFactory factory = ClusterJHelper.getSessionFactory(props);
Session session = factory.getSession();
// Create and initialise an Employee
Employee newEmployee = session.newInstance(Employee.class);
newEmployee.setId(988);
newEmployee.setFirst("John");
newEmployee.setLast("Jones");
newEmployee.setStarted("1 February 2009");
newEmployee.setDepartment(666);
// Write the Employee to the database
session.persist(newEmployee);
// Fetch the Employee from the database
Employee theEmployee = session.find(Employee.class, 988);
// Make some changes to the Employee & write back to the database
theEmployee.setDepartment(777);
theEmployee.setCity("London");
session.updatePersistent(theEmployee);
// Retrieve the set all of Employees in department 777
QueryBuilder builder = session.getQueryBuilder();
QueryDomainType<Employee> domain = builder.createQueryDefinition(Employee.class);
domain.where(domain.get("department").equal(domain.param("department")));
Query<Employee> query = session.createQuery(domain);
query.setParameter("department",777);
List<Employee> results = query.getResultList();
ClusterJ Application Code Example
Copyright© 2011, Oracle. All rights reserved.
ClusterJPA
• Removes ClusterJ limitations:⁃Persistent classes
⁃Relationships
⁃Joins in queries
⁃Lazy loading
⁃Table and index creation from object model
• Implemented as an OpenJPA plugin
• Better JPA performance for insert, update,
Copyright© 2011, Oracle. All rights reserved.© 2011 Oracle Corporation – Proprietary and Confidential
• @Entity(name = "department")
• public class Department {
private int Id;
private String Site;
• public Department(){}
•
@Id public int getId()
{return Id;}
public void setId(int id)
{Id=id;}
•
@Column(name="location")
public String getSite()
{return Site;}
public void setSite(String site)
{Site=site;}
• }
mysql> describe department;
+----------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| id | int(11) | NO | PRI | NULL | |
| location | varchar(255) | YES | | NULL | |
+----------+--------------+------+-----+---------+-------+
Note that OpenJPA will create this table if it doesn’t already exist but user
needs to change the storage engine to NDB:
mysql> ALTER TABLE department=ndb;
ClusterJPA Annotated class example
Copyright© 2011, Oracle. All rights reserved.
EntityManagerFactory entityManagerFactory =
Persistence.createEntityManagerFactory("clusterdb");
EntityManager em = entityManagerFactory.createEntityManager();
EntityTransaction userTransaction = em.getTransaction();
userTransaction.begin();
Department dept = new Department();
dept.setId(101);
dept.setSite(“London”);
em.persist(dept);
userTransaction.commit();
userTransaction.begin();
Department theDepartment = em.find(Department.class, 101);
theDepartment.setSite(“Edinburgh”);
userTransaction.commit();
em.close();
entityManagerFactory.close();
ClusterJPA Application code example
Copyright© 2011, Oracle. All rights reserved.
Performance
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Manager
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Manager 1.1 Features
Delivered as part of MySQL Cluster CGE 7.1
Copyright© 2011, Oracle. All rights reserved.
How Does MySQL Cluster Manager Help ?
Example: Initiating upgrade from MySQL Cluster 6.3 to 7.1
•1 x preliminary check of cluster state
•8 x ssh commands per server
•8 x per-process stop commands
•4 x scp of configuration files (2 x mgmd & 2 x
mysqld)
•8 x per-process start commands
•8 x checks for started and re-joined processes
•8 x process completion verifications
•1 x verify completion of the whole cluster.
•Excludes manual editing of each configuration file.
Total: 46 commands -
2.5 hours of attended operation
Before MySQL Cluster Manager With MySQL Cluster Managerupgrade cluster --package=7.1 mycluster;
Total: 1 Command -
Unattended Operation
• Results
• Reduces the overhead and complexity of managing database clusters
• Reduces the risk of downtime resulting from administrator error
• Automates best practices in database cluster management
Copyright© 2011, Oracle. All rights reserved.
Terms used by MySQL Cluster Manager
• Site: the set of physical hosts which are to run
Cluster processes to be managed by MySQL
Cluster Manager. A site can include 1 or more
clusters.
• Cluster: represents a MySQL Cluster
deployment. A Cluster contains 1 or more
processes running on 1 or more hosts
• Host: Physical machine, running the MySQL
Cluster Manager agent
• Agent: The MySQL Cluster Manager process
running on each host
• Process: an individual MySQL Cluster node;
one of: ndb_mgmd, ndbd, ndbmtd, mysqld &
ndbapi*
• Package: A copy of a MySQL Cluster
installation directory as downloaded from
mysql.com, stored on each host
*ndbapi is a special case, representing a slot for an external application
process to connect to the cluster using the NDB API
Site
Host Host Host Host
Cluster
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
Pro
ce
ss
agent agent agent agent
Cluster
Copyright© 2011, Oracle. All rights reserved.
1. Download MCM from edelivery.oracle.com:
• Package including Cluster
2. Unzip
3. Run agent, define, create & start Cluster!– $> bin¥mcmd –bootstrap
– MySQL Cluster Manager 1.1.2 started
– Connect to MySQL Cluster Manager by running "D:¥Andrew¥Documents¥MySQL¥mcm¥bin¥mcm" -a NOVA:1862
– Configuring default cluster 'mycluster'...
– Starting default cluster 'mycluster'...
– Cluster 'mycluster' started successfully
– ndb_mgmd NOVA:1186
– ndbd NOVA
– ndbd NOVA
– mysqld NOVA:3306
– mysqld NOVA:3307
– ndbapi *
– Connect to the database by running "D:¥Andrew¥Documents¥MySQL¥mcm¥cluster¥bin¥mysql" -h NOVA -P 3306 -u root
4. Connect to Cluster & start using database
•
http://www.clusterdb.com/mysql-cluster/mysql-cluster-manager-1-1-2-creating-a-cluster-is-now-trivial
New in MySQL Cluster Manager 1.1.2
Bootstrap single host Cluster
Copyright© 2011, Oracle. All rights reserved.
Creating & Starting a Cluster
1.Define the site:–Mysql> create site --hosts=192.168.0.10,192.168.0.11,
-> 192.168.0.12,192.168.0.13 mysite;
2.Expand the MySQL Cluster tar-ball(s) from mysql.com
to known directory
3.Define the package(s):•Mysql> add package --basedir=/usr/local/mysql_7_1_15 7.1;
Mysql> add package --basedir=/usr/local/mysql_7_2_1 7.2;
• Note that the basedir should match the directory used
in Step 2.
4.Create the Cluster–Mysql> create cluster --package=7.1
-> [email protected],[email protected],
-> [email protected],[email protected], [email protected],
-> [email protected],[email protected],[email protected]
-> mycluster;
• This is where you define what nodes/processes make
up the Cluster and where they should run
5.Start the Cluster:–Mysql> start cluster mycluster;
agent
1. ndb_mgmd
7. mysqld
192.168.0.10
agent
2. ndb_mgmd
8. mysqld
192.168.0.11
agent
5. ndbd
3. ndbd
192.168.0.12
agent
6. ndbd
4. ndbd
192.168.0.13
mysql
client
Upgrade Cluster
Copyright© 2011, Oracle. All rights reserved.
• Upgrade from MySQL Cluster 7.1.15 to 7.2.1:
• Automatically upgrades each node and
restarts the process – in the correct order to
avoid any loss of service
• Without MySQL Cluster Manager, the
administrator must stop each process in turn,
start the process with the new version and wait
for the node to restart before moving onto the
next one
agent
1. ndb_mgmd
7. mysqld
192.168.0.10
agent
2. ndb_mgmd
8. mysqld
192.168.0.11
agent
5. ndbd
3. ndbd
192.168.0.12
agent
6. ndbd
4. ndbd
192.168.0.13
mysql
client
mysql> upgrade cluster --package=7.2 mycluster;
Check the status of the Cluster
Copyright© 2011, Oracle. All rights reserved.
• Request the status of the Cluster processes–mysql> show status --process mycluster;
–+------+----------+--------------+---------+-----------+
| Id | Process | Host | Status | Nodegroup |
+------+----------+--------------+---------+-----------+
| 1 | ndb_mgmd | 192.168.0.10 | running | |
| 2 | ndb_mgmd | 192.168.0.11 | running | |
| 3 | ndbd | 192.168.0.12 | running | 0 |
| 4 | ndbd | 192.168.0.13 | running | 0 |
| 5 | ndbd | 192.168.0.12 | running | 1 |
| 6 | ndbd | 192.168.0.13 | running | 1 |
| 7 | mysqld | 192.168.0.10 | running | |
| 8 | mysqld | 192.168.0.11 | running | |
+------+----------+--------------+---------+-----------+
• Get the status of the hosts & agents–mysql> list hosts mysite;
–+--------------+-----------+---------+
| Host | Status | Version |
+--------------+-----------+---------+
| 192.168.0.10 | Available | 1.0.1 |
| 192.168.0.11 | Available | 1.0.1 |
| 192.168.0.12 | Available | 1.0.1 |
| 192.168.0.13 | Available | 1.0.1 |
+--------------+-----------+---------+
agent
1. ndb_mgmd
7. mysqld
192.168.0.10
agent
2. ndb_mgmd
8. mysqld
192.168.0.11
agent
5. ndbd
3. ndbd
192.168.0.12
agent
6. ndbd
4. ndbd
192.168.0.13
mysql
client
64
Copyright© 2011, Oracle. All rights reserved.
• Service is maintained for all Cluster processes
(i.e., automatically restart process)
• Without MySQL Cluster Manager, an angel
process would recreate failed ndbd processes
but not mysqld or ndb_mgmd processes
• Cluster continues to function if one or more
agents fail or is stopped (e.g. To upgrade it)– To restore management functions, the agent should be
restarted
agent
1. ndb_mgmd
7. mysqld
192.168.0.10
agent
2. ndb_mgmd
8. mysqld
192.168.0.11
agent
5. ndbd
3. ndbd
192.168.0.12
agent
6. ndbd
4. ndbd
192.168.0.13
HA provided by MySQL Cluster Manager
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Manager
• On-line add-node– mysql> add hosts --hosts=192.168.0.35,192.168.0.36 mysite;
– mysql> add package --basedir=/usr/local/mysql_7_1_15 –
hosts=192.168.0.35,192.168.0.36 7.2;
– mysql> add process --
[email protected],[email protected],[email protected]
68.0.35,[email protected] mycluster;
– mysql> start process --added mycluster;
• Restart optimizations
• Fewer nodes restarted on some parameter changes
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster in Action
Copyright© 2011, Oracle. All rights reserved.
Resources to Get Started
Learn More
Evaluate MySQL Cluster 7.2 Quick Start Guides
MySQL Cluster on the webwww.mysql.com/cluster/
Download Todayhttp://dev.mysql.com/downloads/cluster/
http://labs.mysql.com(memcached)
Linux, Solaris, Windowshttp://www.mysql.com/products/cluster/get-started.html#quickstart
Session Management Webinar: http://tinyurl.com/3dmtf9r
Copyright© 2011, Oracle. All rights reserved.
More Products Releases Than Ever Before
Continuous Innovation
Q2 CY2010 Q3 CY2010 Q4 CY2010 Q1 CY2011
• MySQL Workbench 5.2
GA!
• MySQL Database 5.5
• MySQL Enterprise Backup 3.5
• MySQL Enterprise Monitor 2.3
• MySQL Cluster Manager 1.1
All GA!
A Better MySQL
Q2-4 CY2011
•MySQL Enterprise Monitor 2.2
•MySQL Cluster 7.1
• MySQL Cluster Manager 1.0
All GA!
MySQL Database 5.6
MySQL Cluster 7.2
DMR*
and MySQL Labs!
(“early and often”)
*Development Milestone Release
• MySQL Enterprise
Backup 3.6
• Oracle VM Template
for MySQL Enterprise
Edition
GA!
Copyright© 2011, Oracle. All rights reserved.
Copyright© 2011, Oracle. All rights reserved.
Recommended Hardware
Copyright© 2011, Oracle. All rights reserved.
The Perfect MySQL Server
• 16-24 cores for 5.5 and above
• x86_64 - 64 bit for more memory is important
- Data/Memory ration 1/3 to 1/10 to good (rule of thumb)
- The more the better
• Linux or Solaris best, Windows and Unix also fine.
• RAID 10 for most, RAID 5 OK if very read intensive
• Hardware RAID battery backed up cache critical!
- More disks are always better!
- 4+ recommended, 8-16 can increase IO performance if needed
• At least 2 x NICs for redundancy
• Slaves should be as powerful as the Master
• Oracle Sun X4170 for example
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Hardware Selection -RAM & CPU
• Storage Layer (Data nodes)
• One data node can (7.0+) use 8 cores
• CPU: 2 x 4 core (Nehalem works really well). Faster CPU → faster processing of messages.
• RAM: As much as you need
• a 10GB data set will require 20GB of RAM (because of redundancy
• Each node will then need 2 x 10 / # of data nodes. (2 data nodes → 10GB of RAM → 16GB RAM is good)
• SQL Layer (MySQL Servers)
• CPU: 2 – 16 cores
• RAM: Not as important – 4GB enough (depends on connections and buffers)
Copyright© 2011, Oracle. All rights reserved.
low-end mid-end high-end
1 x SATA 7200 RPM
• For read-mostly
• No redundancy
(but other data node is
the mirror)
1 x SAS 10K RPM
• Heavy duty (many MB/s)
• No redundancy
(but other data node is
the mirror)
4 x SAS 10K RPM
• Heavy duty (many MB/s)
• Disk redundancy (RAID1+0)
hot swap
• REDO, LCP, BACKUP – written sequentially in small chunks (256KB)
• If possible, use Odirect = 1
LCP
REDOLOG
LCP
REDOLOG
LCP / REDOLOG
MySQL Cluster Hardware SelectionDisk Subsystem
for Checkpoints & Logs
Copyright© 2011, Oracle. All rights reserved.
Minimal recommended High-end
2 x SAS 10K RPM (preferably)
• Use High-end for heavy read / write workloads (1000's of 10KB records per sec) of data
(e.g. Content Delivery platforms)
• SSD for TABLESPACE an option to consider
• Having TABLESPACE on separate disk is good for read performance
• Enable WRITE_CACHE on devices
TABLESPACE
LCP
REDOLOG
UNDOLOG
UNDOLOG
LCP
(REDO LOG / UNDO
LOG)
TABLESPACE 1
TABLESPACE 2
4 x SAS 10-15K RPM (preferably)
(REDO LOG)
MySQL Cluster Hardware SelectionDisk Data
Storage
Copyright© 2011, Oracle. All rights reserved.
MySQL Cluster Hardware Selection
Network
• Dedicated >= 1Gb/s networking
– On Oracle Sun CMT servers it may be necessary to bond 4 or more NICs
together because typically many data nodes are on the same physical host.
• Prevent network failures (NIC x 2, Bonding, dual switches)
• Use dedicated network for cluster communication
– Put Data nodes and MySQL Servers on e.g 10.0.1.0 network and let MySQL
listen on a “public” interface.
• No security layer to management node
• Enable port 1186 access only from cluster nodes and administrators
Copyright© 2011, Oracle. All rights reserved.
Case Studies
Copyright© 2011, Oracle. All rights reserved.
• COMPANY OVERVIEW
• Leading provider of communications platforms, solutions & services
• €15.2bn Revenues (2009), 77k employees across 130 countries
• CHALLENGES / OPPORTUNITIES
• Converged services driving migration to next generation HLR / HSS systems
• New IMS platforms for Unified Communications
• Reduce cost per subscriber and accelerate time to value
• SOLUTIONS
• MySQL Cluster Carrier Grade Edition
• MySQL Support & Consulting Services
CUSTOMER PERSPECTIVE
“MySQL Cluster won the performance test hands-down,
and it fitted our needs perfectly. We evaluated shared-disk
clustered databases, but the cost would have been at least
10x more.”
-- François Leygues, Systems Manager
RESULTS
• Scale out on standard ATCA hardware to support 60m+ subscribers on a single platform
• Low latency, high throughput with 99.999%+ availability
• Enabled customers to reduce cost per subscriber and improve margins
• Delivered data management solution at 10x less cost than alternatives
http://www.mysql.com/why-mysql/case-studies/mysql-alcatel-casestudy.php
Copyright© 2011, Oracle. All rights reserved.
• COMPANY OVERVIEW
• Pyro provide comms technology solutions in Core Network, OSS/BSS & VAS
• Deployed in 120+ networks worldwide
• Cell C, one of the largest mobile operators in South Africa
• 560 roaming partners in 186 countries
• CHALLENGES / OPPORTUNITIES
• FIFA 2010 world cup opens up network services to millions of mobile subscribers
• International roaming SDP to support up to 7m roaming subscribers per day
• Offer local pricing with home network functionality
• Minimize cost and time to market
• SOLUTIONS
• MySQL Cluster 7.1 & Services
CUSTOMER PERSPECTIVE
”MySQL Cluster 7.1 gave us the perfect combination of
extreme levels of transaction throughput, low latency &
carrier-grade availability. We also reduced TCO by being
able to scale out on commodity server blades and eliminate
costly shared storage”
-- Phani Naik, Head of Technology at Pyro Group
RESULTS
• Supported subscriber and traffic volumes
• Delivered continuous availability
• Implemented in 25% of the time of typical SDP solutions
• Choice in deployment platforms to eliminate vendor lock-in (migrated from Microsoft)
Copyright© 2011, Oracle. All rights reserved.
• COMPANY OVERVIEW
• Division of Docudesk
• Deliver Document Management SaaS
• CHALLENGES / OPPORTUNITIES
• Provide a single repository for customers to manage, archive, and distribute documents
• Implement scalable, fault tolerant, real time data management back-end
• PHP session state cached for in-service
personalization
• Store document meta-data, text (as BLOBs), ACL, job queues and billing data
• Data volumes growing at 2% per day
• SOLUTION
• MySQL Cluster deployed on EC2
USER PERSPECTIVE
“MySQL Cluster exceeds our requirements for low latency,
high throughput performance with continuous availability, in
a single solution that minimizes complexity and overall
cost.”
-- Casey Brown, Manager of Dev & DBA Services,
Docudesk
RESULTS
• Successfully deployed document management solution, eliminating paper trails from legal processes
• Integrate caching and database into one layer, reducing complexity & cost
• Support workload with 50:50 read/write ratio
• Low latency for real-time user experience and document time-stamping
• Continuous database availability
Copyright© 2011, Oracle. All rights reserved. 53
“Since deploying MySQL Cluster as our eCommerce database, we have had
continuous uptime with linear scalability enabling us to exceed our uptime requirements”
— Sean Collier, CIO & COO, Shopatron Inc
Shopatron: eCommerce Platform
• Applications
• Ecommerce back-end, user authentication, order data & fulfilment, payment data & inventory tracking. Supports several thousand queries per second
• Key business benefits
• Scale quickly and at low cost to meet demand
• Self-healing architecture, reducing TCO
• Why MySQL?
• Low cost scalability
• High read and write throughput
• Extreme availability
http://www.mysql.com/why-mysql/case-studies/mysql_cs_shopatron.php
Copyright© 2011, Oracle. All rights reserved.
• COMPANY OVERVIEW
• UK-based retail and wholesale ISP & Hosting Services
• 2010 awards for best home broadband and customer service
• Acquired by BT in 2007
• CHALLENGES / OPPORTUNITIES
• Enter market for wholesale services, demanding more stringent SLAs
• Re-architect AAA systems for data integrity & continuous availability to support billing sytems
• Consolidate data to for ease of reporting and operating efficiency
• Fast time to market
• SOLUTIONS
• MySQL Cluster
• MySQL Server with InnoDB
CUSTOMER PERSPECTIVE
“Since deploying our latest AAA platform, the MySQL
environment has delivered continuous uptime, enabling us to
exceed our most stringent SLAs”
-- Geoff Mitchell Network Engineer
RESULTS
• Continuous system availability, exceeding wholesale SLAs
• 2x faster time to market for new services
• Agility and scale by separating database from applications
• Improved management & infrastructure efficiency through database consolidation
Copyright© 2011, Oracle. All rights reserved.