mysql ha sharding-fabric
TRANSCRIPT
WEBINAR on
MySQL Fabric For HA and Scaling Out
Presented ByAbdul Manaf
OSSCube Corporate Capabilities
Who Are We
• Global Solutions Provider having offices in the US, UK and India.
• We offer complete solutions in the domain of Digital Transformation, IT Modernization and IoT.
• Appraised at CMMI Level 3, we've been amongst the top 100 Great Places to Work® for three consecutive (2014, 2015 & 2016) years.
• We’ve delivered enterprise solutions to the likes of The New York Times, PVR Cinemas, ACER, Intel, CU Solutions, among others.
Digital Transformation
Practice
• PIM (Product Information Management)• WCM (Web Content Management)• DAM (Digital Asset Management)• e-Commerce
Modernization Practice
• Cloud • Enterprise Mobility• Application Modernization
IOT Practice • SMART Track & Trace• Product Lifecycle Management • Big Data - Analytics
© 2015 OSSCube LLC. All rights reserved.
What We Do
Agenda
• Concept of HA
• Concept of Sharding
• MySQL Fabric Basics
• MySQL Fabric for HA
• MySQL Fabric for Sharding
• Some commands for Configuration & Installation
High Availability
High Availability
Availability Calculation
Principles and Causes of loosing HA
Principles of High Availability• Elimination of single points of failure• Reliable crossover. In multi threaded systems, the crossover point
itself tends to become a single point of failure. • Detection of failures as they occur. If the two principles above are
observed, then a user may never see a failure. But the maintenance activity must.
Scheduled and unscheduled downtimeCauses of losing it (unscheduled downtime)• Failures of hosts , Databases / MySQL• Operating system• The hardware • Maintenance activity that may otherwise cause downtime• And many more ……
MySQL HA Solutions
HA is achieved via redundancy
The primary solutions supported by MySQL include:
• MySQL Native Replication• MySQL Cluster• MySQL with DRBD• Galera Cluster• Oracle VM Template for MySQL• MySQL with Solaris Cluster
So what is MySQL Replication?
Replication copies transactions from the master and replays them to the slave
Sharding
Sharding
• Sharding is a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts called data shards.
• The word shard means a small part of a whole.
(Sharding) Tumblrʼs Size and Growth
Why Sharding?
Write scalability
• No other way to scale writes beyond the limits of one machine
• During peak insert times, you'll likely start hitting lag on slaves before your master shows a concurrency problem
Data size
• Working set won't fit in RAM• SSD performance drops as disk fills up• Risk of completely full disk• Operational difficulties: slow backups, longer to spin up new
slaves• Fault isolation: all of your data in one place = single point of
failure affecting all users
Make Sharding Your Last Resort
• Optimize everything else first, and then if the performance still isn’t good enough, it’s time to take a very bitter medicine
• The application developer has to write more code to be able to handle sharding logic
• Operational issues become more difficult (backing up, adding indexes, changing schema).
MySQL FabricAn extensible and easy-to-use framework for
managing a farm of MySQL server supporting high-availability and sharding
MySQL Fabric: What is it?
General Concepts , MySQL Fabric
• Group : collection of mysqld servers
• Global group : Special groups that store updates that must be propagated to all shards
• Node : Running instance of MySQL Fabric
• Shard : Horizontal partition of data in a table
• Primary : Group member that has been designated master
• Secondary : Group member that is read only
High Availability, MySQL Fabric
• Group of MySQL servers
• Primary server
• Secondary servers
• GTID based replication to work
• MySQL 5.6 +
MySQL Fabric , HA
What Fabric Provides in terms of HA
• Failure detection and promotion
• Routing of database
MySQL Fabric , Sharding
• Used to scale-out the database servers by partitioning the data across multiple MySQL Server "groups".
• Group could contain a single MySQL Server or it could be a HA group.
MySQL Fabric , Sharding
Birds-eye View of a Sharded Database
MySQL Fabric: Configuration
MySQL Fabric: Setup and Teardown
MySQL Fabric: Starting and Stopping
Sharding Architecture
MySQL Fabric: Sharding Setup
MySQL Fabric: Create Groups and add Servers
MySQL Fabric: Create Groups and add Servers
MySQL Fabric: Set up Shard Mapping
MySQL Fabric: Add Shards
Current Limitations
• Sharding is not completely transparent to the application.
• Auto-increment columns cannot be used as a sharding key
• The MySQL Fabric process itself is not fault-tolerant and must be restarted in the event of it failing
• Because the connectors perform the routing function, which could add extra latency
