scaling your web app with mysql replication

Scaling your web app with MySQL replication

Giuseppe MaxiaMySQL Community Team Lead

1This work is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License.

1Thursday, 30 September 2010

Table of contents

Why replication

How to set replication

Using replication

Leveraging replication

From single server to

replication

Adding a slave

Replacing a slave

Replacing the master

Managing replication

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

a simple web application

scheme

database server

web server

clients

r/w requests

database server

web servers

load balancer

clients

r/w requests

scaling web requests

writeread

database load on a simple

web application

85% 15%

readwrite

database load on a successful web

application7

20% 80%

database server

web servers

load balancer

clients

r/w requests✘

scaling up means buying

a bigger database

server

readwrite

the bigger database server will eventually

have the same problem

80%20%

read/writemaster

read/onlyslaves

web servers

load balancer

clients

a web application

scheme with replication

read/writemaster

read/onlyslaves

readwriteread

database load with

replication11

85% 15% 100%

read/writemaster

read/onlyslaves

readwriteread

scaling database load

with replication

85% 15% 100%

Replication assessment

without replication with replication

database handling

performance

Point in Time recovery

failover

write scaling

backup

read scaling

easy harder

high lower (binary logs)

none easy

none possible

none minimal

with downtime without downtime

none easy

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

master

slaveIO thread

SQL thread

client

transaction

binary log

relay log replication concepts

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

1 SHUT DOWN THE DATABASE SERVER

Master

2 MAKE A BACKUP COPY

Master

3 ENABLE THE MASTER

Configuration file[mysqld]log-bin=mysql-binserver-id=1

Master

4 RESTART THE MASTER

Master

5 CREATE REPLICATION USER

SQL commandGRANT REPLICATION SLAVE ON *.* to 'slave_user@'10.10.100.%' IDENTIFIED BY 'slave_pass';

Master

6 INSTALL MySQL on the slave

Slave 1

Make sure that:• You're using the same version of MySQL • You have the same directory structure• The server is not started yet

7 COPY THE MASTER DATA to the slave

Slave 1

8 ENABLE THE SLAVE

Configuration file[mysqld]server-id=2relay-log=mysql-relayread-only# optional:log-bin=mysql-bin

Slave 1

9 START THE SLAVE SERVER

Slave 1

10 INITIALIZE THE SLAVE

SQL commandSET MASTER TOMASTER_HOST=master_IP,MASTER_PORT=3306,MASTER_USER=slave_user,MASTER_PASSWORD='slave_pwd';

Slave 1

11 START THE SLAVE SERVICE

SQL commandSTART SLAVE;

Slave 1

12 CHECK THE SLAVE

SQL commandSHOW SLAVE STATUS \G... Slave_IO_Running: YesSlave_SQL_Running: Yes...

Slave 1

Troubleshooting

• SHOW SLAVE STATUS says SLAVE_IO_RUNNING=No

• Make sure that the slave host can connect to the master

• Make sure that master and slave have different Server-id

• Check the error log of both master and slave

Testing the slave

• Create a table in the master.

• Make sure that the slave has replicated the table.

• Insert data in the master

• read that data in the slave

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

1 NO NEED TO STOP THE MASTER!

Master

2 STOP THE SLAVE

SQL commandSTOP SLAVE IO_THREAD;# wait until the SQL_THREAD# has done everythingSTOP SLAVE SQL_THREAD;# STOP THE SERVER

Slave 1

3 MAKE A COPY OF THE DATA DIRECTORY

Slave 1

4 RESTART THE SLAVE

Slave 1

5 INSTALL MySQL on the new slave

Slave 2

Make sure that:• You're using the same version of MySQL • You have the same directory structure• The server is not started yet

6 COPY THE old slave DATA on the slave

Slave 2

7 ENABLE THE NEW SLAVE

Configuration file[mysqld]server-id=3relay-log=mysql-relayread-only # optional:log-bin=mysql-bin

Slave 2

must be unique!

8 START THE NEW SLAVE

Slave 2

9 CHECK THE SLAVE

SQL commandSHOW SLAVE STATUS \G... Slave_IO_Running: YesSlave_SQL_Running: Yes...

Slave 2

Why it works

• No need to issue a CHANGE MASTER TO command.

• Because we cloned the old slave

• The new slave gets its parameters from the .info files in the data directory

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

From single server application

r/w requests

To replication-aware application

read/writemaster

read/onlyslaves

R/WR/O

load balancer

Single server application$link = mysql_connect(

$server_IP, 'mysql_user', 'mysql_password');$result = mysql_query('INSERT INTO table_name (x) VALUES (1)',$link);$result = mysql_query('SELECT * FROM table_name WHERE x=1',$link);

Making an application aware of replication

connectreadwrite

IPuserpassword

<<R/W database handling>>

connect

IPuserpassword

<<database handling>>db

IPuserpassword

<<read-only database handling>>

connectreadwrite

IPuserpassword

<<R/W database handling>>

Using replication: the WRONG way

R/W split by

statement

Write statement?

Connect to the master

Write to the master

Connect to the next available

Read from slave

Why statement split is wrong

• Breaks transactions

• High risk of inconsistency

• Loses or corrupts data

Using replication:the RIGHT way

R/W split by function

Write function?

Connect to the master

Read and write from master

Connect to the next available

Read from slave

more queries?

Yesmore

queries?

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

•MONITORING• … or die

Sample monitoring

master

Get master binlog and position

get slave status

Running?

Same or later binlog/position?

check table contents

monitoringcontents

GET table CRC

master slave

GET table CRC

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

Slave crashes

are there more slaves?

STOP the master STOP one

add the first slave

add another slave

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

Master crashes

Let all slaves catch up with

execution

FIND the most up to date slave

STOP replication in

all slaves

make it the master

FIND which transactions are

missing from other slaves

connect all slaves to the new master

run missing transactions

to other slaves

Table of contents

Why replication

Using replication

replication

Adding a slave

Replacing a slave

read/writemaster

read/onlyslaves

R/WR/O

load balancer

load balancing

backup master

slaves

STOP SLAVE

remove slave from load balancer

START SLAVE

perform backup

attach slave to load

balancer

Let slave catch up

master

slaves

STOP SLAVE

remove slave from load balancer

START SLAVE

calculate summary

tables

attach slave to load

balancer

Let slave catch up

makesummary

tables

master

innodbnon partitioned

innodbnon partitionedinnodb

partitioned by range

MyISAMpartitioned by range

Partitionsfor heavy statistics

master

innodbnon partitioned

ARCHIVEpartitioned by range

(date)

(product)

(location)

Simulating multiple

dimensions

Semi-synchronous replication

• Available in 5.5 and higher

• Makes sure that at least one slave has copied the data.

• Increases reliability

transaction with regular replication

clientmaster

commit

binary log

execute

returns to client

replication

transaction with semi-

synchronous replication

clientmaster

commit

binary log

execute

returns to client

sends transaction

to slave

gets acknowledgement

relay log

scaling your web app with mysql replication

slave slave

slave service slave

master leveraging replication

master data

slave host

slave serverslave

everythingstop slave

sql command set master

Technology

scaling mysql -- swanseacon.co.uk

jailbreaking mysql replication

scaling php/mysql

mysql replication[1]

mysql replication & cluster

mysql scaling

mysql replication basics

doag 2011: mysql replication

mysql replication devconf 2012

mysql replication formats

mysql replication: what's new in mysql 5.7 and mysql 8 ·...

mysql multi-master replication

froscon 2011: mysql replication

mysql group replication

mysql 5.7 replication news

mysql replication

mysql replication tutorial

mysql replication advanced features - fosdem · mysql...

scaling state machine replication · mysql group...

reliable mysql using replication