setup & operate tungsten replicator

©Continuent 2014

Tungsten University:  Set Up and Operate

Tungsten Replication Robert Hodges, CEO

©Continuent 2014

Introducing Continuent

2

• The leading provider of clustering and replication for open source DBMS

• Our Product: Continuent Tungsten

• Clustering - Commercial-grade HA, performance scaling and data management for MySQL

• Replication - Flexible, high-performance data movement

©Continuent 2014

Introducing Tungsten Replicator

3

©Continuent 2014

Tungsten Replicator Overview

4

Master

(Transactions + Metadata)

Slave

THL

DBMS Logs

Replicator

(Transactions + Metadata)

THLReplicator

Download transactions via network

Apply using JDBC

©Continuent 2014

Master Replication Service

5

Extract Filter Apply

StageExtract Filter Apply

Stage

Pipeline

MySQL Master

Transaction History Log

In-Memory Queue

Slave ReplicatorsBinlog

tcp/ip

©Continuent 2014

Slave Replication Service

6

Extract Filter Apply

StageExtract Filter Apply

StageExtract Filter Apply

Stage

Pipeline

Transaction History Log

In-Memory Queue

Slave DBMS

Master Replicator

tcp/

ip

©Continuent 2014

EC2 Installation Using Github project

1.Set up Github vagrant project

git clone https://github.com/continuent/continuent-vagrant

2.Pick EC2 Vagrantfile and fill in keys

3.Copy examples/MasterSlave/default.pp to manifests/

4.Run launch.sh

7

(Follow directions in README.md)

©Continuent 2014

Tpm installation overview

8

/opt/continuent/etc/tungsten/tungsten.ini

check prereqs [copy code]

configure topology

[defaults]!user=tungsten!home-directory=/opt/continuent!datasource-user=tungsten!datasource-password=secret!application-port=3306!application-user=app_user!application-password=secret!skip-validation-check=MySQLPermissionsCheck!start-and-report=true!![east]!master=db1!slaves=db2,db3!topology=master-slave!

tpm update

(Installation)

©Continuent 2014

What Is Tungsten Doing?

9

©Continuent 2014

Commands to Check Replication

• trepctl services -- All services

• trepctl status -- Overall pipeline status

• trepctl status -name stages -- Stage summary

• trepctl status -name stores -- Store summary

• trepctl status -name tasks -- Task detail

• trepctl [-service name] heartbeat -- Is it Alive?

10

©Continuent 2014

Changing Replicator Configuration

11

©Continuent 2014

Reconfiguring a Replicator

• Edit /etc/tungsten/tungsten.ini

• No restart on local host

trepctl offline!

tpm update --no-restart!

trepctl configure!

trepctl online!

!

• With automatic restart

tpm update

12

Don’t change replicator configuration files directly!

©Continuent 2014

Enabling Parallel Replication

13

©Continuent 2014

Parallel Replication of Sharded Data

14

Extract Filter Apply

StageExtract Filter Apply

StageStage

Pipeline

Transaction History Log

Parallel Queue

Slave DBMS

Extract Filter ApplyExtract Filter ApplyExtract Filter Apply

Master Replicator

tcp/

ip

©Continuent 2014

Fixing Problems

15

©Continuent 2014

The Scene of the Crime

16

trepctl -host db2 status Processing status command... NAME VALUE ---- ----- ... pendingError : Event application failed: seqno=1957469 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on master pendingErrorCode : NONE pendingErrorEventId : mysql-bin.000008:0000000461736666;0 pendingErrorSeqno : 1957469 pendingExceptionMessage: java.sql.SQLException: Statement failed on slave but succeeded on master create table trouble(id int primary key) /* ___SERVICE___ = [university] */ ... Finished status command...

©Continuent 2014

Correcting problems

1.Look at the transaction (thl list -seqno NNN)

2.Look at the replicator message log

3.Fix the problem and go online

OR

4.Skip over the problem and go online

OR

5.Reprovision the slave

17

©Continuent 2014

Tungsten Logs

18

©Continuent 2014

Simple Facts about Logs

• Replicator log = THL (Transaction History Log)

• One THL per replication service

• 2x the size of MySQL binlog

• Default retention of 7 days

• Take service offline to move or delete logs safely

• The thl tool shows you log contents

19

©Continuent 2014

Resetting Replication

20

©Continuent 2014

Resetting/Reinstalling Automatically

Reset

1.tpm reset

Reinstall

1.tpm uninstall --i-am-sure

2.Rerun installation

21

©Continuent 2014

Resetting/Reinstalling Manually

Reset

1.Stop replicator

2.Remove THL files

3.Remove tungsten_<svc> databases

Reinstall

1.All of the above, plus delete /opt/continuent/*

2.Rerun installation

22

©Continuent 2014

Backup and Restore

23

©Continuent 2014

What Backup Tools Does MySQL Have?

24

MySQL Server

Process

File System

DBMS Files

mysqldump (dataset < 10Gb)

xtrabackup (10Gb < dataset < 1Tbtar/rsync (dataset < 1Tb)

file system snapshot (1Tb << dataset)

©Continuent 2014

Advanced Installation Patterns

25

©Continuent 2014

How Do I Install Fan-In Replication?

26

db1Replicator

db3

Service db1

db2Replicator

Service db2

Replicator

Service db1

Service db2

©Continuent 2014

How Do I install Multi-Master?

27

db1 Replicator

Service db1

Service db2

Service db3db3

Service db1

Service db2

Service db3db2

Replicator

Service db1

Service db2

Service db3

©Continuent 2014

Tpm Understands Topologies!

• Use --topology argument!

28

Fan-In StarAll-Masters

--topology=fan-in --topology=all-masters --topology=star

©Continuent 2014

Tpm install from staging directory

29

db1

db2

db3

Staging copy of files

check prereqs copy code

configure topology

tpm configure ... tpm update

©Continuent 2014

Commands to Perform Staging Install

30

$ wget http://downloads.tungsten-replicator.org/download.php?file=tungsten-replicator-2.2.1-403.tar.gz !$ tar -xvzf tungsten-replicator-2.2.1-403.tar.gz !$ ln -s tungsten-replicator-2.2.1-403 tungsten !$ tungsten/tools/tpm configure defaults --reset \ --user=tungsten --home-directory=/opt/continuent \ --datasource-user=tungsten --datasource-password=secret \ --skip-validation-check=MySQLPermissionsCheck \ --start-and-report=true !$ tungsten/tools/tpm configure fanin \ --members=db1,db2,db3 \ --masters=db1,db2 \ --master-services=db1,db2 \ --topology=fan-in !$ tungsten/tools/tpm update

©Continuent 2014

Joining the Community

31

©Continuent 2014

Tungsten Replicator is Open Source

32

• Project home:

http://code.google.com/p/tungsten-replicator/

• Log bugs, find builds, post in replicator discussion group

• Documentation:

https://docs.continuent.com

©Continuent 2014

HeterogeneousOracleMySQL Oracle MySQL MySQL

star

master-slave

fan-in slave all-masters

MySQLOracle Oracle

©Continuent 2014

Wrapping Up

34

©Continuent 2014

More Tungsten University Webinars!

35

• Set Up and Operate Tungsten Clusters(22 May 2014 10am PDT)

• Set Up and Operate Real-Time Data Loading into Hadoop (29 May 2014, 10am PDT)

• Set Up and Operate Open Source Oracle Replication(5 June 2014, 10am PDT)

Send feedback to: tu@continuent.com

©Continuent 2014

Continuent Web Page: http://www.continuent.com

!

Tungsten Replicator 2.0: http://code.google.com/p/tungsten-replicator

Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://flyingclusters.blogspot.com/

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com