replication tips & tricks

of 49 /49
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12 2 Replication Tips & Tricks Mats Kindahl [email protected]

Author: mats-kindahl

Post on 27-Jan-2015

122 views

Category:

Technology


0 download

Embed Size (px)

DESCRIPTION

Presentation on replication tips and tricks given at the MySQL Connect 2012 conference.

TRANSCRIPT

  • 1. Replication Tips & TricksMats [email protected] 2Copyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12

2. Program Agenda Replication setup and status checking Binary log analysis Crash-safe slaves Multi-source replication Slave fail-over using GTID3 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 3. About the PresentationThis presentation will introduce you to some replicationfeatures and also briefly show tips and tricks on how towork with replication. The focus is on short ideas andeach item does not go deeper into details.4 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 4. Replication Architecture Master Slave5 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 5. Reading Slave StatusUnderstanding the fields of SHOW SLAVE STATUS I/O Thread Status SQL Thread Status Slave_IO_Running Slave_SQL_Running Last_IO_Errno Last_SQL_Errno Last_IO_Error Last_SQL_Error Last_IO_Error_Timestamp Last_SQL_Error_Timestamp Master being replicated from Master_Host Master_Port6 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 6. Reading Slave StatusUnderstanding the fields of SHOW SLAVE STATUS Next event to execute in master log coordinates Relay_Master_Log_File + Exec_Master_Log_Pos Next event to execute in relay log coordinatesRelay_Log_File + Relay_Log_Pos Next event to read from masterMaster_Log_File + Read_Master_Log_Pos7 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 7. Reading the Binary LogDecoding the binary log Use mysqlbinlog--hexdumpShow hex dump of event as comment--start-position Start dumping at a position--stop-position Stop dumping after this position# at 275175 =373 16 10#120927 23:11:58 server id 3 end_log_pos 373# Position TimestampTypeMaster IDSizeMaster PosFlags#113 1e c1 64 50 0203 00 00 00 62 00 00 00 75 01 00 00 00 00 Little-endian Query8 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 8. Reading the Binary LogDecoding the binary log# at 275#120927 23:11:58 server id 3 end_log_pos 373# Position TimestampTypeMaster IDSizeMaster Pos Flags#113 1e c1 64 50 0203 00 00 00 62 00 00 00 75 01 00 0000 00#126 c5 03 00 00 00 00 00 00 04 00 00 1a 00 00 00 00 |................|#136 00 00 01 00 00 00 00 00 00 00 00 06 03 73 74 64 |.............std|#146 04 21 00 21 00 08 00 74 65 73 74 00 69 6e 73 65 |.......test.inse|#156 72 74 20 69 6e 74 6f 20 74 32 20 76 61 6c 75 65 |rt.into.t2.value|#166 73 20 28 31 2c 27 74 65 73 74 69 6e 67 27 29 |s..1..testing..|#Querythread_id=965exec_time=0 error_code=0SET TIMESTAMP=1348780318/*!*/;insert into t2 values (1,testing)/*!*/;9 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 9. Reading the Binary LogDecode row events Use mysqlbinlog--verboseDecode row events into pseudo-SQL # at 849 # at 893 #120928 15:19:24 server id 3end_log_pos 893 Table_map: `test`.`t2` mapped to number 48 #120928 15:19:24 server id 3end_log_pos 941 Write_rows: table id 48 flags: STMT_END_F BINLOG3KNlUBMDAAAALAAAAH0DAAAAADAAAAAAAAEABHRlc3QAAnQyAAIDDwIoAAM= 3KNlUBcDAAAAMAAAAK0DAAAAADAAAAAAAAEAAv/8AwAAAA1yb3dzIGFyZSBjb29s /*!*/;10Copyright 2012, Oracle and/or its affiliates. All rights reserved. 10. Reading the Binary LogDecode row events Use mysqlbinlog--verboseDecode row events into pseudo-SQL # at 849 # at 893 #120928 15:19:24 server id 3end_log_pos 893 Table_map: `test`.`t2` mapped to number 48 #120928 15:19:24 server id 3end_log_pos 941 Write_rows: table id 48 flags: STMT_END_F BINLOG3KNlUBMDAAAALAAAAH0DAAAAADAAAAAAAAEABHRlc3QAAnQyAAIDDwIoAAM= 3KNlUBcDAAAAMAAAAK0DAAAAADAAAAAAAAEAAv/8AwAAAA1yb3dzIGFyZSBjb29s /*!*/; ### INSERT INTO test.t2 ### SET ### @1=3 ### @2=rows are cool11Copyright 2012, Oracle and/or its affiliates. All rights reserved. 11. Cloning a Slave Creating a slave image1. Stop the slave2. Write down where slave has stoppedSHOW SLAVE STATUSRelay_Master_Log_File + Exec_Master_Log_Pos3. Backup slavemysqldump slow but safePhysical file copy fast but you need to shut down the server12 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 12. Cloning a Slave Setting up the new slave4. Restore backup on new slave5. Direct the slave to the masterCHANGE MASTERMaster_Host + Master_PortRelay_Master_Log_File + Exec_Master_Log_Pos6. Start slaveSTART SLAVE13 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 13. Cloning a Slave The easy way using MySQL UtilitiesUsing mysqldbcopy mysqldbcopyreplication=slave locking=snapshot source=root:[email protected] destination=root:[email protected] database...14 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 14. Cloning a Slave The easy way using MySQL UtilitiesUsing mysqldbcopy mysqldbcopyreplication=slave locking=lockall source=root:[email protected] destination=root:[email protected] database...15 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 15. Point in time recoveryUse mysqlbinlog --to-datetimeStop reading when reaching dateExample mysqlbinlog --read-from-remote-server e --host=master -uroot --position=192ar! --to-datetime=2009-04-11 12:36:56 psingam reas st master-bin.00002[2-4] | enc mysql -uroot --host=slavetim lly i:a ing icr n ot on W a onmnot16 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 16. Using Relay ServersRelieve master server by creating relay slavesJust keep binary logsDo not store data in tablesUse BLACKHOLE engine--log-slave-updatesSET STORAGE_ENGINE = BLACKHOLEALTER TABLE table ENGINE = BLACKHOLE17 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 17. Using Relay Servers replicate-wild-do-table-wild=*.%_east replicate-wild-do-table-wild=*.%_westFiltering replication streamFiltering done on relay serversreplicatewild*tableWork with cross-database queries18 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 18. Write Your Own Library Write your own utility libraryCreate a utility library to easier work with replicationCommon functions to manage replication:Start/stop slaveChange masterFetch master positionFetch slave execute/read positionFetch master host information19 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 19. Write Your Own Library Basic functions using Bourne shell #Usage:mysql_execsocketsql...#Usage:stop_slavesocket[thread] mysql_exec(){stop_slave(){ sock=$1mysql_exec$1STOPSLAVE$2 shift1} mysqlurootverticalbatch skipcolumnnames#Usage:start_slavesocket[thread] socket="$sock"start_slave(){ exec"$*"mysql_exec$1STARTSLAVE$2 }}20 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 20. Write Your Own Library Basic functions using Bourne shell #Usage:change_mastersocket[host[port[file[pos]]]] change_master(){ host=${2:+MASTER_HOST=$2} port=${3:+,MASTER_PORT=$3} file=${4:+,MASTER_LOG_FILE=$4} pos=${5:+,MASTER_LOG_POS=$5} mysql_exec$1CHANGEMASTERTO$host$port$file$pos } #Usage:fetch_master_possocket fetch_master_pos(){ mysql_exec$1SHOWMASTERSTATUS| grep$1.savepos }24 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 24. Round-Robin Replication A version using Bourne Shell restore_and_start () { cnt=1 socket="/var/run/mysqld/$1.sock" while true cat $1.savepos | { do read host stop_and_save mysqld.$cnt read port cnt=`expr $cnt % 5 + 1` read file restore_and_start mysqld.$cnt read possleep 60 change_master $socket done$host $port $file $pos start_slave $socket } }25 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 25. Crash-safe Slaves Keeping replication information in sync with data FILE TABLETraditional ReplicationPosition stored in fileUpdate after transactionCrash can lose updateTransactional Replication Repository location FILE or TABLEPositions stored in table master_info_repositoryUpdate part of transactionrelay_log_info_repositoryCrash-safe26 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 26. Crash-safe Slaves Steps to set up crash-safe slavesDirect server to use table-based repositoryDefaults to FILE master_info_repository=TABLE relay_log_info_repository=TABLEEnsure that a transactional engine is usedInnoDB default since 5.6.6Before 5.6.6: set the storage engine ALTERTABLEslave_master_infoENGINE=InnoDB ALTERTABLEslave_relay_log_infoENGINE=InnoDB27 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 27. Round-Robin Replication A version using pure SQL CREATE TABLE my_masters (CREATE TABLE current_master ( idx INT PRIMARY KEY, idx INT host CHAR(50) NOT NULL,) ENGINE=InnoDB port INT NOT NULL DEFAULT 3306, log_file CHAR(50), log_pos LONG, UNIQUE INDEX (host,port,user) ) ENGINE=InnoDB CREATE PROCEDURE save_position() BEGINDECLARE l_idx INT UNSIGNED;UPDATE my_masters AS mi, mysql.slave_relay_log_info AS rli SET mi.log_pos = rli.master_log_pos, mi.log_file = rli.master_log_name WHERE idx = (SELECT idx FROM current_master); END28 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 28. Round-Robin Replication In pure SQLCREATE PROCEDURE fetch_next_master(OUT p_host CHAR(50), OUT p_port INT UNSIGNED, OUT p_file CHAR(50), OUT p_pos BIGINT)BEGIN DECLARE l_next_idx INT DEFAULT 1; SELECT idx INTO l_next_idx FROM my_masters Select next index,WHERE idx > (SELECT idx FROM current_master)if there is oneORDER BY idx LIMIT 1; SELECT idx INTO l_next_idx FROM my_mastersSelect the first index, ifWHERE idx >= l_next_idx ORDER BY idx LIMIT 1; there were no next index UPDATE current_master SET idx = l_next_idx; SELECT host, port, log_file, log_pos Fetch master info using INTO p_host, p_port, p_file, p_pos FROM my_masters WHERE idx = l_next_idx;the indexEND29 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 29. Round-Robin Replication In pure SQLCREATE EVENT multi_source ON SCHEDULE EVERY 10 SECOND DOBEGIN DECLARE l_host CHAR(50); DECLARE l_port INT UNSIGNED; DECLARE l_file CHAR(50); DECLARE l_pos BIGINT; SET SQL_LOG_BIN = 0; STOP SLAVE; START TRANSACTION; CALL save_current_position(); CALL fetch_next_master(l_host, l_port, l_file, l_pos); CALL change_master(l_host, l_port, repl_user, xyzzy, l_file, l_pos); COMMIT; START SLAVE;END30 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 30. Different Tables on Master and Slave Upgrading Example: Upgrading a topology without downtime Saving space Not keeping big columns on slave Hiding data Keeping data away from users Note: still available in binary log and relay log just not applied to table Debugging and Auditing Adding data for auditing31 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 31. Different Tables on Master and Slave More columns on slave CREATE TABLE employee ( name CHAR(40),Master CREATE TABLE employee ( name CHAR(40),Slave email CHAR(40)email CHAR(40), changed TIMESTAMPDEFAULT CURRENT_TIMESTAMPON UPDATE CURRENT_TIMESTAMP ) )Same initial columnsStatement-based or row-based replication32 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 32. Different Tables on Master and Slave More columns on master CREATE TABLE employee (CREATE TABLE employee ( name CHAR(40), name CHAR(40), email CHAR(40),email CHAR(40), password CHAR(40)password CHAR(40) ) Master ) SlaveSame initial columnsRow-based replication only33 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 33. Different Tables on Master and Slave Different types of columns on master and slave CREATE TABLE employee (CREATE TABLE employee ( emp_id INT,emp_id TINYINT, name CHAR(40), name CHAR(40), email CHAR(40) email CHAR(40) ) Master ) SlaveTable definitions are identical except that some column type are different34 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 34. Different Tables on Master and Slave Slave type conversions CREATE TABLE employee (CREATE TABLE employee ( emp_id INT,emp_id TINYINT, name CHAR(40), name CHAR(40), email CHAR(40) email CHAR(40) ) Master ) SlaveConsider this statement:St at em INSERT INTO employee VALUES Re entWorks fine (1, Bob, [email protected]),pl -b ica a (500, Alice, [email protected])tio sed Fails silently! n35 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 35. Different Tables on Master and SlaveSlave type conversionsCREATE TABLE employee (CREATE TABLE employee (emp_id TINYINT,emp_id INT,name CHAR(40), name CHAR(40),email CHAR(40) email CHAR(40)) Master ) Slave Consider this statement: RowRe -baplINSERT INTO employee VALUES ica sed tio Throws an error! (1, Bob, [email protected]), n(100, Alice, [email protected])36Copyright 2012, Oracle and/or its affiliates. All rights reserved. 36. Different Tables on Master and Slave Slave type [email protected]@GLOBAL.SLAVE_TYPE_CONVERSIONSType: SET(ALL_LOSSY,ALL_NON_LOSSY)Require slave restartLossy conversions Non-lossy conversionsINTTINYINT TINYINTINTCHAR(64)VARCHAR(32) VARCHAR(32)CHAR(64)DOUBLEFLOAT FLOATDOUBLE37 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 37. Different Tables on Master and Slave Slave type conversionsConversions within same domain possibleINT, TINYINT, SMALLINT, MEDIUMINT, BIGINTCHAR(n), VARCHAR(n), TINYTEXT, TEXT, DECIMAL(n,m), DOUBLE, FLOATALL_NON_LOSSY in SLAVE_TYPE_CONVERSIONSConversion to larger domains allowedALL_LOSSY in SLAVE_TYPE_CONVERSIONSConversion to smaller domains allowed (truncation/rounding may occur)38 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 38. Different Tables on Master and Slave Slave type conversions CREATE TABLE employee ( CREATE TABLE employee ( emp_id TINYINT, emp_id INT, name VARCHAR(40), name VARCHAR(40), email VARCHAR(40) email VARCHAR(40) ) Master) SlaveConsider this statement: Ro w Re -ba pl ica sedINSERT INTO employee VALUES(1, Bob, [email protected]),tio Worksfine!(100, Alice, [email protected]) nSLAVE_TYPE_CONVERSIONS = ALL_NON_LOSSY39 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 39. Different Tables on Master and SlaveSlave type conversionsCREATE TABLE employee ( CREATE TABLE employee (emp_id INT, emp_id TINYINT,name VARCHAR(40), name VARCHAR(40),email VARCHAR(40) email VARCHAR(40)) Master) Slave Consider this statement:Ro w Re -ba pl ica sedINSERT INTO employee VALUES(1, Bob, [email protected]),tio Throws an error!(100, Alice, [email protected]) n SLAVE_TYPE_CONVERSIONS = ALL_NON_LOSSY40Copyright 2012, Oracle and/or its affiliates. All rights reserved. 40. Master and Slave out of sync Slave stopped with a strange error ERROR 1062 (23000): Duplicate entry 3 for key PRIMARY Compare two databases using mysqldbcompare mysqldbcompare [email protected] [email protected] employee41 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 41. Master and Slave out of sync Comparing two databases and finding the differences $ mysqldbcompare --quiet [email protected]:3309 --server2[email protected]:3310 world # Checking databases world on server1 and world on server2 # # # Data differences found among rows: --- world.City +++ world.City @@ -1,5 +1,5 @@ -+-------+------------+--------------+-----------+-------------+ -| ID| Name | CountryCode | District | Population | -+-------+------------+--------------+-----------+-------------+ -| 3048 | Stockholm | SWE | Lisboa | 750348 | -+-------+------------+--------------+-----------+-------------+ ++-------+-----------+--------------+-----------+-------------+ +| ID| Name| CountryCode | District | Population | ++-------+-----------+--------------+-----------+-------------+ +| 3048 | Helsinki | SWE | Lisboa | 750348| ++-------+-----------+--------------+-----------+-------------+42 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 42. Master and Slave out of sync Synchronizing master and slave databases $ mysqldbcompare --quiet --difftype=sql --changes-for=server2> [email protected] [email protected]> world # Checking databases world on server1 and world on server2 # # # Transformation for --changes-for=server2: # # Data differences found among rows: UPDATE world.City SET Name = Stockholm WHERE ID = 3048;43 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 43. Master and Slave out of sync Synchronizing master and slave databases $ mysqldbcompare --quiet --difftype=sql --changes-for=server1> [email protected] [email protected]> world # Checking databases world on server1 and world on server2 # # # Transformation for --changes-for=server1: # # Data differences found among rows: UPDATE world.City SET Name = Helsinki WHERE ID = 3048;44 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 44. Replication Event ChecksumsMasterSlaveEventChecksums45 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 45. Replication Event Checksums Configurations and optionsControlling checksum generation (default is CRC32 since 5.6.6) SETGLOBALBINLOG_CHECKSUM=CRC32Enable verification on events read by dump thread SETGLOBALMASTER_VERIFY_CHECKSUM=ONEnable verification on events read by SQL thread SETGLOBALSLAVE_SQL_VERIFY_CHECKSUM=ON46 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 46. Using Global Transaction ID for Fail-over Handling positions in MySQL 5.5Binary log positions manually handledEach server has its own positionFailing over slaves to new masters difficultWhat is the position to fail over to? CHANGE MASTER TOMASTER_HOST=master1.example.com, MASTER_PORT=3306,MASTER_USER=repl_user, MASTER_PASSWORD=xyzzy,MASTER_LOG_FILE=master-bin.00001, MASTER_LOG_POS=22145; Different fordifferent servers47 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 47. Using Global Transaction ID for Fail-over Enabling Global Transaction IDs MySQL 5.6 has Global Transaction ID[mysqld] Positions independent of servergtid-mode=on GTID handshake disable-gtid-unsafe-statements Done on connection with master log-bin=master-bin Negotiate position automatically log-slave-updates CHANGE MASTER TOMASTER_HOST=master1.example.com, MASTER_PORT=3306,MASTER_USER=repl_user, MASTER_PASSWORD=xyzzy,MASTER_LOG_FILE=master-bin.00001, MASTER_LOG_POS=22145;MASTER_AUTO_POSITION=1;48 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 48. Using Global Transaction ID for Fail-over Failing over to a new masterFailing over slave to another masterSwitch over old master server is onlineFail over old master server is downGTID handshakeAutomatically done when changing master STOP SLAVE; CHANGE MASTER TOMASTER_HOST=master2.example.com, MASTER_PORT=3306; START SLAVE;49 Copyright 2012, Oracle and/or its affiliates. All rights reserved. 49. The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle.50 Copyright 2012, Oracle and/or its affiliates. All rights reserved.