backup & restore - percona...2014/06/18 · logical backups mydumper faster logical backups as...
TRANSCRIPT
![Page 1: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/1.jpg)
Backup & RestoreMaximiliano Bubenick
Sr Remote DBA2014-06-18
![Page 2: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/2.jpg)
www.percona.com
Agenda
● Why backups?● Backup Types● Binary or Raw Backups● Logical Backups● Binlog mirroring● Backups Locks● Examples● Tips
![Page 3: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/3.jpg)
Why Backups?
![Page 4: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/4.jpg)
www.percona.com
Why Backups?
● At some point anything can and will fail● Hardware issues● Application bugs● Operational mistakes● Attacks
![Page 5: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/5.jpg)
www.percona.com
Why Backups?
● Depending on the fail you will need to:● Restore a complete server● Restore a complete DB● A complete table● A few rows
![Page 6: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/6.jpg)
Backup types
![Page 7: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/7.jpg)
www.percona.com
Different Backup types
● Logical● Plain text files● Can be Remote● Slower on large datasets● Sometimes the only option
![Page 8: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/8.jpg)
www.percona.com
Different Backup types
● Binary or Raw● Filesystem level access● Local● Faster on large datasets● Storage engine dependent● Can be useless on filesystem corruption
![Page 9: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/9.jpg)
www.percona.com
Different Backup types
● Binlog● For point in time recovery● PITR even for a single schema
![Page 10: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/10.jpg)
Binary Backups
![Page 11: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/11.jpg)
www.percona.com
Binary Backups
● Tools● Percona XtraBackup● mylvmsnapshot● MySQL Enterprise Backup
![Page 12: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/12.jpg)
www.percona.com
Binary Backups
● Percona XtraBackup● Fast, limited by IO● Almost no locks (--rsync)● Compress on the fly (--compress)● Best option for full server restore or setting up new
slaves
![Page 13: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/13.jpg)
Logical Backups
![Page 14: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/14.jpg)
www.percona.com
Logical Backups
● Tools● mysqldump● Mydumper
![Page 15: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/15.jpg)
www.percona.com
Logical Backups
● mysqldump● General use● Simple to use● By default lock tables● One big output file● Can be piped to mysql client
![Page 16: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/16.jpg)
www.percona.com
Logical Backups
● mysqldump● --opt = --add-drop-table --add-locks --create-options
--quick --extended-insert --lock-tables --set-charset –disable-keys (default)
● --single-transaction (innodb)● --master-data (FTWRL) for binlog coordinates● --dump-slave ● --innodb-optimize-keys (PS only feature)● --triggers (ON) --routines (OFF)
![Page 17: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/17.jpg)
www.percona.com
Logical Backups
● Mydumper● Faster logical backups as is multithread● Almost no locking with innodb tables● Compress on the fly● Doesn't handle Views, Triggers and Procedures ● Separate files per table, one row per line
![Page 18: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/18.jpg)
www.percona.com
Logical Backups
● Mydumper● --threads● --outputdir● --rows, --chunk-filesize● --compress● --less-locking● --kill-long-queries● --use-savepoints
![Page 19: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/19.jpg)
www.percona.com
Logical Backups
● Without chunks
Thread 1 Thread 2 Thread 3 Thread 40
10
20
30
40
50
60
Table D
Table C
Table B
Table A
Tim
e
![Page 20: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/20.jpg)
www.percona.com
Logical Backups
● With chunks (--rows)
Thread 1 Thread 2 Thread 3 Thread 40
5
10
15
20
25
Table D
Table C
Table B
Table A
Tim
e
![Page 21: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/21.jpg)
www.percona.com
Logical Backups
● Less-locking, without it Mydumper needs to keep FTWRL until all non-Innodb tables are dumped
Thread 1 Thread 2 Thread 3 Thread 40
2
4
6
8
10
12
14
Innodb
non-InnoDBTim
e
![Page 22: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/22.jpg)
www.percona.com
Logical Backups
● Mydumper output files ● metadata
Started dump at: 2014-04-18 22:01:30
SHOW MASTER STATUS:
Log: mysql-bin.017436
Pos: 890402821
SHOW SLAVE STATUS:
Host: 192.168.56.101
Log: mysql-bin.017057
Pos: 968001054
Finished dump at: 2014-04-19 03:10:05
![Page 23: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/23.jpg)
www.percona.com
Logical Backups
● Mydumper output files ● database.table.sql
/*!40101 SET NAMES binary*/;
/*!40014 SET FOREIGN_KEY_CHECKS=0*/;
/*!40103 SET TIME_ZONE='+00:00' */;
INSERT INTO `t1` VALUES
(1,"abc"),
(2,"def"),
(4,"abc"),
(5,"abc"),
(6,"abc"),
(7,"abc"),
(8,"abc");
![Page 24: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/24.jpg)
Binlog Mirroring
![Page 25: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/25.jpg)
www.percona.com
Binlog Mirroring
● Mysqlbinlog (5.6)● Works with 5.1 and 5.5 master also● Mirror binlogs on the master on a second server● mysqlbinlog --read-from-remote-server --raw --stop-
never
![Page 26: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/26.jpg)
Backup Locks
![Page 27: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/27.jpg)
www.percona.com
Backup Locks
● LOCK TABLE ... READ LOCAL● Other sessions can read the table but can't write
● LOCAL allows non conflicting INSERTs to MyISAM
● FLUSH TABLES WITH READ LOCK● Global read lock
● Big issue on busy servers and long running selects
● Requires tables be reopened which can be another bottleneck on busy servers
● metadata locking since 5.5● No DDLs to tables used in a transaction
![Page 28: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/28.jpg)
www.percona.com
Backup Locks
● mysqldump ● --lock-tables (default)
– LOCK TABLES READ LOCAL● --lock-all-tables –master-data and –dump-slave
– FLUSH TABLES WITH READ LOCK● --single-transaction (innodb only)
– START TRANSACTION /*!40100 WITH CONSISTENT SNAPSHOT */
● --skip-lock-tables– No locks
![Page 29: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/29.jpg)
www.percona.com
Backup Locks
● Mydumper ● Always needs FTWRL to coordinate threads but for
a small time if all tables are innodb● --less-locking
– LOCK TABLES READ LOCAL for non-innodb– CONSISTENT SNAPSHOT for innodb
● --use-savepoints– Reduce metadata locking issues
● --no-locks– Not ensure a consistent backup
![Page 30: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/30.jpg)
www.percona.com
Backup Locks
● Percona XtraBackup ● FTWRL at the end to copy non transactional files
and get binlog coordinates.● --rsync
– Use rsync twice instead of cp each file, FTWRL for only the second rsync, didn't work with stream
● --no-locks– Not ensure a consistent backup
![Page 31: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/31.jpg)
www.percona.com
Backup Locks
● Percona Server 5.6 new locks for backups● LOCK TABLES FOR BACKUP
● Block updates to non-transactional tables● Block DDL to all tables
● LOCK BINLOG FOR BACKUP● blocks all updates to binary log
● UNLOCK BINLOG
![Page 32: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/32.jpg)
www.percona.com
Backup Locks
● Percona Server 5.6 new locks for backups● Percona XtraBackup 2.2● Mysqldump --lock-for-backup –single-transaction● Mydumper 0.6.2
![Page 33: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/33.jpg)
Examples
![Page 34: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/34.jpg)
www.percona.com
Using Percona XtraBackup
● Simple Backup:innobackupex --rsync --slave-info /backups/
● Prepareinnobackupex --apply-log --use-memory=2G --defaults-file=/etc/my.cnf /backups/latest/
● Restoreinnobackupex --copy-back /backups/latest/
![Page 35: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/35.jpg)
www.percona.com
Using Percona XtraBackup
● Stream Backup:innobackupex –stream=tar ./ | gzip - > xb.tar
● Unpacktar xvif xb.tar.gz
● Can be piped to remote host:
● On destination:
cd /destination/dirnc -l 873 | tar xvif -
● On source:
innobackupex –stream=tar ./ | gzip - | nc <dest_ip> 873
![Page 36: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/36.jpg)
www.percona.com
Using Percona XtraBackup
● Stream Backup:innobackupex --stream=xbstream --compress --parallel=4 ./ > xb.xbs
● Unpackxbstream -x < xb.xbs -C /destination/dir
● Can be piped to remote host:
● On destination:
nc -l 873 | xbstream -x -C /destination/dir
● On source:
innobackupex --stream=xbstream ./ | nc <dest_ip> 873
![Page 37: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/37.jpg)
www.percona.com
Using Mydumper
● Full Backup:mydumper -h x.x.x.x -–less-locking –-compress -o /backup/dir
● Full Restore:myloader -h x.x.x.x -d /backup/dir -o
![Page 38: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/38.jpg)
www.percona.com
Using Mydumper
● Partial Restore:cd /backup/dir
mkdir restore
cp -l db.* ./restore/
cp -l metadata ./restore/
myloader -h x.x.x.x -d /backup/dir/restore/ -o
![Page 39: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/39.jpg)
www.percona.com
Using mysqlbinlog
● Copy binlogs:mysqlbinlog --stop-never --raw --read-from-remote-server --host=127.0.0.1 mysql-bin.000001
![Page 40: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/40.jpg)
Tips
![Page 41: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/41.jpg)
www.percona.com
Tips
● Use the three types of backups● Binary for full restores, new slaves● Logical for partial restores● Binlog for point in time recovery
● Store on more than one server and off-site● Test your backups!!!!● Document restore procedures, script them and
test them!!!
![Page 42: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/42.jpg)
www.percona.com
Tips (cont)
● If taking from a slave run pt-table-checksum● Configuration files, scripts● Do you need to backup everything all days?● Hardlinking backups can save lot of disk space
in some circumstances ● Monitor your Backups
![Page 43: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/43.jpg)
Percona Backup Service
![Page 44: Backup & Restore - Percona...2014/06/18 · Logical Backups Mydumper Faster logical backups as is multithread Almost no locking with innodb tables Compress on the fly Doesn't handle](https://reader033.vdocuments.mx/reader033/viewer/2022050202/5f55efe51b8a1f254d253f4a/html5/thumbnails/44.jpg)
www.percona.com
Percona Backup Service
● Managed MySQL backup and recovery combining our best practices and software tools with your backup servers
● Percona ensures your backups run smoothly and reliably● Backup data sets are always available● Percona manages production recovery● Lower TCO than managing in-house● Any combination of MySQL server and backup locations● Flexible backup configurations per customer● More information at www.percona.com