plam 2015 - evolving backups strategy, devploying pyxbackup
TRANSCRIPT
Evolving Backups Strategy,Devploying pyxbackupJervin Real
Percona Live 2015 - Amsterdam
Managing Backups
Percona ♥ Backups
Why?Why not?
Why?
Everybody ♥ Backups
s9s_backup
Everybody ♥ Backups
s9s_backup
holland backup manager
Everybody ♥ Backups
xtrabackup manager
s9s_backup
holland backup manager
Everybody ♥ Backups
xbackup.shxtrabackup manager
s9s_backup
holland backup manager
Everybody ♥ Backups
xbackup.shxtrabackup manager
s9s_backup
holland backup manager
fromdual_brman
Everybody ♥ Backups
xbackup.shxtrabackup manager
s9s_backup
holland backup manager
fromdual_brman twindb
Everybody ♥ Backups
xbackup.shxtrabackup manager
s9s_backup
holland backup manager
fromdual_brman twindb
… and more
Where things fall short
● dependencies● portability● for non-dbas● babysitting● validation● remote streaming● restore help● encryption● expanded retention sets● full + incremental (+ PS bitmaps)
So we began with xbackup.sh
● bash shell● had most of the initial required features● difficult to maintain and debug● rewritten to pyxbackup
pyxbackuphttps://github.com/dotmanila/pyxbackup
Simple backups
xtrabackup
pyxbackup full|incr
Saving disk space
xtrabackup
compress
pyxbackup --compress --compress-with=gzip full|incr
Validation
xtrabackup
apply-log
compress
pyxbackup --apply-log full|incr
Optional encryption
xtrabackup
apply-log
compress
encrypt
pyxbackup --compress --compress-with=gzip full|incr
Sending copies to remote
xtrabackup
apply-log
compress
encrypt
stream (scp/nc)
pyxbackup --remote-host --remote-push-only full|incr
Validation from remote
xtrabackup
compress
encrypt
stream (scp/nc) apply-log
pyxbackup apply-last
Validation can be run independently
apply-last apply-last
pyxbackup apply-last
Restoring a backup (full/incremental)
restore-set restore-set
pyxbackup restore-set
Binary log streaming to support PITR
binary log stream pyxbackup
pyxbackup binlog-stream
Binlog streaming from master
binary log stream pyxbackup
stream (nc/scp)S
M
pyxbackup --binlog-from-master binlog-stream
Limitations
● python <= 2.7
Limitations
● python <= 2.7● xtrabackup backup only (can be extended for logical dumps, per
database/table restore)
Limitations
● python <= 2.7● xtrabackup backup only (can be extended for logical dumps, per
database/table restore)● local or remote Linux servers only (SSH + Python + xtrabackup)
Future
● xrabackup 2.3
Future
● xrabackup 2.3● python 3.x
Future
● xrabackup 2.3● python 3.x● multiple storage locations (round-robin)
Future
● xrabackup 2.3● python 3.x● multiple storage locations (round-robin)● point in time recovery (some)
Future
● xrabackup 2.3● python 3.x● multiple storage locations (round-robin)● point in time recovery (some)● restore/test database/table
Future
● xrabackup 2.3● python 3.x● multiple storage locations (round-robin)● point in time recovery (some)● restore/test database/table● logical dumps
Future
● xrabackup 2.3● python 3.x● multiple storage locations (round-robin)● point in time recovery (some)● restore/test database/table● logical dumps● unit tests (ahem)
Questions?Yes, #jervinisreal
xtrabackup
apply-log
compress
encrypt
stream (scp/nc)