NBackup and Backup:internals, usage strategy and pitfalls

Dmitry Kuzmenko,

IBSurgeon

About IBSurgeon

• Tools and consulting• Platinum Sponsor of Firebird

Foundation• Founded in 2002: 10 years of

Firebird and InterBase recoveries and consulting

• Based in Moscow, Russia

www.ib-aid.com www.ibsurgeon.com

Simple statements

•The smaller – the faster•How to make database less size?

• no_reserve ! Minus up to 30% of size• only for read-only databases• CORE-3957

• split data

Splitting data

• By tables – problems with complex SQL• Tablespaces – we do not have this• Replication• By databases

• Execute statement on external

• Partial backup• Some tables are read-only• Full backup + partial backup• Hard to restore all these together

Backups

• Speed• DB - read, backup – write• Protocol – services API is the fastest (3-4 times from tcp)• Can’t be stopped by terminating gbak

• Slower than sequential copy• -v – read 34mb/sec, write 33mb/sec, 1 core 60%• no –v - read 60mb/sec, write 56mb/sec, 1 core 100%• If DB size > RAM, can cause RAM consumption on

Windows. Use 2.5.2.

IBBackupSurgeon

Restore

• Create db, fill metadata• Transfer data• Create indices

• Backup – read, db – write• Read 56mb/sec, write 60mb/sec – equal to backup speed

• Db – read, temp – write, read, db – write• Usually 2-4 times slower than backup

Critical time

time

crushDetect what happened

Trying to fix with gfix or FBFirstAid

backup

Error on restore?

restore

What you need to remember

• Database can be corrupted hardly, no reason to fix (or will take too much time)

• Restore can fail, due to logical data corruption (FK, PK, etc)

• Application sufficient data can be broken• Corruption in most cases (99%) causes data loss

• Each action may need to be run 2-3 times

NBackup

• Introduced in Firebird 2.0 (2005, 7 years ago)• Fast as copying file

• 17gb – 4m 30sec, 67mb/sec source and target.• 22gb – 1m 30sec on SSD, 262mb/sec

• Very rarely used• Most still do not understand what is this and how it works, and how to organize 0-1-2-3… level nbackups

• Maybe, most feel comfortable to restore yesterday’s backup on crush?

NBackup sample 1

• 50 users, 10 hours per day, 68k transactions per day, accounting• hourly nbackup – 1-2 minutes, total nbackups – 40gb (year)

5700

220

120

25

0 2000 4000 6000

Size

level 3 hour

level 2 day

level 1 week

Database

NBackup sample 2

• 100 users, 12 hours per day, 224k transactions per day, stock• hourly nbackup – 1-2 minutes

10000

1000

800

300

0 5000 10000 15000

Size

level 3 hour

level 2 day

level 1 week

Database

•0 month• 1 week depend on month

• 2 day depend on week

• 3 hour• 3 hour depend on day

• 3 hour

• 2 day• 3 hour• 3 hour• …

Usual case

DB

FBK FDBrestore

backup

FBK

copy

Using nbackup

DBNBK level 0

FBK

FDB

restore

backup

Problems

• Needs a lot of disk space• Problems with Classic in 2.0, 2.1 – 2.5.1 recommended

• FBDataGuard monitors:• Delta size• Delta age

• -d ON | OFF• On – Windows default, OS cache turned off• Off – Linux default, OS cache turned off• Off on Windows and FB < 2.5.2 can cause RAM

consumption

Summary

• Backup – loosing importance due to slowness and need to restore it• Still good to “renew” DB, change page size, other

options, check logical structure

• Nbackup – fastest way to make copy of db• Page damage can’t be detected, still need to

backup/restore• Lot of space for data• Ability to “restore” more precize state of db

Thank you!

• Questions?

• www.ib-aid.com• support@ib-aid.com