db2 luw logging backup - meetupfiles.meetup.com/20052936/logging y backup.pdfdb2 luw logging. basics...
TRANSCRIPT
1 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING
&
BACKUP
Ana Rivera
IBM Analytics
16 de Diciembre 2016
2 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
Safe Harbor Statement
2
Copyright © IBM Corporation 2016. All rights reserved.
U.S. Government Users Restricted Rights - Use, duplication, or disclosure restricted by GSA ADP Schedule Contract
with IBM Corporation
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES
ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON CURRENT THINKING
REGARDING TRENDS AND DIRECTIONS, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT
NOTICE. FUNCTION DESCRIBED HEREIN MY NEVER BE DELIVERED BY I BM. IBM SHALL NOT BE
RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS
PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS
INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS
FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY
AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.
IBM, the IBM logo, ibm.com and DB2 are trademarks or registered trademarks of International Business
Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms
are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols
indicate U.S. registered or common law trademarks owned by IBM at the time this information was published.
Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at “Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
3 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING AND BACKUP. Agenda
� DB2 Logging. Basics– Overview
– Circular and Archival Logging. Recoverability
– Database Configuration Parameters for Logging
� Crash Recovery
� Infinite Logging
� Monitor
� Backup. Basics.– Backup Types
� Backup Process Model
� Backup Performance.
� Monitor
4 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING
BASICS
5 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS
� Revisamos conceptos 4 Fundamentos de las bases de datos:
� Transaction (UOW): Recoverable sequence of operations within an
application process.
� ACID:– Atomicity : All changes to data are performed as if they are a single operation. All the
changes are performed, or none of them are.
– Consistency : Data is in a consistent state when a transaction starts and when it ends.
– Isolation : Concurrency Control.
– Durability: After a transaction successfully completes, changes to data persist and are
not undone, even in the event of a system failure. Once a transaction is committed, it is
not lost.
� Transaction log files are one of the core strengths of a DBMS.
6 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS
� Transaction logs written by DB2 to record updates to database
� Used during– Rollback
• Reverse changes made by a statement or transaction
– Crash recovery• Redo committed work and undo uncommitted work to make database consistent
– Rollforward recovery• Re-apply changes after a restore is performed
� DB2 uses a Write-Ahead-Logging (WAL) protocol– Log records always written to disk before affected data pages hit disk
– Transactions are logged while they occur, regardless of whether or not the
transactions commit.
– Transactions go from the log buffer to log files (transactional logging) before
any data is written from the buffer pools to the database structures. The files
used to log the transactions are called the TRANSACTION LOGS.
7 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS
� Log Records are initially written into an in-memory log buffer. They
are flushed to disk when:
– The log buffer is full (Monitor!!!!)
– A transaction commits DURABILITY
� Log Records : Log Files. Starting: S0000000.LOG
� Each Log File:– First 2 Pages: Metadata
– Remaining Pages: Log Records
– Number of 4KB in a log file: LOGFILESZ + 2
� DPF : Each database partition has its own set of logs files.
� pureScale: Each member has its own set of logs files.
� BLU MPP: Each database partition has its own set of logs files.
8 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS
� Each log record is identified by a Log Sequence Number (LSN).
� DB2 databases use the log sequence number (LSN) to determine the
order of the operations that generated the log records.
– LSN was increased from 6 bytes to 8 bytes in DB2 9.7
� All DB2 log records begin with a log manager header. – Includes the total log record size, LSN, the log record type, and transaction-
specific information.
– LOG RECORD TYPE : normal, load, backup, compensation, database
migration D
� When log records are written by transactions, extra space is reserved in case a
rollback occurs
– Compensation log records are written during rollback processing
9 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS
� LOG CONTROL FILES:
– When a database restarts after a failure, the
database manager applies transaction information
stored in log files to return the database to a
consistent state.
– To determine which records from the log files need
to be applied to the database, the database
manager uses information recorded in log control
files.
� Two copies:
– SQLOGCTL.(G)LFH1
– SQLOGCTL.(G)LFH2
10 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS.
� LOG PRIMARY AND LOG SECONDARY
– LOGPRIMARY : Number of Primary Log Files.
• Pre allocated logs files. ALWAYS EXISTS. Allocated during database
activation.
– LOGSECOND: Number of secondary Log Files.
• Allocated on demand when not enough space available in primary logs
• Can be set to -1 to enable infinite logging
(LOGPRIMARY + LOGSECOND) <= 256
11 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS. CIRCULAR LOGGING.
� Default for newly created databases.
� Only offline backups are allowed
� Rollforward not allowed
� Contents of log files are not permanent.
– Log Data is overwritten when log records are no longer required for crash
recovery purpose.
DATABASES NO RECOVERABLES
12 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS. ARCHIVE LOGGING (i)
� Enabled by setting: LOGRETAIN, USEREXIT or LOGARCHMETH1/2– LOGRETAIN/USEREXIT discontinued in version 10.1
– USE LOGARCHMETH1/2
� Allows :– Online backup
– Table space backup and restore
– Database and tablespace rollforward
– Rebuild from table space backups.
DATABASES RECOVERABLES
13 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS. ARCHIVE LOGGING (ii)
� Automating database recovery object management.
� AUTO_DEL_REC_OBJ (DB CFG) : ON :– Specifies whether database log files, backup images, and load copy images should be
deleted when their associated recovery history file entry is pruned.
• NUM_DB_BACKUPS (DB CFG) : Number of full database backups to retain
for a database
• REC_HIS_RETENT (DB CFG) : Number of days that historical information
on backups are retained.
Scenario 1. Daily backups
auto_del_rec_obj=ON
rec_his_retentn=0
num_db_backups=3
Keeps three days of history, three full
backups, and all of the incremental backups
and logs in between those backups
Scenario 2. Daily backups
auto_del_rec_obj=ON
rec_his_retentn=15
num_db_backups=3
Keeps 15 days of history, 15 full backups,
and all of the incremental backups and logs
in between those backups.
14 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS. ARCHIVE LOGGING (iii)
� LOG ARCHIVAL COMPRESSION (added in DB2 10.1)
� Log files can be compressed when they are archived.
– Only if archive method is DISK, TSM or VENDOR.
– Enabled with LOGARCHCOMPR1/LOGARCHCOMPR2 DB CFG.
15 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS. ARCHIVE LOGGING (iv)
16 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS.
DB CFG parameters for logging
NEWLOGPATH (Log Path)MIRRORLOGPATHOVERFLOWLOGPATH *LOGRETAIN (removed in DB2 10.1)USEREXIT (removed in DB2 10.1)LOGARCHMETH1/2 *LOGARCHCOMPR1/2 * (new in DB2 10.1)LOGARCHOPT1/2 *NUMARCHRETRY *ARCHRETRYDELAY *FAILARCHPATH *CUR_COMMITLOGBUFSZ **LOGPRIMARY **LOGSECOND * **LOGFILSIZ **
BLK_LOG_DSK_FUL *MAX_LOG *NUM_LOG_SPAN *MINCOMMIT * ** (ignored in DB2 10.1)BLOCKNONLOGGEDLOG_APPL_INFO (new in DB2 10.1)LOG_DDL_STMTS * (new in DB2 10.1)SOFTMAX ** (deprecated in DB2 10.5)PAGE_AGE_TRGT_MCR (new in DB2 10.5)PAGE_AGE_TRGT_GCR (new in DB2 10.5)
* Can be changed dinamically** Updated by Configuration Advisor (AUTOCONFIGURE)
17 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS.
DB CFG parameters for logging (WHERE)
• NEWLOGPATH (LOGPATH) : Database logs are initially created in the following
directory: db_path/instance_name/dbname/NODE0000/LOGSTREAM0000. Y
• ou can change the location in which active log files are placed (and future log files will be placed) by Changing the value of NEWLOGPATH you can change the location.
• LOGARCHMETH1/2 : If you specify both of these parameters, each log file from the
active log path that is set by the LOGPATH configuration parameter is archived twice
• MIRRORLOGPATH : To protect the logs on the primary log path from disk failure or
accidental deletion, you can specify that an identical set of logs be maintained on a secondary (mirror) log path.
• IF MIRRORLOGPATH and LOGARCHMETH2 are set, LOGARCHMETH2 archives log
files from the mirror log path.
• FAILARCHPATH : Alternate directory for the archive log files if there is a problem
with the normal archive path
• NUMARCHRETRY : number of attempts that will be made to archive log files using a
configured log archive method before they are archived to the path specified by the failarchpath configuration parameter.
18 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS.
DB CFG parameters for logging (SIZE)
• LOGPRIMARY • LOGSECOND
• LOGFILSIZ (4k pages) :
Size of each primary and secondary log file.
Maximum: 4 GB
Consider:
• Frequency of archiving required
• Limit of 256 logs (without infinite logging).
• If log file size is very large, it takes longer to physically
allocate it: DB2 tries to avoid it, pre-allocating and renaming
old files.
• Minimum suggested size : 20 – 50 MB
19 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS.
DB CFG parameters for logging (HOW)
• LOGBUFSZ : Amount of memory to use as a buffer for log records before writing these
records to disk.
• BLK_LOG_DSK_FUL : Prevent disk full errors from being generated when DB2 cannot
create a new log file in the active log path. If BLK_LOG_DSK_FUL is set to NO, a
transaction that receives a log disk full error will fail and be rolled back. • MAX_LOG : Percentage of primary log space that can be consumed by one transaction.
(Useful with Infinite logging)
• NUM_LOG_SPAN : Specifies whether there is a limit to how many log files one
transaction can span, and what that limit is (Useful with Infinite Logging)
• MINCOMMIT : Allows you to delay the writing of log records to disk until a minimum
number of commits have been performed (In Version 10.1 and later releases, the value
specified for this configuration parameter is ignored)• BLOCKNONLOGGED : Specifies whether the database manager will allow tables to have
the NOT LOGGED or NOT LOGGED INITIALLY attributes activated.
• LOG_APPL_INFO : specifies that the application information log record is written at the
start of each update transaction• LOG_DDL_STMTS : Extra information regarding Data Definition Language (DDL)
statements will be written to the log.• SOFTMAX / PAGE_AGE_TRGT_MCR / PAGE_AGE_TRGT_GCR . Crash Recovery
20 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING
CRASH RECOVERY
21 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. CRASH RECOVERY (i)
• Crash recovery is required when database terminated abnormally.
• If a failure occurs before all of the changes that are part of the unit of work are completed,
committed, and written to disk, the database is left in an inconsistent and unusable state.
Crash recovery is the process by which the database is moved back to a consistent and
usable state.
• Performed explicitly using RESTART DATABASE command.
• Performed implicitly during first connect if AUTORESTART= ON
• At the time of a database failure resulting from an event such as a power failure, there
might have been changes to the database which:
• Have not been committed, but updated the data in the buffer pool
• Have been committed, but have not been written from the buffer pool to the disk
• Have been committed and written from the buffer pool to the disk.
• When a database is restarted, the log files will be used to perform a crash recovery of the
database which ensures that the database is left in a consistent state :
• All committed transactions are applied to the database and all uncommitted
transactions are not applied to the database).
22 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. CRASH RECOVERY (ii)
• To determine which records from the log file need to be applied to the database, the
database manager uses information recorded in log control files.
• Log control files are periodically written to disk, and, depending on the frequency of this
event, the database manager might be applying log records of committed transactions or
applying log records that describe changes that have already been written from the buffer
pool to disk.
• These log records have no impact on the database, but applying them introduces some
additional processing time into the database restart process.
• Crash Recovery : Two Phases:
• Redo Phase : Logs are used to redo work that may not have yet been persisted to
disk.
• Undo Phase : Uncommited (in-flight) work is rolled back.
• Crash Recovery is an offline operation.
• pureScale crash recovery :
• Member crash recovery
• Group crash recovery
23 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. CRASH RECOVERY.
• Monitor crash recovery: db2 list utilities show detail
24 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. CRASH RECOVERY.
• Monitor crash recovery: db2pd –db xxxx -recovery
db2pd -db sample -recovery
Database Member 0 -- Database SAMPLE -- Active -- Up 0 days 00:00:08 -- Date 201 6-12-15-19.07.09.094499
Recovery:Recovery Status 0x40088005Current LogCurrent LSN 0000000000D4B85FCurrent LRI 00000000000000010000000000003B450000000000D4B85FCurrent LSO 1729424184Job Type CRASH RECOVERYJob ID 1Job Start Time (1481825228) Thu Dec 15 19:07:08 2016Job Description Crash RecoveryInvoker Type UserTotal Phases 2Current Phase 1
Progress:Address PhaseNum Description StartTime CompletedWork TotalWork0x0780000002F5F288 1 Forward Thu Dec 15 19:07:08 39025463 bytes 500858880 bytes0x0780000002F5F410 2 Backward NotStarted 0 bytes 500858880 bytes
25 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. CRASH RECOVERY. TUNING
• SOFTMAX (deprecated v10.5) / PAGE_AGE_TRGT_MCR / PAGE_AGE_TRGT_GCR
• Target duration (in seconds) for changed pages to be kept in the local buffer pool before
they are persisted to table space storage.
• Used when softmax = 0 . Databases migrated from old version still uses SOFTMAX.
• Increasing the value of PAGE_AGE_TRGT_MCR keeps changed pages in memory for a
longer time, allowing more page updates to be buffered before the pages are persisted to
disk.
• This behaviour can help to improve performance but also increases recovery time.
26 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
INFINITE LOGGING
27 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. INFINITE LOGGING (i)
• INFINITE LOGGING :
• No limit on the amount of log space that can be consumed by active transactions
• Enabled by setting LOGSECOND to -1 (can be set dynamically)
• Database must also be configured for archive logging.
• If infinite logging is enabled and space needs to be made available for more active log files,
the database manager archives an active log file and renames it to create a new active log
file.
• If crash recovery is needed when infinite logging is used, log files might need to be
retrieved from the archive log path to complete crash recovery.
• Useful database configuration parameters:
• MAX_LOG : Percentage of primary log space that can be
consumed by one transaction. (Useful with Infinite logging)
• NUM_LOG_SPAN : Specifies whether there is a limit to how
many log files one transaction can span, and what that limit is (Useful with Infinite Logging)
S0000000.LOG
S0000001.LOG
S0000002.LOG
S0000003.LOG
S0000004.LOG
S0001000.LOG
S0001001.LOG
S0001002.LOG
DDDD.....
28 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. INFINITE LOGGING (ii)
• INFINITE LOGGING. IMPLICATIONS OF USE
• Log files may need to be retrieved during rollback and crash recovery :
• It may take a very long time to perform crash recovery.
• OVERFLOWLOGPATH : DB2 can store active log files retrieved from the archive in this
path.
29 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING
MONITOR
30 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
• Many different things can be monitored
• Log Full situations.
• Overall database log Usage
• Log space used at transaction level
• Log read and writes
• Time spent metrics for logging
31 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
• With Data Server Manager (Real Time and History)
32 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
• LOG FULL :
• db2diag.log messages:
2016-12-14-11.12.45.689173+060 E3109A574 LEVEL: Warning
PID : 27001196 TID : 44532 PROC : db2sysc 0
INSTANCE: db2v105 NODE : 000 DB : SAMPLE
APPHDL : 0-402 APPID: *LOCAL.db2v105.161214095607
AUTHID : DB2V105 HOSTNAME: db2blu
EDUID : 44532 EDUNAME: db2agent (SAMPLE) 0
FUNCTION: DB2 UDB, data protection services, sqlpWriteToLog, probe:1660
MESSAGE : ADM1822W The active transaction log is being held by dirty pages.Database performance may be impacted.
2016-12-14-11.16.31.135604+060 E6679A623 LEVEL: ErrorPID : 27001196 TID : 44532 PROC : db2sysc 0INSTANCE: db2v105 NODE : 000 DB : SAMPLEAPPHDL : 0-402 APPID: *LOCAL.db2v105.161214095607AUTHID : DB2V105 HOSTNAME: db2bluEDUID : 44532 EDUNAME: db2agent (SAMPLE) 0FUNCTION: DB2 UDB, data protection services, sqlpgResSpace, probe:2860MESSAGE : ADM1823E The active log is full and is held by application handle
"0-402". Terminate this application by COMMIT, ROLLBACK or FORCEAPPLICATION.
33 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
• MONITOR TABLE FUNCTIONS FOR LOGGING
• MON_GET_TRANSACTION_LOG
• MON_TRANSACTION_LOG_UTILIZATION
• MON_GET_TRANSACTION_LOG (SYSIBMADM.SNAPDETAILLOG and
SYSIBMADM.SNAPDB information)• Returns information about the transaction logging subsystem for the currently connected database
• TOTAL_LOG_AVAILABLE, TOTAL_LOG_USED, SEC_LOG_ALLOCATED
• LOG_READS, LOG_WRITES, LOG_READ_TIME, LOG_WRITE_TIME
• FIRST_ACTIVE_LOG, LAST_ACTIVE_LOG
• NUM_LOG_BUFFER_FULL
• Also includes information previously only available through db2pd
• Currently committed metrics
• Archive logging status
• Current LSN
• Current log chain
• LSN of oldest active transaction
• MON_TRANSACTION_LOG_UTILIZATION (SYSIBMADM.LOG_UTILIZATION)• Returns information about the transaction logging subsystem for the currently connected database.• LOG_UTILIZATION_PERCENT
• TOTAL_LOG_USED_KB
• TOTAL_LOG_AVAILABLE_KB
• TOTAL_LOG_USED_TOP_KB
34 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
• db2pd /home/db2v11 ->db2pd -db rowdb -logs
Database Member 0 -- Database ROWDB -- Active -- Up 1 days 14:40:25 -- Date 2016-12-14-11.59.10.637774
Logs:Current Log Number 283Pages Written 208Cur Commit Disk Log Reads 0Cur Commit Total Log Reads 0Method 1 Archive Status SuccessMethod 1 Next Log to Archive 283Method 1 First Failure n/aMethod 2 Archive Status n/aMethod 2 Next Log to Archive n/aMethod 2 First Failure n/aLog Chain ID 0Current LSO 1285311864Current LSN 0x00000000007B555E
Address StartLSN StartLSO State Size Pages Filename0x0A00050C575E8AF8 00000000007AD017 1280287905 0x00000000 1024 1024 S0000282.LOG0x0A00050C53DEBAD8 00000000007B4012 1284461729 0x00000000 1024 1024 S0000283.LOG0x0A00050C4EA1EFF8 0000000000000000 1288635553 0x00000000 1024 1024 S0000284.LOG0x0A00050C575DF3D8 0000000000000000 1292809377 0x00000000 1024 1024 S0000285.LOG0x0A00050C53DF17F8 0000000000000000 1296983201 0x00000000 1024 1024 S0000286.LOG0x0A00050C53DF3738 0000000000000000 1301157025 0x00000000 1024 1024 S0000287.LOG0x0A00050C53D24F58 0000000000000000 1305330849 0x00000000 1024 1024 S0000288.LOG0x0A00050C4E91D898 0000000000000000 1309504673 0x00000000 1024 1024 S0000289.LOG0x0A00050C4EA1DA78 0000000000000000 1313678497 0x00000000 1024 1024 S0000290.LOG0x0A00050C53D258B8 0000000000000000 1317852321 0x00000000 1024 1024 S0000291.LOG0x0A00050C575EB0D8 0000000000000000 1322026145 0x00000000 1024 1024 S0000292.LOG0x0A00050C53D579D8 0000000000000000 1326199969 0x00000000 1024 1024 S0000293.LOG0x0A00050C575E0AD8 0000000000000000 1330373793 0x00000000 1024 1024 S0000294.LOG
35 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
36 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
EXAMPLES.
37 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
EXAMPLES.
38 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
EXAMPLES.
39 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. MONITORING LOGGING ACTIVITY
EXAMPLES.
40 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING AND BACKUP. Agenda
� DB2 Logging. Basics– Overview
– Circular and Archival Logging. Recoverability
– Database Configuration Parameters for Logging
� Crash Recovery
� Infinite Logging
� Monitor
� Backup. Overview.– Backup Types
� Backup Process Model
� Backup Performance.
� Monitor
41 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 BACKUP
Overview
42 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. OVERVIEW
• Use the BACKUP DATABASE command to take a copy of the
database data and store it on a different medium.
• This backup data can then be used in the case of a failure or damage
to the original data.
• BACKUP COMMAND
• Offline / Online
• Database / Tablespaces
• Full / Incremental/Delta
43 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
BACKUP. DATABASE AVAILABILITY
44 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. ONLINE BACKUPS
• INCLUDE LOGS : Default option for online backups (except no single view backup on DPF)
• RESTORE A BACKUP WITH INCLUDE LOGS:
• Use LOGTARGET option of the RESTORE DATABASE command and specify a fully qualified
path that exists on the DB2 server.
• The restore database utility then writes the log files from the image to the target path. If a log
file with the same name exists in the target path, the restore operation fails and an error is
returned.
45 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. ONLINE BACKUPS
• BACKUP
BACKUP DB <dbname> ONLINE to <backup_path> INCLUDE LOGS
db2 backup db sample online
Backup successful. The timestamp for this backup image is : 20161215195927
• RESTORE AND ROLLFORWARD
RESTORE DB <dbname> ONLINE FROM <backup_path> LOGTARGET <log_path>ROLLFORWARD DB <dbname> TO END OF LOGS AND STOPOVERFLOW LOG PATH <log_path>
// Or, to restore just the LOGSRESTORE DB <dbname> LOGS FROM <backup_path> LOGTARGET <log_target>
EXAMPLE:
46 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. ONLINE BACKUPS
db2 restore db sample into sample2 logtarget defaultDB20000I The RESTORE DATABASE command completed successfully.
db2 rollforward db sample2 query status
Rollforward Status
Input database alias = sample2Number of members have returned status = 1
Member ID = 0Rollforward status = DB pendingNext log file to be read = S0001277.LOGLog files processed = -Last committed transaction = 2016-12-15-19.24.12.000000 UTC
/home/db2v11/db2v11/NODE0000/SQL00003/LOGSTREAM0000 ->ls -ltrtotal 24-rw------- 1 db2v11 db2iadm2 12288 Dec 15 20:29 S0001277.LOG
47 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. ONLINE BACKUPS
db2 rollforward db sample2 to end of logs and complete
Rollforward Status
Input database alias = sample2Number of members have returned status = 1
Member ID = 0Rollforward status = not pendingNext log file to be read =Log files processed = -Last committed transaction = 2016-12-15-19.24.12.000000 UTC
DB20000I The ROLLFORWARD command completed successfully.
db2 connect to sample2
Database Connection Information
Database server = DB2/AIX64 11.1.0SQL authorization ID = DB2V11Local database alias = SAMPLE2
48 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. ONLINE BACKUPS
db2blu (db2v11) /datossd/prueba/backup ->db2 list utilities show detail
ID = 8567Type = RESTOREDatabase Name = SAMPLE2Member Number = 0Description = dbStart Time = 12/15/2016 20:01:27.523907State = ExecutingInvocation Type = UserProgress Monitoring:
Completed Work = 342695936 bytesStart Time = 12/15/2016 20:01:27.523913
49 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
BACKUP TYPES.
FULL/INCREMENTAL/DELTA
• INCREMENTAL/DELTA : TRACKMOD DATABASE CONFIGURATION PARAMETER : ON
50 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
50
� Full backup
� Incremental backups (what has changed since the last full backup?)
� Delta backups (what has changed since the last backup of any kind?)
BACKUP TYPES.
FULL/INCREMENTAL/DELTA
51 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
51
Recovery Using Database Log files
� Requires all archived database log files
Full backup Database logs
Recovery
Time
52 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
52
Recovery Using Delta Backups
� Requires full backup, all delta backups, and database log files
Full backup Delta1 Delta2 Delta3 Database logs
Recovery
Time
53 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
53
Recovery Using Incremental Backups
� Requires full backup, latest incremental backup, and database log
files
Full backup Incr1 Incr2 Incr3 Database logs
Recovery
Time
54 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
54
Recovery Using Incremental Backups
55 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP.
56 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. DATABASE/TABLESPACES
• Backup tablespace level: Available only for databases configured with archive logging
• New option in db2 v11.1 : backup database no tablespace
57 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. BACKUP PROCESS MODEL
db2agent EDU starts and :• Calculate and create an optimal number of db2bm and db2med EDUs (Based on #cpus, #session,
#files)
• Allocate an optimal number of optimally sized backup buffers• Display the selected tunable values to the db2diag.log
• Allocate message queues for inter-EDU communication
• Create a list of table spaces that are sorted by decreasing size• Coordinate the backup process by sending control messages to the db2bm and db2med EDUs
• The db2bm EDU acquires a backup buffer from the empty queue, and then fills the buffer by
reading
58 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. PERFORMANCE (i)
• Balanced table spaces
• Data distributed between table spaces.
• During the backup, DB2 assigns one db2bm EDU to back up each table space. If
most of the data in a database is in one table space, the backup is serialized while
only
• the one table space is being backed up.
• Maximun parallelism : Having data equally distributed between multiple
tablespaces
• Difference between SMS and DMS tablespaces.
• SMS table space, every page in the table is included in the backup image.
• DMS table space, only extents that are marked as used at the beginning of the
backup are included in the backup image.
59 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. PERFORMANCE (ii)
• Buffer Size and number of Buffers.
• Minimum buffer size is 8 (4 KB) pages
• Maximum buffer size is 16384 (4 KB) pages
• TIP : 90% UTIL_HEAP_SZ : OFFLINE BACKUP / 50% UTIL_HEAP_SZ: ONLINE
BACKUP
• Number of buffers: Depending on number of db2bm and db2med
DB2DIAG.LOG INFORMATION :
MESSAGE : Performance statisticsDATA #1 : String, 1080 bytes
Parallelism = 5Number of buffers = 5Buffer size = 2363392 (577 4kB pages)
60 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. MONITOR PERFORMANCE
• REGISTRY VARIABLE (db2 v10.1) : DB2_BAR_STATS
• Information on db2diag.log :
• BM. The db2bm EDU ID.
• Total. Length of time that each EDU existed.
• I/O. Time that was spent performing read or write I/O.
• MsgQ. Time that was spent waiting for an I/O buffer.
• WaitQ. Time that was spent waiting for a state machine control message.
• Buffers. Number of I/O buffers that were processed.
• Kbytes. Quantity of data that was processed.
• MC. The db2med EDU ID
• Compr. Time that was spent performing the compression operation.
• Compr Bytes. Quantity of uncompressed data that was compressed.
61 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. MONITOR PERFORMANCE
• EXAMPLE : 28.28 *100 / 53.58 = 52,8 % COMPRESSION TIME!
62 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. MONITOR PERFORMANCE
• EXAMPLE : WAITQ
63 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. DATABASE RECOVERY FILES
The recovery history file contains a summary of the backup information that can be used
to determine recovery options.
It is used to track recovery-related events such as backup and restore operations, among
others.
The recovery history file file is located in the database directory.
The table space change history file, contains information that can be used to determine
which log files are required for the recovery of a particular table space.
64 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW BACKUP. DATABASE RECOVERY FILES
db2 list history backup all for db sample
----------------------------------------------------------------------------
Comment: DB2 BACKUP SAMPLE ONLINE
Start Time: 20161215193108
End Time: 20161215193112
Status: A
---------------------------------------------------------------------------
-
EID: 1678 Location: /datossd/prueba/backup
Op Obj Timestamp+Sequence Type Dev Earliest Log Current Log Backup ID
-- --- ------------------ ---- --- ------------ ------------ --------------
B D 20161215195927001 N D S0001276.LOG S0001276.LOG
---------------------------------------------------------------------------
-
Contains 5 tablespace(s):
00001 SYSCATSPACE
00002 USERSPACE1
00003 IBMDB2SAMPLEREL
00004 IBMDB2SAMPLEXML
00005 SYSTOOLSPACE
---------------------------------------------------------------------------
65 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING AND BACKUP. Agenda
� DB2 Logging. Basics– Overview
– Circular and Archival Logging. Recoverability
– Database Configuration Parameters for Logging
� Crash Recovery
� Infinite Logging
� Monitor
� Backup. Overview.– Backup Types
� Backup Process Model
� Backup Performance.
� Monitor
66 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
DB2 LUW LOGGING. BASICS. LOGGING EDUs
EDU NAME DESCRIPTION
db2loggr Log reader; also responsible for other things (soft checkpoints, reclaiming log
space, suspending logging, etc)
db2loggw Log writer
db2logts Tracks what table spaces have log records in what log files (so unnecessary log files
can be skipped during table space rollforward)
db2lfr Log file reader for processing individual log files; used as anLFR/Shredder pair
db2shred Shredder; extracts log record from log pages received from the LFR EDU (one per
scan)
db2redom Processes redo log records during recovery and assigns them to redo workers for
processing
db2redow Processes redo log records during recovery at the request of the redo master;
multiple redo workers can exist
db2logmgr Log manager. Manages log files (archives/retrieves) for a recoverable database.
Also does log file preallocation for LOGRETAIN=ON databases
67 © 2016 IBM Corporation
Meetup DB2 LUW - Madrid
GRACIAS
Ana Rivera
IBM Analytics
16 de Diciembre 2016