CHANGE DATA CAPTURE& CHANGE TRACKING

DEEP DIVE

W. Kevin Hazzard

LinchpinPeople.com

Group Principal

CHANGE DATA CAPTURE

• Turn it onEXECUTE sys.sp_cdc_enable_db;

• Which generates for the whole database

• A capture job

• A cleanup job

• Then include at least one table withEXECUTE sys.sp_cdc_enable_table …;

• Which creates a capture instance containing

• A change table

• An all-changes function

• A net-changes function (optional)

transactions

warehouse

sources

changes

capture job

archivesector

ETL

analysis

datamart

ETL

CAPTURE INSTANCE

• Contains• The capture table

• cdc.fn_cdc_get_all_changes_<instance name> function

• cdc.fn_cdc_get_net_changes_<instance name> function (optional)

• Maximum two per source table

SYS.SP_CDC_ENABLE_TABLE

• @source_schema

• @source_name

• @supports_net_changes

• @role_name

• @filegroup_name

• @index_name

• @captured_column_list

• @allow_partition_switch

• @capture_instance

DEMONSTRATION

Enabling Change Data Capture (CDC)

Creating a CDC Capture Instance

Querying CDC Tables as Things Change

SQL SERVER AGENT JOBS

• cdc.Capture_capture• Starts with SQL Agent

• RAISERROR(22801, 10, -1);

• EXEC sys.sp_MScdc_capture_job;

• Just a wrapper for sys.sp_cdc_scan

• cdc.Capture_cleanup• Runs at 02:00 daily

• EXEC sys.sp_MScdc_cleanup_job;

CDC DATA RETENTION

• Based on LSN Validity Intervals

• Database

• Capture Instance

• The cleanup job deletes CDC data acording to retention policySELECT * FROM msdb.dbo.cdc_jobs WHERE job_type = N'cleanup';

• You can start the cleanup job manuallyEXEC sys.sp_cdc_start_job @job_type = N'cleanup';

HANDLING SCHEMA CHANGES

• Deleted and new columns are handled well

• Modified columns require specific steps:

• Stop the capture and cleanup jobs

• Change the schema as necessary

• Generate new capture instances for all modified tables

• Process all data in the old and new capture instances

• Manually run the cleanup job

• Delete the old capture instance

• Turn the capture and cleanup jobs back on

CHANGE TRACKING

• Turn it on for one databaseALTER DATABASE <DBNAME> SET CHANGE_TRACKING = ON;

• Then enable a tableALTER TABLE <TBLNAME> ENABLE CHANGE_TRACKING;

transactionswarehouse

sources archivesector

ETL

sync

CHANGE TRACKING TABLES & FUNCTIONS

• sys.change_tracking_databases

• sys.change_tracking_tables

• CHANGETABLE(CHANGES)

• CHANGETABLE(VERSION)

• CHANGE_TRACKING_CURRENT_VERSION()

• CHANGE_TRACKING_MIN_VALID_VERSION()

• CHANGE_TRACKING_IS_COLUMN_IN_MASK()

• WITH CHANGE_TRACKING_CONTEXT

DEMONSTRATION

Enabling Change Tracking (CT) for a Database

Enabling CT on a Table

Querying CT Tables and Functions as Things Change

HANDLING SCHEMA CHANGES

• No modifications to the primary key are allowed including related indexes

• Dropping columns is OK but they may still appear in the change data

• When adding columns, changes are tracked but the metadata change is not reported

• Switching partitions will fail on change tracked changes

• Data type changes are not tracked

WHICH ONE IS RIGHT FOR ME?

Change Data Capture

• Works in Enterprise Edition only

• Stores every discrete change

• Storage intensive

• Good for auditing

• Requires SQL Server agent

• No special serialization required

Change Tracking

• Works in all versions of SQL Server

• Returns differences from current

• Storage light

• Good for device synchronization

• No job agent required

• Operates best with snapshot isolation