Thank you to our Sponsors

Breaking Data – Terry Bunio

@Tbunio

Media Sponsor:

When Good Data Goes Bad…

Who Am I?

• Terry Bunio

• Data Base Administrator

- Oracle, SQL Server 6,6.5,7,2000,2005,2012, Informix, ADABAS

• Sharepoint fan

• Data Modeler/Architect

- Investors Group, LPL Financial, Manitoba Blue Cross, Assante

Financial, CI Funds, Mackenzie Financial

- Normalized and Dimensional

• Agilist

- Innovation Gamer, Team Member, SQL Developer, Test writer,

Sticky Sticker, Project Manager, PMO on SAP Implementation

My Blog – www.agilevoyageur.com

Breaking Data

SQL Saturday Winnipeg

• November 22nd – Red River Community College

- Downtown Campus

• First SQL Saturday ever in Winnipeg

• 3rd in Canada after Toronto and Vancouver

• 20 Sessions

• 4 Tracks

- Business Intelligence

- DBA

- Developer

- New Database Technology

March 2 – 3, 2015

Call for Speakers Open!

www.prairiedevcon.com

Question?

• What is broken data?

• How do we fix it?

Objectives

• Mine

- Hopefully introduce a couple of ideas you can take back and

improve on

• Yours?

Three types of broken data

• Inconsistent - Easy

• Incoherent - Moderate

• Ineffectual - Hard

Inconsistent

Inconsistent

• All Data must have a structure

• Domain

- “In data management and database analysis, a data

domain refers to all the values which a data element may

contain.” - Wikipedia

Inconsistent

• Domains are a simple way to ensure data consistency

• Many times this is overlooked due to tools that don’t

promote it

- Hand-rolled SQL DDL and scripts

• Use tools that require you to define Data Domains

- Erwin

- Oracle SQL Data Modeler

• FREE!

• http://www.oracle.com/technetwork/developer-

tools/datamodeler/overview/index.html

Inconsistent

Inconsistent

• You try and find inconsistencies in that model!

• Luckily I have used Oracle Data Modeler and defined the

following Domains

Incoherent

Incoherence

Incoherent

• Many databases remain coherent by using Foreign Key

Constraints

- These constraints ensure records can’t be stored in one table

unless the row they refer to in another table already exists

- These constraints are usually enabled all the time

- They can slow down performance and cause the data to be

Ineffectual

Incoherent

• Most databases create the constraints and leave them

enabled all the time

• Constraint Double-Whammy

- Slows down actual inserted/modification of data

- Further slows down code as you validate the code values before

you try to insert/update to avoid throwing database exception

Incoherent

• Alternative approach

- Leave Constraints disabled

- Attempt to re-enable them periodically to report on any invalid

data – Daily or Weekly

• You can then correct that data

- Disable constraints again

• In the past this process wasn’t possible due to the length

of time such a process would take

• only takes 75 minutes to re-enable 616 Foreign Key

constraints on a 1.1 Terabyte MSSQL 2012 database.

Thanks Microsoft!

Incoherent

• Demo

Ineffectual

• There are three types of database Performance Tuning

that you can do to make your data less ineffectual

- Execution Plan / Statistics IO

- SQL Profiler

- OStress

Execution Plan

• Demo

Execution Plan

• 1.sql

Execution Plan

• You then get an Execution Plan tab

• Execution Plan process has actually got very good at

recommending indexes.

• Anyone remember MSSQL Index Wizard?

How to read Execution Plan

• Index Seek >> Index Scan >> Table Scan

• Look for steps that are a large percentage of the overall

query

- See if those steps are using the right indexes

• Hover over each step to get details

How to read Execution Plan

• Cached plan size – how much memory the plan

generated by this query will take up in stored procedure

cache. This is a useful number when investigating cache

performance issues because you'll be able to see which

plans are taking up more memory.

• Estimated Operator Cost – Overall percentage cost of

the step

How to read Execution Plan

• Estimated Subtree Cost – tells us the accumulated

optimizer cost assigned to this step and all previous

steps, but remember to read from right to left. This

number is meaningless in the real world, but is a

mathematical evaluation used by the query optimizer to

determine the cost of the operator in question; it

represents the amount of time that the optimizer thinks

this operator will take.

• Estimated number of rows – calculated based on the

statistics available to the optimizer for the table or index

in question.

SET STATISTICS IO ON

SET STATISTICS IO ON

• DBCC FREEPROCCACHE

SQL Profiler

• Demo

SQL Profiler

SQL Profiler

SQL Profiler

SQL Profiler

SQL Profiler

• You can save traces and replay those traces to simulate

load

• There are some limitations though

- Replay RPC events as remote procedure calls

- Replay attention

- Replay DTC transactions

- Replay as part of an automated scripts === SCALABLE Tool

OStress

• Comprised of two utilties:

- Read80Trace

• Required in order to convert trace files into RML files

- OSTRESS

• Multithreaded ODBC-based query utility. The OSTRESS utility reads input

from a command-line parameter. The command-line parameter can be an

RML file that is produced by the Read80Trace utility or a standard go-

delimited .SQL script file. In stress mode, one thread is created for each

connection, and all threads run as fast as possible without synchronization

among the threads. You can use this mode to generate a specific type of

stress load on the server.

Ostress

• First we need to download and install the tools on the

server where we want to run our trace files

- http://www.microsoft.com/en-

us/search/Results.aspx?q=ostress&form=DLC

Ostress

• Demo

Convert Trace Files to RML files

• DOS Command

- CD c:\Program Files\Microsoft Corporation\RMLUtils

- ReadTrace –Ic:\TraceFiles\TraceSample.trc –oc:\RMLFiles –T28

• T28 flag

- Important as it allows you to replay the RML file against SQL

Server

Ostress

• Now you can simply run Ostress with those RML files

- OSTRESS -creplay.ini -mreplay -T88 –ic:\RMLFiles\*rml –

oc:\RMLFiles\ReplayResult

Specific use

• You can run and compare Ostress results when you

upgrade SQL Server or other system software and

hardware!

• You can compare them using the following command:

- ReadTrace –Ic:\TraceFiles\*.trc –oc:\TraceFiles\ReplayResult\ –

dods –f

• You can answer confidently whether the new server can

handle the current production load and stress

Review

• The power of this structure is that we can now automate

hundreds of threads to replay loads on the database

• This can now also become part of automated

testing/continuous integration processes

Whew…

Questions?