Platinum Sponsors

Silver Sponsors

Gold Sponsors

DATABASE UNIT TESTING MADE EASY WITH VISUAL STUDIO

Sanil Mhatre

Database Developer, Scottrade

About Me

Database Developer –Scottrade (@scottradejobs)

MCTS – MS SQL Server 2008 Database

Development

http://SQLWithSanil.com

@SQLSuperGuru

MSDN Forums & SQLServerCentral.com

SQL Saturday #154 | St.Louis | Sept 15

Pre-requisites

MS Visual Studio Team system framework & tools

MS Visual Studio Team System (VSTS) 2010 & 2012

Ultimate & Premium Editions – create, modify & run

database unit tests

Professional Edition – run database unit tests but cannot

create or modify them

MS Visual Studio Team System 2008 Database

Edition

Agenda

Software Testing – Basics

Unit Testing – Basics

Database Unit Testing – Principles,

Terminology & Levels

Demo – Implement Database Unit Testing

with VSTS 2010 & MS SQL Server 2008 R2

New features in VSTS 2012

Software Testing

Meet Requirements & Expectations Deterministic & Predictable outcome

Types of Software Testing

Criteria Black Box Testing White Box Testing

Definition

Method in which the internal design/ implementation of the item being tested is NOT known to the tester

Method in which the internal design/ implementation of the item being tested is known to the tester.

Levels Applicable To

Mainly applicable to higher levels of testing:Acceptance TestingSystem Testing

Mainly applicable to lower levels of testing:Unit TestingIntegration Testing

ResponsibilityGenerally, independent Software Testers

Generally, Software Developers

Programming Knowledge Not Required Required

Implementation Knowledge Not Required Required

Basis for Test CasesRequirement Specifications Detail Design

Levels of Software Testing

Unit Testing

Integration Testing

System Testing

System Integration Testing

What is Unit Testing ?

Validation & Verification Methodology

Belongs to the White Box testing category

Test individual units of Source Code

Unit – smallest testable part of application

Method, Function, Class

Tests created before or during development

Tests conducted during development

Goals of Database Unit Testing

Ensure quality & accuracy of database

deliverables

Establish a baseline state of your

database

Verify Subsequent changes made to

database objects

Identify regression issues throughout the

application lifecycle

Database Unit Testing Terminology

• Isolated Environment (database)

• Local development, testing & validation

• Created by Script, VSTS Project or Restoring backup

SandBox

• Exercises a targeted code path

• Multiple assertions

TestCase

Database Unit Testing Terminology• Simple Condition to

indicate success or failure

• Returns a True or False

• Verifies Expected Interface

• Verifies extraneous Results

• Verifies data modification behavior

Assertion

Database Unit Testing Terminology

• Collection of all test cases for a given stored procedure

• Tests interface, code & behavior

Unit Test

• TSQL Scripts that executes a test case

• May include assertions like;• Return values• Output parameters• Expected Errors

Test Script

Database Unit Testing Terminology

• Logical & named grouping of related unit tests

• Multiple test lists can be created

• Can be run as a unit• A test suite• Builds & Check in

Policy• Deprecated in VSTS

2012

Test List

Database Unit Testing Terminology

• TSQL Scripts• Prepare database for unit

testing• Returns database back to

pre test state after testing• Framework also provides

common Initialize & Cleanup Scripts

• Pre Test and Post Test Scripts

• Data Generation Plans

Setup &

Cleanup

Scripts

Database Unit Testing Principles

Isolation• Outcome of one test must not affect the

outcome of any other test(s)• Each test must return the database back to its

previous state after execution

Independence• One test case must not depend on any other

test case• Each test case must be run independently of

any other test case• Test case assertion must be independent of

the order of it’s execution

Database Unit Testing Principles

Characteristics of Ideal Unit Test• Exercise all possible code paths• Evaluate all possible conditions• Validate correct behavior for each

path/conditionConsiderable efforts for exhaustive testing• Non Trivial stored procedures• Number of input parameters• Permutations of optional parameters• Error code paths

Testable Interfaces of a Stored Procedure

• Name• Data Type (Length)• Optionality

Input & Output

Parameters

• Success/failureReturn Code

• SET NOCOUNT ON/OFFRow Count

Testable Interfaces of a Stored Procedure

• One or more SELECT statements• Number of Columns• Column Ordinal position• Column Name & Data Type

Result Set

• PRINT Messages• RaiseError – Severity 10 & below

Informational

Messages

• RaiseError – Severity 10 & above

Error Messages

Unit Test Conditions

Predefined Test conditions in VSTS 2010 Data Checksum Empty ResultSet/Non Empty Result Set Execution Time Expected Schema Row Count Scalar Value Inconclusive

Levels of Unit Testing

Factors Affecting the choice of Appropriate Level of Unit testing

Costs of Unit test development & maintenance

Added Value of Quality

Application Criticality & Frequency of use

Project Deadlines

Levels of Unit Testing• Smallest amount of testing

• Ensures error free compiles

• No code is exercised• Consists of single test case with SET FMTONLY ON

• Executes with NULL inputs

• Intrinsic Assertion – No Errors

Minimal

Levels of Unit Testing

• Exercises commonly used code paths & conditions

• One step above Minimal • Tests Empty/Non-Empty

Result sets• Some assertion of

typical behavior & expected interface

• Not a thorough & complete test

Marginal

Levels of Unit Testing• Exercises all code paths &

conditions• Test cases for all

permutations of optional parameters

• Multiple Assertions to completely verify interface & behavior

• Could also verify column names of result set

• It’s an exhaustive unit test to verify all but unexpected errors

Complete

Levels of Unit Testing

Possible to develop and un-testable stored

procedure ?

A search stored procedure with 8 optional

parameters will need over 2000 test cases

Numerous found/not found conditions

Error validation

Possibly an indicator of flawed design

Consider refactoring to facilitate testability

Demo

Prerequisite : Database Project Exists in

VSTS

Create a Unit Test Project in VSTS

Configure SandBox

Create Tests

Run Tests

New in VSTS 2012

Redesigned Unit Test Explorer

Support for 3rd party testing frameworks

Code Analysis improvements

Continuous test runner

Deprecated Features

Test lists

Generate Unit Test Wizards

Conclusion

Software Testing - Basics

Unit Testing – Basics

Database Unit Testing – Detailed

discussion on Principles, Goals & Levels

Demo – Implementation of Database Unit

Testing with VSTS 2010 & SQL Server 2008

R2

Resources

MSDN

http://

msdn.microsoft.com/en-us/library/aa833283.aspx

http://

msdn.microsoft.com/en-us/library/aa833423.aspx

Wikipedia

http://en.wikipedia.org/wiki/Software_testing

Image -

http://i.swebee.com/content/error-404-road.jpg

THANK YOU !

Questions & Feedback