BIOPRESENTATION

International Conference OnSoftware Testing Analysis & Review

November 15-19, 2004Anaheim, CA USA

T14

November 18, 2004 1:30 PM

AUTOMATED DATABASE

TESTING WITH NUNIT

Alan CorwinProcess Builder, Inc.

Alan Corwin Alan Corwin has been a software developer, trainer, and course designer for nearly thirty years. He is president and founder of Process Builder, CIO of OptionBots, and teaches High Performance Data Integration in the University of Washington’s Advanced Web Development program.

1

Automated Database Testing with NUnit

Planned Test-Driven Development

2

Planned?Isn’t test-driven development planned?Yes. Although planning is often decried as a waste of time by the XP community, simply writing your tests before you write your code is a form of planning. It’s the next level of planning that few XPersare willing to take – planning to do a series of standard tests for an entire application.

3

Test GoalsTest the Generic Database RequirementsProvide a Framework for More Complex TestsSupport Rapid Development (Daily Deliverables!)Support Test-Driven Development Allow Tests to be run both by developers and testers.Test engineers and developers should be able to quickly and easily add test cases as needed.Eliminate unnecessary communication cycles.

4

Test Design FeaturesTests are fully automated.Tests are black box (ignorant of the underlying data structure)Tests are infinitely expandable.Tests remain valid even when the implementation changes.

5

Tools RequiredVisualStudio.NET (Enterprise Architect Edition)NUnit XML Spy (Optional)

6

The Test ProcessDeveloper Builds the Test DataSet ClassesDeveloper Creates the Test Data FileThe Test Data Files are placed in a common directory.Developer constructs and executes the basic test classes.Test Engineers develop test cases and add them to the test file.Tests are continually added.Each time the product is changed or a test case is added, all of the tests are rerun.

Developers run all tests when the code changes.Test engineers run all tests when new test cases are added.

7

Test DataCreate the Test Data DatasetCreate the Test Data Data File

8

What is the Purpose of the test DataSet?

To hold the values necessary to robustly test the Create, Read, Update, and Delete functions.To provide a strongly typed DataSet to increase reliability and reduce coding.To simply the generation and initial population of the test data file.

9

Why do you need a DataSet?First, you don’t “need” a DataSet. They are just a good idea.DataSets reduce the amount of code required to construct the tests.DataSets increase the readability of code.DataSets eliminate many kinds of errors in the test code itself.

10

Create the Test DataSetEach DataSet is based on an underlying table, view, or stored procedure.Start the DataSet Wizard by adding a new DataSetto your test project.Drag the underlying object to the designer.Switch to the XML view.Modify the primary key field.Duplicate the data fields.Add “Update” to the end of each new field name.Save.

Demo Here!

11

What Just Happened?The format for the test data file was defined.An XSD (XML schema definition file) was just created.A Strongly Typed DataSet class was generated (3000 or more lines of highly reliable code) – requires that Generate DataSet be turned on (it is by default).

12

Generate the Data FileHere’s where XML Spy comes in handy.Open the Schema file you just created in XML SpyChoose the ‘Generate Sample XML File’ command from the DTD/Schema menu.When the dialog appears, select your options including the number of rows to generate, and click OK.Save the file in your public TestData directory as an XML file.Edit the data rows as needed.

Demo Here!

13

The Test Data FileXML Format Means

Editable with a wide variety of tools (including Notepad)Built to a known standard supported by every software vendorEasy to transfer to any other data formatEasy to read and understand..NET-ready

For proper testing, you need both a green bar data file and a red bar data file.

14

NUnit BasicsWhat is NUnit?NUnit LimitationsObtaining and Installing the ProductConstructing A Simple TestRunning Tests

15

What Is NUnit?A Public Domain Test Harness for .NET

A Set of Classes to Facilitate Test Development and ExecutionA Command Line InterfaceA GUI to Run the Tests from Windows

A Tool For Developer-Centered TestingDeveloped as JUnit, Ported to .NET, then Rewritten in C#

16

NUnit LimitationsYou can’t test exception propagation easily.

Hard Code TestsRed Bar Tests

Can’t debug when you run from the UI(This was a much longer list until version 2.1

which corrected most of the problems.)

Unlike user interfaces, database and business objects should throw rather than handle exceptions.

17

Obtaining and Installing NUnithttp://www.nunit.orgGo to the Downloads PageRun the Executable

(That’s all there is to installing the program. Note, however that each project only knows about the test harness when it contains a reference to the NUnit libraries.)

18

Constructing a Simple TestAdding the Harness ReferenceCreating a Test ClassAdding Test Fixture DecorationsWrite the Test Logic

19

Adding the Harness ReferenceRight-click the References icon for the test project in the Solution Explorer and select Add Reference.When the Open dialogue appears, navigate to the NUnit Installation directory.Select all of the DLLs and click Open.(I’m not sure they are all needed, but the documentation does not specify which are needed.)

20

Creating a Test ClassAdd a new class to the project.Import the NUnit Framework

Imports NUnit.Framework (VB.NET)using NUnit.Framework; (C#)

Add a TestFixture Decoration to the class declaration.

<TestFixture()> (VB -- must be on the same line as the class declaration)[TestFixture] (C# -- Must be on the declaration line or the line before)

21

Write the Test LogicGet the Test DataFor Each Test Case

Create the RecordVerify the RecordUpdate the RecordVerify the RecordDelete the RecordVerify the Deletion

The test logic is often in a different class than the test execution.

That is the approach that we use because it makes debugging both the test and the application easier.

22

Running TestsFrom the GUI

Green Bar TestsRed Bar Tests

From the Code

23

Process Change ResultsAs a result of adding NUnit and the basic database test process demonstrated here, we were able to shrink delivery times significantly while testing our database applications hundreds of times more thoroughly than we could before. To our credit and embarrassment, our prototypes are now more robust and bug-free than our finished applications used to be. (And our finished applications were always more robust than the majority of what you see.)

24

Current Deliverables for One Programmer for One Four Hour Work Period

One DataSet ClassOne Command Helper ClassFive Stored ProceduresOne Business Object Class for Individual ObjectsOne Business Object Class for the DataSet

One Test Class for Each Business Object ClassOne Test Data File containing at least 3 test casesA Data Entry form for the individual object class.A List form for the DataSetClass

25

SummaryNUnit is free, flexible, and easy to use.XML test data files are easy to create and use.NUnit supports a simple, standard approach to database testing that addresses the needs of both the test team and the development team.The result of this approach is an automated test suite that creates significant gains in quality and productivity.

26

Technical AppendixTest LogicSet TestsAdditional TestsStandardizing Test ClassesResourcesAutomation of Test Automation

27

Test Logic: Get the DataThree Details Make getting the Data Easy

Our test data is in an XML file.We have a strongly typed DataSet class that matches that XML file.Microsoft provides the ReadXML method.

VBDim myTestData As New MyTestDataSetName()myTestData.ReadXML(fullPathOfTestFile)

C#MyTestDataSetName myTestData = New MyTestDataSetName();myTestData.ReadXML(fullPathOfTestFile);

28

Test Logic: Get Each Test Case

A Strongly Typed DataSet consists of Strongly Typed DataRows. VBFor Each rowInstance _

As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _in _testSet.Tables[0].Rows

_testRow = rowInstance ‘Assign to class variable for common use.Next rowInstance

C#foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow rowInstance in _testSet.Tables[0].Rows)

{_testRow = rowInstance;} // assign to a class variable for common use.

These Row classes were generated when the Test Data DataSet was created.

29

Test Logic: The CRUD TestsThe Logic of your CRUD tests will depend on the structure of your application, but the set of tests will always have to be done within the test data loop. VBFor Each rowInstance _

As BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow _in _testSet.Tables[0].Rows

_testRow = rowInstance ‘Assign to class variable for common use.RunCrudTestSuite()

Next rowInstance

C#foreach (BackSpreadsParameterSetsTestData.SampleBusinessObjectsRow rowInstance in _testSet.Tables[0].Rows){

_testRow = rowInstance; // assign to a class variable for common use. RunCrudTestSuite();

}

A better design would pass the test row to the CRUD test routine..

30

Our Basic CRUD Testsprivate void RunCrudTestSuite(){

int recordID = theNewID();TestInitialValues(recordID);UpdateRecord(recordID);TestUpdateValues(recordID);DeleteRecord(recordID);TestDeletion (recordID);

}

The recordID ties the tests to the same database object.

31

theNewID()protected override int theNewID(){

SampleBusinessObject testObject = new SampleBusinessObject();testObject.Name = _testRow.name;testObject.OptionBotID = _testRow.optionBotID;testObject.Save();Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");return testObject.ID;

}

Access to the database is allowed only through the business objects.A new business object is created.The properies are set from the test data row.The business object is saved.A check is made to see that no exceptions occurred.The id of the new record is returned.

Note the Assertion. This is an NUnit Assertion.

32

TestInitialValues()protected override void TestInitialValues(int idOfObject){

SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);Assert.AreEqual(testObject.Name, _testRow.name, “Wrong name.");Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotID, “Wrong OptionBot ID!");Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");

}

Tests are very simple.You put a value into the field.You check to make sure that the same value was recorded and retrieved.

33

UpdateRecord()protected override void UpdateRecord(int idOfObject){

SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);testObject.Name = _testRow.nameUpdate;testObject.OptionBotID = _testRow.optionBotIDUpdate;testObject.Save();Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");return testObject.ID;

}

This is very similar to creating the record except that we start with an existing object.Note that we are populating the business object with the update values from the test data row.

34

TestUpdateValues()protected override void TestUpdateValues(int idOfObject){

SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");Assert.AreEqual(testObject.Name, _testRow.nameUpdate, “Wrong name.");Assert.AreEqual(testObject.OptionBotID, _testRow.optionBotIDUpdate, “Wrong OptionBot

ID!");}

Update test is almost identical to the initial values test.Note that we are testing against the update values.

35

DeleteRecord()

protected override void DeleteRecord(int idOfObject){

SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);testObject.Delete();

}

DeleteRecord is very simple. The logic for the deletion is in the business object so our test code simply calls the objects Delete Method.

36

TestDeletion()protected override void TestDeletion(int idOfObject){

SampleBusinessObject testObject = new SampleBusinessObject(idOfObject);Assert.AreEqual(testObject.Exceptions.Count, 0, "An object loading error occurred.");Assert.AreEqual(testObject.IsNew, true, “The object was not deleted.");

}

Update test is almost identical to the initial values test.Note that we are testing against the update values.

37

The Test DecorationNote that although we had to add a TestFixture decoration to the class declaration, none of the individual tests have had any additional decorations. These cannot be run from the NUnit GUI. That is intentional; the CRUD regression tests are designed to be run as a unit.Add a test in your Test Fixture class to run the whole set and add the declaration as shown on the next screen.

38

Test Decoration ExampleThe test decoration allows the NUnit GUI to recognize a method as a test.

[Test]public void TestBackSpreadParameterSets(){

SampleObjectTest _thisTest = new SampleObjectTest(@"H:\TestData\OptionBot\SampleObjects.xml");_thisTest.RunAllTests();

}

In VB.NET, the correct decoration is ‘<Test()>’.

39

Test Initialization ExampleNUnit allows you to specify a set of actions that are performed when the Test Fixture class is initialized.

40

The Basic Persistent Set TestsGet the SetSelect SubsetsCount the Set

41

Additional TestsAre Driven by Customer Requirements Are added at the specification of the test engineer.Are added by the developer.Additional Test Example

42

Standardizing the Test ClassesTest Class Interfaces are used to standardize the language used for the test methods.Abstract Test Classes capture the common elements of test classes. The latest versions of our test classes are currently available at http://www.ProcessBuilder.com for the following languages

VB.NETC#

43

NUnit ResourcesNUnit Home Page, http://www.nunit.orgNUnit Resources http://www.testdriven.com/modules/news/

44

Automating Test AutomationNote that this presentation is about how to automate the execution of tests, not the creation of those tests. The creation of those tests is clearly automatable.Extracting MetaDataCode Generation Alternatives