Testing as a Driver forTesting as a Driver forDevelopment ChangeDevelopment Change

Wall Street SystemsWall Street Systems

Graham Thomas

                                             

2

AbstractAbstract

You have established a testing process but reached the point where to make further improvements you have to change the way development works.

So how do you go about changing the development lifecycle? What are the key areas to address, and how do you cope with the traditional problems of the relationship between the developers and the testers, organisational priorities assigned to testing, and the commercial realities of a software house.

This presentation is case study of work over the last year, kicking-off a change program, and the initiatives that have resulted.

3

AgendaAgenda

Improving the testing process Analysis of the results Development Change Program Achievements to date Work still to do

4

Test Process ImprovementTest Process Improvement

Test Strategy

Test Process

S

R

A

D

B U

I

A

UnitTesting

IntegrationTesting

SystemTesting

AcceptanceTesting

Stage Entry Stage Execution Stage Exit

5

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

29/10/2001 13/11/2001 28/11/2001 13/12/2001 02/01/2002 17/01/2002 01/02/2002 18/02/2002

TST52 System Availability

Test Process ImprovementTest Process Improvement

Fault Database

Process Metrics

6

Test Process AnalysisTest Process Analysis

STAGE ENTRY

Some software components delivered late

Incomplete documentation

Test environment availability

STAGE EXIT

High percentage of integration faults

Leading to a lack of test coverage

Project priorities steer testing

7

Change ProgramChange Program

SoftwareManagement

ProgrammeOffice

Development Oracle/UnixDev. Env.

Unit/Int.Testing

TimelyDocuments

ReleaseMaintenance

ProcureCM Tool

SoftwareVersioning

ConfigurableItem Control

ChangeControl

Testing AutomatedTesting

RQMTSMgmt

EnvelopeReg. Test

Client BuildTesting

ReleaseManagement

Dev.Lifecycle

ChangeManagement

ProgrammeManagement

Short Term Mid Term Long TermStream

8

Change Program ModelsChange Program Models

Incremental

Step Change

Thin Threads

9

Achievements To DateAchievements To Date

Development LifecycleRequirements processTest AutomationUnit Testing ToolsConfiguration Management Development Partnership Programme

10

Client Requirements Feasibility Design

Receive & Log Request

Validate Request

Request further info

Valid?

Business Process Analysis

Technical Infrastructure

Analysis

Workload Capacity Analysis

Auth.?Reject

Request

Review Commitment with Client

AuthorisedRequest

Cost Estimate

FA?

Design

Functional Specification

ReviewFunctional

Specification

ProgramSpecification

Build & Unit Test

EnhancementRequest

Additional Details

Planning Costings & Timing Info

Request Rejected

Client Acceptance

Implementation

Design?

Y

Y

Y

Y

ReviewProgram

Specification

RequirementsWorkshop

Release Test ImplementationBuild & Unit Test

Design UnitTest

Plan

Code & Review

Unit Test & Review

Integration Test & Review

Release Test

RT?

Test Pack Impact

Analysis

Review Results

Execute Release Test

Review Results

Update Release Test

Baseline

Implementation

Implementation Analysis

Software Management

Controlled Software Delivery

Upgrade Process

On-site Testing

Review Results

Y

Y

Client

EnvelopeTesting

ReviewResults

Upg?

Controlled Software Delivery

On-siteTest?

Y

Software Management

Development LifecycleDevelopment Lifecycle

Requirements Feasibility Design Build &

Unit Test Release

Test Implement

Requirements Feasibility Design Build &

Unit Test Release

Test Implement

EnvelopeTest

ControlledDelivery

Modified for Small items

Base Development Model

11

RequirementsRequirements

Tightened up definitionTesting SpecificationAcceptance CriteriaClosely Monitored ProcessDrives development activityClearly defined targets

12

Test AutomationTest Automation

Generated scripts for key functionality

Run on multiple PC’s controlled through a master

Developed the suite to run unattended and overnight

Successfully execute over 8,000 transactions in a single run

13

Unit Testing ToolsUnit Testing Tools

Suite of unit testing toolsAutomatic Static Source Code AnalysisRuntime Error Analysis Automatic Performance Analysis and

OptimisationAutomatic Code Coverage AnalysisAutomatic Error Detection and Recovery

14

Configuration Management Configuration Management

Identification of CI’sBranching strategyEnvironmental

requirementsStage 1

Secure source CI’s. Check Out/Check In 3 month time-box

Main

Release

Dev.

Client

15

Development PartnershipDevelopment Partnership

Assisted Development

Hosted Test Lab

Remote Project Support

Hardware Lab

Testing Partnership

Requirements Feasibility Design Build &

Unit Test Release

Test Implement

Base Development Model

16

Still to DoStill to Do

Roll-out Unit Testing ToolsCM Phase 2 – Process ChangeIntroduce Process Workflow with CRMIncrease coverage of automation suiteLeverage automation in other areasReview testing activities across the whole

company

17

The Story So FarThe Story So Far

It is hard enough just changing the testing process

You need sponsorship and support to re-engineer the development lifecycle

Change is an inclusive processChange is continual

It is harder not to change !

18

Where Value was AddedWhere Value was Added

Successes Test Process Improvement On-going Change Program Identified two-speed lifecycle Tool evaluation for CM & Unit Testing Broadened the view of quality from QC to QA

Failures Point solutions only No quick wins Pick & Mix approach by development

19

Contact DetailsContact Details

Graham Thomas

graham@badgerscroft.com

+44 (0)7973 387 853

www.badgerscroft.com