service validation and testing - a ca service...

TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING

Malcolm RyderCA SERVICES / ARCHITECT

MARCH 2010

Service Validation and Testing:A CA Service Management Process Map

Copyright © 2010 CA. All rights reserved. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. This document is for your infor-mational purposes only. To the extent permitted by applicable law, CA provides this document “As Is” without warranty of any kind, including, without limitation, any implied warranties of merchantability or fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage, direct or indirect, from the use of this document including, without limitation, lost profits, business interruption, goodwill or lost data, even if CA is expressly advised of such damages.

Table of Contents

TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING 1

Executive SummaryChallenge

The Information Technology Infrastructure Library version 3 (ITIL® V3) process framework approaches IT Service Management (ITSM) in terms of the lifecycle of a service. The Service Lifecycle is an organization model providing insight into the way ITSM is structured, and embodies critical guidance for IT organizations seeking to improve service quality and align more closely with business goals to create value for the business and its customers.

However, ITIL V3 best practice guidelines across the five phases of the service lifecycle are complex and challenging to interpret, and they are not designed to provide definitive advice about implementing ITSM processes. Many IT organizations consequently undertake an ITIL journey without a firm idea of their goals and the path to achieve those goals.

One of the key ITIL management processes, Service Validation and Testing, presents its own special challenges. Participants in the service quality assurance effort must coordinate the priorities and extent of testing with the type and timing of business needs to be met; consequently, the measure of success in validation and testing extends beyond the technical aspect to the matter of business value.

Opportunity

The primary objective of the Service Validation and Testing process is to ensure that a service offering, as designed, deployed and supported, has the quality expected to meet customer expectations.This quality is a strategic process outcome. CA has developed a unique approach to representing the ITIL framework and its interdependent IT Service Management (ITSM) processes at a high level in the form of an easy-to-use subway map. This map is an ideal starting point for understanding and communicating about ITIL in support of successful program planning and implementation.

Benefits

Following the Validation and Testing map provides guidance to:

• Improve business process capabilities

• Effectively upgrade the IT infrastructure in timely alignment with business needs

• Foster the efficient delivery of multiple IT services

• Improve the quality of technical support

Executive Summary 1

SECTION 1:

Finding the Right Path to IT Service Excellence 3

SECTION 2:

Service Validation and Testing 5Adopt Best PracticesValidate & VerifyTracking Validations Versus RequirementsPerform TestsReport / Closure

SECTION 3:

Benefits 9A Key to Achieving IT Service Excellence

SECTION 4:

Conclusions 10

SECTION 5:

About the Author 10

2 TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING 3

Simplifying ITILThe ITIL V3 process framework focuses on the service lifecycle and the way that service management components are structured and linked. It embodies critical guidance for IT organizations that are seeking to improve service quality and align more closely with business goals.

But, the ITIL V3 best-practice guidelines across the five phases of the service lifecycle are complex and challenging to interpret. Moreover, they are not designed to provide definitive advice about implementing IT Service Management (ITSM) processes. Many IT organizations consequently undertake an ITIL journey without a firm idea of their goals and the path to achieve those goals.

CA has developed a unique approach to charting the ITIL journey through a visual representation of the ITIL framework and its interdependent ITSM processes modeled after an urban subway system. This three-part map (Figure A) presents an easy-to-navigate, high-level view of the ITIL terrain. IT executives, strategists and implementers can use these Service Management process maps along with the family of CA Service Management process map technology briefs that expand on them. The maps and technology briefs provide a common reference point for understanding and communicating about ITIL and help you with program planning and implementation.

How to Use the CA Service Management Process MapsCA’s Service Management process maps (Figure A) illustrate every process (or track), each activity (or station) and the key relationships that are relevant to navigating continuous IT service improvement. The ITIL quality cycle takes the form of a “circle” with each Plan-Do- Check-Act (P-D-C-A) step as a process integration point (junction) on the line. Junctions serve both as reference points when assessing process maturity, and as a means to consider the implications of implementing a process in isolation.

Strategic controls (Service Portfolio Management, Demand Management and Financial Management) are needed to reduce risk and optimize integration across the service lifecycle, as illustrated on the three points of the triangle centered in the P-D-C-A quality circle (seen more easily in Figure B). These strategic controls help in evaluating, prioritizing and assuring the appropriate levels of financial and human resources for existing and new services.

This paper is part of a series of Service Management Process Map technology briefs. Each brief explains how to navigate a particular ITIL process journey, reviewing each process activity that must be addressed in order to achieve process objectives. Along each journey careful attention is paid to how technology plays a critical role in both integrating ITIL processes and automating ITIL process activities.

SECTION 1: CHALLENGE Finding the Right Path to IT Service Excellence

Most organizations are highly sensitive to quality levels in IT deployments, and they already rely on continuously managed processes and practices that are expected to provide some assurance of the fitness of deployed components and systems for their purpose. For that matter, validation most often means verifying that component and systems specifications are appropriate to established norms for operational risk. Testing takes place under the watchful eye of domain and subject matter experts in the line of duty for resolving problems and executing changes.

Yet as we see in the process interactions shown on our subway map, the circuit from planned modifications to deployment releases is an incomplete journey. This is because builds and installations may or may not go according to expectations. But once utilization begins, quality requirements differ again, and acceptance of the deployments and releases depends on further steps defined more from a business user’s perspective than from a technical one.

Service Transition

Service Validation and Testing

Service Asset and ConfigurationManagement

Change Management

Transition Planning and Support

Release and DeploymentManagement

Key Intersections

Strategic Controls

Strategic Inputs

Continual ServiceImprovement

Prioritize

BuildSchedule

ImpactAnalysis

Execute(Emergency/Standard)

Schedule Change

CAB Review

Evaluation/Decision

RFCAnalysis

Categorize

Planning

Preparation

DevelopStrategy

Preparationand Planning

KnowledgeTransfer

Monitorand Report

ManageBuild/Release

FinancialManagement

DemandManagement

Service PortfolioManagement

Deploy

Verify

PerformTests

Validate &Verify

AdoptBest Practices

EnsureReleaseValue

Report/Closure

AssureQuality

Audit

StatusReports

Maintain AccurateService Configurations

CoordinateResources

BusinessResponsiveness

ConfigurationControl

IdentifyConfigurations

Manageand Plan

DO

CHECK

PLAN

ACT

TransitionPlanning and

Support


Support

Release andDeployment

Management


Management

ChangeManagement

ChangeManagement

Service Asset andConfigurationManagement


ServiceValidation

and Testing

ServiceValidation

and Testing

FIGURE A

CA has developed three maps: Service Design, Service Transition and Service Operation since most ITSM discussions focus on these critical ITIL disciplines

Service Transition

Service Validation and Testing

Service Asset and ConfigurationManagement

Change Management

Transition Planning and Support

Release and DeploymentManagement

Key Intersections

Strategic Controls

Strategic Inputs


Prioritize

BuildSchedule

ImpactAnalysis

Execute(Emergency/Standard)

Schedule Change

CAB Review

Evaluation/Decision

RFCAnalysis

Categorize

Planning

Preparation

DevelopStrategy

Preparationand Planning

KnowledgeTransfer

Monitorand Report

ManageBuild/Release

FinancialManagement

DemandManagement


Deploy

Verify

PerformTests

Validate &Verify

AdoptBest Practices

EnsureReleaseValue

Report/Closure

AssureQuality

Audit

StatusReports

Maintain AccurateService Configurations

CoordinateResources

BusinessResponsiveness

ConfigurationControl

IdentifyConfigurations

Manageand Plan

DO

CHECK

PLAN

ACT


Support


Support


Management


Management

ChangeManagement

ChangeManagement



ServiceValidation

and Testing

ServiceValidation

and Testing

FIGURE B

The Service Transition map represents a journey of developing and improving capabilities for the transition of new and modified services to production.

Service Design

Service Catalog Management

Service Level Management

IT Service Continuity Management

Capacity Management

Info. Security Management

Availability Management

Key Intersections

Strategic Controls

Strategic Inputs


Service LevelManagementService LevelManagement

IT Service Continuity

Management

IT Service Continuity

Management

CapacityManagement

CapacityManagement

Info. SecurityManagementInfo. SecurityManagement

AvailabilityManagementAvailability

Management

Service CatalogManagement

Service CatalogManagement

DemandManagement


FinancialManagement

Reviewand Audit

ManageSecurity Incidents

OptimizeAvailability

Mitigate Risk

MonitorServices

Methods/Techniques

DocumentService

Definition

Build CatalogContents

BusinessServiceViews

TechnicalServiceViews

PublishLive Services

ReportAchievements

ManageIssues

CatalogOperational

Services

MonitorPerformance

Design SLAFramework

ReviseSLAs / OLAs

Meet BusinessRequirements

ServiceReview

CustomerSatisfaction

DetermineVulnerabilities

Assess Risk

Build

Test

MaintainBusinessServices

SpecifyContinuity

Requirements

MonitorDemand

Adjust and Tune

DeliverRequiredResources

Analyze

Test

ProactiveManagement

AnalyzePerformance

Build Plan

ForecastRequirements

Set SecurityControls

Assess &Classify Assets

MaintainPolicy

Model/Trend

DO

CHECK

PLAN

ACT

Service Operation

Problem Management

Incident Management

Event Management

Request Fulfillment

Knowledge Management

Access Management

Key Intersections

Strategic Controls

Strategic Inputs


InformedDecisions

Fulfillment

Approval(Financial, Compliance)

Capture Info.

Prevent andEliminateProblems

Record

Resolve

Monitor / Track

Service Request(Incl. Self-Service)

Access Request

Verify

ProblemControl

KnownErrors

Record

Review/Action

Select Response

Filter /Correlate

Detect(Incl. Fault Detection)

Raise Incident

Investigate

Diagnose

Escalate

Resolve/Recover

Work Around

RestoreService

DeliverStandardized

Services

Automateand Control

SecureService Access

ErrorControl

ExecutivePolicy

ProvideRights

Transfer/Disseminate

Store Info.

DemandManagement

Transformto Usable

Knowledge

FinancialManagement


DO

CHECK

PLAN

ACT

ProblemManagement

ProblemManagement

KnowledgeManagement

KnowledgeManagement

AccessManagement

AccessManagement

RequestFulfillment

RequestFulfillment

IncidentManagement

IncidentManagement

EventManagement

EventManagement


Alignment and compliance with the business is the overall goal of validation and testing, mandating explicit awareness of business requirements and coordination with them. Key points in this coordination include:

• Cooperation of specific business and technical roles

• Documentation of requirements at different levels of service, and corresponding documentation of the plan for validations

• Tracking the validations versus the requirements

Another question that comes up is about how much testing is enough. Too much is difficult to justify, and too little is too risky. One of the main outcomes of the Service Validation and Testing process is an increase in the ability to recognize a return on the investment in testing, which encourages the inclusion of right-sized effective testing in the business-level planning and budgeting for IT Services development.

SECTION 2: OPPORTUNITY Service Validation and TestingThe Service Validation and Testing process is responsible for planning and coordinating tests to ensure that specifications for the service design are met and validated through delivery of the service and, including co-operation with the Release and Deployment process, to manage and limit risks that could result from insufficient utility and warranty of the service in operation. Service Validation and Testing takes the technical proficiencies and plans of quality assurance for IT and strategically places them in the larger context of service quality for the business.

The strategic issue is that the business must be able to grow and adapt in order to maintain its success and critical competencies in its market ecosystem of customers, partners and competitors. This makes both Capacity and Change strategic areas of risk, and consequently it makes investments in service deliveries and service modifications part of financial management as well as business risk management.

The ultimate goal of the Service Validation and Testing track (Figure C) is a full alignment of Service characteristics to the utility and warranty requirements of the business. The subway map for this journey plots the key stations along the course in this way:

• Adopt Best Practices

• Validate and Verify

• Perform Tests

• Closure and Reporting

FIGURE C

The Service Validation and Testing track represents management’s attention to how verification of service characteristics is proactively aligned with the requirements for utilization and warranty of an upcoming new deployment of service


Adopt Best PracticesThe key practices of Service Validation and Testing will be

• A model for identifying business alignment

• An integrated set of test plans

• Training, knowledge transfer, and communications

• Verification of results through pre-defined tracking and reporting

These practices are represented in the following table, which illustrates the incorporation of testing within a larger process framework (inputs, activities, and eventual outcomes) of identifying business value from quality assurance.

The inputs for this high-level process express two things: a proactive identification of the customer’s view of a service, and the perspectives of the testing organization as driven by that view. Explicit definitions and standardized formulations of the relevant factors across the process are reusable, which allows for direct comparisons of different testing occasions. The results of such comparisons create an awareness that supports prioritization of service deliveries; and they inform analyses for the improvement of quality management procedures.

In the table above, items such as Service Level Packages (SLPs), Test Plans, and Reports are typical artifacts of the ITIL management process and can be researched in detail within most substantive ITIL publications. A service management organization will want to reach a level of confidence that the various items exist and are clearly related to each other. This will call for assigning clear responsibilities for routine production and review of the items.

Validate & VerifyThe outstanding feature of Service Validation and Testing is its deliberate synchronization of test types with defined business requirements, agnostic to particular technologies and to the variations of hardware and software. The following illustration presents an overview of this synchronization.

Tracking Validations Versus Requirements

• Specs decompose from top down—business to service to supplier

• Tests build from bottom up—components to functions to business agreements

SPECIFICATIONS VALIDATION TESTS KEY ROLES

Business Requirements Alignment to Business Agreements

• Customer

• Compliance Manager

Service Requirements User Acceptance • Service Level Manager

• Test Manager

• Test Manager User

Service Solution Design Operational Readiness • Change Manager

• Security Manager

Service Release Requirements

Release Package • Release Manger

Development Components with Assembly • Technical and Subject Matter Experts

Qualified Supplier(s)

Perform TestsThe essence of manageable testing is the ability to define tests as repeatable, measurable procedures.

This introduces the aspect of Key Performance Indicators (KPIs). As noted in most literature on performance management, KPIs are metrics that address efficiency, effectiveness and cost issues traceable to the design of an executing process and related to the outputs of that process. Indicators identify output characteristics that correlate to desired outcomes, and the means of driving those indicators will be seen as Critical Success Factors (CSFs).

Accordingly, testing will feature standards, scripts (methods) and metrics that are defined and logically related to each other before test execution actually begins.

Further, as a matter of setting scope to testing as well as methodological compliance, policies will be established to govern the decisions about when testing must occur and what type must occur. As identified in the table above, key roles in the organization take on responsibility and authority as stakeholders in the specifications to be validated. In that light, cooperation amongst the roles is expected and directed by policies, while also being institutionalized as integrations of their respective management processes. At points of process integration, the processes may feed each other as participants in shared workflow. In the ITIL subway map of processes, this is notable especially where the Build and Release of a service package involves validation and testing.

QUALITY ASSURANCE

Inputs Validations Outcomes

Service Quality Policy• Value Expectations

Service Package• Description of Designed IT Service

Delivered Value• Available and Supportable

Service Level Package• Intended Utility and

Warranty

Utility Level(fit for purpose)

Warranty Level(fit for use)

Release for Targeted Line of Service

Requirements and Targets• Measures

Usability Compliance Certification

Test Strategy• Plans, Model

• Resources• Methods, Scripts

Specifications Buisness Needs Quality Rating and Results Report


SECTION 3: BENEFITS

ITIL v3 describes Service Design, Service Transition (deployment), and Service Operation as three major phases in the service lifecycle. Within Service Transition, the Service Validation and Testing effort is one of several high-level management processes that link the appropriate design of the service to the smooth operability of the service.

IT Services are the building blocks of business processes and business services. A successful delivery of a service will feature the right thing arriving at the right place at the right time for the right reason. Validation and Testing confirms that the service is fit to its target purpose and fit to its offered use. The business uses multiple services to support its own performance and growth. Executives own the service portfolio, set service goals and control the investment in the service. This executive demand and influence thus should sponsor the qualification of the service as a primary concern of financial and risk management for the business.

The executive view of a successful service implementation includes concerns about the control, compatibility and delivery of the service. These three factors translate, respectively, into management of the quality, risk, and value of the provided service, as deliberately generated from end-to-end throughout the service lifecycle .

The goal of Service Validation and Testing is to ensure that specifications for the service design are met and validated through delivery of the service and, including co-operation with the Release and Deployment process, to manage and limit risks that could result from insufficient utility and warranty of the service in operation.

A Key to Achieving IT Service ExcellenceAutomating ITSM through technology can help your organization reduce the amount ofresources required to achieve ITIL v3 best practices. This assists your IT department in improving the quality of its services while embarking upon a continuous IT service excellence program focused on fostering business growth.

As you reach the end point of the Validation and Testing journey outlined in the CA Service Transition process map, your organization should have a better handle on the steps needed to promote successful service deployments. Specifically, bringing deployment efforts in line with ITIL best practices can help you:

• Improve business process capabilities

• Effectively upgrade the IT infrastructure in timely alignment with business needs

• Foster the efficient delivery of multiple IT services

• Improve the quality of technical support

One of the most important aspects of these process integration points is the documentation that occurs and is used from validation and testing. For example, a test plan usually precedes release and deployment activities, and an update of service records in a configuration management database (CMDB) usually follows deployment but precedes a post-implementation review (PIR). This arrangement highlights the cooperation expected between managers of services, configurations, changes and releases.

Report / ClosureDocumentation of results is also a fundamental deliverable of the Service Validation and Testing effort, as testing itself is expected to be the source of evidence that the service is not ready for delivery and may go through another iteration of development or modification before it can succeed.

As part of proper management of testing itself, these cycles of tests need directives identifying and justifying whether the scope and depth of testing is to be different or not in the next cycle versus the prior one. Because test cycling directly affects the turnaround time between development and release, the management of the test iterations affects the ability of the organization to meet service level agreements with customers.

Test reporting explicitly establishes the reasons why something is tested and why it is deemed suitable for release or not. As material for decision support, the reporting affects resource allocations as well as readiness for compliance auditing.

10 TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING

This document shall not serve to (i) affect the rights and/or obligations of CA or its licensees under any existing or future license agreement or services agreement relating to any CA software product; or (ii) amend any product documentation or specifications for any CA software product. The development, release and timing of any features or functionality described in this document remain at CA’s sole discretion. Notwithstanding anything in this document to the contrary, upon the general availability of any future CA product release referenced in this document, CA may make such release available to new licensees in the form of a regularly scheduled major product release. Such releases may be made available to current licensees of the product who are active subscribers to CA maintenance and support, on a when and if-available basis. In the event of a conflict between the terms of this paragraph and any other information contained in this document, the terms of this paragraph shall govern.

Copyright © 2010 CA. All rights reserved. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Intel and Intel Itanium are trademarks or registered trademarks of Intel Corporation in the United States and other countries. Microsoft, SQL Server .Net , Visual Studio, Internet Explorer, Windows Vista and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. ITIL® is a Registered Trade Mark, and a Registered Community Trade Mark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. Red Hat® and Red Hat® Enterprise Linux® are trade-marks or registered trademarks of Red Hat, Inc in the United States and/or other countries. SUSE is a registered trademark of Novell, Inc., in the United States and other countries. Eclipse is a trademark of the Eclipse Foundation, Inc. SPARC is a trademark or registered trademark of SPARC International, Inc. in the United States and other countries. Adobe and Dreamweaver are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. Mozilla® and Firefox® are registered trademarks of The Mozilla Foundation in the United States and/or other countries. Solaris is a register trademark of Sun Microsystems in the United States and other countries. IBM, z/OS, z/Architecture, CICS, WebSphere. DB2, MQSeries, System p5 , System p6, AIX and DB2 Universal Database are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. THIS DOCUMENT IS FOR YOUR INFORMATIONAL PURPOSES ONLY. TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENT “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT WILL CA BE LIABLE FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USE OF THIS DOCUMENT, INCLUDING, WITHOUT LIMITATION, LOST PROFITS, LOST INVESTMENT, BUSINESS INTERRUPTION, GOODWILL OR LOST DATA, EVEN IF CA IS EXPRESSLY ADVISED IN ADVANCE OF THE POSSIBILITY OF SUCH DAMAGES.

About CA: CA (NASDAQ: CA) is the world’s leading independent IT management software company. With CA’s Enterprise IT Management (EITM) vision and expertise, organizations can more effectively govern, manage and secure IT to optimize business performance and sustain competitive advantage. For more information, visit www.ca.com.

ID1688-032910

SECTION 4: CONCLUSION

Service Validation and Testing facilitates ITSM by managing the certification of IT in the form of services needed by the business. However, doing so requires rigorous processes, appropriate communication plans and support commitments—as well as an outline detailing where to start and how to proceed through the journey.

Following the steps outlined in the CA Service Transition process map gives organizations a clear view of how their Service Validation and Testing journey will take shape, illustrating the key stops en route to achieving effective requirements alignment. This journey results in:

• Strategic re-orientation of testing towards business value

• Better prioritization and investment in services

• Improved risk management

• Stronger integration with other ITSM and ITIL best practices

SECTION 5: ABOUT THE AUTHOR

Malcolm Ryder has over 25 years experience in the IT industry, with expertise in the areasof service transition, IT value management, and IT strategy. For the last 15 years, Malcolm has worked in consulting and solution strategy roles with a heavy emphasis on service management systems with vendors, service providers and end-user customers. Malcolm has been a co-developer of multiple market-leading commercial ITSM solutions since the mid ’80s.

www.ca.com

service validation and testing - a ca service...

Documents