© 2013 ibm corporation risk based validation january 2014

© 2013 IBM Corporation

Risk Based Validation

January 2014


Questions

Do you wish to be able to stay current with the latest releases of Maximo?

Do you wish to save costs in validation and upgrade?

If you answered yes to either or both of these questions….

stay tuned, we are here to help in achieving these objectives!


Topics

Risk-Based Validation Program – Overview

Approach

Analysis Questions

Supportive Documentation

History and 2013 Hi-lights

Developing a Risk Based Validation Strategy – Best Practices

Discussion Questions


Risk-Based Validation

For many years, the FDA has promoted a shift to a risk-based approach to validation– Software validation should be focused on “a justified and documented risk

assessment and a determination of the potential of the system to affect product quality and safety, and record integrity.”

Practical motivations behind risk-based validation:– Companies can more rigorously test the areas of the software system that pose

the greatest risk to product quality and patient safety based on intended use– Overall validation costs are reduced and efficiency is increased – An industry-wide shift to risk-based validation would lead to faster adoption of

innovation in manufacturing and new technological advances

While the risk-based approach is taking hold in regulatory industries, it can and should be used by any and all industries


This unique program supports customers’ implementation of a risk-based approach to Maximo validation for all releases (e.g. new installs, upgrades, fix packs)

– Enables moving to latest fix pack/versions– Enables reduction of costs and efforts– Shifts $ to other areas as needed

• Customer validation is “mapped” to IBM validation; eliminate duplications

• IBM provides supportive evidence from detailed test documentation

• Risk-based testing is an effective way to prioritize risks for building validation centered around high risk areas in your business

• Early risk assessment results in more effective testing

• IBM provides documentation and assistance, including:• Automated & manual test execution reports Summary report of test strategy• Pareto analysis of defects Traceability matrices• Mapping analysis for test coverage

Other benefits :– Develop best regression testing practices, automation practices– Directly influence IBM test efforts for Maximo by providing scenarios to include in IBM testing

5 IBM Confidential

Risk Based Validation Program - Overview


Approach

1. Gather business requirements

2. Categorize requirements

3. Perform risk based testing approach• Analyze your requirements/test cases against IBM test cases, proposed

reductions where duplications exist

4. If needed, perform audit IBM quality management system, to support acceptance of IBM validation and verification data

5. Provide supporting documentation to support elimination of tests

6. Result - develop consistent validation strategy to be used for all release updates

Criticality/Development Method OOTB Configured Customized

Critical Risk-based Investigating Investigating

Non-Critical Risk-based Investigating Investigating

Other Risk-based Investigating Investigating


History and 2013 Hi-lights

• Focus Areas:

• Maximo and Calibration; Maximo 7.5 upgrade and later versions of 7.1.x

• Wide range of business requirements (200 to 2000)

• Benefits stem mostly from OOTB areas (critical and/or non critical)

• Concentration in Life Sciences/regulatory sectors

• Customer references available on request

• Reductions in validation costs range from 20-82%

• Provided and helped companies develop ongoing validation best practices

• 2013 Updates:

• Doubled number of customers as compared to 2012, reductions hold same

• Based on customer input, updated Fix Pack read-me, to allow required impact analysis

• Looking to work with business partners for expansion

• Interest with Nuclear and Oil and Gas customers

• Published QPP content on the ISM Library


Business requirements mapped to IBM results

8


Business Requirement mapped to IBM test case

Example client business requirements mapped to IBM Test cases

9


Business requirement traced to automated test result : System shall allow work order to be auto generated from a Preventive Maintenance record

10 IBM Confidential

Sample Data – Business Requirement


Mapping Document

11


Enhanced Fix pack Documentation

12


Updated Fix pack readme updated to provide list of APARs provided by module vs. by APAR prior to changes


Fix pack documentation example

Fix pack test status tracking spreadsheet

14


Fix Pack Documentation – List of Class File Changes

List of file changes per fix pack

15


Fix pack documentation – Quality plan summary

Quality plan summary information

16


Test Results

17


Sample Data – Test Execution Summary Report

Test Execution summary reports


Sample Data – Look up of specific test case


Developing a Risk Based Validation Strategy – Best Practices

Determine regression “strategy” to be used for all releases– Articulate exactly what’s been tested/not tested, giving accountability to testing efforts

Areas to focus:– Analyzing the most important/critical functionalities: Priority Indicator– Analyzing the areas where the probability of failure is high: Defect Indicator– Analyzing the impact of failure of a particular functionality: Severity Indicator

Analyze fix packs thoughtfully– Review areas that are impacted; assess risk – Often can reduce testing

Include user “exploratory” testing – All testing in the world does not replace having your own users test – Include in test planning, set of users testing the release for x man-days

Include testing for newly configured or customized areas

Consider Automation– Speed up validation and more easily trace results to initial risk assessment– Best used for areas that are repeated in testing


Discussion Questions

What are roadblocks to getting onto the latest release cycle?

Are there challenges in your validation strategy?

What percentage of your application is OOTB?

How can you streamline your regression testing? – What validation takes up the most time?– Are there areas that are there duplications? Do groups duplicate testing?

Are there quality risks that you have had in past?

Do you really need to test all Maximo areas, for every release? Where are the risk areas? What can you do about the risk areas?

Automation?

Performance?

© 2013 ibm corporation risk based validation january 2014

Documents

validation software

riskbased approach

acceptance of ibm validation

ibm confidential risk

maximo validation

building validation

ibm corporation approach

greatest risk