mcif- per kroll

© 2009 IBM Corporation

PPM06

Measure Your Results by Applying Measured Capability Improvement Framework

and Rational Insight

Per KrollChief Solutions Architect

MCIF, [email protected]

PPM06

© 2009 IBM Corporation

IBM Rational Software Conference 2009

PPM06: Measure Your Results by Applying MCIF and Rational Insight

Discussion Topics

Stop guessing, start measuring

Introducing IBM Rational Insight

Demo

Parting thoughts



Challenges to the effective delivery of softwareA history of cost overruns, schedule slips and quality issues

“Business leaders and CIOs are under pressure to enable their teams to become catalysts for change. However, change is outpacing their ability to deliver.

They are reorienting their teams to focus on ROI and quantified business outcomes and to mitigate risk and reduce costs.”

– IBM CIO Survey, 2008

Growing focuson business outcomes

“Only 22% of executives felt that their IT and business strategy were tightly integrated”

“49% of budgets suffer overruns and 62% fail to meet their scheduled”

“34% of software projects are deemed successful costing $300B annually”

30% of project costs are due to rework and poor execution of requirements”

“50% of outsourced projects are expected to under perform”



Software delivery is a business process that can be continuously improved

Source: Gartner, “Making the Difference: The 2008 CIO Agenda,” Jan. 2008

Requirements

Design

Implementation

Verification

Deployment

CIO’s top priority over last three years: “Improving Business Processes”



Measurement is key to transforming a business process

Toyota: No work without process No process without metrics No metrics without measurement No measurement without analysis No analysis without improvement



Software Measurement Status - Today

Fortune 500 companies with productivity measures: 30%

Fortune 500 companies with quality measures: 45%

Fortune 500 companies with complete measures: 15%

Fortune 500 companies with missing measures: 85%

Number of software measurement personnel 5,500

Number of software projects measured: 160,000

Number of software projects not measured 50,000,000

- Capers Jones (2009)



Does It Help to Measure?Companies that measure: Companies that don’t:

On-time projects: 75% On-time projects: 45%

Late projects: 20% Late projects: 40%

Cancelled projects: 5% Cancelled projects: 15%

Defect removal: > 95% Defect removal: Unknown

Cost estimates: Accurate Cost estimates: Optimistic

User satisfaction: High User satisfaction: Low

Software status: High Software status: Low

Staff morale: High Staff morale: Low

- Capers Jones (2009)



Yet the wrong metrics can incent negative behavior



Lessons from the factory world: Knobs and meters

One sets the knobs hoping to achieve optimal meter readings The meter readings are called outcome measures

Sometimes you need additional measures to ensure the system has responded to the knobs, these are called control measures

Setting

Better Results



Meters for software and systems delivery improvement Business value

Return on Investment (ROI) Return on Assets (ROA) Profit …

Operational objectives Productivity Time to market Quality Predictability …

Practice-based control measures Test Management: Defect density, test coverage Iterative Development: Velocity, iteration burn down Continuous integration: Build stability, Build frequency …

Operational Efficiency0% 100%

Practice Adoption0% 100%

Business Value$0M $15M



Leverage a Control Framework to Manage to Expected Business Results

Operational Level

Practice Level

Business Level

Process Enactment / Governance Enforcement / Process AwarenessJazz Platform

Business Objectives

Process Definition / PracticesRational Method Composer

Operational Objectives

feedback

feedback

feedback

Fe

ed

ba

ck

Performance Measurement(IBM Rational Insight)

Value Metricse.g., ROI, ROA for SSD

Operational Effectiveness Metricse.g., Time to market, productivity

Practice Adoption/Maturity

SubjectiveIBM Rational

Self-Check

Practice ArtifactsObjective



A Systematic Approach to Software Excellence

A phased approach that helps teams

Adopt an incremental, measured approach to transformation

Focus on the core practices that matter most

Accelerate adoption through out-of-the-box assets

Articulate capability improvements in terms of business value

Support any method Optimized for Agile practices

Empower teams to measure, manage and incrementally improve their software delivery capability.

Establish business and operational objectives

Prioritize practices and define roadmap

Accelerate adoption with tools and services

Report, analyze and act on results

Phase1

Phase2

Phase3

Phase4

Start next incremental improvement



Leverage MCIF Assets to Incrementally Definethe Control Framework

Operational Level

Practice Level

Business Level

Process Enactment / Governance Enforcement / Process AwarenessJazz Platform

Business Objectives

Process Definition / PracticesRational Method Composer

Operational Objectives

feedback

feedback

feedback

fee

db

ack

Performance Measurement(IBM Rational Insight)

Value Metricse.g., ROI, ROA for SSD

Operational Effectiveness Metricse.g., Time to market, productivity

Practice Adoption/Maturity

SubjectiveIBM Rational

Self-Check

Practice ArtifactsObjective

Phase 4: Rational Insight

Phase 3-4:Self-Check

Phase 3: Rational Method Composer

Phase 2:Health Assessment

Phase 1: Exec Business

Value Workshop

Phase 3:Rapid Deployment Package

NEW

ENHANCED



Unifying Platform - Jazz Operationalize capability improvement

Define Business andOperational Objectives

Define Practices

Operationalize processes, accountability, and policies

acrossthe organization

Measure, monitor and report on practice adoption, and business

and operational results

Phase

4

Phase

3

Phase

1/2

Client Integrations Server Integrations

FutureIBM

Capabilities

3rd-PartyJazz

Capabilities

DeliverEnduringQuality

AccelerateChange &Delivery

Ensure Security &

ComplianceManage

ArchitectureManageEvolving

Requirements

ImproveProject

Success

QueryStorage

Collaboration

DiscoveryData Warehousing

Administration: Users, projects, process

Best Practices

JAZZ SERVICES



Discussion Topics



Demo

Parting thoughts



The Danger of Information Blind Spots

Lack of Relevant Timely Information

Information Overload

Inability to combine various data points for a single view of analysis E.g. combining cross-functional

requirements, balance risk, and learning from failures

Inability to know “what” to monitor and “how” to achieve expected results

“77% of managers are aware of bad decisions that they have made because they did not have access to accurate information”

Source: Business Intelligence Forum, Strategic decisions still taken based on gut feel, 2004



Measure and manage software delivery performance

Measure process and project outcomes Real-time intelligence based on IT industry

best-practice metrics, dashboards and models

Inform decisions and drill into issues Alerts and automated analysis focuses owner

to take action on root causes, before mitigation costs balloon 1000%

Take real-time action on relevant data Proven BI-backbone automates collection and

analysis to improve lifecycle productivity 10-15%

Deliver performance and predictability with integrated lifecycle intelligence

Rational Insight

Integrated lifecycle

intelligence

“Organizations exercising world-classperformance management practices enjoy

a 2.4 times market returns oftypical companies”

BusinessWeek Study:The Payoff of Pervasive Performance Management

Control Settings

ResultsBetter



Measure performance against business objectives IBM Rational Insight

Monitor and analyze project and process performance Collaborate across disparate development

artifacts in the context of operational and business objectives

Automate reporting and measurement by leveraging Jazz compliant open REST interfaces and a Cognos based reporting platform

Report on relevant data with a built-in library of industry best practice metrics and dashboards

Continual process improvement to effect organizational outcomes Process improvement and best practice

guidance through the Measured Capability Improvement Framework (MCIF)

Built on best-in-class business intelligence platform from IBM Cognos

“Until now we were gathering metrics by manual methods. Rational Insight simplifies the presentation... Overall great tool to have for optimizing project efforts and giving management a birds eye view of the project performance at all stages.”

Real-time information in a

single view across projects, products and geographies

Automated project and process measurement can improve team productively by 15%

“Through the automation and dashboard capabilities in Insight, our team is saving over 40 hours a month on our reporting activities. We also can offer new capabilities such as “on-demand” dashboards which would have been very labor

intensive to do without Insight”



Rational Insight – SSD Performance Management

Open Lifecycle Service Integrations

Best Practice Guidance

ClearCase

MS Project

Quality Manager

Partners

Cognos based report serverprovides a consolidate view

ClearQuest

RequisitePro

Team Concert

Insight

Powered by

Measured Capability Improvement Framework

(MCIF)

Powered By Cognos

3rd Party Integrations

Rational Insight Data Warehouse

ODBC/JDBCXML



Rational Insight

Project Management

Project Management

Build Management

Build Management

Architecture Management

Architecture Management

Requirements Management

Requirements Management

Change Management

Change Management

Configuration Management

Configuration Management

Portfolio Management

Portfolio Management

Quality Management

Quality Management

CxO

Project Manager

Collaborate across disparate development artifactsIn the context of business objectives

Process Lead



High Level Business Objectives mapped to Operational Objectives Reduce Costs Increase Market Share Improve Customer Satisfaction

Line of Business goals driving operational performance initiatives Manage and Optimize Resources Develop Transparently Implement Test Driven Development

Project Team goals mapped to business objectives Build Management and Health Automated Testing Sprint/Iteration Velocity or Burndown Requirements Traceability

Collaborate across disparate development artifactsIn the context of business objectives

CxO

Process Lead

Project Manager



Multiple Roles Come With Multiple Usage Patterns…

View dashboards and reports online…

View reports on mobile devices…

Embed charts and reports in spreadsheets / presentations…

Search for reports using standard paradigms…

Easily create and customize graphical dashboards…



Extending the Insight platform with Cognos Go!Increase business intelligence user adoption rates within your organization by letting users view and consume reports, scorecards, and other BI content using familiar applications or devices, such as a BlackBerry®, search engine, MS Office application, or Web browser.

Increase business intelligence user adoption rates within your organization by letting users view and consume reports, scorecards, and other BI content using familiar applications or devices, such as a BlackBerry®, search engine, MS Office application, or Web browser.

IBM Cognos 8 Go! Dashboard lets users build and edit dynamic and interactive dashboards using a drag-and-drop Flash interface.

IBM Cognos 8 Go! Dashboard lets users build and edit dynamic and interactive dashboards using a drag-and-drop Flash interface.

IBM Cognos 8 Go! Mobile lets users securely receive and interact with reports and analysis through their BlackBerry® handheld devices.

IBM Cognos 8 Go! Mobile lets users securely receive and interact with reports and analysis through their BlackBerry® handheld devices.

IBM Cognos 8 Go! Search lets users find existing reports in IBM Cognos 8 Business Intelligence more quickly. It also lets you integrate BI content into enterprise search applications such as those provided by Autonomy, Fast, Google, and IBM.

IBM Cognos 8 Go! Search lets users find existing reports in IBM Cognos 8 Business Intelligence more quickly. It also lets you integrate BI content into enterprise search applications such as those provided by Autonomy, Fast, Google, and IBM.

IBM Cognos 8 Go! Office lets your users view, interact with, and refresh BI content within MS Office applications such as Microsoft® Excel®, PowerPoint®, and Word®.

IBM Cognos 8 Go! Office lets your users view, interact with, and refresh BI content within MS Office applications such as Microsoft® Excel®, PowerPoint®, and Word®.



Discussion Topics



Demo

Parting thoughts


PPM06: Measure Your Results by Applying MCIF and Rational Insight 25



This top-level dashboard showcases schedule

variance, project health, size, and % complete for each project. Drill down on project for more info.

This next set of slides showcases the executive performance measurement dashboard we are using to govern Rational’s development teams leveraging Rational Insight using MCIF-style dashboards. Note that some data has been

added, changed or removed to avoid revealing confidential data.

Project health by region. Drill down for regional information, including headcount

and cost-related information.

Dashboards are organized around business objectives to ensure focus on potential problem areas preventing

us to reach stated business objectives.



Project Health and Practice Control MetricsIterative

Development2-Level

PlanningContinuousIntegration



Discussion Topics



Demo

Parting thoughts



Progress checkpoints

Confidently deliver incremental improvement

Reach goals and maximize value with improvement roadmap Identify customer business challenge and align to

operational objectives Apply proven best practice to achieve objectives

Drive continuous and measurable improvement End-to-end lifecycle framework drives 15-25%

productivity improvement of selected priorities

Check results and iterate as needed Progress checkpoints and data analysis

determines results and improvements still needed

Drive necessary change at lower cost with a proven, repeatable approach

Measured Capability Improvement Framework

Data analysis

Continuousand measurable

improvement

Learning opportunities

“IBM’s advice was not ‘product centric’ but rather focused on the business outcomes for Qantas and how we could go about achieving those.”

Brad Tate, Manager Enterprise SolutionsQantas

Rational Insight



Additional MCIF and Rational Insight Resources MCIF on IBM.COM

http://www.ibm.com/software/rational/mcif

Insight on IBM.COMhttp://www-01.ibm.com/software/rational/products/insight

Executive Dashboard on Jazz.nethttps://jazz.net/projects/executive-dashboard/

Self-Check article on developerWorkshttp://www.ibm.com/developerworks/rational/library/edge/08/may08/kroll_krebs/index.html

RMC 7.5 Practices on developerWorkshttp://www.ibm.com/developerworks/rational/practices/

IBM Rational Insight



© Copyright IBM Corporation 2009. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

mcif- per kroll

Technology

business process toyota

business processthat

business processes

number of software projects

terms of business value

software measurement

rational insight phase

results phase