fusion kpiworkshop 09 website
DESCRIPTION
KPIs for businessTRANSCRIPT
KPI based Scorecard
Design your own
Wednesday September 23
2.00 PM – 6.00 PM
Mustang 6
Workshop overview
• This session is designed for professionals who operate in the field of IT
performance management and allows them to create a KPI based
scorecard using KPIs that actually make sense to the business
• The following are the objectives for this session:
– Understand the principles of successful IT Performance Management
– Are able to identify what KPIs are relevant to their business
– Learn to identify quality KPIs
– Using a Software as a Service environment can configure a scorecard and
dashboard
– Are able to interpret information in a dashboard and use this for effective
decision making
• This workshop focuses on the process for the creation of an IT KPI based
scorecard and will not go into the details of data collection, data quality or
data transformation
2
Workshop Agenda
3
Session Title Description Type Time
1 The current state
of IT Performance
management in
your organization
Introduction of the participants and
Discussion on maturity of IT Performance
Management in your organization
Assignment 2.00 – 2.10 PM
2 Current state of
the industry
An overview of the current state of the IT
Performance Management industry
Presentation and
Group Discussion
2.10 – 2.30PM
3 IT Performance
Management
An overview of the IT Performance
Management process, terminology and
concepts
Presentation 2.30 – 3.00 PM
4a How to Defining IT Performance management
objectives and identifying key processes
relevant to your organization
Presentation 3.00 – 3.45
4b Define / Measure Define requirements for IT Performance
Management: objectives, audience, scope
Individual Exercise 3.45 – 4.30 PM
5 Scorecard Design Create an IT KPI based Scorecard based
on the current state of your organization
and your requirements
Presentation and
Group Exercise
4.30 – 5.30 PM
6 Review of
Scorecards /
Dashboards
Analysis of a number of Scorecard and
Dashboard examples
Presentation and
Group Discussion
5.30 – 5.45 PM
7 Questions and
Answers
Q and A session and definition of next
steps
Group Discussion 5.45 – 6.00 PM
CURRENT STATE OF YOUR
ORGANIZATIONS‟ IT
PERFORMANCE
Session 1
The current state of IT Performance Management
in your organization
Agree
Agree to some extent
Disagree
Not sure
Q1) What is your opinion on the following statement:
“ For too many organizations IT is a black box. Projects and systems are so
complex that few CIOs can predict a direct impact on the business, making
it hard to win budget and resources even in prosperous times”
Source: Information Week 2008 - Hunting the Elusive IT Dashboard
The current state of IT Performance Management
in your organization
They don‟t have a clear picture
They could, but information is fragmented and dispersed over
tools and reports
They do, but a lot of effort goes into consolidating various
reports into one view
They have a holistic picture of their overall IT performance
thanks to one or few regular concise reports
There is no clear trend to be distinguished
Q2) To what extent does IT management in your organization / your
customers have a holistic picture on and decision making information
about the performance of IT?
The current state of IT Performance Management
in your organization
Q3) How would you rate the maturity level of your organization (or that of
your customers) in terms of its IT performance management.
Level 0: IT Performance Management is non-existent.
Level 1: There is ad hoc monitoring in isolated areas.
Level 2: Some measures are set with a clear link to business
goals but are not communicated. Measurement processes
emerge, but are not consistently applied.
Level 3: Efficiency and effectiveness are measured and
communicated and linked to business goals and the IT
strategic plan. Continuous improvement is emerging.
Level 4: There is an integrated performance measurement
system linking IT performance to business goals by global
application of a documented framework. Continuous
improvement is a way of life.
The current state of IT Performance Management
in your organization
Reduction of IT cost
Establish the progress toward achieving goals
Optimal resource allocation
Be compliant with internal or external regulations
Require an insight into performance against
service levels
Identify internal improvement opportunities
Other
Q4) What is the main driver for IT Performance Management?
The current state of IT Performance Management
in your organization
Having support from the business
Proper guidance on the implementation
Having a formalized service management framework in
place (ITIL, COBIT)
Having a useful software application to capture and
convey performance measures
Having an industry standard for metrics (common
language)
Having the proper resources in place (budget, people,
infrastructure)
Other. Please specify:
Q5) What do you consider to be the key success factors for effective IT
performance management?
The current state of IT Performance Management
in your organization
Business Intelligence (e.g. IBM, Hyperion, SAP)
Manual reporting (e.g. Excel)
Internally developed software solution
IT performance solution from external vendor (Metricus, M42 )
IT service management solutions (HP OpenView, BMC
Remedy…)
Service Level Management Monitoring Tools (Digital fuel,
Oblicore...)
Other. Please specify:
We don‟t use any tools to report IT performance
Q6) Which tools are you currently using to report IT performance?
CURRENT STATE OF IT
PERFORMANCE
MANAGEMENT
Session 2
IT Performance Management market maturity
12
“For too many organizations IT is a black box.
Projects and systems are so complex that few
CIOs can predict a direct impact on the business,
making it hard to win budget and resources even
in prosperous times.
And when the CIO can't get a clear picture of the
real-time data that underlies critical applications,
infrastructure, and projects, IT too often ends up
reacting to issues after users and customers are
having problems.” (Information Week March
2008)
Source: Information Week 2008 - Hunting the Elusive IT Dashboard
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs’
survey and interviews among 99 IT executives and consultants
65% of IT Managers agree with an InformationWeek statement that says IT managers
have no idea what is happening inside their IT organization (according to an ITpreneurs
survey)
IT Performance Management market maturity
13
The majority of organizations are not reporting on IT Performance due to the time
and energy involved in doing this
Collecting data
from various sources
Converting data
into logical numbers
Building complex
Excel sheets
Highly Ineffective
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
They don't have a clear picture
They have a holistic picture of their overall IT performance thanks to one or few regular
concise reports
They do, but a lot of effort goes into consolidating various reports into one view
They could, but information is fragmented and dispersed over tools and reports
Percentage 2009 Percentage 2008
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews amongst IT executives and consultants
IT Performance Management market maturity
Level 0 IT Performance Management is non-existent.
Level 1 There is ad hoc monitoring in isolated areas.
Level 2 Some measures are set with a clear link to
business goals but are not communicated.
Measurement processes emerge, but are not
consistently applied.
Level 3 Efficiency and effectiveness are measured and
communicated and linked to business goals and
the IT strategic plan. Continuous improvement is
emerging.
Level 4 There is an integrated performance measurement
system linking IT performance to business goals by
global application of a documented framework.
Continuous improvement is a way of life.
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Level 0 Level 1 Level 2 Level 3 Level 4
IT Performance Measurement is emerging and the basics are getting in place but
there is no proper communication as yet
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews amongst IT executives and consultants
Note: this is a self assessment
14
IT Performance Management market maturity
0% 20% 40% 60% 80%
Other
Be compliant with internal or external regulations
Identify internal improvement opportunities
Require an insight into performance against service levels
Optimal resource allocation
Establish the progress toward achieving goals
Reduction of IT cost
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews IT executives and consultants
The reduction of cost and the creation of clarity on the performance of IT in relation
to business goals are the main drivers for IT Performance management initiatives
15
IT Performance Management market maturity
Organizations look at IT best practices for support in identifying metrics and realize
they need to have both resources in place and buy-in from the business
Source: “Trends in IT Performance Management,” 2008/9, ITpreneurs
Survey and interviews amongst IT executives and consultants
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Other
Proper guidance on the implementation
Having a useful software application to capture and convey performance measures
Having an industry standard for metrics (common language)
Having support from the business
Having the proper resources in place (budget, people, infrastructure)
Having a formalized service management framework in place (ITIL, COBIT)
percentage 2009 percentage 2008
16
IT Performance Management market maturity
Source: “Trends in IT Performance Management,” 2009, ITpreneurs
Survey and interviews IT executives and consultants
Manual reporting using Excel is favored in most organizations today followed
by the reporting capabilities provided by service management tools
0% 10% 20% 30% 40% 50% 60% 70% 80%
IT performance solution from external vendor (Metricus, M42 )
We don't use any tools to report IT performance
Other
Service Level Management Monitoring Tools (Digital fuel, Oblicore)
Business Intelligence (e.g. IBM, Hyperion, SAP)
Internally developed software solution
IT service management solutions (HP OpenView, BMC Remedy)
Manual reporting (e.g. Excel)
Percentage 2009 percentage 2008
17
IT PERFORMANCE
MANAGEMENT PROCESS
Session 3
18
The IT Performance Management Process
IT Performance Management is about measuring, improving, and
demonstrating the value of IT
“IT Performance Management is the effective combination of methods,
metrics, data, and tools that enables organizations to define KPIs that are
relevant to them, understand their current performance against
predetermined goals, and enables organizations to build on this information,
initiate improvement activities, and achieve optimal IT performance in line
with business requirements”
* Metricus definition for IT Performance Management
19
The IT Performance Management Process
Performance measurement is the process of assessing progress toward achieving
predetermined goals*. Performance Management builds on that process, adding the relevant
communication and action on the progress achieved against these predetermined goals.
20
Measure Manage Improve
IT Performance Management is closely related to IT performance
measurement.
They are sometimes mistaken for each other. Strictly speaking, Performance
Management is the larger domain and includes performance measurement as a
component.
IT Performance Management
IT Performance Measurement
Define
* Wikipedia
The IT Performance Management Process
21
Define
Measure
Manage
Improve
Define what is important
to you and what you
should measure
Measure your IT performance
through best practice KPIs
and performance data
Manage the ongoing process and
present decision making
information to relevant
stakeholders in your organization
Initiate improvement
activities based on historic
data and scenario planning
In this workshop we will touch on the first three phases of the IT Performance
Management lifecycle: define, measure and manage
What drives IT Performance Management
Just like other business departments, IT has to continuously improve and ensure alignment with
the business
Ultimately the only way for IT management to demonstrate value and control is by defining,
measuring and managing IT performance
A great idea, but the idea often gets stuck at not being able to successfully measure IT
performance and not being able to bring everything together into a view that allows the IT
management to take informed IT decisions.
What is not defined cannot be controlled.
What is not controlled cannot be measured
what is not measured cannot be managed or improved
“
”
22
What drives IT Performance Management
23
Business
Strategy
IT Strategy
IT Processes
IT ActivitiesIT Performance
Analysis
ITIL/COBIT
Process
Scorecards
CIO Dashboard /
IT process health
IT Balanced
Scorecards
drives
drives
drives
IT Performance
Metrics and
Measures
Business Strategy drives IT
strategy and results in
defining, managing and
optimizing IT processes and
activities
This needs to be balanced
against the availability of
quality data and processes
that can provide information
from the bottom up
Structured KPIs drive performance
management for various users
24
IT KPIs (that are aligned with the business strategy) play a role at each
decision-making level in the organization and enable the organization to
take informed IT decisions in line with business requirements
Business Leadership
IT management
IT departments
IT Balanced
Scorecard
IT process health
dashboards
Functional and
process scorecards
Timely,
accurate,
and
quality
IT metrics
and
measures
(KPIs)
Business
Strategy
IT
Strategy
Processes and
Activities
drives
drives
From Measures to Decisions
25
Measures KPIsOperational
Data
Dashboards
Scorecards
IT Balanced
ScorecardDecisions!
An Operational Data Source (ODS) represents data collected by a system or
process associated with the delivery and support of IT services.
A comprehensive understanding of ODS's within the context of the information ecosystem of an
organization is essential to the structured definition and development of IT Service Metrics.
e.g. FrontRange HEAT,
HP Service Manager,
ServiceCenter, CA
Unicenter, Service Plus,
Servicedesk, Numara
Footprint / Trackit, In
house applications, Open
Source applications
e.g. IBM Tivoli, HP
OpenView suite, CA
Unicenter, Nagios, Open
Source Applications
Enterprise Systems
Management
Decentralized,
disparate data related
to specific
users/processes
e.g. SAP, Oracle Financials
Enterprise Resource
Planning
Internal IT applications
developed specifically
for IT support/delivery
functions
Service Desk Manual Custom
ODS examples
Operational Data Source
26
IT
processes
and
activities
Data
WarehouseIT strategy
Data
Integration
KPI based decision
making information
Insight
Change
Data
integration
process
?
Data Management
Business objectives change, which has an impact on the requirements
for data. Thus keep in mind that this process is dynamic and requires
continual management
27
Measures are distinct sets of data derived from mathematical calculations.
Measures are quantifiable, for example, size, volume, or percentage,
and involve aggregation of data elements, for example, sum, average,
min, max, or count.
In and of themselves, measures may or may not be meaningful.
However, measures represent building blocks for the metrics
required to make business decisions.
Examples:
Calculation Measure
Sum (incidents) Volume of incidents
Sum (Incidents Resolved) where „Met Target‟ = „Y‟ Volume of incidents resolved within target resolution time
Average (Database Availability) % Database Availability
Sum (IT Resources) where „Trained‟ = „Y‟ Number of IT resources trained
Measures
28
Metrics consist of one or more measures combined with a mathematical
calculation and a standard presentation (format) for the output.
Volume of Incidents
Volume of Incidents Resolved
Within Target Resolution Time
Measures
Categorization
Priority = „Critical‟
Time
Daily
Dimensions
Metric
% Critical Incidents
Resolved
Within Target
division
facilitates
comparisonfunctional
categorization
Metrics are associated with two dimensions: a time dimension and a functional categorization
dimension.
Metrics are used in the quantitative and periodic assessment of a process that is to be
measured.
Metrics should be associated with targets that are set based on specific business objectives.
Metrics are associated with procedures to determine the measures required and procedures for
the interpretation of metric results.
Metrics
29
Acronyms that are used for „metrics‟
30
Impact related - Key Performance
Indicator (KPI), Critical Success Factor
(CSF), Outcome Measure, Performance
Indicator
Operational related – IT Service Level
Agreement – SLA, IT Operational Level
Agreement (OLA), and Service Level
Objective (SLO)
Term „Metric‟ often used interchangeably
with „Target‟, „Benchmark‟ and „Goal‟
Clear definitions are required in order to
facilitate the necessary communications,
and to set appropriate expectations, with
users.
IT Performance
Metric
CSFOutcome
MeasureKPI
OLO SLA SLO
Numerous terms and acronyms exists for labeling „metrics‟
Characteristics of good metrics
31
Metric Presentation
32
Concept Description
Dashboard • A dashboard is a graphical display of the status of a
selected set of key metrics
Scorecard • Scorecards are the consolidated tabular and graphical
display of sets of metrics related to particular business
functions.
Visualization • Visualization reporting solutions show relationships
between selected metrics and assist in performing
impact analysis
Interactive Data Analysis • Online interactive analysis of data providing „slice and
dice‟ and „drill-down‟, „drill-through‟ capabilities. Known
as OLAP (Online Analytical Processing).
Scheduled Reports • Individual reports related to specific data requirements.
Operational Reports • Reports available directly from operational systems
used to support processes
Monitoring • Near real-time display of detailed data related to the
delivery of a service.
Metric Presentation
33
Concept
Dashboard
Scorecard
Visualization
Interactive Data Analysis
Scheduled Reports
Operational Reports
MonitoringOperations
Executive
Managers
Functional
Managers
Process Managers
In theory various users in the organization have access to
information that is timely and relevant only to them
Metric Presentation
34
Concept
Dashboard
Scorecard
Visualization
Interactive Data Analysis
Scheduled Reports
Operational Reports
MonitoringOperations
Executive
Managers
Functional
Managers
Process Managers
In practice many IT managers want to have access to very
detailed reports. Also, scorecards or interactive data reports are
only sporadically used
“A dashboard is a visual
display of the most
important information
needed to achieve one or
more objectives;
consolidated and arranged
on a single screen so the
information can be
monitored at a glance”
Intelligent Enterprise, 2004
Green IT dashboard
Dashboards
35
Highly visual, data aggregated from
various sources
Provide a one glance view on the
current state of the
organization/process or activities.
Typically used by CIOs, IT
Management and business users.
„Scorecards‟ focus on the consolidated presentation of metrics and
presents an accurate view of the here and now compared with predefined
goals
Graphical representation of trends
and alignment with defined targets is
provided.
They are supported by static and
interactive reports, as well as
diagrammatic representations of
metric performance and linkages
between metrics.
Categorization of metrics within a
scorecard is typically related to
functional or process views.
Service Desk Scorecard
Metricus Scorecard from the Service Desk Module
Scorecards
36
DEFINE REQUIREMENTS FOR
IT PERFORMANCE
MANAGEMENT
Session 4
37
Method 1) GQM – goal, question, metrics
Source: L.Buglione & A.Abran © 2000
GQM is a technique to derive measures for project control starting from
high-level goals, passing through the decomposition in several questions
to answer.
Author: Victor Basili in the early „80s (NASA Goddard Space Flight Center)
Measurement object: software projects
Application: GQM is one of the most well-known and used measurement
approaches for establishing a measurement program
38
Method 1) GQM – goal, question, metrics
Process Governance
Complete Correct & Valid Detailed
Are mandatory
fields filled out
Are data entered according to process policies & standards
Do text fields contain
appropriate level of
detailsDoes the data reflect what actually happened
Process Effectiveness
Incident Problem Change
Reduce MTTR Decrease days to root cause determination
Decrease problems due to change
Decrease days to implement permanent fixes
Business Objectives
Increase Pace ofService Delivery
Improve ServiceAvailability & Stability
Deliver Projectson Time / on budget
… Reduce outage minutes
Reduce number of incidents
…
Reduce MTBF Decrease issues during change implementation
Using the Goal Driven Method
Source: David Chiu of BMO Financial 39
Method 2a) Balanced Scorecard
BSc is a multidimensional framework for “translating (organizational) strategyinto action” at all levels of an enterprise, by linking objectives, initiatives and measures to an organisation‟s strategy
Authors: Kaplan & Norton (HBS) in the early „90s, originating from a French methodcalled Tableau du Bord (turn of 20th century)
Main measurement object: whole organisation / a SBU
Application: BSc is one of the most relevant management practices of last 75 years, according Harvard Business Review Journal
Source: L.Buglione & A.Abran © 2000 40
Business strategy
IT strategy
Method 2b) Balanced Scorecard and COBIT
COBIT provides a bridge between business risks, control needs and technical by
providing a comprehensive framework for delivering value while managing risk
and control over data and information throughout the lifecycle.
Authors: ISACA in the „90s, originating from being an „auditors check list‟
Main measurement object: The entire IT organization: IT resources need to be managed by a set of naturally grouped processes. COBIT provides a framework that achieves this objective.
Source: COBIT 4.1, ISACA 42
Method 3) COBIT
Aligning business goals with IT goals using COBIT, example for DS5
Source: COBIT 4.1, ISACA 43
Assignment
Business Goals
IT Goals
IT process goals
IT KPIs
Step 1 define business goals
We are going to use the following model from COBIT to define a process based KPI
scorecard
IT
management
IT
departments
IT process
health
dashboards
Functional and
process
scorecards
drives
Step 2 define IT goals
Step 3 process
goals
Step
4
KPIsStep 5
Scorecard
44
Use the following template for step 1, 2, 3
45
Business goal
IT goals
Processes
45
Case description
At the Coffee Company, coffee is not merely a bean or a beverage.
It is the furthest thing from a product or a commodity. Coffee
involves relationships and responsibilities. It is a process involving
high standards and tough decisions. More than anything, coffee
has the connective force to enrich peoples‟ lives.
The Coffee Company is a gourmet coffee company, serving high-
quality coffee blends and a specialty selection of coffee by the cup.
Espresso and related food and beverage products complement the
offerings. Along with hot and cold liquid servings, the Coffee
Company offers gourmet beans in retail bags through a retail
channel as well as an online channel. All offerings are provided in an
atmosphere that whispers of the warmth and convenience of your
living room.
From 1991 to date, the Coffee Company experienced and realized
significant growth in size, revenue, and number of stores. From only
one store in downtown Toronto, the company has grown to 12,000
retail stores and 1 online store.
the current economic situation has made the Coffee Company
shift its focus from expansion to consolidating business with
its existing stores and focus on ensuring the uninterrupted
availability of its coffee products and services.
The coffee production process
Farming,
drying,
extractingRoasting Distribution
Retail
stores
Online
store
customers
After roasting, coffee is
packaged and sent to
one of our five regional
distribution centers in
Vancouver, Rotterdam,
Singapore, London and
Houston. The distribution
centers supply our retail
stores manage the
distribution from the
regional internet sales.
Our outlets are company
owned and franchise
locations worldwide. The
distribution centers
supply our stores twice a
week or more depending
on demand. Each
retailer is strongly
connected to the
community as well to
ensure delivery of fresh
products such as breads,
bagels, etc.
Coffee grows best in a
warm, humid climate
with a relatively stable
temperature of about
27C all year round.
The world's coffee
plantations are
therefore found in the
so-called coffee-belt
that straddles the
equator between the
tropics of Cancer and
Capricorn.
Un-roasted 'green' beans are
shipped to Rotterdam in the
Netherlands and Vancouver,
Canada from locations as
close as Hawaii and as far
away as Indonesia. Varietals
are then roasted or mixed
with other beans to create
blends, such as our popular
VOC or Da Vinci blends.
Fragmented information from IT
Retail store managers are getting
tired of constantly having to wait
more than 5 minutes on the
telephone with the service desk.
Before they get someone on the
line..They have a shop to run!
A change weekend is always
exciting for the IT organization..
Will there by any store with
problems with their online
systems or does everything go
well? In case of issues, the
challenge is to find out what went
wrong and how to revert. Testing
plans are not maintained
centrally and there is no review
of implemented changes
Our CMDB is not very
easy to use and as a
result its‟ not kept up to
date alwaysI am the incident manager, and I
am proud of closing incidents
within targets by my first level
support team
Change management is a
rush job, and our change
manager agrees to all
requests from the
business as they feel
everything is important.
So as a result they
don‟t call but bypass
the service desk and go
directly to the
specialists
It seems that we keep
reopening incidents as our
solutions don‟t seem to
work, what‟s going on. We
need a knowledge base..
CREATE AN IT KPI BASED
SCORECARD
Session 5
49
Tips and Tricks for selecting KPIs
CIO/Executive
ManagementConsulting
“Middlemen”IT Service
Transition Team
1. Show me how
well we are
managing issues
in IT
2a. Sure! Will
this scorecard
do?2b. We don‟t know
if it‟s possible but
that‟s not our
problem, we need
the sale.
3. We don‟t
have the
processes or
data in place…
There is a gap between theory and practice, and there are not that many metric practitioners out
there
What looks and sounds nice is often not practical and possible.
Don‟t be put in a situation with „middlemen.‟ Work direct with the users of KPIs to ensure that
what is requested is actually feasible.
When you are presented with a KPI wish list that looks too good to
be true, it probably is.
50
Tips and Tricks for selecting KPIs
51
Use only use KPIs when you know the data is available and only use
KPIs if the results can be tied to higher level operational, tactical or
strategic objectives
The horse and the cart: Only use metrics when you
know the data is available.
Confirm with all stakeholders the actual need and
planned use of each of the reported metrics
The requirement for metrics should not drive the
requirement for data; process and strategy should.
Tips and Tricks for selecting KPIs
52
Ensure that your „effort-to-insight ratio‟ remains positive
The cost for collecting data for a metric should weigh up against the insight that you are
retrieving from the metric.
The cost categories are data collection, Business Intelligence, and report development.
Costs are predominantly qualitative whilst benefits are both qualitative and quantitative.
Cost of
data collection
Insight
provided by
the KPI
Tips and Tricks for selecting KPIs
53
Garbage in = Garbage out: Poor data quality in IT services negatively affects
the ability for informed decision making throughout the entire organization.
Data Quality
Quality of
IT Metrics
Low
Low High
High
IT metrics should be developed and
presented with the same rigor as financial
accounting metrics.
Consider: Integrity of data, consistency
with prior periods reporting, materiality
(Value of data must exceed cost of
reporting)
Tips and Tricks for selecting KPIs
54
Focus on consolidating the data required to
support metrics and scorecards.
Don‟t assume „one approach fits all‟ for data
integration.
Build a 'data resource network,‟ including
developers, support and operations (DBAs),
and application SMEs.
Word
Excel
PowerPoint
Face to Face
Telephone
Meetings
Memos/Letters
Help Desk
Monitoring
Systems
Unstructured Structured
Recognize that data integration will be the greatest challenge in developing
KPIs, scorecards, and supporting reports.
Tips and Tricks for selecting KPIs
55
Less is more, and it is recommended to start with only a handful of KPIs
rather than going with dozens of them
KPIs are put in place to be used to manage and
too many KPIs make managing to them difficult
For every KPI processes need to be in place to
collect data for every KPI and do something
with the results of the KPI.
Start small, keep it simple, and build upon
achieved successes.
Recognize that metrics will change over time
based on the changing values required from the
metrics and the growing maturity of the
organization
Tips and Tricks for selecting KPIs
56
Audience Rationale
Executive Senior
Management
• Implementing metrics and scorecards solutions is a strategic initiative that
requires C-level sponsorship (CIO, CEO).
• This is generally recognized as the #1 Critical Success Factor to incorporate
any Performance Management Initiative or implementing Business Intelligence
software
Identify Key People • The people that really matter to a process must always be part of it. This is
often a group so small that you can call the members out by name and count
them on one hand.
• Key people need to be identified for requirements gathering, ownership of
metrics and reports, assistance with developing metric/scorecard solutions,
assistance in architecting and configuring the infrastructure required, and
support for data collection and metrics presentation.
• Performance Management software solutions can provide guidance and
templates for scorecards and reports for specific users in your organization
Identify Metric Users • Ask users what they need!
• Collect key information for users.
• Map metrics, scorecards, and reports to users.
• Develop an authorization and application security model.
• Keep it simple! Start on the premise of allowing users to see all information
and selectively revoke rights where appropriate.
Involve the right people when you are designing your IT Performance
Management initiative
Metrics that are used need to be SMART:
Specific
Measurable
Action oriented
Realistic
Time bound
Now we have identified the Processes that have our key interest and that
are important for us when we want to meet the objectives of the business.
The next step is the identification of specific KPIs for the selected
processes.
First some guidelines around metrics:
Metric Requirements - SMART -
57
The definition of an IT Performance Metric needs to be structured in alignment
with defined attributes and valid measures
Good Example Poor Example
[% Calls Abandoned]. A fundamental Call
Center/Service Desk metric, this divides the
number of calls offered by the number of calls
abandoned. Both measures are readily available
and attributes such as time dimension,
categorization fields, targets, user base, and so on
are straightforward to obtain. (To be discussed
more in the section – „IT Performance Metrics –
Attributes.‟
Time used to resolve unavailable services.
What services are included? What is the definition of
unavailable? How is unavailable time equated with
effort to resolve? Is it linked through incidents? If so,
are the processes in place to record time?
This helps avoid the „100,000 feet‟ syndrome of documenting
„wish lists‟ of what sound like nice metrics to have, but in reality
are not meaningful, not possible, too costly, and so on.
Metric Requirements - SMART -
58
IT Performance Metrics need to be quantitative, allowing users to actually
measure progress and performance against the IT Performance Metric .
The provision of an IT KPI should prompt the users to ask
questions.
Supporting data needs to be directly available and/or
mechanisms available to obtain support data.
Typically, quantitative metrics are supportable while qualitative
metrics are hard to support.
Good Example Poor Example
[SRs Open] is a valuable IT service metric
but needs to be supported by details of
open service requests.
% Projects with Predefined Benefit
is often based on after the fact
„guestimates‟ as to whether predefined
benefits were available. For the metric to
be of value, information on what predefined
project benefits need to be stored in
conjunction with baseline project details.
Metric Requirements - SMART -
59
An IT Performance Metric should contain information that can be directly acted
upon.
This could involve questions being asked as to why a metric is a specific
value, providing information on achieving defined levels of service, or
automatically instigating remediation action within a particular IT service
process.
Good Example Poor Example
[% Service Requests Resolved On
Initial Contact] is often a metric contained
within a Service Level Agreement (SLA).
Failing to meet a defined contractual
target, say 85% will require action to be
taken in order to avoid possible penalties.
Incidents Created
While this is often interesting to look at,
particularly across a period of time, and it is
certainly easy to measure, it‟s hard to make
an educated decision based solely on the
results in isolation. This is actually a
measure that needs to be categorized with
IT performance reference data, such as
service or classification, in order to be of
value.
Metric Requirements - SMART -
60
An IT Performance Metric should be realistic from a data perspective, that is, the
data associated with underlying measures in the calculation of the metric needs
to be available.
An IT Performance Metric also has to be justifiable from the initial and ongoing
costs. The effort in collecting the metric logically is lower than the value derived
from decisions related to the metric.
Good Example Poor Example
[% Incidents Escalated]
The volume of incidents and an indicator if
an incident was escalated should be
readily available from the operational data
source associated with Incident
Management.
% of Problems Recorded and Tracked
It is difficult to detect that a problem is not
recorded or tracked unless it actually has
been recorded and tracked!
Metric Requirements - SMART -
61
An IT Performance Metric should be collected at regular time intervals, that is,
have an associated time dimension.
This may be hourly, weekly, monthly, quarterly, and so on.
Metrics without a time dimension should be referred to as milestones.
Good Example Milestones
[% Incidents Caused by Changes] is a
metric that should be analyzed over time
to ensure a downward trend. Data should
be available every day.
% of Data Elements Contained within
Enterprise Data Model. The creation of
an Enterprise Data Model is a largely
investigative activity aimed at
understanding what data elements exist.
Consequently, the denominator for this
metric, the number of data elements,
cannot be determined until an Enterprise
Data Model actually exists. As a result, this
should be an IT Performance Milestone –
„Enterprise Data Model Complete.‟
.
Metric Requirements - SMART -
62
Assignment
Business Goals
IT Goals
IT process goals
IT KPIs
Step 1 define business goals
Based on identified processes, we are going to look at and select a number of IT KPIs
and create a scorecard for this:
IT
management
IT
departments
IT process
health
dashboards
Functional and
process
scorecards
drives
Step 2 define IT goals
Step 3 process
goals
Step
4
KPIsStep 5
Designing a
Scorecard63
Assignment
Step 1: identify the most relevant KPIs for
Change, Service desk / incident management
Process name:
Metric Importance (1-10) Rationale (why)
Step 2: write down which KPIs
are most relevant for your IT KPI
based scorecard and why
64
Note: keep the
„tips and tricks‟ in
mind!
Use the following template for step 4
Process name:
Metric Importance
(1-10)
Rationale (why)
65
% IT Costs - Travel Reduce travel cost by providing viable alternatives, such as Web conferences or VOIP and e-learning
education instead of classroom training.
Energy Cost per IT Facility Energy cost per IT facility gives insight into the energy cost per facility. Monitoring this information
periodically helps organizations in optimizing the energy consumption.
% Projects Related to
Business
Projects should not be created if they are not directly related to specific IT or business objectives. The
combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%.
% Inactive Application
Accounts
Inactive accounts represent overhead on security systems, both from a performance and an
administration perspective. Importantly, they make the task of auditing security more difficult due to
unnecessary overhead. If effective security management processes are in place, the number of
inactive accounts should be minimal.
% IT Resources - Contractors IT departments will almost always have a requirement for contractors. The level of contractors needs
to be balanced with the potential impact on overall IT resource moral as well as costs associated with
projects.
The % of IT costs that can be directly connected to business value drivers, for example, sales/services increase due to increased connectivity
This is a difficult metric to measure; however, the actual process of attempting to measure this metric will yield results through mapping IT costs to business value drivers.
The average cost per user license for a particular application and types of applications
It is common for an IT organization to purchase far more licenses than required. For example, it is estimated that 80% of Business Intelligence software licenses are not used. Reducing the average cost per user license is a combination of ensuring that the correct number of users are determined and that costs are negotiated efficiently with IT vendors.
A measure indicating how satisfied the business is with the allocation of costs from IT
This metric can be used to measure the effectiveness of the allocation of costs from IT to the business. While subjective in nature as the business will rarely be satisfied with IT costs, it can be used to gauge improvements in the distribution of costs.
The % of IT staff that has participated in IT-related training over the last 12 months.
It is important within any organization that resources receive the necessary training to ensure that they are aware of current knowledge and to assist with career development. This metric provides details on what % of IT personnel have received training over the previous 12 months. The level of training is also significantly proportional to effectiveness and the costs associated with IT support.
KPI Examples: Cost Control
% Availability SLAs Met The % of services delivered for which the requirements in the Service Level Agreement for the availability of
services is met
% CIs Monitored The % of configuration items that are actively monitored by an infrastructure monitoring tool. Only configuration
items that are candidates for monitoring i.e. infrastructure components such as servers, networks and databases
are included in this metric.
% Database Backup Failures The number of scheduled backups that failed in a selected time period
% Database Restoration
Failures
The % of database restoration attempts that failed during a selected time period.
% Databases Backup/Restore
Verification
The % of databases that are scheduled for backup for which backup testing and verification of restoration has
been performed
% Databases Scheduled for
Backup
The % of databases that required backup that have successfully been schedule for backup.
% Server Backup Failures The % of total server backups that failed during a selected time period. Failure is defined by the non completion
of a backup. This includes both scheduled backup's that did not start and backups that failed during the backup
process.
% Server Restoration Failures The % of server backup restoration attempts that failed during a selected time period
% Servers Backup/Restore
Verification
The % of total servers that have had a full backup and restoration tested and validated.
% Servers Scheduled for
Backup
The % of services in an IT infrastructure that are scheduled for regular backups. The nature of backups will vary
across servers based on the software and applications hosted on a server. This metric looks at all servers and
provides an enterprise IT perspective.
Average Training Days for IT
Resources
The average number of training days for IT resources over the previous 12 months
IT Continuity Plan Reviews Number of reviews per period of the IT continuity plan
KPI Examples: Ensure continuous service
Projects Completed on Time The number of projects completed that were completed within the target end date.
Projects Completed Within Budget The number of projects completed where the total actual costs were less than the total budgeted costs.
Projects Completed on Time and
Within Budget
The number of projects that were completed by the target completion date and had total actual costs less than
budgeted costs.
Projects Open with Milestones
Missed
The number of projects currently open where one or more milestones related to the project have not been met.
Projects Scheduled The number of projects at a given point in time that are scheduled to be started in the future. Scheduled projects
are those that have been created, subsequently approved, but not yet commenced.
Projects Open The number of projects that are currently open, that is, implementation or development has started but has not
been completed.
Projects Completed The number of projects marked as completed during a specified timeframe. The definition of completed is that
the intended functionality to be provided by the project has been accepted by the users of the functionality.
Projects Championed by Business The number of IT projects created were initiated from the business and that have funding and full support from
the business.
Projects Completed with
Documentation and Testing Plans
A measure of the number of projects that have appropriate documentation associated with implementation tasks,
such as testing and post-implementation tasks, such as support and training.
Projects Created The number of new projects created.
% Projects Related to Business Projects should not be created if they are not directly related to specific IT or business objectives. The
combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%.
Projects - Predefined Benefit The number of projects created that have specified defined tangible benefits to the business or IT if successfully
implemented.
KPI Examples: Project Management
% IT Processes Meeting QA
Objectives
The % of IT processes that meet the QQ objectives that were defined
% IT Processes Reviewed by QA The % of IT processes that have been reviewed by conducting a formal QA
% IT Resources - Quality Training The % of IT resources who have received IT Quality related training within the previous 12 months.
% Projects Meeting QA
Objectives
The % of IT projects that meet the QQ objectives that were defined
% Projects Receiving QA Review The % of IT projects that have been reviewed by conducting a formal QA
% Stakeholders Understanding IT
Policy
The % of stakeholders that understand the IT policy
KPI Examples: Manage Quality
% CIs with Operations
Documentation
The percentage of Configuration Items that have associated operations documentation
% Incidents Resulting from
Inadequate Documentation
The percentage of incidents that can be appointed to inadequacies in the documentation made available with
new or changed applications
% SRs Resulting from Inadequate
Training
The % of SRs created due to the requestor not having sufficient training in functionality related to an IT service
e.g. configuration of new email accounts in Outlook or accessing functionality within an ERP system or on the
corporate portal.
% SRs Resulting from Lack of
Documentation
The % of service requests created as a result of lack of documentation related to an IT service. For example, a
user calls the service desk because they cannot find information on the corporate portal on how to connect to the
instant messaging system.
% Training Attendance New
Applications
The % of stakeholders that attended training for a new application after release
Average Change Documentation
Update Time
Average time it takes for documentation to be updated after a change
Customer Satisfaction -
Training/Documentation
The average customer satisfaction score for IT Training/Documentation surveys sent in a selected period. This
is representative of the overall satisfaction and confident that the business has in IT's ability to delivery training
and IT related documentation required by the business to use IT services. A good range of values for
measurement is 0 to 10
Incidents Resulting from
Inadequate Documentation
The volume of incidents that can be appointed to inadequacies in the documentation made available with new or
changed applications
SRs Resulting from Inadequate
Training
The volume of Service Requests that can be appointed to inadequacies in the training that was presented to
stakeholders after release of the new application or after a change made to the applicatoin
KPI Examples: Enable Operation and use
Changes Created The number of requests for change (RFC's) created. In IT Service Support, a change is the addition,
modification, or removal of approved, supported or baseline hardware or software components. This can
include network, application, environment, and system components, or other IT components, including
documentation. All changes should relate to a configuration item.
Changes Implemented The % of critical priority changes implemented within an agreed target time.
Average Change implementation time The average length of time required to implement requests for change.
% Changes Implemented within Target - Critical The % of critical priority changes implemented within an agreed target time.
% Changes Implemented within Target - High The % of high priority changes implemented within an agreed target time.
% Changes Implemented within Target - Medium The % of medium priority changes implemented within an agreed target time.
Average Cost per Change Average cost per change
% changes failed The % of changes that failed during the implementation phase of Change Management.
% emergency changes The % of changes implemented that classified as emergency changes. Emergency changes are those
which require circumvention of routine change management processes due to the urgency of business
requirements and changes to the IT infrastructure. A comprehensive Change Management process will
include a process for handling Emergency Changes
% high risk changes The % of changes created where the risk of incidents occurring within the IT infrastructure is high. A
comprehensive Change Management process will include a process for handling High Risk Changes
% changes rejected The % of change requests that are analyzed by IT and subsequently rejected.
% changes due to CMBD issues The % of changes that were made as a result of incorrect information provided by the CMDB
% changes post implementation feedback The % changes that received feedback after completion of the implementatoin
% changes post implementation review The % of changes that were reviewed post implementatoin
% changes process compliance The % of changes that correctly followed the defined change management process
% changes audited The % of changes that were audited post implementation
% changes causing incidents The % of changes implemented that resulted in one or more incidents occurring.
% changes causing problems The % of changes implemented that resulted in one or more problems incurring
% changes implemented without a back out plan The % of changes that were implemented without a defined back out plan
KPI Examples: Change Management
% changes implemented without CI
testing
The % of changes that were implemented without proper CI testing
% changes implemented without testing The % of changes implemented that did not go through the formal testing phase incorporated within the
Change Management process.
% changes requiring scheduled outages The% of changes that require a service outage to be scheduled in order for implementation of the change to
occur.
% changes specified inaccurately The % of changes that were not specified correctly
% changes with incorrect data The % of changes that were closed but contained 1 or more incorrect data components e.g. wrong
categorization fields entered, incorrect history, timestamps entered incorrectly, missing solution or closure
description, etc.
% Changes without sign off The % of changes that were incorporated without formal sign off
KPI Examples: Change Management
% IT Level 1 Support with
Backup
An important IT metric that is often used as a benchmark between IT organizations. The lack of backup
personnel can directly effect the delivery and support of IT services, which in turn can effect the overall
operations of an organization. This is a particularly important metric for the Service Desk because of the
customer visibility involved.
% SRs Resolved on Initial
Contact
An important service desk metric as the ideal scenario is that service requests related to phone calls are
resolved during the first call, minimizing the impact on the customer and the amount of process required by IT
support. This metric is also a key driver of customer satisfaction because it is an 'interaction' metric - one
whereby the service desk is exposed to the customer.
IT Resources - Level 1 Support The number of IT resources classified as Level 1 Support. Level 1 Support typically refers to agents at the
Service Desk who are the first point of contact for service requests, incidents, and change requests from the
business. They will try to resolve 'how-to' questions, routine simple changes, such as password resets, and
ensure that more complex issues are assigned (dispatched) to the correct Level 2 and 3 resources.
% Calls Abandoned Provides insight into how efficient the Service Desk is at answering IT support telephone calls from customers. A
high abandonment rate may indicate inadequate staffing or inefficient processes in terms of the collection of
information from the customer. Also, an important metric in terms of customer satisfaction because while small
minorities of these abandoned calls are accidents or mistakes, the majority represent either dissatisfied
customers (those who hung up) or customers who are about to become dissatisfied (those whose calls were
terminated).
Calls Answered Measures the number of calls that were answered by the Service Desk. This is related to <Calls - Offered> and
<Calls - Abandoned> in that the difference between <Calls - Offered> and <Calls - Abandoned> are mostly the
calls that were answered (the exception are the calls that were terminated during the initial pre-recordings). So,
as the % of calls abandoned decreases, the % of calls answered should increase. Each call answered becomes
a candidate for a service request, incident, or change request to be created.
Call Answer Time A measure of how long it took to be answered from the time the customer was placed in a queue to the actual
answer time. The time placed in a queue is normally immediately after the automated menu messages.
Customer Satisfaction - Service
Operations
The results of survey responses for Service Delivery surveys sent.
Avg Daily Incidents Handled per
Service Desk Agent
Relates to the calls that Service Desks can handle and is an important metric relevant to the planning of Service
Desk resources.
Average Cost per Call Average cost per call to the Service Desk.
% IT Staff Turnover - Level 1
Support
High Service Desk staff turnover will have a negative effect on key service request resolution metrics, such as [%
SRs Resolved on Initial] and [% SRs Resolved within Target].
KPI Examples: Service Desk
% Incidents Misrouted 'Misrouted incidents add to the overall resolution time and, as such, impact the customer and add cost to IT operations. A high number and
% of misrouted service requests could indicate fundamental flaws and problems in the incident management process.
% Incidents Reopened Primarily an operational metric that provides information on the effectiveness of the Service Desk and IT Operations resolution process.
The ideal process is that an incident is resolved and then automatically closed within a set timeframe. If an issue recurs, a new incident is
created. Incidents should not be reopened after initial resolution if the issue logged has been appropriately actioned and resolved.
% Changes Causing Incidents If the change management process is functioning effectively and the CMDB is accurate, this metric should be very low because all changes
should occur during scheduled outage times. Note: When analyzing this metric, care needs to be taken as if the incident root-cause
analysis process is failing then this metric will be artificially low.
% Incidents Dispatched to Level 2 Costs associated with level 2 are typically higher than for level 1 due to the additional expertise required. As IT operations becomes more
efficient, this metric should fall, resulting in faster resolution times and reduced costs.
% Incidents Resolved by Known Errors Known error information assists in the incident resolution by providing an IT operations resource with answers to questions previously
asked. This assists in achieving target resolution rates and reducing overall costs. This is stated as an 'above target' metric, meaning that
while the overall volume of incidents and incidents resolved by known errors should reduce, but the % of incidents solved by known errors
should increase.
% Incidents Dispatched to Level 3 Costs associated with level 3 are typically higher than for level 1 and 2 because of the additional expertise required. As IT operations
become more efficient, this metric should fall, resulting in faster resolution times and reduced costs.
% Incidents Resolved by Workaround While workaround solutions play an important part in maintaining the overall stability of IT services delivery, this metric should decrease as
an IT environment matures.
% Incidents Dispatched Provides an indication of IT operations efficiency in processing incidents. Ideally, over time, the volume dispatched decreases as a % of the
total incidents created.
% Incidents Void Void incidents should be monitored to ensure that upward deviations from longer term averages do not occur. Such deviations may
represent data quality or data entry problems. They could also highlight issues with automated incident creation mechanisms.
Average Incident Dispatch to Own
Duration
This metric indicates how responsive IT support staff are at accepting ownership of incidents. Increasing trends can indicate issues with the
process of service request ownership. Hourly or daily spikes can indicate staff rostering issues. Over time, the [Incidents - Dispatch to Own
Duration] should decrease, but increase relative to the [Incidents - Create to Dispatch] duration. This will result in the [Incidents - Create to
Own] duration decreasing.
% Incidents Owned within Target -
Critical
Prompt ownership of critical priority incidents indicates efficient service support processes and increases the probability that incidents will
be resolved within the target time. Ownership of critical priority incidents within target has increased importance due to the customer
visibility into processes within IT, that is, because of the potential business of critical priority service requests, the customer will have high
expectations regarding ownership and subsequent resolution.
Incidents created The volume of incidents that are created
% incidents resolved by 1st
level The % of incidents that were resolved by the first level support team
% incidents resolved within target critical The % of critical incidents resolved within the defined target time. Also referred to as 'Resolution Met' or „TTR Met' (Time to Resolve Met).
A typical target time for resolution of critical priority incidents is 2 hours.
KPI Examples: Incident Management
% incidents resolved within target – high The % of high priority incidents resolved within the defined target time. Also referred to as 'Resolution Met' or
„TTR Met' (Time to Resolve Met). A typical target time for resolution of critical priority incidents is 2 hours.
% incidents resolved within target – low The % of low priority incidents that were owned within the defined target time. Also referred to as 'Ownership
Met‟, „TTO Met' (Time to Own Met), or 'Accept Met'. A typical target time for ownership of low incidents is 4
hours% incidents caused by changes The % of incidents caused by the implementation of a change. Refer [Changes - Created]
% incidents caused by CMDB issues The % of incidents that were caused by data errors within the configuration management database.
% incidents linked to testing errors The % of incidents that can be linked to errors made during testing
% incidents auto generated The % of incidents that were automatically generated by IT monitoring tools.
% incidents resulting from inadequate
documentation
The % of incidents that are the outcome of inadequate documentation
% incidents resolved by known errors The % of incidents that were resolved by access to information available regarding known errors. This
information is typically stored in a 'Known Error' or Knowledge Management database. The source of the
information should be Problem Management processes where known errors are identified.
KPI Examples: Incident Management
% Problems with Incorrect Data Shows the % of problems that were identified as having one or more invalid data components. Poor data quality
associated with operational IT systems adds significant cost to a business.
Problems Open - Missed Target
Resolution - Critical
Used to monitor the volume of critical problems that are open and have missed target resolution. Increasing
volumes highlight risk to the business and potential resource or IT infrastructure issues.
% Incidents Linked to Problems In theory, all incidents are caused by problems. But, in the practical context of implementing IT service
frameworks, not all incidents result in a problem record and resolution through the formal problem management
process. This metric provides an indication of the % of incidents that are linked to the resolution of a problem.
The goal of effective incident and problem management processes should be to increase the % over time while
decreasing the overall volume.
Average Problem Create to
Close Duration
Can be used over time to track the efficiency of IT operations in resolving and closing problems. Increasing
trends can indicate resource issues and instability of the IT infrastructure. However, care needs to be taken when
analyzing this measure because some problems can be open for extended periods of time due to difficulties in
capturing the data required for problem diagnosis
Problems Created Used to monitor the overall number of problems generated within an IT infrastructure. A high level of problems
indicates inherit instability and potential problems with all IT service processes as problems are minimized via
effective incident, configuration, and change management.
% Proactive Problems Used to monitor how many problems are initiated by IT operations in a pro-active manner, that is, they start a
new project because they believe that something could go wrong or they want to ensure that a given future
problem does not happen. The presence of proactive problems indicates a mature, efficient problem
management process.
Problems Open Used to help identify resource issues within IT operations and assist with analyzing the overall stability of the IT
infrastructure.
% Problems Completed within
Target - Critical
Provides an indication of the effectiveness of Problem Management processes. Can also be used as an indicator
of how many serious problem exists - those that require more time to resolve because of extensive analysis and
diagnosis requirements.
% Problems Root Cause
Identified
One of the fundamental goals of problem management is to identify the root cause of the related incidents and/or
service requests. Efforts can be made to rectify the root causes, and information can then subsequently be fed
into knowledge bases and known errors.
% Problems RFC Created Provides information on how many problems require requests for change in order to be closed. The
implementation of one of more changes associated with a problem can be a sign that tangible outcomes are
occurring because of problem management processes or that the IT infrastructure requires significant change to
correct issues. Given this, this metric needs to be analyzed with respect to the overall maturity of IT services
delivery.
Average Problems per Release Can be used to track the overall effectiveness with which IT implements releases. Problems related to releases
can indicate serious flaws in the Release Management process.
KPI Examples: Problem Management
ANALYSIS OF A NUMBER OF
SCORECARD AND
DASHBOARD EXAMPLES
Session 6
77
A number of example dashboards/scorecards for
review
View a number of examples from Metricus
78
CollaborateProvide comments with graphs, manage issues and documents
IT KPIs – 25 practical, pragmatic, usable KPIs
TrendingTrending information to analyze KPIs in detail
SettingsManage definition, set target, tolerance, data criteria
Metricus – IT Balanced Scorecard
79
Drill down into the detailsBenchmark different service desk sites across the world and drill down into the details of available data
IT KPIs – 25 practical, pragmatic, usable KPIs for managing your service desk operations across the world
DashboardsCall handling information, information on Service Requests and Incidents
Metricus – Service Desk
80
IT KPIs – Every Module includes at least 25 KPIs with information on the practical use, adoption and benefits of KPIs.
Analyze to the greatest extent possible Dynamic charts allow you to analyze data and create charts that are most relevant to you
Add your own KPIs
Simply add your own KPIs or modify KPIs to the Module and publish them to a dashboard
Metricus – Service Request Management
81
Comprehensive DashboardsA typical Metricus Dashboard Template that provides a comprehensive picture on your current IT performance and historical information. These two elements together allow you to take adequate decisions for the future
Six Sigma charts
For selected KPIs you have the ability to create Six Sigma control charts or histograms. This functionality helps you reduce defects in your processes
Metricus – Change Management
82
For Visio experts
Metricus allows you to create and upload Visio charts and link KPIs to Visio components. For every Module specific templates are available but you can also create your own
Manage KPI categories
For all areas in the IT balanced scorecard KPIs are available. We categorized them according to customer feedback, but you can create your own sub categories
Metricus – IT Balanced Scorecard
83
IT Performance Management is
strongly advocated in ITIL.
Measuring, Managing, and Improving ITIL
processes has its roots in the Deming Circle
that is advocated in ITIL. The Plan, Do,
Check, and Act (PDCA) Cycle requires to
define a baseline and continuously measure
progress against that baseline.
ITIL addresses the use and benefits of KPIs
and IT Performance Management in every
process, as well as having an entire lifecycle
phase fully dedicated to IT Performance
Management, Continual Service
Improvement (CSI).
ITIL Service Lifecycle © Crown Copyright 2007 Reproduced under license from OGC
IT Performance Management and best practices:
ITIL®
84
The 7-Step Improvement Process © Crown Copyright 2007 Reproduced under license from OGC
The 7-Step Improvement Process is the most important element of Continual
Service Improvement. This process follows the same logical steps as defined in the IT Performance Management lifecycle and
can help you define what is important to you, measure progress against defined goals, and effectively
utilize your resources to optimize your IT organization.
IT Performance Management and best practices:
ITIL®
85
An improvement is not noted until measured
against a baseline
Justification for an ITIL project requires a
measured baseline and an improvement target
Without metrics, an ITIL project will soon lose
steam and eventually fail
A process is not implemented
until measured
ITIL Service Lifecycle © Crown Copyright 2007 Reproduced under license from OGC
Weaknesses
The ITIL publications are limited in the specification of
KPIs to be used.
There is not a lot of consistency in the definition of KPIs
and information on KPI utilization between the various
ITIL domains.
The Continual Service Improvement is intended to be
used throughout all phases of the service lifecycle,
however in reality is only considered late in most
implementation projects.
Strengths
ITIL is strong in terms of processes surrounding service
support. This has translated into broadly accepted sets of
KPIs to measure Incident Management, Change
Management, Problem Management, and so on.
ITIL emphasizes effective Configuration Management. An
accurate CMS is fundamental to the delivery of IT
Performance Management.
IT Performance Management and best practices:
ITIL®
86
ISO/IEC 20000 provides an external
quality mark for IT Service
Management organizations
ISO/IEC 20000 promotes the adoption of an integrated process approach to effectively deliver managed services to meet business and customer requirements.
Adopting ISO/IEC 20000 formalizes the measurement component of IT processes for organizations because they have to demonstrate and attest control over IT processes.
This includes requirements such as “A process should be in place to identify, measure, report, and manage improvement activities,” and “Reports shall be produced to meet customer needs, including trend information, satisfaction analysis, and so on.”
Service Delivery Processes
Capacity Management
Service Continuity and
Availability Management
Service Level Management
Service Reporting
Information Security
Management
Budgeting and Accounting
for IT Services
Release Processes
Release Management
Resolution Processes
Incident Management
Problem Management
Relationship Processes
Business Relationship
Management
Supplier Management
Control Processes
Configuration Management
Change Management
Service Delivery Processes
Capacity Management
Service Continuity and
Availability Management
Service Level Management
Service Reporting
Information Security
Management
Budgeting and Accounting
for IT Services
Release Processes
Release Management
Resolution Processes
Incident Management
Problem Management
Relationship Processes
Business Relationship
Management
Supplier Management
Control Processes
Configuration Management
Change Management
IT Performance Management and best practices:
ISO/IEC 20000
87
There are no KPIs in ISO/IEC 20000.
Even though ISO/IEC 20000 formalizes
many of the operational IT processes, it
does not provide guidance on good KPI
selection and adoption.
ISO/IEC 20000 formalizes the need for
well-defined and thought-through
process adoption.
There are many documentation
requirements in ISO/IEC 20000 and, as a
result, the chances of finding the data
required for effective IT Performance
Management are likely to be high.
A specific process called „service
reporting‟ dictates reporting
requirements.
Strengths Weaknesses
IT Performance Management and best practices:
ISO/IEC 20000
88
Thank you
Contact details:Arjan Woertman
+31 (0)10 71 10 260insert photo
89