Demystifying IT KPIs

An archestra notebook.

© 2013 Malcolm Ryder / archestra

Finding the Keys

Although "measuring IT" has an enormous literature of references, the sheer volume of that reference information can create the most difficult challenge to the practice. Why? Because, with more and more and more points to consider, the real primary question - "how much measurement is enough" -- may not be confidently answered. However, there is universal consensus that the "trees in the forest" are always key performance indicators, or KPIs.The idea of "key" suggests that there is a process of filtering out indicators that don't matter as much, leaving a manageably fewer number that matter a lot. What is the logic of this filtration?The filtering can take place on several different terms, as follows.

Defining Performance

To begin with, if you have a consistent definition of "performance", you can discover, borrow, and/or invent KPIs that will make sense in your shop whether or not they make sense elsewhere.

To best use the idea of performance, emphasize a context-neutral definition, such as follows (and note that there are two parts included).

Performance is a measure of the degree to which progress has been made:

(a.) relative to a measurable target,

(b.) with the method that was chosen to cause the progress.

This neutrality makes it clear that performance can be identified on many different levels and at many locations of activity, without being mistaken for quality, risk, or other characteristics.

The importance of that observation is that although a single target can be the highest point of interest, performance tends to involve multiple perspectives.

The Semantics of What Matters vs. What Counts

Usual impact

Actual impact(higher)

Usual impact

Actual impact(lower)

TARGET X

Method 1 Method 2

In this hypothetical scenario, we see a high performance BYMethod 1 OF effort TO Target X, and a low performance BYMethod 2 OF effort TO Target X. But meanwhile the absoluteperformance of Method 2 is higher than that of Method 1.

Enough is enough

To avoid having to investigate and track unnecessary indicators, think of "Key Performance" indicators, not key "Performance Indicators".

A key performance is an effort formed from:

• the amount of impact necessary by each of…

• the smallest number of things known together to …

• ordinarily suffice for …

• creating the required levels of progress towards …

• a defined target.

ESCAPE VELOCITY:

FUEL THRUST TIME

X Y Z

(minimum amounts of fewest elements)

Criticality Defined neutrally as above, the idea of performance is meaningless until both a measurable target has been defined, and a method of proceeding has been specified.

This dual aspect is crucial to acknowledge because, in real life, we “perform” with what we currently do have. That is, we presently pursue goals with things that we actually already have, not with the imaginary. This is the objective basis of performance.

That said, one of the main reasons for measuring performance is to identify whether there is a need to do things differently. If we are disappointed, that finding can mean changing the current actuals. But if we are already satisfied, it can also mean revising our ideals.

The ideals can be the methods used, or the target itself. Either way, the point is to identify what must make the most important difference. To do that, performance measurement needs to be objective, but the users of the measurements may have an agenda.

The agenda may be to make decisions about methods, or to validate whether targets are reasonable, or other reasons. Objective measurements can tell multiple different stories. Yet even when they do, they need to maintain a distinction between actuals and ideals, as well as current versus future.

Current

Ideal

Future

Actual

Often dominates

targets

Includes most

indicators

Trend without changes

Goals with or without changes

IT Measurement: The Who Cares Test

Business cares about IT only because IT is a way to accomplish more labor both faster and affordably. (That accomplishment allows the business to do other fairly important things.)

And, there is an IT organization only because bringing resources and accountability under centralized management creates consistencies of awareness -- consistencies that let the business approach obtain IT the way a consumer approaches a preferred provider of services.

Because of that, most performance targets of the IT organization need to quantify ambitions, agreements, or expectations about the following IT abilities: to provide, to be preferred, and to keep up with varying demand and priorities.

Recognizing Progress

So, what indicators about progress can be recognized?

Indicators are nothing but directions versus targets.

Directions are simple: moving away, moving towards, or standing still.

Indicators, whether historical or predictive, are usually quantitative signals (even if only a binary on/off or true/false) that represent direction.

But this is where another issue arises about how things are gettingdone (not how much got done)...

Driving Progress

The practical perspective is that things get done by selecting a resource (created by combining people, processes and tools) --then giving that resource a responsibility (consisting of a scope, direction and priority) during a specified timeframe -- and having them operate on the responsibility.

Too often, what is left out of the picture is two other things: a support mechanism adequate for the responsible resource, and the impact of management supervision on the responsible resource.

There are four big variables in the formula for the activities that "drive" progress towards a target: • resource, • responsibility (including timing), • support, • and supervision. Logically, these variables are important only in how they affect the major abilities (provision, preference, priorities) that describe the business demand for IT. Certain combinations of variables may prove to be "critical" to success.

A Layered Performance Architecture

Overall, the “measurement" of the performance shows multiple layers of indicators, about whether operations were:

• composed adequately,

• behaved correctly,

• and did the right kind of work,

• for the right reason.

But if you don't know what the right reasons are, the rest of it is arbitrary.

The reasons are business-level objectives. (See examples following.)

GROWTH HEALTH RECOVERY

Increase Volume

Update Portfolio

Modify Agreements

Add Partner

Improve Operation

Restore Brand

Enter Segment

Lower Costs

Exit Location

Improve Operation

Restore Brand

Enter Segment

Goals

Plan

Preferred participants must now provide according to priorities.

with generic objectives

(example)

IT aligns with the business on how IT is to participate in the plan.

The business finds value in IT’s participation in several ways:• immediate certainties (agreements)• sustained attentiveness (expectations)• long-term compatibility (ambitions)

This creates operational objectives for IT that require appropriate organization, capabilities and staging (success factors).

The Same Page

The Same PagePerformance logic:

Example: to hold the status of Preferred by the business, the IT organization can bring higher-quality processes to the objective of enforcing Agreements.

As a way to assure the quality of the process, standards can be identified and adopted.• This means that, for example, a “pace of standards adoption” can be a target

for IT. Along the way, there can be indicators of progress towards that IT target, which will be monitored.

By hitting its targets, IT can reinforce the opportunities that the business wants for accomplishing a business objective such as “improved operations”.• An opportunity can be specified as a target for Business. An example target

could be a level of “delivery speed”. Along the way there can be indicators of progress towards that business target, which will be monitored.

Sourcing the KPIs

A generalized understanding of performance indicators becomes practical when it is clear that relevant information is already being collected.

The selection of information to use as indicators should be adequate when drawn from the essential factors allowing operations to drive progress:

• Resources

• Responsibility

• Support

• Supervision

Areas containing operational Targets that are often recorded for Objectives

Of Provision Of Preferences Re: Priorities

Supervision Monitoring results Survey results Reports published

Support Infrastructure & staffing events Standards adoption Programs managed

Responsibility SLA compliance Services planning Fulfillment rates

Resource Assets managed Training and IT Change Assignments distributed

Areas (that contain Indicators) addressing operational Targets

Agreements Expectations Ambitions

Priorities Initiative adoptions Proposal approvals Strategy alignment

Preferences Process management User requests by category Roadmap milestones

Provision Contract compliance Delivery rates R&D projects

© 2013 Malcolm Ryder / archestra