challenges in business performance measurement: the case of a
DESCRIPTION
TRANSCRIPT
Challenges in Business Performance Challenges in Business Performance Measurement: The Case of a Measurement: The Case of a
Corporate IT FunctionCorporate IT Function
Stephen Corea & Andy Watters (Warwick University, UK)
Presentation OutlinePresentation Outline
Research Motivation
Theoretical Review
Methodology
Case Study: GITS
Findings & Discussion
Conclusion
Business performance measurement (PM) – presenting relevant information to management staff for assessing the organization's progress towards achieving strategic/operational aims
Several major PM frameworks proposed recently: field dominated by prescriptive, top-down perspective (formal derivation from strategy)
Research BackgroundResearch Background
RESEARCH AIM:
exploratory study to understand the challenges in mounting dashboard based PM practices in a corporate IS Dept
Significant no. of PM initiatives fail to take root or adequately deliver expected benefits (70% according to McCunn, 1998)
Need for greater understanding of on-the-ground challenges and bottom-up perspective of challenges involved
1. Robust PM system should take a ‘balanced’ approach (Kaplan & Norton, 1992)
2. Identification/population of useful performance measures (or metrics) to capture progress towards goal attainment is a key but not easily satisfied criteria (Neely et al, 1997):
Theory: PM ImplementationTheory: PM Implementation
Key proposed principles of PM design
Should be grounded in strategy: performance metrics must be derive from predefined strategy/objectives (De Tonia & Tonchia, 2001)
Should incorporate a high proportion of ‘leading’ indicators (Eckerson, 2006) i.e. in comparison to ‘lag indicators’
Theory: PM DesignTheory: PM Design 22 most-cited recommendations for designing
measures (Neely e al, 1997)
1. Be derived from strategy2. Be simple to understand3. Provide timely and accurate feedback4. Be actionable: based on quantities that
can be influenced or controlled5. Reflect the business process: both
customer and supplier should be involved in defining
6. Relate to specific targets7. Be relevant8. Be predictive: part of a closed
management loop9. Be clearly defined10. Have visual impact11. Should focus on improvement12. Be consistent (i.e. should maintain
significance as time goes by)
1. Provide fast feedback2. Have an explicit purpose3. Be based on explicitly defined formula
and source of data4. Employ ratios rather than absolute
numbers5. Use data which are automatically
collected as part of a process6. Be reported in a simple consistent
format7. Be based on trends rather than
snapshots8. Provide information9. Be precise – be exact about what is
being measured10. Be objective – not based on opinion
3. Systemic aspect of effective PM: process rationalisation, shared understanding & staff commitment, IT support (data capture/collection, processing & presentation)
4. Dashboards: visual impact, data quality and timeliness (Few, 2005; Dixon et al., 1999)
5. PM for the IT Function: need for spread of measures across 3 categories (Stanwick & Stanwick, 2005) – (i) efficiency; (ii) effectiveness; (iii) productivity
Theory: PM ImplementationTheory: PM Implementation
Interpretive case study method (Walsham, 1995): indepth single-site case study, aimed at theoretical generalisation
Case organisation/unit: GITS (Group IT Services), the corporate IT function of Multicorp (a pseudonym), a multi-national manufacturer of tobacco-based products
Data Gathering & Analysis
multiple site visits: June to August 2006
27 semi-structured interviews, dashboards/documents review, informal conversations
inductive analysis (Glaser & Strauss, 1967): identifying patterned regularities (common themes, issues or dilemmas)
Research MethodResearch Method
Case Study: Multicorp /GITSCase Study: Multicorp /GITS
GITS (Groups IT Services) corporate IT function, formed in 2004
transform supply of IT support across Multicorp’s worldwide business units from traditional geographically-localised model into a centralised, ‘shared services’ model
to achieve cost-savings of £100 million by 2009
Major manufacturer of cigarettes & tobacco-based products 300+ brands sold in 180 ‘end-markets’ (i.e. country-specific regions);
factories in 54 countries
stable industry competitive conditions: strategy is strongly focused on efficiencies (cost-savings) across business units (esp. support)
Case Study: GITSCase Study: GITS
World Class People
Service
Quality
Irresistible Value
Cost Savings
SDS 26
Volume
£100mCSS 4.5
100%
Supply Side
Strategy & structure
vision: irresistible value
client-facing units & management team
critical need to monitor performance
3 dashboards in use
“Two years ago GITS had less than 100 staff, the management team could sit in a room and discuss in detail operational issues throughout the department. Now we’re more than 500 strong, and are doing far more things. We haven’t a clue what is going on out there, and don’t know what operational things we should be looking at.” (Manager)
GITS: Leadership DashboardGITS: Leadership Dashboard
Performance-to-date
Customer Satisfaction
Managed Volume
Cost Reduction
e2e SLA Performance
Mean Time To Fix
Application Services Operations
Technical Services Operations
GITS: Application Services DashboardGITS: Application Services Dashboard
On Targe t
Within 10% o f Targe t 2004
More than 10% Below Target Actual Actual Budget Actual BudgetFinancial Responsible Unit
Managed Volume- Ongoing Tim £'Ms 18.2 14 0.9 2.4 10.1 14.0Cost Reduction Tim £'Ms 7.2 7 2.3 4.1 4.4 7.6Cost Avoidance Tim £'Ms 3.5 4 0.3 1.7 4.0Overhead Rate Tim % 10 8 8.0% 9.5% 8.0% 8.0
Service MetricsE2E SLA Performance David % N/A 99.5% 100.0% 98.5% 99.7% 98.5% 99.5%Timeliness of First Response David Hours N/A 1.90 0.6 1.9 2.3 1.9 1.9Problem Resolution David Days N/A 1.00 0.1 1.0 0.9 1.0 1.0
Customer PerceptionProject Evaluation David /5 N/A 3.50 4.0 3.5 4.0 3.5 3.5Customer Satisfaction Survey Simon /5 3.5 3.50 3.3 3.5 3.4 3.5 3.3Improvement Evaluation David /5 N/A 3.50 3.5 3.5 3.5
Operational EfficiencyCoverage Level Within Agreed Response TimeDan % N/A 80.0% 95% 80% 97% 80% 80%Incident Evaluation David # N/A 3.50 4.0 3.5 3.7 3.5 3.5Service Office Response Time David % N/A 95.0% 100.0% 95.0% 90.8% 95.0% 95.0%
EAS/ES Demand Fulfilment Thorsten % N/A 90.0% 98% 95% 95% 90% 90%Resource Utilisation Thorsten % 100 100.0% 113% 100% 93% 100% 100%Permanent Headcount Mike # 184 220 208 208 208 208 220
Staff Development Rating Mike /32 20 22 22 22 22 22 22Prince 2 Accreditation Mike % 62.0% 80.0% 99% 75% 97% 75% 80%Leading Through Change Participation Simon % N/A 95.0% 98% 100% 98% 100% 98%
Current Period 2005 Year To Date Current Year End Forecast
P&O
2005 Year End Budget
Driving for R
esults B
uilding
Global
On Targe t
Within 10% o f Targe t 2004
More than 10% Below Target Actual Actual Budget Actual BudgetFinancial Responsible Unit
Managed Volume- Ongoing Tim £'Ms 18.2 14 0.9 2.4 10.1 14.0Cost Reduction Tim £'Ms 7.2 7 2.3 4.1 4.4 7.6Cost Avoidance Tim £'Ms 3.5 4 0.3 1.7 4.0Overhead Rate Tim % 10 8 8.0% 9.5% 8.0% 8.0
Service MetricsE2E SLA Performance David % N/A 99.5% 100.0% 98.5% 99.7% 98.5% 99.5%Timeliness of First Response David Hours N/A 1.90 0.6 1.9 2.3 1.9 1.9Problem Resolution David Days N/A 1.00 0.1 1.0 0.9 1.0 1.0
Customer PerceptionProject Evaluation David /5 N/A 3.50 4.0 3.5 4.0 3.5 3.5Customer Satisfaction Survey Simon /5 3.5 3.50 3.3 3.5 3.4 3.5 3.3Improvement Evaluation David /5 N/A 3.50 3.5 3.5 3.5
Operational EfficiencyCoverage Level Within Agreed Response TimeDan % N/A 80.0% 95% 80% 97% 80% 80%Incident Evaluation David # N/A 3.50 4.0 3.5 3.7 3.5 3.5Service Office Response Time David % N/A 95.0% 100.0% 95.0% 90.8% 95.0% 95.0%
EAS/ES Demand Fulfilment Thorsten % N/A 90.0% 98% 95% 95% 90% 90%Resource Utilisation Thorsten % 100 100.0% 113% 100% 93% 100% 100%Permanent Headcount Mike # 184 220 208 208 208 208 220
Staff Development Rating Mike /32 20 22 22 22 22 22 22Prince 2 Accreditation Mike % 62.0% 80.0% 99% 75% 97% 75% 80%Leading Through Change Participation Simon % N/A 95.0% 98% 100% 98% 100% 98%
Current Period 2005 Year To Date Current Year End Forecast
P&O
2005 Year End Budget
Driving for R
esults B
uilding
Global
GITS: Technical Services DashboardGITS: Technical Services Dashboard
Technical Services Dashboard
Reporting Period: April VISION
ANALYTICS SUSTAINABLE ORGANISATION
On Target % Achieved Team Building SDS Your Voice 6 Sigma
Objectives 0.00% People ? ?
Signed Off On Target KCC Progress Audit Memos
Memos missing Deadlines
Strategy Risk & Control 78% 16 0
To Date £ Target Stretch Key Roles No. Filled Contractor %
Cost Reduction Resourcing 23 21 13%Cost Avoidance
Transformation Plans 0
Governance in place
TS Update Stakeholders mapped
Governance & Comms
OPERATIONAL
No. Lines Performance P&L YtoD Availability CSS
Service Lines ? AVAILABILITYE2E FIX SCOREGITS OVERALL CSS
Budget Forecast To date Variance Forecast To date Variance
Financial Ops. £2,222,333.44 £1,111,111.22 £1,233,222
No. No. Quality Improvement
Current Budgets
ForecastedExpenditure
% Projects Complete
£ Benefits
Projects
Revenue
Deliver irresistible value through...
Expenses
Technical Services Dashboard
Reporting Period: April VISION
ANALYTICS SUSTAINABLE ORGANISATION
On Target % Achieved Team Building SDS Your Voice 6 Sigma
Objectives 0.00% People ? ?
Signed Off On Target KCC Progress Audit Memos
Memos missing Deadlines
Strategy Risk & Control 78% 16 0
To Date £ Target Stretch Key Roles No. Filled Contractor %
Cost Reduction Resourcing 23 21 13%Cost Avoidance
Transformation Plans 0
Governance in place
TS Update Stakeholders mapped
Governance & Comms
OPERATIONAL
No. Lines Performance P&L YtoD Availability CSS
Service Lines ? AVAILABILITYE2E FIX SCOREGITS OVERALL CSS
Budget Forecast To date Variance Forecast To date Variance
Financial Ops. £2,222,333.44 £1,111,111.22 £1,233,222
No. No. Quality Improvement
Current Budgets
ForecastedExpenditure
% Projects Complete
£ Benefits
Projects
Revenue
Deliver irresistible value through...
Expenses
Inadequacies in dashboard population and scope of measurement
difficulties obtaining timely & accurate data
areas of performance left untracked (in scope & time)
deficient in leading indicators (heavily lagged-oriented): lack of predictive capacity to take proactive interventions
““I’ve no idea what drives the numbers. I’m not sure if anyone has” (manager)
Case: FindingsCase: Findings
lack of clarity or common understanding regarding definition of certain measures, e.g.
Constitution of measures: e.g. managed volume
(i) “We count managed volume against our target only when services have been transferred to GITS, and the first invoice sent to the end-market”;
(ii) “Managed volume is just that: services which we (GITS) manage. It doesn’t matter if we haven’t billed the customer yet.”
Progress towards targets: e.g. cost savings
(i) “We claim that we have achieved a cost saving when we sign a contract with an outsource provider to provide the service at a cost lower next year than our current deal”
(ii) “Cost savings are claimed when we release next years’ price list to the end markets in May, with confirmation in early December.”
Case: FindingsCase: Findings
Relation of measurement to strategy: difference between Leadership & AS/TS dashboards
Application Services & Technical Services dashboards reported self-chosen operational targets beyond existing strategy
Case: FindingsCase: Findings
non-strategy derived measures were seen by managers as useful aspects for monitoring operational health
“There are quite a few measures which don’t directly relate to strategy or targets, but we think it is worthwhile to keep track of them. It helps to know the operational health of the business.” (Application Services manager)
political value or ‘signaling’ ““We benchmark the charge rates of our project managers against external
consultancy providers; we’re less expensive, and the difference is classed as Cost Avoidance. It helps us demonstrate our value to the business.” (Technical Services manager)
Lack of systemisation in data collection and measurement (cum dashboard) design
no top-down mandate or formal programme / framework guiding the implementation of these practices: need for process rationalisation and information systems infrastructure (Bourne et. al, 2003)
difficulty identifying leading indicators (Neely et al., 2000; Eckerson, 2006)
Case: DiscussionCase: Discussion
Need for re-orientation of fundamental aim/focus
from a tool for simply monitoring/reporting to one of learning what factors drive results (i.e. to be able to influence/control)
Re-thinking major PM tenets/principles
1. Notion of ‘balance’ in balanced measurement
financial vs. non-financial ‘lever’ reporting vs. predication/learning ‘lever’ (lagging
vs. leading indicators)
Case: DiscussionCase: Discussion
2. Existing strategy as the source for deriving measures: a case for de-coupling strategy from measurement?
cost-focussed strategies promote financial and discourage non-financial indicators
‘de-politicizing’ of measurement promote transformation of PM towards a learning
rather than simply a monitoring tool
Results of this exploratory study suggest a need for further research & theoretical development to extend & deepen understanding of the complex nature of PM
What does ‘balanced’ measurement imply Relationship between strategy and measurement
Questions?
Thank you
ConclusionConclusion
END OF PRESENTATIONEND OF PRESENTATION
ReferencesReferences1.1. Kaplan, R. and Norton, D. (1992). Kaplan, R. and Norton, D. (1992). “The balanced scorecard – measures that drive “The balanced scorecard – measures that drive
performance.” Harvard Business Review. January-February, 71-29.performance.” Harvard Business Review. January-February, 71-29.
2.2. McCunn, P. (1998) The Balanced Scorecard: the eleventh commandment. McCunn, P. (1998) The Balanced Scorecard: the eleventh commandment. Management Accounting 34-36.Management Accounting 34-36.
3.3. Neely, A., Richards, H., Mills, J., Platts, K. and Bourne, M. (1997) Designing Neely, A., Richards, H., Mills, J., Platts, K. and Bourne, M. (1997) Designing performance measures: a structured approach. International Journal of performance measures: a structured approach. International Journal of
4.4. Operations & Production Management 17, 1131-1152.Operations & Production Management 17, 1131-1152.
5.5. De Toni, A. and Tonchia, S. (2001) Performance measurement systems - models, De Toni, A. and Tonchia, S. (2001) Performance measurement systems - models, characteristics and measures. International Journal of Productions and Operations characteristics and measures. International Journal of Productions and Operations Management 1, 347-354.Management 1, 347-354.
6.6. Eckerson, W. (2006) Performance Dashboards: Measuring Monitoring and Managing Eckerson, W. (2006) Performance Dashboards: Measuring Monitoring and Managing Your Business, edn. New Jersey: John Wiley & Sons.Your Business, edn. New Jersey: John Wiley & Sons.
7.7. Few, S. (2005) Dashboard Design: Beyond Meters, Gauges, ad Traffic Lights. Few, S. (2005) Dashboard Design: Beyond Meters, Gauges, ad Traffic Lights. Business Intelligence Journal 10, 18-24.Business Intelligence Journal 10, 18-24.
8.8. Dixon, J.R., Nanni, A.J. and Vollmann, T.E. (1990) The new performance challenge: Dixon, J.R., Nanni, A.J. and Vollmann, T.E. (1990) The new performance challenge: Measuring operations for world-class competition, Homewood, IL: Dow Jones-Irwin.Measuring operations for world-class competition, Homewood, IL: Dow Jones-Irwin.
9.9. Stanwick, P. and Stanwick, S. (2005) IT Performance: How Do You Measure a Moving Stanwick, P. and Stanwick, S. (2005) IT Performance: How Do You Measure a Moving Target? The Journal of Corporate Accounting and Finance 13, 19-24.Target? The Journal of Corporate Accounting and Finance 13, 19-24.
10.10. Walsham, G. (1995). Interpretive case studies in IS research: nature and method. Walsham, G. (1995). Interpretive case studies in IS research: nature and method. European Journal of Information Systems, 4:2, 74-81.European Journal of Information Systems, 4:2, 74-81.
11.11. Glaser, B. and Strauss, A. (1967) The discovery of grounded theory, Aldine, Chicago.Glaser, B. and Strauss, A. (1967) The discovery of grounded theory, Aldine, Chicago.
12.12. Bourne, M., Franco, M. and Wilkes, J. (2003) Corporate Performance Management. Bourne, M., Franco, M. and Wilkes, J. (2003) Corporate Performance Management. Measuring Business Excellence 7, 15-21.Measuring Business Excellence 7, 15-21.