bititci et 2007 pms a development quide - tqm magazine

The need for integrated performancemeasurement

Business performance improvements arisingfrom increased manufacturing integrationcontinue to be one of the primary competitiveissues of the 1990s. Recent research intomanufacturing systems integration (Carrieand MacIntosh, 1992) has identified the needfor effective deployment of business objectivesdown through the organization and the sub-sequent measurement of performance incritical areas as key elements of sustainablecompetitive advantage. Other researchershave also noted the links between perfor-mance measures and strategic plans and/orcritical success factors of the business. Theresearch by Grady (1991) and Eccles andPyburn (1992) supports the same conclusionsdrawn in the earlier research programme.

There is already considerable work beingcarried out by the accounting profession onperformance measurement. Indeed, mostmanufacturing organizations have extensiveperformance measurement systems based oncost and financial accounting practices.Recent innovations, such as activity-basedcosting, overcome some of the difficultiesassociated with traditional methods but stilldo not promote continuous improvement andstrategic orientation. Notable work has beencarried out by Kaplan (1983; 1990) andJohnson and Kaplan (1987) in recognition ofthese weaknesses. Neely (1993) summarizesthe shortcomings of the current accountingpractices with respect to performance mea-surement in manufacturing enterprises.

Because financial measures that are cur-rently in place are not supporting the changeprocess, there is a case for new styles of mea-surement systems that are appropriate to theneeds of the modern business. Green (1991),McNair and Mosconi (1987), Drucker(1990) and Russell (1992) show that there is aneed for alignment of financial and non-financial measures that fit within a strategicframework.

Furthermore, based on research carriedout by Bititci (1993), Bititci and Swenson(1993), Blenkinsop and Burns (1991) andGelders et al. (1993) there is evidence that,even in companies where quality-orientedperformance measures are employed, theseare still being used in a manner which doesnot promote integration. There have beenseveral cases cited where the companys

46

The TQM MagazineVolume 9 Number 1 1997 pp. 4653 MCB University Press ISSN 0954-478X

TechniquesIntegratedperformancemeasurement systems:an audit anddevelopment guide

Umit S. BititciAllan S. Carrie andLiam McDevitt

The authorsUmit S. Bititci is the Director of the Centre for StrategicManufacturing at the University of Strathclyde, Glasgow,UK. Allan S. Carrie is a Professor and Liam McDevitt is aResearch Assistant at the University of Strathclyde.

AbstractThe performance management process is seen as a closedloop control system which deploys policy and strategy, andobtains feedback from various levels in order to managethe performance of the business. The performance measurement system is the information system which is atthe heart of the performance management process and itis of critical importance to the effective and efficientfunctioning of the performance management system.Research identifies two critical elements with respect tothe content and structure of the performance measure-ment system: integrity and deployment. The viable systems model (VSM) provides a framework for assessingthe integrity of the performance measurement system,while the reference model developed for integratedperformance measurement systems provides a frameworkagainst which performance measurement systems can bedesigned and audited.

strategy, improvement projects and perfor-mance measures were in conflict.

In summary, the need for an integrated setof performance measures which supportrather than contradict business objectives isnow clearly established (Bititci, 1994).

The performance management process

A paper which was presented by the authorsat the International Federation for Informa-tion Processing Working Group 5.7 Work-ing Conference in Seattle (Bititci, 1995)asserted that performance managementshould be viewed as a key business processwhich is central to the future wellbeing andprosperity of any manufacturing enterprise.

The performance management process isthe process by which the company managesits performance in line with its corporate andfunctional strategies and objectives. Theobjective of this process is to provide a proac-tive closed loop control system, where thecorporate and functional strategies aredeployed to all business processes, activities,tasks and personnel, and feedback is obtainedthrough the performance measurement system to enable appropriate managementdecisions (Figure 1).

In essence, the performance managementprocess defines how an organization usesvarious systems to manage its performance.At the heart of the performance managementprocess, there is an information system whichenables the closed loop deployment andfeedback system. This information system isthe performance measurement system whichshould integrate all relevant information

from the relevant systems. In this contextintegration means that the performancemeasurement system should enable the cor-rect deployment of the strategic and tacticalobjectives of the business as well as providinga structured framework to allow the relevantinformation to feed back to the appropriatepoints to facilitate the decision and controlprocesses.

This information system, i.e. the perfor-mance measurement system, to be effective inachieving its objectives as described above,should take account of the strategic and environmental factors relating to the businessas well as considering the structure of theorganization, its processes, functions andtheir relationships. Similarly, the effectivenessof the performance management processwhich makes use of this information systemdepends on how this information is used tomanage the performance of the business.Figure 2 summarizes this view of the perfor-mance management process.

The structure of the performancemeasurement system

The research to date revealed two criticalconsiderations with respect to the structure ofperformance measurement systems: integrity;and deployment

IntegrityIntegrity refers to the ability of the perfor-mance measurement system to promoteintegration between various areas of the busi-ness. Previous research on manufacturingsystems integration undertaken by the same

47

Integrated performance measurement systems

Umit S. Bititci, Allan S. Carrie and Liam McDevitt

The TQM Magazine

Volume 9 Number 1 1997 4653

Vision

Businessobjectives

Strategicgoals

Criticalsuccess factors

Critical tasksAction plan

Performance measures

DEPLOYMENT

FEEDBACK

Figure 1 The closed loop deployment and feedbacksystem

Theinformation

system

The performancemanagement process

The performancemeasurement system

Strategy Environment Structure Processes Relationships

Informationtechnology

What ismeasured?

Who usesthe measures?

ResponsibilitiesHow systems areused to manage

performance

Reportingstructure

Attitudes

Culturalissues

Behaviouralissues

Figure 2 The performance management process and the position of theperformance measurement system

consortium identified the value of the viableSystems Model (VSM) (Beer, 1979; Blenkin-sop, 1993) as a framework for assessing asystems integrity. The theory behind the VSMis that for any system to be viable it must havefive components. The VSM identifies thesefive components as five systems. These sys-tems are discussed in the followingparagraphs with respect to the integrity of theperformance measurement system.

System 1This is the operational unit which producesthe goods or services. In other words it repre-sents the productive function of the organiza-tion. It could be interpreted that system 1consists of business activities such as orderprocessing, credit control, production plan-ning, manufacturing, shipping, invoicing, etc.From a performance measurement systemsperspective system 1 will consist of perfor-mance measures which objectively measurethe performance of an individual businessactivity. For example: order processing order processing lead-

time, order processing accuracy; credit control credit control leadtime,

average time taken to register new customers;

production planning plan stability; manufacturing manufacturing leadtimes,

set-up times, yield, utilization.

System 2This is the local management system whichco-ordinates the activities of operational units.This system represents the business processwhich contains the business activities of system 1. For example, an order fulfilmentprocess may contain all of the business activi-ties listed above. From a performance measurement systems perspective, system 2will consist of performance measures whichobjectively measure the performance of anindividual business process. In other words,the performance measure applied to theprocess is a function of the collective perfor-mance of its constituent activities. For example, for the order fulfilment processshown in Figure 3, performance againstmeasures such as order fulfilment leadtimes,delivery performance and process cost are alla function of the performance of the activitiescontained within that process.

System 3This system represents the tactical manage-ment system which manages the operations ofsystems 1 and 2 by setting targets and priori-ties. This is the management system which isresponsible for the performance of businessprocesses and activities in line with therequirements of the higher level systems.From a performance measurement systemsperspective this is the system which deploysthe strategic policies and priorities to opera-tional systems (system 1) as well as monitor-ing the performance of these operationalsystems.

System 4This is the developmental system whichconcerns itself with the external environmentand therefore the future. Its focus is onimprovement. This is the system which identi-fies the changes necessary to the lower levelsystems. This system, by focusing externally,identifies the improvement gaps and setsstrategies to fulfil corporate objectives. Theperformance measures used in this systemtend to be externally focused and compara-tive, e.g. delivery performance with respect tothe competitors or market requirements.

It would be appropriate to combine theexternal focus of this system with the resultscategory of the European Foundation forQuality Management (EFQM) model fortotal quality management (TQM) where thefocus is on customer satisfaction, peoplesatisfaction, business results and societysatisfaction. From a reference model pointof view it may be appropriate to expect externally-oriented measures correspondingto each one of the four areas.

System 5This is the boss: this system sets the direction,the corporate policy and the objectives theorganization would be adopting in the future.With respect to the performance measure-ment system, this system sets the corporatepriorities and targets.

The combination of systems 3, 4 and 5 isknown as the meta system, which is responsi-ble for identifying and managing change.

Figure 3 attempts to illustrate the applica-tion of the VSM to an organization asdescribed in the preceding paragraphs.

The VSM places great importance oncommunications between its individual components. It stresses that the higher levelsystems amplify (i.e. deploy) higher level

48



The TQM Magazine


objectives in to local objectives and that during this deployment process the objectivesare transduced (i.e. translated into locallyrelevant terms). The research presented inthis paper has adopted the term deploymentas being a function of amplification and transduction. Further:

DeploymentDeployment refers to the deployment ofbusiness objectives and policies throughoutthe hierarchical structure of the organizationas illustrated in Figure 2. The objective ofdeployment, in this context, is to ensure that: performance measures used at various

levels of the organization reflect the busi-ness objectives and policies;

deployment is consistent through thehierarchy of the organization;

deployment is relevant and correct withrespect to the impact and influence ofindividual business areas (i.e. processes,functions and activities).

A reference model

The reference model developed by theresearch team at the University of Strathclydeis based on the two facets of the performancemeasurement system already identified asintegrity and deployment.

The VSM requires its five systems to interactand co-ordinate in a controlled fashion. Theexplanation of the five systems and the inter-pretation of the five systems with respect toperformance measurement systems, providedin the previous section, suggest that the busi-ness directions and policies are set by system5 using external information provided bysystem 4, and that these policies are deployedthrough systems 4, 3 and 2 to the operationalsystems (i.e. system 1s).

However, VSM itself does not clearlyestablish the exact requirements for correctdeployment but, when it is combined with thethree basic objectives specified with respect todeployment a reference model is developed. Apictorial view of this reference model is pro-vided in Figure 4.

The audit method

Having identified and defined the key charac-teristics of an integrated performance mea-surement system (i.e. integrity and deploy-ment), the researchers at the University ofStrathclyde have developed and tested amethod for auditing the integrity and deploy-ment of the performance measurement system as defined in the reference model. Theaudit process consists of three phases. Theseare: data collection; integrity audit; anddeployment audit.

49



The TQM Magazine


System 3Monitors and manages

the performance of systems 1 and 2

System 5Direction, policy and

strategy setting

The metasystem

System 1Order

processing

System 1Planning

System 1Creditcontrol

System 1Shipping

System 1Invoicing

System 1Manufac-

turing

System 2The process

The

exte

rnal

env

ironm

ent

System 4Monitors the external environment

and internal performanceSets improvement targets

Figure 3 The VSM configuration and its interpretation to the structure of a performance measurement system

Data collection phaseThe data collection phase, using a workbookdeveloped by the research team, collects allinformation relevant to the performancemanagement process of the business. Duringthe early stages of the research programme itbecame apparent that most businesses, smallor large, consist of discrete business unitswhich in turn may be treated as stand-alonebusinesses. The common terminology applied to these business units are strategicbusiness units (SBUs). The configuration ofSBUs within an organization could varydramatically.

In the first instance the data collectionphase requires the identification of the indi-vidual SBUs and their strategic requirementsin terms of the qualifiers and the differen-tiators. The performance measurementsystem should ensure that the prioritiesbetween the above two criteria are balancedand appropriately managed in a dynamicenvironment.

The information collected includes: Identification of strategic business units

(SBUs) within the business.

The market requirements in terms ofqualifiers and differentiators for each SBU.

The development plans or objectives forthe business or each of its SBUs.

The performance measures used andreviewed by the executive managementteam within the business, i.e. the strategicperformance measures.

The performance measures used andreviewed within each function of the busi-ness, i.e. the functional or operationalperformance measures.

Personal objectives and any associatedincentive schemes for the executives, man-agers, supervisors and operational person-nel.

Review, reporting and performanceresponsibilities associated with measuresused at all levels.

Once the data are collected they are analysedto conduct the integrity and deploymentaudits respectively. An example of the auditprocess and output is provided in the follow-ing section.

50



The TQM Magazine


Environmental requirements andcompetitive position

Business results, customer satisfaction,people satisfaction, society satisfaction

System 5: Sets direction, strategy, policyand objectives based on current performanceand the external indicators obtained from

system 4

Theenvironment

Market,customers,

people, society,shareholders,

etc.

Business objectivesDo the business objectives reflect the

environmental requirements and gaps?

System 4: Provides external intelligencewith respect to the strategic objectives of

the business. Expresses strategic objectivesin measurable terms and sets targets

Strategic measuresDo the performance measured and prioritiesused at a strategic level reflect the businessobjectives, environmental requirements and

associated priorities?

System 3: Deploys strategic objectives andpriorities to co-ordinating (system 2) and

operational (system 1) measures. Monitorsand manages the performance of the process

Co-ordinating/process measuresDo the measures and priorities at this level

reflect the environmental requirements,business objectives, the strategic measures

and priorities?

System 2: Employs process-orientedperformance measures to ensure that

individual business activities and tasks areco-ordinated effectively and efficiently

Operational measuresDo the measures and priorities employed atbusiness activity level reflect the higher level

objectives, measures and priorities?

System 1: Employs performance measuresto measure the performance of each business

activity or task

Integrity Deployment

Figure 4 A reference model for integrated performance measurement systems

Performance measurement systemsaudit for a small engineeringmanufacturing business

The purpose of this section is to illustrate theapplication of the audit techniques developedin a real life case study. The company concerned is a traditional engineering company manufacturing blowing equipment.The company, although autonomous, ispart of a larger group. It employs some 100people from a single site based in Glasgow,UK.

Data collectionData were collected using standard datacollection forms developed by the researchteam. During data collection three SBUs wereidentified within the business, although priorto the audit the company did not formallyrecognize these SBUs. These were:(1) Non-competitive markets, where the

differentiator is production flexibility andthe qualifier is product quality.

(2) Competitive markets, where the differen-tiator is price and the qualifier is onceagain product quality.

(3) Spares and service business, where thedifferentiator is delivery speed and thequalifiers are price and quality.

Integrity auditIn order to conduct the integrity audit in acomprehensive, rigorous and repeatable

manner, the research team has compiled achecklist to search for objective evidencesupporting the existence of all five systemswithin the VSM. Following the analysis ofthese data against this checklist various gapswere identified with respect to the complete-ness and integrity of the performance mea-surement system. An extract from the resultsof the integrity audit carried out on the collab-orating organization illustrates the nature ofthe conclusions obtained from such an audit(Table I).

The deployment auditIn order to audit the deployment of the per-formance measurement system a three-stageaudit method has been developed: Stage one focuses on the requirements of

each SBU environment and through aseries of matrices assesses the deploymentof environmental requirements throughbusiness objectives to strategic and func-tional performance measures.

Stage two focuses on the business objectivesand through a series of similar sets ofmatrices assesses their deployment throughthe strategic performance measures tofunctional levels.

Stage three focuses on the deployment ofstrategic performance measures to thefunctional levels, again through the use ofsimilar matrices.

51



The TQM Magazine


Table I Extracts from an integrity audit

System Description

System 5 A policy-making system is in place where the business objectives and strategy are stated in the form of a business planThere is no evidence of the business plan and objectives being based on external competitiveposition and actual performance of the business

System 4 There is no evidence of a formal or informal mechanism for identifying the companys currentperformance with respect to the external environmentThe strategic objectives are not expressed in measurable terms with associated targets withthe exception of the objective relating to reduction of material costs

System 3 There is no evidence of a system which prioritizes and deploys the business objectives to system2 and 1. Any deployment seems to be random (see deployment audit results)

System 2 There is no evidence of system 2/co-ordination type measuresThere is no evidence of the recognition of the key business processes

System 1 There are numerous measures at this level; however, none of these are linked in any way to thebusiness objectivesMeasures at this level do not have targets associated with themMeasures at this level are not prioritized

The audit process is designed to be rigorousin order to ensure repeatability. A scoringsystem has been devised to measure thestrength of deployment which results in arigorous approach and repeatable conclu-sions. An extract from the results of thedeployment audit carried out in the collabo-rating organization illustrates the nature of theconclusions obtained from such an audit.

Extracts from the deployment audit:(1) Stage 1: SBU 1 market requirements:

Differentiator production flexibility is: not deployed as a business objective; not deployed as a strategic perform-

ance measure; marginally deployed within the

manufacturing function with nodirect measure of performance andassociated target.

Qualifier product functional perfor-mance is: partially deployed as a business

objective; not deployed as a strategic perfor-

mance measure; marginally deployed at the manufac-

turing function and is not deployedin any other function. Expecteddeployment path for this measurewould have included manufacturingand technical functions.

(2) Stage 2: SBU 1 business objectives: Business objective reduce production

and inventory costs by range rational-ization is: not deployed as a strategic measure; partially deployed in the manufac-

turing function. Expected deploy-ment path for this measure wouldhave been the technical function.

(3) Stage 3: SBU 1 strategic measures: Strategic measure cost is strongly

deployed across all functions. Expecteddeployment path for this measurewould have included all functions.

The terminology can be defined as follows: strongly deployed deployment is measur-

able with an associated target; partially deployed deployment is measur-

able with no target or priority; marginally deployed deployment is

indirect;

expected deployment path identifies thefunctions where the reference model wouldexpect to see a strong deployment.

Summary and conclusions

The research and development effort is aimedat development of practical tools and tech-niques to facilitate more scientific and precisemanagement of business performance. To this end, the first stage of the work producedan audit method designed to assess theintegrity and relevance of the performancemeasurement system used in an organization.Figure 5 summarizes the output from the firststage of the research.

More specifically the research and develop-ment work presented in this paper may besummarized as follows: The research project carried out at the

University of Strathclyde focusing onperformance measurement systems identi-fied the need to study performance man-agement as a process.

The performance management process isseen as a closed loop control system whichdeploys policy and strategy, and obtainsfeedback from various levels in order tomanage the performance of the business.

The performance measurement system isthe information system which is at theheart of the performance managementprocess and it is of critical importance tothe effective and efficient functioning of theperformance management system.

Research identified two critical elementswith respect to the content and structure of

52



The TQM Magazine


Gaps in thedesign of theperformancemeasurement

systemDeployment

gaps

Recommendations

Output

Integrity audit

Audit method

Deployment audit

Reference model

VSMintegrity

framework

Deploymentframework

Potentialdeployment

paths

PMs byfunction and by

criteria

Figure 5 Stage 1 research output from the University of Strathclyde

the performance measurement system:integrity and deployment.

The viable systems model (VSM) providesa framework for assessing the integrity ofthe performance measurement system.

Within the VSM framework, deployment isan integral element which largely corre-sponds to the amplification and transduc-tion elements of the VSM.

The reference model developed for inte-grated performance measurement systemsprovides a framework against which perfor-mance measurement systems can bedesigned and audited.

The audit methods developed to assess theintegrity and deployment of the perfor-mance measurement system provide simple but rigorous and repeatable toolswhich may be used to improve the effec-tiveness and efficiency of the performancemanagement process.

The complicated theoretical basis of auditmethods is made transparent to the userthrough the use of clearly designed work-books which take the user step-by-stepthrough the process.

References

Beer, S. (1979), The Heart of Enterprise, John Wiley & Sons,Chichester.

Bititci, U.S. (1993), Integrated Performance Measures: TheKey to Business Integration and Improvement, 9thNational Conference on Manufacturing Research,Bath, September.

Bititci, U.S. (1994), Measuring your way to profit,Management Decision, Vol. 32 No. 6, pp. 16-24.

Bititci, U.S. (1995), Performance Measurement for Perfor-mance Management, IFIP WG5.7, Working Confer-ence, Seattle, WA, August.

Bititci, U.S. and Swenson, H. (1993), Use of performancemeasures at strategic and operational levels,unpublished research report, University of Strath-clyde, Glasgow.

Blenkinsop, S.A. (1993), Organizational Aspects of Infor-mation Processing Systems, PhD Thesis, Universityof Loughborough, Loughborough.

Blenkinsop, S. and Burns, N.D. (1991), PerformanceMeasurement as an Integrating Factor in Manufac-turing Enterprises, 7th National Conference onManufacturing Research, September, pp. 231-6.

Carrie, A.S. and MacIntosh (1992), UK research inmanufacturing systems integration, in Pels, H.J.and Worthman, J.C. (Eds), Integration in ProductionManagement Systems, North Holland, Amsterdam,pp. 323-36.

Drucker, P.E. (1990), The emerging theory of manufactur-ing, Harvard Business Review, May/June, pp. 94-102.

Eccles, R.G. and Pyburn, P.J. (1992), Creating a compre-hensive system to measure performance, Manage-ment Accounting, October, pp. 41-4.

Gelders, L., Mannaerts, P. and Maes, J. (1993), Manufactur-ing Strategy and Performance Indicators, Proceed-ings of Industrial Engineering and ProductionManagement 93, Mons, Belgium, June.

Grady, M.W. (1991), Performance measurement, imple-menting strategy, Management Accounting, June,pp. 49-53.

Johnson, H.T. and Kaplan, R.S. (1987), Relevance Lost theRise and Fall of Management Accounting, HarvardBusiness School Press, Boston, MA.

Kaplan, R.S. (1983), Measuring performance a newchallenge for managerial accounting research, TheAccounting Review, Vol. 58 No. 4, pp. 686-705.

Kaplan, R.S. (1990), Measures for Manufacturing Excel-lence, Harvard Business School Press, Boston, MA.

McNair, C.J. and Mosconi, W. (1987), Measuring perfor-mance in advanced manufacturing environment,Management Accounting, July, pp. 28-31.

Neely, A.D. (1993), Performance Measurement SystemDesign, Theory and Practice, Manufacturing Engin-eering Group, University of Cambridge, Cambridge,April.

Russell, R. (1992), The Role of Performance Measurementin Manufacturing Excellence, Conference, 27thAnnual British Production and Inventory ControlSociety, Birmingham, November.

53



The TQM Magazine


Commentary

There is an old saying that if you dont measure it you wont manage it. And we know the simplersomething is to measure the easier it is to manage.

bititci et 2007 pms a development quide - tqm magazine

Documents