performance-based measurement: action for organizations and hpt accountability

11
7 Performance Improvement, vol. 49, no. 1, January 2010 ©2010 International Society for Performance Improvement Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/pfi.20116 PERFORMANCE-BASED MEASUREMENT: ACTION FOR ORGANIZATIONS AND HPT ACCOUNTABILITY Josephine A. Larbi-Apau, MPS, MPhil James L. Moseley, CPT, EdD Basic measurements and applications of six selected general but critical operational performance-based indicators—effectiveness, efficiency, productivity, profitability, return on investment, and benefit-cost ratio—are presented. With each measurement, goals and potential impact are explored. Errors, risks, limitations to measurements, and a final check for validity, applicability, accountability, and usability of these measurements are discussed. This article provides a simple, practical guide to performance measurement for organizations and human performance technology practitioners’ accountability and continuous improvement. A PERFORMANCE-BASED MEASUREMENT SYSTEM is critical for achieving accountability and results. It explains on a regular basis the measurement of output, results (outcomes), and efficiency of services, products, or programs (Spais, 2005). Regular but fundamental meas- urements such as input, output, cost, revenue, process, and impact are essential to maximize benefits and mini- mize consequences. Nonetheless, these measures alone have minimal or no meaning to the organization if they are not related to critical performance and outcome measures such as resource productivity, management effi- ciency, and team effectiveness. Profitability, return on investment of performance-based programs, and cost- benefit of training interventions have to be measured for accountability. All businesses, establishments, and organ- izations make conscious decisions to measure critical per- formance and match the results against standards or benchmarks to be competitive and successful. Similarly, micro, macro, and mega projects and programs apply performance measures to determine successes, results, capacities, and accountability. Performance measure- ments are employed by all sorts of organizations—public and private, developed and developing, industrial and service, and profit and nonprofit. Performance-based measurement helps to analyze and evaluate critical areas of human and organizational per- formance. These measurements enable practitioners to manage and improve services, determine the true value of product and projects, balance satisfaction and profit, and establish cost-effectiveness (van Aken, Letens, Coleman, Farris, & van Goubergen, 2005; Marr & Neely, 2004; Tangen, 2004). Performance measurement estab- lishes viable alliances and global competitiveness, as well as improves and justifies investments and actions. It is vital for decision making, working out project time lines, and estimating effects of projects and programs, includ- ing tangible and intangible results. Regular measurement of performance improves deci- sion making and allows managers to manage and keep focus. Kennerly and Neely (2003) and Neely (1999) have emphasized a lack of sufficient guidance for practitioners on how to evaluate, manage, and update performance measurement systems over time. Tangen (2004) suggests practical and meaningful measurements to fulfill the unique needs of practitioners. All management, depart- ments, divisions, teams, and units within the organization do not have to trade one strategy for another when they work in collaboration to determine what, why, how, and

Upload: josephine-a-larbi-apau

Post on 06-Jul-2016

226 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Performance-based measurement: Action for organizations and HPT accountability

7

Performance Improvement, vol. 49, no. 1, January 2010©2010 International Society for Performance Improvement

Published online in Wiley InterScience (www.interscience.wiley.com) • DOI: 10.1002/pfi.20116

PERFORMANCE-BASED MEASUREMENT:ACTION FOR ORGANIZATIONS AND HPT ACCOUNTABILITY

Josephine A. Larbi-Apau, MPS, MPhil James L. Moseley, CPT,EdD

Basic measurements and applications of six selected general but critical operational

performance-based indicators—effectiveness, efficiency, productivity, profitability, return on

investment, and benefit-cost ratio—are presented. With each measurement, goals and potential

impact are explored. Errors, risks, limitations to measurements, and a final check for validity,

applicability, accountability, and usability of these measurements are discussed. This article

provides a simple, practical guide to performance measurement for organizations and human

performance technology practitioners’ accountability and continuous improvement.

A PERFORMANCE-BASED MEASUREMENT SYSTEMis critical for achieving accountability and results. Itexplains on a regular basis the measurement of output,results (outcomes), and efficiency of services, products, orprograms (Spais, 2005). Regular but fundamental meas-urements such as input, output, cost, revenue, process,and impact are essential to maximize benefits and mini-mize consequences. Nonetheless, these measures alonehave minimal or no meaning to the organization if theyare not related to critical performance and outcomemeasures such as resource productivity, management effi-ciency, and team effectiveness. Profitability, return oninvestment of performance-based programs, and cost-benefit of training interventions have to be measured foraccountability. All businesses, establishments, and organ-izations make conscious decisions to measure critical per-formance and match the results against standards orbenchmarks to be competitive and successful. Similarly,micro, macro, and mega projects and programs applyperformance measures to determine successes, results,capacities, and accountability. Performance measure-ments are employed by all sorts of organizations—publicand private, developed and developing, industrial andservice, and profit and nonprofit.

Performance-based measurement helps to analyze andevaluate critical areas of human and organizational per-formance. These measurements enable practitioners tomanage and improve services, determine the true value of product and projects, balance satisfaction and profit,and establish cost-effectiveness (van Aken, Letens,Coleman, Farris, & van Goubergen, 2005; Marr & Neely,2004; Tangen, 2004). Performance measurement estab-lishes viable alliances and global competitiveness, as wellas improves and justifies investments and actions. It isvital for decision making, working out project time lines,and estimating effects of projects and programs, includ-ing tangible and intangible results.

Regular measurement of performance improves deci-sion making and allows managers to manage and keepfocus. Kennerly and Neely (2003) and Neely (1999) haveemphasized a lack of sufficient guidance for practitionerson how to evaluate, manage, and update performancemeasurement systems over time. Tangen (2004) suggestspractical and meaningful measurements to fulfill theunique needs of practitioners. All management, depart-ments, divisions, teams, and units within the organizationdo not have to trade one strategy for another when theywork in collaboration to determine what, why, how, and

Page 2: Performance-based measurement: Action for organizations and HPT accountability

8 www.ispi.org • DOI: 10.1002/pfi • JANUARY 2010

when to measure for maximum effect. Six performance-based measurements and potential impacts are simply andpractically presented in this article for human performancetechnology (HPT) practitioners and organizations: effec-tiveness, efficiency, profitability, productivity, return oninvestment, and benefit-cost ratio. The performance-basedmeasurement illustration in Figure 1 is a continuum: fromno action, to minimal action, to maximum action that istaken with the purpose of increasing individual and orga-nizational accountability. In each case, the arrows representa different indicator in a direction of no action to higheraccountability and results.

This diagram helps evaluators determine where they areand where they would want to be as far as accountability isconcerned. This representation is not a matrix, as would betypically interpreted on x- and y-axes, and the indicatorsare not arranged in a hierarchical order; rather it is aframework designed to guide and interpret financial-basedperformance measurement on the x-axis and nonfinancial-based measurement on the y-axis. Effectiveness, effi-ciency, and productivity, which are typically nonfinancialindicators, can be determined on the y-axis, while benefit-to-cost ratio, return on investment, and profitability, thefinancial indicators, are determined on the x-axis. Theseindicators are interrelated; hence, the evaluator has tospecify what to measure based on the intended purpose,task to perform, and performance value. For example,

being more effective and efficient increases productivity.High levels of effectiveness imply improvement in deci-sion making and the quality of services, programs, orprojects. Profitability is a function of increased produc-tivity and efficiency, and profitability and productivity aretypical concepts of efficiency.

PERFORMANCE-BASEDMEASUREMENTS, GOALS, AND POTENTIAL IMPACTThe six performance indicators are presented relative to their specific goals, potential impacts, and effects.The selection of these six is based on the frequency ofapplication, purpose, and criticality. In measuring theseindicators, first determine the set of performance measurements (in quality and quantity) and how theywill be focused to balance and align to personal and orga-nizational goals or benchmarks. Second, determine howthese measures will be deployed and implemented toeffect change and improve performance. Third, focusmore on outcomes than output. Outcome-focused measurements determine to what extent a product, pro-gram, or service contributes to achieving desired results.Outcome-based measurements provide a clearer pictureof the organization’s performance. Measurement of these

FIGURE 1. DIRECTION OF CRITICAL PERFORMANCE-BASED MEASUREMENTS FOR ACCOUNTABILITY

Page 3: Performance-based measurement: Action for organizations and HPT accountability

Performance Improvement • Volume 49 • Number 1 • DOI: 10.1002/pfi 9

indicators can be more complex than presented in thisarticle, but the general methods, goals, and potentialeffects are comparable for the intended purpose. We nowturn to the performance-based indicators.

EffectivenessEffectiveness explains the ability to produce actual results,output, or effects based on planned or standard output. Itis an outcome of doing the right thing versus doing thething right. Effectiveness measures apply to performanceof leadership, management, services, customer retention,product quality, and decision making. It is the ability todefine strategic goals and align them with missions of theorganization, the degree of stakeholder’s commitmentand collaboration to reach a common purpose, and howwell and timely services and products are made accessibleor affordable (or both) at the right place. The power ofeffectiveness is generally measured as the ratio of actualoutput over planned output and is expressed as:

Effectiveness =Actual or standard output

Planned or expected output

Effectiveness measurement provides a general over-view of performance in comparison with the orga-nization’s benchmark or defined standard. To measureeffectiveness:

1. Plan expected output, for example, the number ofclients to attend to in a day or tasks to perform.

2. Determine actual output or standard output, such asthe number of actual tasks performed.

3. Divide the standard output by the planned or expectedoutput.

4. Convert to percentage or monetary value for a clearerpicture.

For example, the effectiveness of performance can bebased on the number of customers attended to on time (9 out of 10), number of legal cases resolved per week (2 out of 3), or tasks completed per day (12 out of 12) bycomparing the actual against the planned output for the day, week, month, or year. Effectiveness expresses theactual production of or the power and capacity to achieveor produce an effect.

Potential Impact. Improved customer service, trackingand retention, and daily performance are potentialimpacts of this measure. Effectiveness measures improvedecision making and put employees back in control oftheir actions and performance. This measurement alsoimproves self-confidence and achievement, includingevery workday practice and experience when desired

results are accomplished. Effectiveness is an indicator ofthe improved value of the organization’s program, project,or product. It expresses general success and relates directlyto the actual output against expected output, such as thegeneral performances of the four vital stakeholders:employees, customers, suppliers, and shareholders.

EfficiencyEfficiency is quantitative and measures improved or in-creased services and products generated without chang-ing the inputs. It is in contrast to effectiveness, where thething is done right instead of doing the right thing. It isone of the firm’s critical performance measurements andinvolves selecting the appropriate inputs (Mester, 2003).Efficiency is used to determine best practices and costsassociated with practices or operations. It is intended,among others, to determine the true cost of products andservices and customers’ sensitivity to changes in price.According to Silva and Stefanou (2007), “Long-run effi-ciency measures indicate the relative efficiency of bothvariable and dynamic factors while short-run measures of efficiency indicate whether variable inputs areemployed efficiently in the production process” (p. 402).Generally efficiency is measured as the ratio of amount ofwork output or operation (usually energy, time, ormoney) over amount of work input or operation and isexpressed as a percentage, ratio, or fraction:

Efficiency (%) = Amount of work output

� 100Amount of work input

To measure efficiency:

1. Determine the parameter to measure, such as theactivity, work, or project.

2. Identify the variables to include or select, for example,output values and input levels.

3. Measure selected variables, for instance, base expendi-ture in relation to a particular year, employee inputand output at a particular time, or speed with whichcustomer needs are responded to.

4. Compute the ratio of the quantity or value of outputand the corresponding variable input.

5. Multiply the results by 100 to convert to a percentage.

Outcome-based measurementsprovide a clearer picture of theorganization’s performance.

Page 4: Performance-based measurement: Action for organizations and HPT accountability

10 www.ispi.org • DOI: 10.1002/pfi • JANUARY 2010

6. Compare results to a standard or benchmark of anorganization or business that is operating at the samelevel and context.

7. Interpret this result to determine the organization’s orindividual’s efficiency level.

Examples of outcome-focused efficiency measure-ments are cost per satisfied client served in a restaurant,management time per cost of client who consulted toimprove a business plan, and employee time per actualamount of pizza delivered to satisfied customer. Othermeasures are allocative and technical efficiencies.Allocative efficiency refers to how inputs are chosen andmixed to produce an output. Technical efficiency explainshow technological inputs are varied to produce an out-put. Efficiency can be applied to long-term or short-termgoals. Short-term measures indicate the employment ofvariable factors, while long-term measures indicate therelative efficiency of both variable inputs and investmentin quasi-fixed factors (Silva & Stefanou, 2007).

Potential Impact. The potential effects of efficiencymeasures are improved practices, reduced errors or min-imized waste and costs, and time saved. Measuring effi-ciency enables organizations to deliver top-qualityservices to clients and ensures delivery of services orproducts at minimum possible cost. For example, Rosko,Mutter, and Maryland (2008) suggest that a direct mea-sure of hospital inefficiency helps to clarify the impact ofenvironmental factors on important dimensions of hos-pital performance, such as eliminating medical errors andunnecessary procedures.

Efficiency improves the organization’s ability to retainor attract price-sensitive customers, that is, the ability todetermine the cost at which customers can be retained andare satisfied to pay (demand) for products and services.Efficiency measurement means maximizing output andprofits and establishing the true value of the product orservice delivered. The relative efficiency of products, serv-ices, or performance can be compared to previous perfor-mance outcome to improve practices and productivity.

ProfitabilityProfitability measures the quantitative differencesbetween revenue obtained from an output and the costsassociated with the use of inputs in producing that out-put. Revenue is estimated by multiplying the price atwhich the goods or services are sold by the number ofunits sold. Similarly, cost is calculated by multiplying theprice at which the inputs are purchased. The goals ofprofitability measurement include the ability to effec-

tively determine cost and resource allocations, as well as viable alliances and partnerships. Profitability meas-urement helps to manage and balance satisfaction, cost-effectiveness, and efficiency. Profitability determinesachievement of balance in profit by factors such asemployee salaries and incentives, customer retention, anderrors in supply. Profitability is estimated as total earningsless total expenses (or potential total earnings less poten-tial total expenses) and is given by:

Profitability = Total earnings or revenue – Total expenses or costs

Examples of profitability (profit) measurements follow:

• Gross margin: Gross profit divided by revenue

• Research and development (R&D) to sales: R&Dexpense divided by revenue

• Net profit margin: Net income (after taxes) divided byrevenue

• Return on equity: Net profit divided by average share-holder equity for the specified period

• Operating margin is equal to operating incomedivided by revenue

• Interest coverage ratio/earnings before interest:Earnings before interest and tax divided by interestexpense

For example, net profit for a service or product can becalculated as the difference between total earnings and thetotal expenses or cost plus administrative and overheadcharges or costs. To calculate net profit:

1. Determine total earnings.

2. Calculate total costs of producing a service or product.

3. Calculate administrative cost relative to that productor service.

4. Determine all overhead charges relative to that prod-uct or service.

5. Sum total cost (production, administration, and overhead).

6. Subtract total cost from total earnings.

7. Interpret the results, and make a decision.

Marginal profit is estimated as a ratio of the netincome by sales or earnings. Profit is maximized at apoint where the marginal cost equals the marginal rev-enue (MC = MR) and measures how much is retainedfrom every dollar earned.

Page 5: Performance-based measurement: Action for organizations and HPT accountability

Performance Improvement • Volume 49 • Number 1 • DOI: 10.1002/pfi 11

Potential Impact. The effects of this measure includeimproved financial management and transactions.Making profit means expansion and improved practices.Maximizing profits leads to enhanced employee salaries,incentives, and general remunerations. It is an indicatorfor increased return on net worth and equity of stake-holders.

ProductivityProductivity is generally a quantitative measure of outputand input used to produce an output. “Productivity is aconcept in close relation to profitability, economicgrowth, efficiency, surplus value, quality, performance,partial productivity, and need” (Saari, 2006, p. 1). It canbe measured as total productivity, in which case allresources are considered, or partial productivity, whenoutput is measured against specific selected units ofresource or input factor such as in labor productivity orcapital productivity. Usually productivity measurement isused to determine and allocate resources (human andnonhuman). Productivity can be used to track perfor-mance and reward systems and organizational effective-ness and to compare staff or employee performance. Itcan be used to control sectors by function or product,determine relative benefits of different and variedresources, or compare with competitors. This measure-ment takes into account the value of output and inputs,which differentiates it from allocative efficiency.Generally the total factor productivity is expressed as:

Total productivity = Total output

Unit input or total inputs

Usually estimates of the values of input and output aredependent on the base price index since subsequentprices change over time. This means that the price that isused to estimate the value today may not correspond tothe same value tomorrow. Remember the time value ofmoney. To measure productivity:

1. Identify the input and output you want to measure,such as management, leadership, or type of products.

2. Measure unit quantity of inputs used in levels or values.

3. Measure quantity output in levels or values, such asgallons of milk produced or number of customersserved.

4. Estimate the ratio of the output per unit of input usedfor each enterprise, product, or project or for the totalenterprise, product, or project.

5. The result is the estimated productivity level for thatenterprise, product, or project.

Examples of productivity measures are students taughtper teacher (student-to-teacher ratio), patients seen perphysician (patients-to-physician ratio), customers han-dled per employee, and percentage of orders completed.Productivity is used to determine the value added by theprocess, product, or service. It is a ratio of the value addedby the inputs such as capital, human factor, and othervariable inputs consumed: Output = f (capital, labor,other variables).

Potential Impact. Increased productivity meansimproved content and reward systems, improved staffperformance levels, and better control systems. Itimplies increased functionality, accountability, globalcompetitiveness, and organizational production. Pro-ductivity tracks worker effectiveness and indicates howefficiently inputs or resources are used. For example,Pritchard, Harrell, DiazGranados, and Guzman (2008)conducted a meta-analysis a using productivity meas-urement and enhancement system (ProMES) interven-tion in organizations and found that productivityimprovements were large with the application of theintervention. They reported that work lives and generalproductivity can be improved when these measurementsare related to the job, organizational practices, and man-agement support. The ProMES is an intervention systemthat measures a set of defined objectives and quantifi-able productivity indicators and provides feedback forimprovement.

Return on InvestmentReturn on investment (ROI) can be qualitative or quanti-tative. It measures the incremental gain or loss (return)from an action, project, or program relative to investmentcost of that action, project or program. ROI compares themonetary benefits to costs (Phillips & Phillips, 2008;Pershing, 2006). The higher the ROI percentage, the bet-ter the rate of the return on investment of an action, proj-ect, or program. However, excessive use of ROI mayconflict with strategic objectives and strategic building(Tangen, 2004). It is mainly a financial measure and maynot be directly related to qualitative human performance.The goals of ROI include the ability to make a soundinvestment decision or justify the usefulness of invest-ment action. It also helps to manage human capital orresources effectively. As a stand-alone measure, it does not help to improve performance. ROI is expressed as apercentage:

ROI (%) = Investment gains – Investment cost

� 100Investment cost

Page 6: Performance-based measurement: Action for organizations and HPT accountability

12 www.ispi.org • DOI: 10.1002/pfi • JANUARY 2010

To estimate financial ROI:

1. Calculate the total investment costs of the specific pro-gram you want to measure.

2. Determine the total financial (investment) gains fromthe design and implementation (plus the evaluation ifit will be conducted) of the project.

3. Subtract the total investment cost from the totalinvestment gains.

4. Divide by the total investment cost.

5. Multiply by 100 to obtain the percentage score.

For example, the net ROI for a new performanceimprovement support software training that cost$650,000 over its productive life and yielded $900,000 incumulative gains can be calculated as:

ROI (%) = $900,000 – $650,000

� 100 = 38.46%$650,000

In monetary terms, this means a cumulative gain of$38.46 per every $100 spent on the project.

Potential Impact. Potential gains from ROI measures areenhanced investment and financial decision making. Itinfluences other decision-making functions such as skillsand training, human resource selection, and capital devel-opment. ROI improves business investment judgmentand self-justification; however, as a single calculation, itdoes not provide insight into future business perfor-mance or how to improve business results, and it does notmeasure and analyze risk factors.

Return on investment has been applied in estimatingsocial benefits and impact in charitable organizations(Polonsky & Grau, 2008) and also to quantify the value indifferent companies and organizations (Phillips &Phillips, 2008; Burkett, 2008). Polonsky and Grau (2008)report that social value could provide critical informationto help in decision making and developing internalbenchmarks to improve social performance in nonprofitcharitable organizations. In a comprehensive case studyto measure the ROI of a program in a global company,Burkett (2008) reports a link between improved produc-tivity and labor efficiency. Both tangible results, such asincreased operational capacity, and intangible results,such as improved clarity of priorities, roles, and responsi-bilities of managers, were reported in this study.

Benefit-Cost Ratio Benefit-cost ratio (BCR), or cost-benefit ratio, evaluatesthe need to implement a course of action. This measureallows the evaluator to put a quantitative value on bothtangible (physical) and intangible (such as social) costs

and benefits of an action. The goal is to determinewhether to embark on a project or follow a course ofaction. BCR allows the practitioner to determine projectworth to make sound changes or implementation deci-sions. It helps to work out project time lines and paybackperiods and to estimate physical and social or environ-mental effects of the project. It is a general tool for deter-mining costs and benefits of planned change. BCR isestimated as:

Benefit-cost ratio = Value of projected benefits

Value of associated costs

To measure BCR:

1. Determine all projected costs associated with the pro-gram or project.

2. Characterize all projected benefits of the project orprogram.

3. Determine the duration in weeks, months, or yearsthat the project or program will travel.

4. Compare the cumulative total to determine the vari-ance in value (the difference).

5. Express projected benefits and projected costs as aratio.

Table 1 summarizes the cumulative BCR estimates of afour-week performance support training program. Fromthe table, the total cumulative BCR equals 1.3 point ben-efit to 1 point cost (or simply 1.3), which in monetaryvalue means that for every $1.00 invested, $1.30 was gen-erated as an investment gain. BCR does not provide the causality in variance; therefore, further analysis suchas payback period is required for better decision making.Payback period is the length of time or period required torecover the original investment cost.

Potential Impact. Benefit-cost ratio improves investmentand financial decision making, as well as change deci-sions and functions. It is used to compare actual costs andbenefits such as development and implementation of a

Return on investment hasbeen applied in estimatingsocial benefits and impact incharitable organizations.

Page 7: Performance-based measurement: Action for organizations and HPT accountability

Performance Improvement • Volume 49 • Number 1 • DOI: 10.1002/pfi 13

new system, project, or training. BCR is based on pro-jected estimates during the planning and analysis stages. Itcould be used to increase savings, reduce the need toemploy more workers to perform the same job, or avoidunnecessary training costs. In a social context, it could beused to qualitatively improve response to customerdemands and make better decisions in community service.

Levina, Belfield, Muennig, and Rouse (2007) estimatedthe public returns to investments in African Americanmale high school dropouts and reported a benefit-to-costratio of 2.83. This result implies that increased investmentin these cohorts could yield higher dividends. Table 2summarizes the general methods for each of the criticalperformance-based measurements, aligned with sug-gested goals and potential impacts and effects on theorganization and HPT practices.

ERRORS, RISKS, AND LIMITATIONS TOPERFORMANCE-BASED MEASUREMENTSPerformance measurement requires data collection,analysis, and reporting for both simple and complex fixes.Only data required should be collected since too muchinformation results in information overload and timewasting and too little information is inadequate for theintended purpose. Error determination, risk factors, andlimitations of these indicators are crucial for accuracy and reliability of the measurements.

ErrorsErrors of duplications in costs and overestimations mustbe particularly checked. Costs must be accurate and veri-

fiable; for example, communal costs and expenses that areshared between numbers of services or costs associatedwith free services can result in errors in performance-based measurement. Other inaccuracies could come fromunaccounted resources, divested resources (e.g., redeploy-ing human resource from production to service), andinappropriate links between personal and organizationalgoals and strategies, which could result in inconsistenciesand unrealistic measurements. Multiple measures of thesame performance indicators can result from failure torecognize and differentiate roles played by stakeholders,suppliers, or other clients.

RisksRisks are potential partial outcomes or elements that couldresult in superfluous occurrences in measurements if theyare not reduced or prevented. Examples of risk factors arefailure to identify, track, or measure the performance of aniche or segment of customers or overlooking small proj-ects or costs that could undercut the overall outcome of aprogram. Risk factors can be physical, such as when substi-tutes for production are not tracked or when some finan-cial investment is unauthorized and therefore cannot beaccounted for. Other risk factors are associated with vio-lated social costs that are misapplied or diverted. For exam-ple, in a nonprofit or charitable organization, funds couldbe diverted for emergency relief and overlooked.

LimitationsLimitations of performance-based measurements includethe inability to determine causes of failure or nonperfor-mance. Also, effects of uncertainties and natural phenom-ena cannot be measured directly. In addition, motivation,

CUMULATIVE BENEFIT-COST DRAFT OF PROJECTED 4-WEEK PERFORMANCE TABLE 1 SOFTWARE PROGRAM

CUMULATIVE BENEFIT-COST DRAFT

DESCRIPTION WEEK 1 WEEK 2 WEEK 3 WEEK 4 TOTAL

Projected benefits 0 0 $30,000 $35,000 $65,000

Projected costs $10,000 $15,000 $18,000 $20,000 $63,000

Difference ($10,000)a ($15,000)a $12,000 $15,000 $2,000

BCR 1.6 2.3 1.3

aNo projected benefits for weeks 1 and 2; negative difference.

Page 8: Performance-based measurement: Action for organizations and HPT accountability

14 www.ispi.org • DOI: 10.1002/pfi • JANUARY 2010

CRITICAL PERFORMANCE-BASED MEASUREMENTS, ALIGNED WITH SUGGESTED TABLE 2 GOALS AND POTENTIAL IMPACT

TYPE OF MEASUREMENT GENERAL METHOD USED SUGGESTED GOALS POTENTIAL IMPACT

Effectiveness: Ability to Effectiveness = Actual output/produce actual results, output, Planned outputor effect based on planned output or standard

Efficiency: Improved or Efficiency (%) = Amount of workincreased services or products output/Amount of work input generated with same units of � 100 input

Profitability: Measures Profit = Revenue – Costdifferences between the revenue obtained from output and the cost associated with the use of the input

Productivity: Measures output Total factor productivity = Total from production processes or output/Unit input or inputsresources used to produce that output

Return on investment (ROI): ROI (%) = Investment gains – Incremental gain or loss Investment cost/Investment cost(return) from an action, � 100project, or program relative to investment cost of that action, project, or program

Benefit-cost ratio (BCR): BCR = Value of benefits/Value Evaluates quantitatively the of costs associated with itneed to implement a course of action

Measures performance of:

Team project

Leadership

Management

Service and customer retention

Product quality

Determine best practices

Lower cost and save time

Determine true value of product or service

Determine price sensitivity

Improve service, customer satisfaction

Determine cost and resource allocation

Determine viable alliances andpartnerships

Balance satisfaction and profit

Determine supplier errors

Manage costs

Determine labor resourcesrequired to produce an output

Determine global competitiveness

Track performance and rewardsystems

Control units by functions

Compare staff performance levels

Make sound investment judgments

For self-justification

Manage human capital investment

Self-investment

Justify investment action

Determine project worth

For making decision

Work out project time line

Work out payback period

Estimate intangible effects

Improved customer and servicetracking and retention

Puts employees back in control

Improved self-confidence andperformance

Improved best practices

Cost reduction and time saving(without waste)

True value established––controlled spending

Improved financial transactions

Enhanced employee salary andincentives

Increased return on net worthand equity

Improved reward systems

Improved staff performance

Better controlled system andaccountability

Improved decision making

Improved skills and training

Increased human capital andother resources

Improved decision making

Improved financial decision

Improved change decision

Page 9: Performance-based measurement: Action for organizations and HPT accountability

Performance Improvement • Volume 49 • Number 1 • DOI: 10.1002/pfi 15

good leadership, decision making, incentives, and otherhuman factors that could potentially influence perfor-mance cannot be wholly captured or accounted for bythese measurements; for example, crime prevention, ille-gal drug use, sexual harassment, time spent on private e-mailing and texting, or pilfering on the job. Accordingto Artley and Stroh (2001), performance objectives thatare measured and expressed only in numerical quotas do not fix defective processes. A further analysis of thechallenges and required changes in processes is whatdetermines the action for improving performance andachieving the desired results.

RecapPerformance measurement is complex when differentsystems, products, and processes are integrated. The prac-titioner or evaluator should always be guided by the pur-pose, extent of measurement, details required, availableresources, and time factor. Accessibility of relevant sup-porting data, associated cost, definable human factors,and feasibility should guide the decision to measure andimplement the results.

THE FINAL CHECKA final check to ascertain validity, usability, applicability,and accountability is critical to the overall process.Performance measurement is meaningless if it is notrelated to organizational or individual goals, practices,and performance. Hatry (1999) argues that “if the rightthings are not measured, or are measured inaccurately,those using the data will be misled and bad decisions willlikely follow” (p. 3). Performance measures should be val-idated through team processes as a means to increaseusability and validity. Teamwork is vital for effectiveapplication, coordination, and alignment of objectives,strategies, and results. Team collaboration is required towork within schedule and maintain a standard of perfor-mance. It ensures buy-in and ownership and avoids con-

flicts with other performance objectives and results.Teamwork is essential for determining levels of successand the rationale for general improvement.

The SMART test can be used to guide objective appli-cations and reduction in subjectivity and bias. SMARTmeans the measure must be specific (S), clear, and tar-geted without ambiguities. The selected performanceindicators should be measurable (M) and quantifiable insuch a way as to be compared with similar and previousdata or measures. The attainable (A) variable responds tothe achievability, reasonability, and credibility of themeasurement. Within the constraints and contexts of per-sonal and organizational goals, it is important that themeasurement is realistic (R) and cost-effective, as well astimely (T) for the preferred outcome.

Exhibit 1 provides a checklist for a final check of themeasurements based on the SMART model. Standards arealigned with criteria and decision points. The decision-making process is dependent not only on the standardsbut also on motivation, incentives, good leadership andjudgment, and other human factors that cannot be cap-tured in the measurements but have potential influence onperformance. It is imperative that leadership, divisions,and teams track, monitor, and modify these indicators relative to standards and benchmarks for potential causesand solutions. Performance-based measurements canalways be planned, selected, and modified to reflect anorganization’s goals and strategies. The selected criticalfew of performance measures should be emphasized tobalance the organization’s internal and external perfor-mance needs and requirements (Artley & Stroh, 2001).

Validating and confirming the results requires:

• Corroborating the rationale of the measurement withthe organization’s broad strategic goals and specificobjectives.

• Establishing the use of particular measurements forspecific or large effects.

• Rechecking the factors and determining how they aremeasured as a unit of a factor or in combination withother factors to avoid the risk of unrealistic multiplemeasurements.

• Determining how these measures will interpret intofactors that can contribute to the overall success of theprogram, project, product, or service.

• Reviewing the infrastructures that support the changeand potential implementation of the results.

• Confirming the tangible and intangible effects of themeasurement to the overall transformation andimprovement in performance.

• Interpreting, reporting, implementing, tracking, mon-itoring, and evaluating the results.

Multiple measures of thesame performance indicatorscan result from failure torecognize and differentiateroles played by stakeholders,suppliers, or other clients.

Page 10: Performance-based measurement: Action for organizations and HPT accountability

16 www.ispi.org • DOI: 10.1002/pfi • JANUARY 2010

The context and accuracy of the measurement are crit-ical elements for translating the results into the desiredeffects. Van Aken et al. (2005) argue that a top-down con-trol performance model ignores organizational realitieswhen it uses a strategic management system withoutaddressing the key contextual external environment.

CONCLUSIONPerformance-based measurement is a critical element inachieving management results. These measures help HPTpractitioners and organization managers make informed

decisions on best practices. They provide accountabilityand benchmarks in deciding what should be discarded,changed, or improved. Accounting for an organization’soperations, performance practices, and achievementrequires measurements such as effectiveness, efficiency,profitability, productivity, return on investment, andcost-benefit ratio to improve human and general perfor-mance. These measurements should be applied to estab-lished personal and organizational goals, standards, orbenchmarks. Focusing on outcomes can help to deter-mine the causality of gaps in performance. A balancedapproach aligned with defined strategies and best prac-

EXHIBIT 1 FINAL CHECKLIST FOR ACCOUNTABILITY, VALIDITY, APPLICABILITY, AND USABILITY

This checklist is for conducting the final performance test for accountability, applicability, validity, and usability. Make a decision to implement the performance by tracking, monitoring, and modifying the results of your measurement for the intended purpose. Check the appropriate oval: Excellent, Very good, Good, and Fair. Interpret, make a decision–determine the causes and solutions.

STANDARD DECISION POINTS

EXCELLENT VERY GOOD GOOD FAIR CRITERIA 90–100% 80–89% 70–79% LESS THAN 70% TRACK? MONITOR? MODIFY?

SPECIFIC?

Clear goal

Targeted

Well-defined

MEASURABLE?

Quantifiable

Qualitative

Comparable

ATTAINABLE?

Cost-effective

Context-fit

Content-fit

RELIABLE?

Credible

Manageable

Meaningful

TIMELINESS?

Efficient

Feasible

Best-fit

Page 11: Performance-based measurement: Action for organizations and HPT accountability

Performance Improvement • Volume 49 • Number 1 • DOI: 10.1002/pfi 17

tices from the customer, employee, and organization’spoint of view will ensure overall improvement in perfor-mance and systemic change.

References

Artley, W., & Stroh, S. (2001, September). The performance-based management handbook: Establishing an integrated perfor-mance measurement system (Vol. 2). Performance-BasedManagement Special Interest Group. Retrieved August 20,2008, from http://www.orau.gov/pbm/pbmhandbook/Volume%205.pdf.

Burkett, H. (2008). Measuring ROI in a career development ini-tiative: A global company. In P.P. Phillips & J.J. Phillips (Eds.),ROI in action casebook. San Francisco: Jossey-Bass/Pfeiffer.

Hatry, H. (1999). Performance measurement: Getting results.Washington, DC: Urban Institute Press.

Kennerly, M., & Neely, A. (2003). Measuring performance in achanging business environment. International Journal ofOperations and Production Management, 23(2), 213–229.

Levina, H.M., Belfield, C., Muennig, P., & Rouse, C. (2007). Thepublic returns to public educational investments in African-American males. Economics of Education Review, 26, 700–709.

Marr, B., & Neely, A. (2004). Managing and measuring forvalue: The case of call center performance. Bedfordshire: Schoolof Management and Fijitsu, Cranfield University.

Mester, L.J. (2003, July). Applying efficiency measurement tech-niques to central banks (Working Paper No. 03–13). TheWharton School, University of Pennsylvania. Retrieved fromhttp://fic.wharton.upenn.edu/fic/papers/03/0325.pdf.

Neely, A. (1999). The performance measurement revolution:Why now and what next? International Journal of Operationsand Production Management, 19(2), 205–228.

Pershing, J.A. (Ed.). (2006). Handbook of human performancetechnology (3rd ed.). San Francisco: Jossey-Bass.

Phillips, P.P., & Phillips, J.J. (Eds.). (2008). ROI in action case-book. San Francisco: Jossey-Bass/Pfeiffer.

Polonsky, M.J., & Grau, S.L. (2008). Evaluating the social valueof charitable organizations: A conceptual foundation. Journalof Macromarketing, 28(2), 130–140.

Pritchard, R.D., Harrell, M.M., DiazGranados, D., & Guzman,M.J. (2008). The productivity measurement and enhancementsystem: A meta-analysis. Journal of Applied Psychology, 93(3),540–567.

Rosko, M.D., Mutter, R.L., & Maryland, R. (2008). Stochasticfrontier analysis of hospital inefficiency: A review of empiricalissues and assessment of robustness. Medical Care Researchand Review, 65(2), 131–166.

Saari, S. (2006). Productivity: Theory and measurement in busi-ness. Retrieved June 13, 2008, from http://www.mido.fi/index_tiedostot/Productivity_EPC2006_Saari.pdf.

Silva, E., & Stefanou, S.E. (2007). Dynamic efficiency measure-ment: Theory and application. American Journal ofAgricultural Economics, 89(2), 398–419.

Spais, G.S. (2005). A fundamental formula for leadership effi-ciency—Weighting the contribution of “Marketing Paideia.”Journal of Strategic Marketing, 13(1), 61–77.

Tangen, S. (2004). Professional practice: Performance measure-ment: From philosophy to practice. International Journal ofProductivity and Performance Management, 53(8), 726–737.

van Aken, E.M., Letens, G., Coleman, G.D., Farris, J., & vanGoubergen, D. (2005). Assessing maturity and effectiveness ofenterprise performance measurement systems. InternationalJournal of Productivity and Performance Management, 54(5/6),400–418.

JOSEPHINE A. LARBI-APAU, MPS, MPhil, is a doctoral candidate in instructional technology andRumble Fellow at Wayne State University College of Education. She is a program and training manager,agribusiness consultant, and assistant director of education. She is a graduate of Cornell University,Ithaca, New York, and the University of Ghana, Legon. She may be reached at [email protected].

JAMES L. MOSELEY, CPT, EdD, is an associate professor in instructional technology at Wayne StateUniversity College of Education, and a performance consultant with the Lake Group. He is the coauthorof six books and numerous articles. In addition to Training Older Workers and Learners, three of hisbooks—Fundamentals of Performance Technology: A Guide for Improving People, Process, andPerformance (ISPI, 2000, 2004), Performance Improvement Interventions: Enhancing People and Process Through Performance Technology (ISPI, 2001), and Confirmative Evaluation: PracticalStrategies for Valuing Continuous Improvement (Pfeiffer, 2004)—have received the ISPI Award ofExcellence for Outstanding Instructional Communication. He may be reached at [email protected].