7082903 store 24 balanced scorecard

8/14/2019 7082903 Store 24 Balanced Scorecard

1/44

08-081

Testing Strategy with Multiple Performance Measures Evidence from a BalancedScorecard at Store24Dennis Campbell Srikant M. Datar Susan L. Kulp V.G. Narayanan

Copyright 2008 by Dennis Campbell, Srikant M. Datar, Susan L. Kulp, and V.G.Narayanan Working papers are in draft form. This working paper is distributed for

purposes of comment and discussion only. It may not be reproduced withoutpermission of the copyright holder. Copies of working papers are available fromthe author.


2/44

Testing Strategy with Multiple Performance Measures Evidence from a BalancedScorecard at Store24* Dennis Campbell Srikant Datar Harvard Business School SusanL. Kulp George Washington University V.G. Narayanan Harvard Business SchoolCurrent Draft: February 2008ABSTRACT: We analyze balanced scorecard data from a convenience store chain,Store24, during the implementation of an innovative, but ultimately unsuccessfulstrategy. Quarterly strategic reviews, based in part on the firm's balancedscorecard, led executives at Store24 to identify problems with, and eventually

abandon, this strategy over a two year period. We find that formal statisticaltests of the hypotheses underlying the firm's balanced scorecard and strategy mapreveal problems with the strategy on a timelier basis. We also test alternativehypotheses to those underlying the firm's formal strategy map and scorecard thatare consistent with concerns expressed by some of Store24's top executives duringthe initial stages of implementing the new strategy. Our analysis demonstratesthat this firm's balanced scorecard contained useful and timely information fordistinguishing between these alternatives. These results provide some of the firstfield-based evidence on the potential for a firm's balanced scorecard to provideuseful information for detecting problems in its strategy.

I.

Introduction

This study investigates the role of the balanced scorecard in generating usefulinformation for testing and validating an organization's strategy. Numerous casestudies of balanced scorecard

implementations document their use in translating organizational strategies toobjectives and measures, communicating strategic objectives to employees,evaluating the performance of business units, and aligning the incentives ofemployees across business units and functions.1 Field-based and experimentalresearch in the accounting literature has also focused on these uses of balancedscorecards (Malina and

The authors thank Store24 for use of its data. We thank Chris Ittner, RobertKaplan, Ken Koga, Joan Luft, Michael Maher, Ella Mae Matsumura Tatiana Sandino,Philip Stocken, Dan Weiss, two anonymous referees, and seminar participants at theAAA Annual Meeting in Orlando, Boston University, the EIASM conference, HarvardUniversity, Management Accounting Section Mid-year Meeting in San Diego, MichiganState University, Ohio State University, University of Arizona, UCLA, Universityof Michigan, University of Southern California, and the University of Wisconsinfor their helpful comments and suggestions. 1 See Kaplan (1998), Campbell and Lane(2006), or many of the organizations documented in Kaplan and Norton (2006) forexamples.

*

1


3/44

Selto 2001; Lipe and Salterio 2000: Ittner et. al. 2003; Banker et. al. 2004;Campbell 2008). In addition to these uses, the literature on the balancedscorecard has long argued that these measurement systems can play a role infacilitating feedback and learning through the testing, validation, and revisionof the underlying strategic assumptions embedded in the scorecard (Kaplan andNorton 1996, 2004, 2008). Despite these long-standing claims, there has beencomparatively little research on this potential learning and feedback role ofbalanced scorecards. The testing and validation of assumptions underlying balanced

scorecards is an important topic given the considerable uncertainty faced bydecision-makers in designing these measurement systems. The balanced scorecardframework advocates choosing performance metrics related to key financial andcustomer objectives, the firm's internal processes for achieving these objectives,and organizational capabilities necessary to execute its internal processes.Further, performance measures should be

explicitly linked via a "strategy-map" of hypothesized "cause-and-effect"relationships that depict the firm's strategy (Kaplan and Norton 2000; 2004).Improvements in measures of organizational capabilities are expected to driveimprovements in the execution of internal processes which in turn lead to customerand financial outcomes. In this way, the balanced scorecard framework explicitlyrecognizes

interrelationships between strategy-specific measures of financial and customeroutcomes and inputoriented "performance drivers" related to the firm's internalprocesses and organizational capabilities. However, managers must formulatestrategies, and select related objectives and measures, based on exanteexpectations about how the strategy will translate into financial performance.Thus, the strategic objectives and performance measures chosen for anorganization's scorecard are often based on uncertain hypotheses about howmeasured performance against these objectives ultimately leads to financialperformance. Proponents of the balanced scorecard concept have long recognizedthis uncertainty. To mitigate against it, and as a mechanism for strategicfeedback and learning, they have advocated for formal testing of the hypothesizedlinkages among the performance measures included in an organization's scorecard

(Kaplan and Norton 1996; 2004, 2008). Theoretically, multiple performance measuresselected based on

2


4/44

an organization's unique strategy, coupled with a set of hypothesizedrelationships among these performance measures that describes the strategy (e.g. a"strategy map" or "business model"), should provide the necessary data forstrategy validation and testing via standard statistical analysis techniques.Particularly in organizations with large numbers of similar operating units,statistical analysis can potentially be used to explicitly test the hypothesizedlinkages among the performance measures in the balanced scorecard and provideearly feedback about the validity of the underlying strategic assumptions.

However, there are at least three reasons why such an approach may not provideuseful information even in a setting where improvements in nonfinancialperformance measures are expected to lead improvements in financial performancewith a short time-lag. First, nonfinancial performance measures related to afirm's unique strategy may be noisy indicators of true underlying strategicperformance limiting their usefulness for statistical testing purposes. Consistentwith this notion, the problem of "quantifying qualitative information" hasfrequently been noted by decision-makers as a significant challenge inimplementing balanced scorecards (Ittner and Larcker 1998). Second, an

organization's balanced scorecard and associated strategy map may not capture alldimensions necessary for a strategy to succeed. Explicit (e.g. technologyinvestment) and implicit (e.g. difficulty in

measurement) costs of information collection may limit the set of performancemeasures that are ultimately included in an organization's measurement system tothose related to dimensions of strategy that are easiest to measure (Goold andQuinn 1990). Finally, statistical analysis of the hypothesized linkages amongperformance measures in a balanced scorecard may not yield incremental informationrelative to alternative mechanisms used to monitor performance withinorganizations. For example, organizations frequently use formal strategy reviewmeetings to assess whether strategy is progressing as intended. Ongoing monitoringover time of measured performance against strategic objectives may help decision-makers implicitly test the hypotheses underlying the organization's balancedscorecard even absent formal statistical analysis. Given these considerations, theextent to which balanced scorecards provide useful information for testing and

validating an organization's strategy is an open empirical question.

3


5/44

We investigate this issue by analyzing balanced scorecard data from a conveniencestore chain, Store24, during the implementation of an innovative, but ultimatelyunsuccessful strategy. In FY 1998 Store24 initiated a new store-level strategy todifferentiate itself by improving customer experiences. There was, however,significant variation in how much and how well individual stores executed againstStore24's implementation plan, in how customers valued this strategy, and infinancial performance across stores. Though store24 monitored store performancevia a set of performance measures formulated in a balanced scorecard, they did not

rely on, nor did they conduct, formal statistical tests of the hypothesizedrelationships among the performance measures in the scorecard. Rather, quarterlystrategic reviews, based in part on the firm's balanced scorecard, led executivesat Store24 to identify problems with, and eventually abandon, this strategy over atwo year period after which they reverted back to a traditional strategy thatemphasized speed of service and operational efficiency. Our objectives in thispaper are to explore whether, when, and how information about problems with thisstrategy was captured in the firm's balanced scorecard. Doing so may, in turn,provide evidence for or against claims in the balanced scorecard literature thatthese measurement systems provide useful and timely information for testing theefficacy of an organization's strategy. Additionally, it may

stimulate new theories about the role of multidimensional performance measurement

systems in the strategic feedback and learning processes of organizations and theconditions under which formal analysis of the data generated by these systemsprovides incremental learning relative to standard strategic review practices. Toachieve these research objectives, we exploit a unique feature of our researchsetting. Namely, Store24 as a research site offers three natural benchmarksagainst which we can gauge the efficacy of the information in the firm's balancedscorecard in detecting problems in its strategy: (1) the explicit hypothesesunderlying the firm's balanced scorecard and strategy map; (2) implicitalternatives to these hypotheses based on management concerns about the merits ofthe strategy; and (3) perceived problems with the strategy revealed in the firm'sformal strategy review processes.

4


6/44

We find that statistical tests of the hypotheses underlying the firm's balancedscorecard and strategy map reveal problems with the strategy. Contrary to theexplicit hypotheses underlying the firm's balanced scorecard and strategy map,nonfinancial performance measures related to the strategy are not, and in somecases are even negative, drivers of financial performance. We further findevidence for and against several alternative hypotheses to those underlying thefirm's formal strategy map and scorecard that are consistent with concernsexpressed by some of Store24's top executives during the initial stages of

implementing the new strategy. Field-evidence based on interviews at our researchsite reveals that concerns about the strategy centered on issues of formulation,implementation, and fit with the organization's existing level of employeecapabilities. As part of the customer perspective of its balanced scorecard,Store24 executives measured the extent to which individual stores provided anentertaining experience (i.e., a strategy-specific customer outcome measure).Concerns about whether the strategy was well formulated arose from disagreementamong executives about the merits of this choice of strategic objective forStore24. In particular, some top executives where concerned that, even if theorganization could achieve this strategic objective, financial returns would notfollow. We find that this is indeed the case. On average, store-level performanceon Store24's strategy-specific customer outcome metric is negatively related tostore-level financial performance even after controlling for a variety of

location- and store-specific factors. Concerns about whether the strategy was wellimplemented arose from disagreement among executives about the merits of theoperating standards chosen to implement the new strategy. Store24 executivesdeveloped a store-level action plan to implement the strategy, mapped the actionplan into operating standards, and measured store-level conformance with thesestandards as part of the internal process perspective of its scorecard (i.e. astrategy-specific input measure). Thus, all stores worked on executing againstthese operating standards to implement the new strategy. There was, however,

significant variation in how well the strategy was implemented in different storesand in how customers experienced the implementation. Some top executives whereconcerned that, even if stores executed

well against these operating standards, this would not result in achievement ofthe strategic objective of

5


7/44

providing an entertaining in-store experience from the customer's perspective. Wefind that the firm's scorecard metrics reveal evidence against this alternativehypothesis and in favor of the explicit hypothesis underlying the firm's strategymap. On average, store-level performance on Store24's strategyspecific input-metric is positively related to store-level performance on the firm's strategy-specific customer outcome metric. Concerns about the fit of the strategy with theorganization's existing capabilities arose from disagreement among executivesabout whether the skill levels of store-level employees were sufficient to

implement, and derive economic benefits from, the strategy. Consistent with theseconcerns, our results indicate that cross-sectional differences in measures ofemployee capabilities in the firm's scorecard account for differences in thesuccess of Store24s strategy. Low employee skill levels do not directly affectstrategy implementation. But in stores with low employee skills, even when outcomemeasures are high, financial performance is poor. Conversely, in stores with highemployee skills, when outcome measures are high, financial performance is strong.These results are consistent with a "poor fit"

hypothesis in which regardless of how thoroughly Store24 implements its strategy,for the strategy to succeed, store level employee capabilities need to be high.Collectively, these results provide evidence that this firm's balanced scorecardcontained relevant information for detecting strategic problems and for

distinguishing among implicit alternative hypotheses (relative to those explicitlyarticulated in the scorecard) related to strategy formulation, implementation, andfit problems. However, they do not provide evidence on whether formal analysis ofthe data generated by Store24's balanced scorecard provides incremental learningrelative to the firm's quarterly strategic review process which did not rely onsuch analysis. Using a sub-sample of quarterly data from almost one-year prior tothe quarter in which Store24 executives decided to abandon the strategy, albeit ona more limited set of performance measures due to data availability, we findresults that are consistent with those noted above. We view these results asproviding evidence that formal analysis of the data generated by Store24'sbalanced scorecard provides timely information about strategic problems relativeto the firm's quarterly strategy review process.

6


8/44


9/44

non-financial performance metrics (Rucci et al. 1998, Kaplan and Norton 1996;2000). Except for Ittner and Larcker (1998b), prior empirical work typicallyignores potential nonlinearities in relationships among performance measures.Moreover, these studies do not examine interactions among non-financialperformance measures as a source of nonlinearity that may moderate theserelationships (Ittner and Larcker 1998a). The results in this paper are subject tothe caveat that the field-based nature of our research limits the generalizabilityof our findings. However, the unique nature of a firms strategy dictates that the

performance measures and links between these measures, articulated in the firmsbalanced scorecard, are likely to be firm-specific. Future research should provideadditional evidence from other settings of the extent to which business model-based performance measurement systems such as the balanced scorecard captureinformation useful for monitoring strategic progress. The remainder of the paperproceeds as follows. In section II we present our research site and describe thefirm's strategy and related balanced scorecard implementation. Section IIIpresents our empirical research design and results. We conclude the paper insection IV.

II.

Research Setting

Store24 is a privately held convenience store retailer in New England, the 4thlargest in the region. Its stores, located through Massachusetts, New Hampshire,Rhode Island, and Connecticut, are grouped into nine geographic divisions, eachwith its own division manager. Stores are homogenous in many aspects of theiroperations including compensation, technology, management structure, and productpricing, but they vary in size, geographic location, market demographics, andproduct mix. The companys primary product categories include cigarettes,beverages, snacks, prepared foods, and lottery tickets. Revenues totaledapproximately $180 million in fiscal year 1998 (May 1, 1998 to April 30, 1999).Store24 employed 800 people including 740 store managers and crew and 60 corporatelevel employees. The skills and experience of these employees vary widely overalland across stores.

8


10/44

Store24 operates in a mature environment with competition from convenience stores,gasoline retailers, and drug stores. Traditionally, convenience store retailingfocused on short-term productivity (e.g., inventory and cash control). As theconvenience store industry matured and competition intensified, marketing,customer service, and brand name emerged as differentiating factors. Before FY1998 and after FY 1999, Store24 did not differentiate itself; rather it focused onexcelling at traditional service quality metrics such as physical environment(cleanliness and store layout) and quality of the customer experience (fast,

friendly service) (Fitzsimmons and Fitzsimmons, 2001). During FYs 1998 and 1999(that is from May 1, 1998 to April 30, 2000), Store24 formulated a strategy aimedat increasing same-store sales and margins because growing via new sites wasdifficult. Location is a primary driver of store performance. However, we arestymied on the growth front due to a lack of acceptable new sites. This has led toa focus on optimizing our existing sites through an increasing emphasis on store-level marketing and operations, explained Store24s CFO. To achieve its goals,Store24 changed its strategy to creating entertaining in-store atmospheres thatwould differentiate its stores from those of competitors. The DifferentiationStrategy Store24 implemented this new, innovative store-level strategy during thefirst quarter of FY 1998 (i.e., beginning May 1, 1998). It aimed to differentiateits stores while maintaining performance on traditional productivity measures.Successful retailers, such as Disney stores, offer fun and interactive shopping

experiences. Store24s CEO believed that adopting a similar strategy would improvefinancial performance. Store24 provided a fun in-store atmosphere by emphasizingspecific themes. Store-level strategy execution centered on a large display case(i.e., endcap) featuring themeoriented promotional items and store decorationsthat fostered employee interaction with customers. For example, during the oldmovie theme stores featured life-size cutouts of movie stars, endcaps containedhigh-margin videos of old movies, and old movies became a conversation piece. Thethemes sought to attract urban adults between the ages of 14 and 29 years, agrowing market segment and Store24s target market. A senior manager explained,The [Differentiation] strategy was really playing off of the urban,

9


11/44

young adult market. Marketers know that this demographic gets bored easily andneeds to be stimulated. We wanted this group to always see new and differentthings in the store. In contrast to the basic service quality component, store-managers were accorded autonomy in implementing the differentiation strategy. Thatis, although all stores were required to implement the new strategy, how theyimplemented or how much they implemented varied across stores. Corporate defined atheme and provided the endcaps, but store employees possessed considerableflexibility in strategy execution. Thus, manager and crew skills were at least as

important as theme choice to the strategys success. Store24s controllerexplained, Our best managers really took the strategy to heart. The strategyserved as an outlet for manager and crew creativity. However, other managers putminimal effort into this strategy and even stocked traditional items such as chipson the endcaps saying they needed the product space. The differentiationstrategy, as originally conceived, centered on the physical environment. But theinteraction between store employees and customers was crucial to the strategyssuccess. Senior management intended the themes and promotions to serve as pointsof interaction that would help Store24 establish relationships with customers andcross-sell high margin products. Explained a senior executive, The endcaps anddisplays under the [differentiation strategy] had the dual intention of building arapport with customers and bumping up the average sales per customer. We felt thatstore management and crew could use the displays as ice-breakers in talking with

customers. In addition, the margins on the promotional items featured under the[differentiation] strategy were typically two to four times the margins of ourtraditional products. When customers were browsing or window shopping we

encouraged store crew to direct the customers attention to these promotionalitems. Store24 looked to its differentiation strategy to attract new customersand increase store sales, specifically, sales of highermargin, strategy-specificproducts, and thereby boost store profits. Balanced Scorecard PerformanceMeasurement System Store24 used a balanced scorecard-based performance measurementsystem. The company

collected information on a variety of performance measures at various levels ofthe organization and at

10


12/44

various frequencies. Store24's balanced scorecard along with its performancedimensions ("parameters"), performance measures, organizational levels ofmeasurement (e.g. store vs. corporate), and frequencies of measurement (e.g.quarterly vs. annual) are provided in Figure 1. Performance measures wereorganized around the four traditional balanced scorecard perspectives (financial,customer, internal, and learning & growth). Store24 selected a variety of

traditional accounting based measures of performance for the financial perspective

of its scorecard. In addition to sales growth and asset utilization metrics,Store24 also monitored several different profitability metrics including grossprofit, controllable contribution, EBITDA, and return on capital deployed. Allfinancial performance metrics were collected and monitored on a quarterly basis.With the exception of G&A overhead and return on capital deployed which weremeasured at the corporate level, all other financial metrics were collected at thestore-level but might also be aggregated at the regional or corporate level formonitoring and review purposes.3 Performance measures in the customer perspectiveof the balanced scorecard were monitored quarterly but, underscoring thedifficulty of collecting customer information within stores, were measuredprimarily at the corporate level. As Store24s CFO explained Our customers areinformationally

anonymous. This is a high-transaction, low-ring environment. Our stores see anaverage of 8,000 customers per week and an average check-size of $5. The vastmajority of transactions are cash-based. Store24 contracted with a third-partyresearch firm which conducted quarterly telephone surveys of selfidentifiedconvenience store customers in the companys major markets to assess thelikelihood of customers shopping at Store24, name recognition of Store24, and, forself-identified Store24 customers, the quality of merchandise, price, and storecleanliness. In order to measure the extent to which the company was achieving itsstrategic objective of making its stores fun, entertaining places to shop, Store24also captured a strategy-specific customer outcome metric related to itsdifferentiation strategy: the proportion of self-identified Store24 customers thatrated their shopping experiences at Store24 highly

3

Store24 eventually moved towards store-level allocation of portions of its capitalinvestments, valued at historical cost, for measurement of return on investedcapital at the store-level. However, this occurred later than the period we studyin this paper, and we do not have access to data on store-level return-on-capitalmetrics.

11


13/44

on this dimension. While metrics for the customer-perspective were not routinelycollected at the storelevel as part of the balanced scorecard, a feature we willexploit for our later empirical tests is that Store24 did commission a one-timesurvey project to obtain store-level customer outcome metrics. Between the 1st and4th quarters of FY 1999, the same third-party research firm that conductedexternal surveys for the company solicited feedback from customers at 65 storesabout Store24, its product selection, and other factors that would persuade themto shop at Store24 more often. Customers ranked unique attributes related to the

differentiation strategy that they found appealing; among these was fun place toshop, entertaining, and unexpected. The latter measures capture whethercustomers observe and value the new strategy. Throughout the remainder of thepaper, we refer to these metrics as strategyspecific customer outcome measures ofthe differentiation strategy. The internal perspective of Store24s balancedscorecard primarily contained measures of storelevel execution of the companysunique operating standards. Store24 translated the components of its strategy intoa set of store-level operating standards and measured store-level conformance tothese standards via walk-through audits. During these announced visits, managementevaluated store

performance on various dimensions including in-store image, in-stock position, andstore appearance. The walk-through audit score quantified the store-level

implementation of Store24s strategy. For FYs 1998 and 1999, the standardsincluded for audit reflected both the differentiation strategy as well astraditional dimensions of basic convenience store operations and service quality.A stores differentiation score referred to a separate measure of conformance toonly standards related to the differentiation strategy such as actions in terms ofthemes and products that would make Store24 a fun and entertaining place to shop.This measure of strategy inputs captured the store-level activities that seniormanagement believed drove the success of the differentiation strategy. Senior andmid-level corporate management measured performance by conducting walk-throughaudits twice per quarter.4 Points were awarded based on compliance with 78operating standards selected by senior management. A percentage score is

4

We omit the mystery shop scores due to their correlation with walk-through auditscores and data availability. We cannot disaggregate mystery shop scores intobasic service quality and differentiation strategy implementation measures.

12


14/44

calculated by dividing total awarded points by total potential points. Store24also measured conformance to store-level operating standards through monthlysurprise visits or mystery shops. The primary role of the mystery shop review,which consisted of twenty high-level questions, was to ensure the validity of themore detailed walk-through audit scores. Scores on the announced and unannouncedvisits are significantly and positively correlated. Reflecting one focus of thedifferentiation strategy on driving sales of higher-margin, non-traditionalproducts, the company also monitored net gross profit from new concepts at the

corporate level as part of the internal perspective of its scorecard. Senior anddivision management considered employee capabilities critical to consistentimplementation of the store-level strategy. Accordingly, Store24 selected variousmeasures of employee capabilities in the learning & growth perspective of thescorecard. Measures of the tenure of store managers and crew were included inStore24s scorecard consistent with managements belief that the retention ofexperienced employees was necessary for strong store-level operationalperformance. Employee capabilities were also directly measured through bi-annualevaluations of manager and crew skill levels. These evaluations were conductedduring the 2nd and 4th quarters of each fiscal year. Managers were rated, on afive-point scale, on many dimensions including ability to retain, train, andinteract with crew; customer service; merchandising; time and labor management;maintaining store safety; and technology use. A store managers skill rating was

the average score across all dimensions. Crew skills were rated on a five-pointscale along similar dimensions; all non-management employee scores were averagedto devise a stores crew skill rating. In addition to measures of employee

capabilities, the learning & growth perspective of Store24s balanced scorecardcontained corporate and regional measures of employee satisfaction and informationtechnology use. Store manager and crew compensation was tied to, for example,store-level profit and strategy implementation measures. To encourageimplementation of the differentiation strategy specifically,

employee rewards were based on both the differentiation score and total walk-through audit score. As a result of these measures and incentives, all storesimplemented the new strategy. But, implementation of the differentiation strategy

was not straightforward. Beyond the physical environment and stocking of

13


15/44

new products, it required store staff to establish relationships with customersand sell high-margin products. Implementation of the strategy varied significantlyamong stores. Even when stores

implemented the strategy well, there was variation in how customers experiencedthe new strategy. There was also significant variation across stores inprofitability. We exploit these variations in our empirical tests to examinewhether Store24s balanced scorecard contained useful information for drawing

conclusions about both strategy formulation and implementation. Strategy Map andHypotheses Underlying the Balanced Scorecard The performance measures included inStore24s balanced scorecard were selected based on an underlying strategy mapwhich described senior managements assumptions about cause-and-effectrelationships across the four perspectives of the scorecard. In particular, thestrategy map detailed how input metrics selected for the learning & growth andinternal process perspectives of the scorecard were linked to outcome metricsselected for the financial and customer perspectives of the scorecard via ahypothesized set of cause-effect relationships. Store24s strategy-map, includingthe objectives and related performance measures in each perspective, isillustrated in Figure 2. The strategy map at Store24 had a simple structure.Starting with the learning & growth

perspective, measures and objectives in each perspective were hypothesized to bedrivers of those in the next perspective. The strategy map did not capture allpossible relationships among the performance measures in the balanced scorecardbut, rather, focused on managements primary hypotheses about how their chosenobjectives ultimately led to financial performance. The strategy map reflectedseveral straightforward and specific hypotheses about how the differentiationstrategy would result in financial performance. First, improvements in measures ofthe unique internal processes chosen by senior management to implement thestrategy were expected to lead to improved financial performance via a two-stepprocess: improvements in measures of strategy-inputs would lead to improvements instrategy-specific customer outcome measures which would lead, in turn, to improvedfinancial performance. As a starting point, it is worth considering the assumedlink between

14


16/44

the internal process and financial perspectives of the scorecard (the dashed linein Figure 3) before considering the intermediate performance measures in thecustomer perspective. H1: Ceteris Paribus strategy inputs are positively relatedto financial performance. Figure 3 illustrates this and subsequent hypothesesunderlying Store24s strategy map and balanced scorecard. H1 could be rejected ifthe input metrics show no (or a negative) relationship with financial performance.Underscoring the importance of the assumed linkages across the perspectives ofStore24s balanced scorecard, this could occur if either of the following two

hypotheses in the strategy map were rejected. H2: Ceteris Paribus strategy inputsare positively related to strategy-specific customer outcomes. H3: Ceteris Paribusstrategy-specific customer outcomes are positively related to financialperformance. As discussed below, the extent to which these hypotheses arevalidated depends on whether the differentiation strategy was well formulatedand/or well implemented. Finally, improvements in measures of organizationalcapabilities were expected to drive improvements in measures of the uniqueinternal processes chosen by senior management to implement the strategy (e.g.strategy inputs). H4: Ceteris Paribus measures of employee capabilities arepositively related to measures of strategy inputs. H1-H4 collectively representthe explicit hypotheses underlying Store24s strategy map and balanced scorecard.Alternatives to the Hypotheses Underlying the Strategy Map and Balanced ScorecardWe interviewed several members of the top management team at Store24 including the

CEO, CFO, COO, Controller, V.P. of Marketing, and others to gauge the extent towhich there was consensus about the differentiation strategy at the time thestrategy map was developed and/or in the early stages of implementing thedifferentiation strategy. Our interviews suggest that views about the merits ofthe differentiation strategy were not unanimous among top management even once thestrategy map was

15


17/44

developed. The interviews reveal that concerns about the differentiation strategycentered on issues of its formulation, implementation, and fit with Store24's thenexisting level of employee capabilities. These concerns were discussed openlyamong the top management team early in the strategy development process. Concernsabout whether the strategy was well implemented arose from disagreement amongexecutives about the merits of the operating standards chosen to implement the newstrategy. In

particular, some executives at Store24 were uncertain early on about whether theoperating standards for the differentiation strategy were specific enough todeliver the differentiated customer experience intended by the strategy. In arepresentative comment, Store24s CFO noted there was a potential disconnectbetween how we intended to execute the strategy at the operational level andunderstanding how the strategy helped customers and sales. This uncertainty amongthe top management team

suggests an implicit alternative to H2: strategy inputs are unrelated to strategy-specific customer outcomes. Thus, rejection of H2 would provide evidence that thedifferentiation strategy was not well implemented in the sense that the store-level execution of the action plan and internal processes selected by seniormanagement to implement the strategy, as captured by the strategy input measure,

do not result in the strategy-specific customer outcomes intended by the strategy.In the customer perspective of its balanced scorecard, the performance metric thatcaptured the primary strategic objective of the differentiation strategy measuredthe extent to which individual stores provided an entertaining experience (i.e., astrategy-specific customer outcome measure). Concerns about whether the strategywas well formulated arose from disagreement among executives about the merits ofthis choice of strategic objective for Store24. In a representative commentexpressing the uncertainty that surrounded this choice of strategic objective, onesenior manager noted that we needed to differentiate our stores, but some of uswerent sure this is what our customers wanted. We were throwing a wild card ofentertainment into a business that is about fast-efficient service. Reflectingthis uncertainty, some top executives where concerned that, even if theorganization could achieve this strategic objective, financial returns would not

follow or might even decline. This suggests an implicit alternative to H3:strategy-

16


18/44


19/44

Despite the relative degree of consensus among top management about the role ofemployee capabilities in the differentiation strategy, the strategy map did notdirectly reflect these hypotheses. Rather, these hypotheses remained implicit andoutside of the formal strategy map. Strategy Change and the Quarterly ReviewProcess Store24 followed the differentiation strategy during FYs 1998 and 1999.During this time, management monitored the scorecard. Formal monitoring of thescorecard metrics took place during quarterly strategic review meetings which, inactuality, focused on both strategic and operational issues within Store24. These

meetings were attended by corporate and regional management with the intent ofmonitoring strategic progress and identifying any operational issues that mighthinder such progress. The balanced scorecard was the primary, but not sole, sourceof performance information in these meetings. During these meetings, the balancedscorecard served as a focus for identifying areas in need of improvement. Duringeach quarterly meeting, overall performance on each of the different perspectiveswas given a grade of A+/-, B+/-, C+/-, D+/-, or F. The grades given forperformance in each perspective reflected the consensus judgment of management inattendance5. Once each perspective was graded, reasons for abnormally high or lowperformance were discussed to identify whether the strategy represented in thescorecard and strategy map were progressing as intended. For example, it was notuncommon for financial performance to be strong while nonfinancial metrics in thecustomer, internal, or learning & growth perspectives fell below targeted

performance or vice versa. During the two year period the strategy was beingimplemented patterns were observed in the scorecard metrics that led topmanagement to solicit additional customer feedback and, ultimately, to questionthe validity of their strategic assumptions. Store-level execution of operatingstandards

(strategy-inputs) declined and then gradually increased over this period (Figure4), and the strategyspecific customer outcome measure followed the same pattern.In each quarter of FY 1999 Store24 posted a higher profit than in thecorresponding quarter of FY 1998 (Figure 4). Store24 management,We do not have historical information on the grade awarded for each perspective ineach quarter. The grades were not formally recorded for use outside of thequarterly meetings. Rather, they primarily served as a way to focus management

attention during the meetings on areas of needed improvement.5

18


20/44

however, did not feel that they could attribute the strong financial performanceto the new strategy for two reasons. First, performance on the strategy input andoutcome metrics showed a declining pattern during most quarters suggesting apotential disconnect between the strategy and observed financial performance.Second, growth in profits closely tracked industry averages suggesting thepossibility that the strategy was not differentiating Store24 as strongly asintended. In FY 2000, based on negative customer

feedback, Store24 concluded that the differentiation strategy had failed andrefocused its strategy on traditional service quality activities.6 See Figure 5for a timeline of events related to Store24s strategy change. Based only ontrends in the balanced scorecard metrics, it was difficult for management todefinitively disentangle problems with strategy formulation from those withstrategy implementation. That is, it wasnt easy to pinpoint why the strategyfailed.

III.

Empirical Tests and Results

In this section, we test the explicit and implicit hypotheses underlying Store24s

balanced scorecard to examine whether, when, and how its multiple performancemeasures captured information about problems with the differentiation strategy.Our sample consists of financial, non-financial and

customer performance measures for 65 stores during fiscal years 1998 and 1999(i.e., during implementation of the differentiation strategy). To obtain scores onstore-level differentiation, we

disaggregate the walk-through audit scores into their constituent components. Wehave data for storelevel implementation of the differentiation strategy for thefourth quarter of FY 1998 and the second and third quarters of FY 1999.7 Wesupplement Store24s balanced scorecard data with information on store competitionand demographics gathered during the same time period. Empirical Variables

Financial Performance

6 7

Store24 received negative feedback from in-store comment cards, telephone surveysand focus groups. Unfortunately, after abandoning the differentiation strategy,Store24 did not maintain consistent historical data on the performance measuresrelated to this strategy. We were only able to obtain data on these performancemeasures for these quarters.

19


21/44

To improve its financial performance, Store24 can: i) increase customers; ii)increase spending per customer; or iii) increase the efficiency and effectivenessof store personnel (decrease costs). The measure of controllable contribution(Profit) from the financial perspective of Store24s scorecard summarizes thesecategories at the store level; it is defined as revenues (Sales) from generalmerchandise, lottery tickets, money orders, and phone cards less cost-of-goodssold, utilities expense, and labor expense. This measure reflects the financialcomponents that Store24 believes store-level management can influence and is the

primary measure used by management to evaluate overall store financialperformance. We measure Profit as annual operating profit during FY 1999. This isthe period we are able to match with available strategy input measures, strategyoutcome measures, and measures of employee capabilities. FY 1999 is the secondyear of Store24's differentiation strategy, allowing enough time for any start-upproblems in implementation to be worked out. In all analyses, we scale Profit bysquare feet of store selling space.8 Non-financial Performance Measures Measure ofStrategy Inputs. We disaggregate stores total operational audit scores intoscores that reflect the stores compliance with operating standards (strategyinput measures) for the differentiation strategy.9 Input_Diff reflects a storespercentage score on operating standards related to the

differentiation strategy; it reflects how well each store executed this strategy.

We use the strategy input measure taken at the beginning of FY 1999 in all ourempirical analyses (Input_Diff). 10 Measure of Basic Operational Compliance.During the walk-through audit, Store24 management also measures basic servicequality items such as in-store image, fast service, and in-stock position.Input_Basic is the average percentage score on operating standards related tobasic service quality taken over the same period as our measure of strategy-specific inputs.

We find similar results for all of our subsequent analyses when store-level EBITDAis used rather than controllable contribution. 9 Due to extra credit points forstrong implementation of Differentiation, a stores score on Input_Diff can reach135%. Employees were compensated based on a separate measure of this strategynormalized by total available points. Thus, they were induced to invest in this

implementation. 10 Mystery shop scores are positively and significantly correlatedwith walk-through audit scores and cannot be disaggregated. Adding mystery shopscores to the analyses does not change the results.

8

20


22/44

Measure of Strategy-Specific Customer Outcomes. A third-party research firmconducted in-store customer interviews at a subset of stores throughout FY 1999.11Customers rated the attributes they liked most about this particular Store24,including whether Store24 was entertaining, a fun place to shop, andunexpected. We collect the metrics specific to the differentiation strategy;these metrics comprise a reliable set as evidenced by a Cronbachs coefficientalpha of 0.9596. Each attribute is measured as the proportion of surveyedcustomers who stated that they liked this characteristic about a particular store;

Outcome_Diff is the average of these measures. Outcome_Diff reflects whether

customers observe and value the new strategy; it represents a strategy-specificcustomer outcome measure resulting from implementation of the differentiationstrategy (strategy input measures). Employee Capabilities. We take the measures ofmanager and crew skills (MgrSkill, CrewSkill) from the learning & growthperspective of the scorecard as our primary measures of the firms employeecapabilities. The inclusion of tenure metrics in Store24s scorecard reflects theidea that experience may capture dimensions of employee capabilities that are notdirectly measured in the skill ratings. For this reason, we also include Store24smeasures of manager and crew tenure (MgrTenure, CrewTenure) in our analyses. Inall subsequent empirical tests, we use the skill and tenure metrics taken in thebeginning of FY 1999.12 Were Store24s senior management simply to infer skill

ratings from actual store performance, a stores manager and crew skills ratingswould reflect store performance rather than exogenous skill levels. As shown inTable 2, neither manager nor crew skills exhibit significant univariatecorrelations with Profit. Thus, on average, senior executives do not providehigher skill ratings to employees in better performing stores. Data on individualemployee skill ratings for a sample of 20 stores reveals variation in skillratings across individual employees within a particular store, reflecting seniormanagements desire to identify individual skills rather than infer skill-levelfrom store performance.

11 12

Data was collected for approximately 15-20 stores per quarter. Our results are

invariant to the use of average skills and tenure throughout FY 1999 rather thantaking the skill metrics at the beginning of FY 1999.

21


23/44

Control variables. Store24 collects demographic information for the half-mileradius around each store. Many of these demographics relate to population and foottraffic in the trading area of a given store and are highly correlated. Becausemany of these variables are correlated we use factor analysis to identify theunderlying constructs and find one population factor with an eigenvalue greaterthan one. Population represents daily activity around the store location. Itcomprises primarily the student population (pre-high school, high school, andcollege), pedestrian count rating, and population density. Income is an estimate

of the median level of annual disposable income available to a family for groceryand convenience store purchases in the surrounding area which Store24 obtains froma third-party research firm. Because we expect high income and/or large populationareas to offer more sales potential, these variables should relate positively tofinancial performance. Finally, having more competing stores in the area isexpected to be associated with lower financial performance. To control for thiseffect, we include Competition which reflects the number of competing storeswithin a half-mile radius of each store. We also control for unobservable locationcharacteristics by including rent per square foot (Rent). Store24 pays a premiumto rent facilities in locations with, for example, high visibility. Cross-sectional differences in Rent should capture store location differences which wedo not directly control for in our analyses. Finally, we include a measure ofstore size (SQFT), measured as square feet of retail selling space, and a variable

that indicates whether a store is open 24 hours per day (24Hours). Methodology Wetest the baseline hypothesis in Store24s balanced scorecard, H1, by estimatingthe following equation:PROFITi t = 0 + 1 Input _ Diffi + 2 MgrSkilli + 3CrewSkilli + 4 MgrTenurei + 5 CrewTenurei + 6 Input _ Basici + 7 Competitioni + 8 Populationi + 9 Incomei + 10 24 Hoursi + 11 SquareFeeti + 12 Renti + i

(1)

Where PROFITi denotes controllable contribution for store i during FY 1999. Weestimate this equation using OLS on a cross-sectional sample of 65 stores. Toreduce collinearity due to the inclusion of the interaction terms and to maintaininterpretability of the coefficients, we mean center the interaction variables

prior to estimation (Aiken and West 1991).

22


24/44


25/44

priori reason to believe that strategy-specific outcomes should be driven by thesefactors. However, we have estimated Equation 3 using the same controls as inEquations 1 and 2 and results are substantively similar.13

23


26/44

of H3; it is consistent with bad implementation of a good strategy. Strategy-specific customer outcomes (more entertaining stores) are associated with higherfinancial performance; however the strategy input measures do not lead to higherlevels of strategy-specific customer outcomes. To test the implicit assumptionsunderlying managements beliefs about the complementary impact of Store24semployee capabilities on the relationships between input, outcome and financialperformance measures, we rely on the interaction terms between the strategy-specific measures and the measures of employee capabilities (e.g. skill and

tenure). Significant coefficients on these variables indicate that the level ofemployee capabilities impacts the relationships among input measures, outcomemeasures and financial performance (H5 and H6). Finally, we use equation (4) toinvestigate the final explicit hypothesis in Store24s balanced scorecard (H4) byexamining the relationship between performance on the strategy input metric(Input_Diff) and the level of employee capabilities (MgrSkill, CrewSkill,MgrTenure, CrewTenure). We include MgrSkill, CrewSkill MgrTenure, and CrewTenurein equations (2) and (3) to account for any main effects of employee capabilitieson store financial performance.15 Although scaling by store size (Square Feet)alleviates concerns with heteroskedasticity, we calculate p-values based on bothOLS standard errors and Mackinnon and Whites (1985) heteroskedasticity consistentHC3 standard errors with no substantive differences in results.16 RESULTSDescriptive Statistics Table 1 provides descriptive statistics and Table 2

presents the correlation matrix for the sample of 65 stores. Note that the storesexhibit wide cross-sectional variability in both Store24s input measure(Input_Diff) and outcome measure (Outcome_Diff). Outcome_Diff is negativelyrelated to Profit.15

The univariate correlations suggest that

Additionally, the outcome measure is significantly

Managers with high skills may, for example, more effectively manage labor andinventory costs which would have a direct effect on store-level financialperformance. 16 Whites test for heteroskedasticity is not as reliable in small

samples (Mackinnon and White 1985, Long and Ervin 2000). Long and Ervin (1997)suggest using the HC3 estimator for standard errors when heteroskedasticity issuspected. Although we have no a priori reason to suspect heteroskedasticity, wecheck p-values based on HC3 estimators for robustness (untabulated).

24


27/44

positively related to Store24s input measure (Input_Diff). Together, thisprovides preliminary evidence that the differentiation strategy was wellimplemented, as Store24s view of good implementation corresponds to the customeroutcome, but possibly poorly formulated due to the negative relation of thecustomer outcome with financial performance. Since stores vary on other factorsthat might affect financial performance (e.g., location and skills) we refrainfrom making conclusions based on these univariate tests. Competition, Population,Income, Sqft and Rent all exhibit significant correlations with Profit. Thus,

these seem to be powerful controls for unobserved location characteristics thatmight affect store performance. Tests of H1 (Linking Internal Processes toFinancial Performance) Table 3 reports the results of estimating the relationshipbetween Profit and Store24s assessment of stores internal conformance withstrategic operating standards. On average, the strategy input metric, Input_Diff,is not associated with Profit. This suggests that store-level effort to implementthe new strategy was not translating into store-level profits. Manager skills andtenure significantly and positively relate to profit as does population in thesurrounding area; competition is negatively related to profit. Compliance withbasic operating standards (Input_Basic) is positively and significantly related toprofit with each 1% increase in this measure corresponding with a $2.16 increasein annual profit per square foot all else equal. These results highlight that thehypothesized link in the scorecard between internal implementation of the action

plans related to the new strategy and financial performance does not exist (H1).However, it is unclear whether the strategy was poorly formulated or poorlyimplemented. Tests of H2 and H3 (Distinguishing between Problems of Formulationvs. Implementation) Table 4 contains results from estimation of equation (3). Onaverage, Store24s input metric (Input_Diff) positively relates to the outcomemeasure (p


28/44

Table 5. On average, the outcome measure of the differentiation strategyimplementation is negatively related to Profit (p


29/44

We further examine the interaction between crew skills and the outcome measureusing post-hoc probing as suggested by Aiken and West (1991).19 Panel B of Table 5illustrates the estimated relationship between the outcome measure (Outcome_Diff )and financial performance (Profit), conditional on high (1 point above the meanrating), mean, and low (1 point below the mean rating) crew skills, respectively.We compute the standard errors for each estimated relationship in Panel A of Table5 conditional on the level of crew skills and adjust t-statistics accordinglyprior to inference. The outcome measure negatively impacts Profit in stores with

low and average skills. However, these negative impacts seem to be mitigated instores with high crew skills where there is a positive relationship between theoutcome measure and Profit. Overall, the results suggest problems with the fit ofthe differentiation strategy with Store24s employee capabilities. Crew skillsdetermine the magnitude of the relationship between strategy outcomes andfinancial performance, but the relationship is only greater than zero for highlevels of crew skills.20 Results of H4 (Linking the Learning & Growth and InternalProcess Perspectives) The results of tests of the drivers of the input metrics arepresented in Table 6. On average, crew skills and tenure are not significantlyrelated to strategy execution at the store-level; manager skills, but not managertenure, are positively and significantly related to store-level strategy execution(Input_Diff) (p


30/44

Did the Balanced Scorecard Contain Timely Information about Problems with theStrategy? Collectively, the results reported in Tables 2-6 provide evidence thatthis firms balanced scorecard contained relevant information for detectingstrategic problems and for distinguishing among implicit alternative hypotheses(relative to those explicitly articulated in the scorecard) related to strategyformulation, implementation, and fit problems. However, they do not provideevidence on whether formal analysis of the data generated by Store24's balancedscorecard provides incremental learning relative to the firm's quarterly strategic

review process which did not rely on such analysis. To shed light on this issue,we estimate a version of equation (1) using quarterly, rather than annual, Profitmeasured during the first quarter of FY 1999 almost a full year prior to thedecision to abandon the strategy. Strategy-specific customer outcomes were notavailable at the store level at this point, so we focus on the direct link betweenthe internal process and financial perspectives of Store24s scorecard. Although,it would have been possible for Store24 to perform a similar test even sooner,this is the earliest quarter that we are able to line up Store24s strategy-input,employee capability, and financial performance metrics. We supplement thespecification in equation (1) with interaction terms between the strategy-inputand employee capability metrics.21 As shown in Table 7, we find results that areconsistent with those noted earlier. On average,

the input metric, Input_Diff, is not associated with Profit. However, therelationship between Profit and Input_Diff is increasing in the level of crewskills. Panel B of Table 7 shows that the relationship between Profit andInput_Diff is significant and positive in stores with high crew skills andsignificant and negative in those with low crew skills. Despite the use of earlierquarterly data, these results are largely consistent with those reported in Tables3-5 offering evidence that formal analysis of the data generated by Store24'sbalanced scorecard provides timely information about strategic problems relativeto the firm's quarterly strategy review process.

When we run the same analysis, but use the annual rather than quarterly data onProfit, we obtain qualitatively similar results. That is, if we repeat theanalysis reported in Table 3 with the inclusion of interaction terms between

Input_Diff and measures of employee capabilities, we find no relationship betweenProfit and Input_Diff on average and a negative (positive) relationship in storeswith low (high) crew skills.

21

28


31/44

What did Store24 Learn about Performance Drivers in the Balanced Scorecard?Executives at Store24 eventually refocused the companys strategy on traditionalconvenience store operating activities related to speed, efficiency, and serviceafter abandoning the differentiation strategy. While our analysis providesevidence that the companys balanced scorecard contained useful and timelyinformation for detecting problems in its strategy, the results also suggest thatStore24 executives eventually learned about problems with the strategy despite alack of reliance on such formal analysis. In particular, the results in Tables 3

and 7 show that store level execution of the operating standards related to thedifferentiation strategy (Input_Diff) are not related to financial performancewhile store level execution of basic operating standards (Input_Basic) arestrongly and positively related to financial performance. Thus, the operatingstandards which Store24 executives eventually abandoned were not, while those theyretained were, drivers of financial performance. Similarly and perhapsunsurprisingly given the earlier results in the paper, in untabulated analyses wefind that once the differentiation strategy was abandoned, the updated internalprocess metric capturing overall store compliance with operating standards (e.g.the walk-through audit) becomes a stronger predictor of financial performance.Furthermore, this increase in predictive ability is

concentrated primarily in stores with low crew skills where the differentiation

strategy was least effective. Overall, these results provide evidence that Store24executives learned about the underlying drivers of store performance despite alack of reliance on formal statistical analysis of the assumed relationshipsunderlying their scorecard. However, the earlier results in the paper show thatsuch analysis could have yielded more timely information as well as more detail onwhy the strategy was not working as planned.

IV.

Discussion and Conclusion

Our research investigates whether, when, and how information about problems with afirms strategy was captured in the multiple performance measures of its balanced

scorecard. We analyze balanced scorecard data from a convenience store chain,Store24, during the implementation of an innovative, but ultimately unsuccessfulstrategy. Our results demonstrate that formal statistical tests of

29


32/44

the hypotheses underlying the firm's balanced scorecard and strategy map revealproblems with the strategy on a timelier basis relative to the quarterly reviewprocess that eventually led management to question and abandon the strategy. Ouranalysis also demonstrates that this firm's balanced scorecard contained usefuland timely information for distinguishing between several alternatives to thehypotheses underlying the firms scorecard and strategy map that are consistentwith concerns expressed by the top management team early in the development andimplementation of the strategy. These results provide some of the first field-

based evidence on the potential for a firm's balanced scorecard to provide usefulinformation for detecting problems in its strategy. En route, we document that theextent to which non-financial performance measures predict future financialperformance depends on characteristics of the underlying strategy captured bythose measures. Little, or no, relationship between firm-specific non-financialmetrics and accounting returns may be informative about (1) the firms strategyformulation, (2) its strategy implementation, or (3) the strategys fit withinternal capabilities. We provide some of the first field-based empirical evidenceon the potential for the multiple measures in a balanced scorecard to distinguishbetween these three alternatives. Companies develop assumptions about the links inbusiness-model based measurement systems like the balanced scorecard based on exante expectations (Ittner and Larcker 1998). Our findings indicate that non-financial and financial measures and the hypothesized links between them can be

used more extensively for continuous hypothesis testing ex post. Building on priorresearch illustrating the use of balanced scorecards data to communicate strategy(Selto and Malina 2001; Banker et. al. 2004), we use Store24s balanced scorecarddata to study how the system can be used to test strategy performance. Ourfindings suggest that ongoing tests of these relationships are important to ensurethat hypothesized links are valid. Such investigation can potentially revealspecific aspects of a strategys merits as well as its shortcomings; it can helpdistinguish between strategic problems related to formulation, implementation, orfit of the strategy with the firms internal capabilities. If a companyconsistently applies its scorecard

30


33/44

across multiple units, these tests can be performed at an early stage, prior tocollecting an extensive longitudinal sample. The results in this paper are subjectto the caveat that the field-based nature of our research limits thegeneralizability of our findings. However, the unique nature of any firmsstrategy dictates that the performance measures and links between these measures,articulated in the firms business model, are likely to be firm-specific. Futureresearch should provide additional evidence from other settings of the extent towhich business model-based performance measurement systems, such as the balanced

scorecard, capture information useful for strategy testing and validation.Throughout the paper, we rely on notions of strategy-formulation, strategy-implementation, and fit in interpreting our results. We recognize that there is astrong interrelationship between these

concepts. We define tests related to strategy-formulation as analyzing whether,given the capabilities available to Store24, their choice of strategy was sound.Similarly, our tests related to implementation refer to the efficacy of Store24'sunique internal processes in achieving its strategic objectives given itsavailable capabilities. Our point is not to belabor the distinction betweenformulation, implementation, and fit, but rather to identify how multiple measuresin a balanced scorecard might systematically be used to test how well differentdrivers of performance are working to achieve strategic objectives and superior

financial performance. References 1. Aiken, L.S. and S.G. West. 1991. MultipleRegression: Testing and Interpreting Interactions. London: Sage Publications. 2.Banker, R. D., G. Potter, and D. Srinivasan. 2001. An Empirical Investigation ofan Incentive Plan that Includes Nonfinancial Performance Measures. The AccountingReview 75 (1). 3. Banker, R. D., H. Chang, and M. Pizzini. 2004. The BalancedScorecard: Judgemental Effects of Performance Measures Linked to Strategy. TheAccounting Review 79 (1). 4. Campbell, D. and D. Lane. 2006. "China ResourcesCorporation (A): 6S Management."

Harvard Business School Case 107-013.5. Campbell, D. 2008. Nonfinancial Performance Measures and Promotion-BasedIncentives. Journal of Accounting Research. Forthcoming. 6. Cstore News, 2000. 7.Fitzsimmons, J.A. and M.J. Fitzsimmons. 2001. Service Management: Operations,

Strategy, and Information Technology. New York: McGraw-Hill. 8. Goold, M. andQuinn, J. 1990. The Paradox of Strategic Controls. Strategic Management Journal,11 (1), 43-57 9. Ittner, C.D. and D.F. Larcker. 1998a. Innovations in PerformanceMeasurement: Trends and Research Implications. Journal of Management AccountingResearch 6: 205-238.

31


34/44

10. Ittner, C.D. and D.F. Larcker. 1998b. Are non-financial measures leadingindicators of financial performance?: An analysis of customer satisfaction.Journal of Accounting Research 36: 1-35. 11. Ittner, C.D. and D.F. Larcker. 2001.Assessing Empirical Research in Managerial Accounting: A Value-Based ManagementPerspective. Journal of Accounting and Economics 32: 349-410. 12. Ittner, C.,Larcker, D., and Meyer, M. 2003. "Subjectivity and the Weighting of PerformanceMeasures: Evidence From a Balanced Scorecard." The Accounting Review. 78(3) : 725-758 13. Ittner, C.D. and D.F. Larcker 2005. Moving from Strategic Measurement to

Strategic Data Analysis. Controlling Strategy: Management, Accounting, andPerformance Measurement. Edited by C. Chapman. Oxford University Press. 14.Kaplan, R.S. and D.P. Norton. 1992. The Balanced Scorecard Measures that driveperformance. Harvard Business Review 70 (1): 71-79. 15. ___. 1996 The BalancedScorecard: Translating Strategy into Action. Boston, MA: Harvard Business SchoolPress. 16. Kaplan, R. S. 1998 "Mobil USM&R (A): Linking the Balanced Scorecard."Harvard Business School Case 197-025. 17. ___. 2000 The Strategy FocusedOrganization. Boston, MA: Harvard Business School Press. 18. ___. 2004 StrategyMaps: Converting Intangible Assets into Tangible Outcomes. Boston:

Harvard Business School Publishing 19. ___. 2006 Alignment: Using the BalancedScorecard to Create Corporate Synergies. Boston: Harvard Business School Press 20.___. 2008 The Execution Premium: Linking Strategy to Operations for Competitive

Advantage. Harvard Business School Press21. Nagar, V. and M. V. Rajan. 2001. The Revenue Implications of Financial andOperational Measures of Product Quality. The Accounting Review 76 (4): 495-513.22. Mackinnon, J.G. and White, H. 1985. Some Heteroskedasticity ConsistentCovariance Matrix Estimators with Improved Finite Sample Properties. Journal ofEconometrics. 29: 53-57. 23. Selto, F. and M. Malina 2001. Communicating Strategy:An Empirical Study of the Effectiveness of the Balanced Scorecard. Journal ofManagement Accounting Research. 13: 47-90

32


35/44

FIGURE 1 Store24 Balanced ScorecardFinancial Perspective Parameter Return on Capital Deployed G&A Overhead EBITDAControllable Contribution Gross Profit Growth Sales Growth Inventory TurnoverCustomer Perspective Parameter Loyalty - Recommend Store24 Primary ConvenienceStore Enjoyable Experience Measurement % would recommend Store24 and % will visitStore24 soon based on telephone survey % stating Store24 as their primaryconvenience store based on telephone survey % viewing Store24 as fun and/orentertaining place to shop based on telephone survey Level Corporate Corporate

Corporate Frequency Quarterly Quarterly Quarterly Measurement EBITDA divided byvalue of equipment and leaseholds Average G&A cost per store Controllablecontribution less rental or lease cost Gross profit less utilities and laborexpense Growth in gross profit from same quarter in prior year Growth in salesfrom same quarter in prior year Days inventory for general merchandise andcigarettes Level Corporate Corporate Store Store Store Store Store FrequencyQuarterly Quarterly Quarterly Quarterly Quarterly Quarterly Quarterly

Internal Perspective Parameter Concept Development Operational Excellence BanBoredom Measurement Net gross profit $ from new concepts Walk-through audit andmystery shopper ratings of compliance with basic operating standards Walk-throughaudit and mystery shopper ratings of compliance with Ban Boredom implementationstandards Level Corporate Store Store Frequency Quarterly Quarterly Quarterly

Learning and Growth Perspective Parameter Manager Skills Crew Skills ManagerTenure Crew Tenure Employee Satisfaction Information System Use Measurement Skillrating of store managers Average Skill rating of non-management store employeesNumber of years manager has been with Store24 Averge number of years with Store24for non-management store employees Gallup survey of employee satisfaction on 5-point scale Regional manager evaluation of store utilization of front and back-office technology Level Store Store Store Store Corporate Regional Frequency Every6-months Every 6-months Quarterly Quarterly Every 6-months Every 6-months

33


36/44

FIGURE 2 Store24 Strategy MapFinancial PerspectiveROI Return on Capital Deployed*

EBITDA

EBITDA* Asset Utilization Inventory turns*

Contribution

Controllable Contribution*

Gross Profit Increase Sales

Gross Profit Growth*

Sales or Sales Growth* Basic Requirements

Customer Perspective

Differentiators Enjoyable Experience Interesting Promotions Telephone Surveys*

Quality, Value, Cleanliness, Selection

Friendly Interactions

Telephone Surveys*

Internal Perspective

Differentiate In-Store Experience Create fun, entertaining instore atmospheres BanBoredom Walk-Through Audits* Net Gross Profit from New Concepts*

Increase Customer Value Enhance the customer experience with flawless operations

Walk-Through Audits* Mystery shoppers*

Learning & Growth Perspective

Competencies Required competencies are built on capable employees

Technology Focus on technology is on information systems use

Climate for Action Ability to implement relies heavily on employee satisfactionGallup poll*

Skills evaluation* Employee Tenure*

Technology evaluation sheet*

* Measures

34


37/44

FIGURE 3 Summary of Hypotheses Underlying the Scorecard and Strategy MapFinancial Perspective Financial Performance H3 H1 Customer Perspective H6Strategy-Specific Customer Outcomes

H2 Strategy Inputs H4 H5

Internal Process Perspective

Learning and growth Perspective

Employee Capabilities

FIGURE 4 Store24s Scorecard Metrics during Differentiation PeriodStrategy-Spcific Customer Outcome Measure

Strategy-Specific Input Measure6.20 Enjoyable Experience Rating

124% Walk-Through Audit Score 124% 123% 123% 122% 122% 121% 121% 120% 120% Q1 FY99Q2 FY99 Q3 FY99 Q4 FY99 Quarter

6.10 6.00 5.90 5.80 5.70 5.60 Q1 FY99 Q2 FY99 Quarter Q3 FY99 Q4 FY99

Average Operating Profit

$5,400 $5,200 Operating Profit $5,000 $4,800 $4,600 $4,400 $4,200 $4,000 Q1 FY98Q2 FY98 Q3 FY98 Q4 FY98 Q1 FY99 Q2 FY99 Q3 FY99 Q4 FY99 Quarter

* **

Operating profit (e.g. controllable contribution) is scaled by the number of weeksin each respective quarter. Note that operating profit in convenience storeretailing exhibits strong quarterly seasonality.

35


38/44

FIGURE 5 Timeline of Events Related to Store24s Strategy ChangeQ1 FY 1998 Q3 FY 1999 Q4 FY 1999 Q1 FY 2000

Store24 implementsdifferentiation strategy.

Customer Monitors and enforcesstore-level strategy implementation using walk-through audits

Customer focus

Translates strategy

to a set of operating standards and measures store-level implementation of thesestandards using walk-through audits.

feedback surveys suggest differentiation strategy is not resonating withcustomers.

groups confirm that differentiation strategy is not resonating with customers.basic service operations.

Store24

Store 24 refocuses on

updates performance measures to only reflect basic service operations

Monitors customer

feedback about strategy through in-store comment cards and telephone surveys.

TABLE 1 Descriptive Statistics for the Sample of 65 Stores Used in EmpiricalAnalyses

Variable Mean SD Min Median Max 133.93 54.88 51.88 121.63 349.49 Profit 108.1622.39 46.43 117.85 135.71 Input_Diff 89.89 5.58 71.21 89.60 99.26 Input_Basic27.98 9.88 2.56 26.85 51.87 Outcome_Diff 3.27 0.63 1.21 3.27 4.38 MgrSkill 3.350.43 2.75 3.24 4.51 CrewSkill 45.2 57.95 0.20 22.51 265.76 MgrTenure 13.4 16.382.27 8.48 89.66 CrewTenure 3.87 1.38 1.65 3.68 11.13 Competition -0.06 0.90 -1.27-0.28 3.06 Population 2,588.35 532.20 1,700.00 2,499.00 4,230.00 Income 0.85 0.361.00 1.00 24hours 2,139.05 374.78 1,333.00 2,133.00 2,919.00 Sqft 23.73 15.90 4.7619.02 85.71 Rent Profit = Revenue from general merchandise, lottery tickets, moneyorders, and phone cards less expenses related to cost-of-goods sold, utilities,and labor, scaled by square feet of the store; Input_Diff = measure of store-levelimplementation of Differentiation strategy measured as percentage compliance withoperating standards related to the differentiation strategy; Outcome_Diff =customer (Outcome) measure of Differentiation strategy; MgrSkill and CrewSkill

=Average of bi-annual measures of the manager and crew skills in basic storeoperations, rated on a five-point scale; MgrTenure and CrewTenure = Months oftenure with Store24 for managers and average months of tenure with the company fornon-management employees (crew) respectively; Input_Basic = measure of percentagecompliance with operating standards related to standard convenience storeoperations; Competition = number of competitors within the trading area of astore; Population = store location factor score capturing items related topopulation density and foot traffic around the stores trading area; Income =Measure of median annual disposable income available for grocery and conveniencestore purchases in the stores trading area 24hours = 1 if store is open 24 hoursper day, 0 otherwise; Sqft = square footage of the store; and Rent = monthly rent


39/44

per square foot for store.

36


40/44

TABLE 2 Correlation Matrix for 65 Stores during FY 1999Profit Outcome_Diff Input_Diff MgrSkill CrewSkill MgrTenure CrewTenure Input_BasicCompetition Population Income 24hours Sqft Rent Profit 1 -0.4125* -0.1293 0.09110.0949 0.3460* 0.0815 0.1022 -0.3920* 0.4452* 0.2223* -0.096 -0.5790* 0.6526*Outcome_Diff 1 0.2433* 0.0682 -0.0401 -0.1274 -0.0133 0.1007 0.2477* -0.0848-0.4648* 0.0209 0.2280* -0.4003* Input_Diff MgrSkill CrewSkill MgrTenureCrewTenure Input_Basic Competition Population Income 24hours Sqft

1 0.183 0.0031 0.0575 -0.0046 0.2612* 0.0419 -0.2750* -0.1132 0.2731* 0.1885-0.3130*

1 0.2551* 0.2162* 0.1847 0.3514* 0.2896* -0.1605 -0.2698* 0.2067* 0.1402 -0.1444

1 -0.1059 0.12 0.1446 -0.1647 0.0515 0.0142 0.0612 -0.0159 0.0329

1 0.3266* 0.0483 0.1677 -0.0363 0.0323 -0.1923 0.0266 0.1956

1 0.1386 -0.0427 -0.0089 0.0118 -0.0949 0.063 0.172

1 0.0138 -0.0592 0.0434 0.2005 0.1552 -0.0654

1 -0.1303 -0.3821* 0.0998 0.3145* -0.4161*

1 0.0281 -0.1776 -0.181 0.3904*

1 -0.0095 -0.0661 0.4206*

1 0.035 -0.1363

1 -0.5487*

37


41/44

TABLE 3 Linking the Internal Process and Financial Perspectives (DependentVariable = Profit; Adjusted R2 = 0.72)Intercept Input_Diff MgrSkill CrewSkill MgrTenure CrewTenure Input_BasicCompetition Population Income 24hours Square Feet (00's) Rent per Square FootCoefficient 22.49 0.02 16.87 2.02 0.34 -0.41 2.16 -7.91 22.63 0.006 -0.92 -0.060.32 Standard Error 89.05 0.19 8.93 7.36 0.08 0.25 1.05 3.56 4.68 0.009 11.42 0.010.38 Two-Sided p-Value 0.80 0.92 0.06 0.79 0.00 0.11 0.05 0.03 0.00 0.48 0.94 0.000.40

All bolded coefficients are significant at least at the 10% level using a two-tailed test.

TABLE 4

Linking Internal Processes to Customer Outcomes(Dependent Variable = Outcome_Diff; Adjusted R2 = 0.13)Intercept Input_Diff Input_Diff x MgrSkill Input_Diff x CrewSkill Input_Diff xMgrTenure Input_Diff x CrewTenure MgrSkill CrewSkill MgrTenure CrewTenureCoefficient 18.10 0.11 -0.11 -0.02 0.002 -0.001 1.99 -2.03 -0.04 0.05 StandardError 10.50 0.06 0.13 0.13 0.001 0.004 2.93 2.38 0.03 0.06 Two-Sided p-Value 0.090.07 0.39 0.86 0.184 0.834 0.50 0.40 0.11 0.42


38


42/44

TABLE 5

Panel A: Linking Customer Outcomes to Financial Performance(Dependent Variable = Profit; Adjusted R2 = 0.76)Intercept Outcome_Diff Outcome_Diff x MgrSkill Outcome_Diff x CrewSkillOutcome_Diff x MgrTenure Outcome_Diff x CrewTenure MgrSkill CrewSkill MgrTenureCrewTenure Input_Basic Competition Population Income 24hours Square Feet (00's)Rent per Square Foot Coefficient 37.12 -0.92 0.64 2.00 0.00 -0.02 21.00 3.78 0.28

-0.37 2.47 -7.51 22.50 -0.01 -5.28 -0.07 0.06 Standard Error 88.53 0.46 0.79 0.840.01 0.04 8.55 6.99 0.07 0.26 1.00 3.35 4.38 0.009 11.58 0.01 0.38 Two-Sided p-Value 0.68 0.05 0.42 0.02 0.64 0.65 0.02 0.59 0.00 0.17 0.02 0.03 0.00 0.93 0.650.00 0.88


Panel B: Summary of Moderating Effect of Crew Skills Two-sided p-value for test of1 + 3 = 0

Low Crew Skills Mean Crew Skills High Crew Skills Coefficient -2.92 -0.92 1.08Two-Sided p-Value 0.000 0.098 0.020

TABLE 6

Linking the Learning & Growth and Internal Process Perspectives (DependentVariable = Input_Diff; Adjusted R2 = 0.09)Intercept MgrSkill CrewSkill MgrTenure CrewTenure Coefficient 55.90 16.41 0.14-0.03 -0.12 Standard Error 34.13 5.14 9.12 0.05 0.18 Two-Sided p-Value 0.11 0.000.99 0.53 0.49


39


43/44

TABLE 7

Panel A: Did the Balanced Scorecard Contain Timely Information about Problems withthe Strategy?(Dependent Variable = FY 1998 fourth-quarter Profit; Adjusted R2 = 0.66)Intercept Input_Diff Input_Diff x MgrSkill Input_Diff x CrewSkill Input_Diff xMgrTenure Input_Diff x CrewTenure MgrSkill CrewSkill MgrTenure CrewTenureInput_Basic Competition Population Income 24hours Square Feet (00's) Rent per

Square Foot Coefficient 14.76 0.002 0.09 0.17 0.00 0.00 5.57 0.11 0.09 -0.10 0.39-2.12 4.27 2.78 -1.60 -0.02 0.10 Standard Error 21.39 0.05 0.07 0.09 0.00 0.002.54 2.18 0.02 0.06 0.21 0.60 1.28 1.21 4.32 0.00 0.12 Two-Sided p-Value 0.49 0.980.26 0.05 0.73 0.91 0.03 0.96 0.00 0.13 0.07 0.00 0.00 0.03 0.71 0.00 0.40


Panel B: Summary of Moderating Effect of Crew SkillsLow Crew Skills Mean Crew Skills High Crew Skills Coefficient -0.17 0.002 0.18Two-Sided p-Value 0.098 0.976 0.084

40


44/44

7082903 store 24 balanced scorecard

Documents