08-081

Copyright © 2008 by Dennis Campbell, Srikant M. Datar, Susan L. Kulp, and V.G. Narayanan

Working papers are in draft form. This working paper is distributed for purposes of comment and discussion only. It may not be reproduced without permission of the copyright holder. Copies of working papers are available from the author.

Testing Strategy with Multiple Performance Measures Evidence from a Balanced Scorecard at Store24 Dennis Campbell Srikant M. Datar Susan L. Kulp V.G. Narayanan

1

Testing Strategy with Multiple Performance Measures Evidence from a Balanced Scorecard at Store24*

Dennis Campbell

Srikant Datar Harvard Business School

Susan L. Kulp

George Washington University

V.G. Narayanan Harvard Business School

Current Draft: February 2008

ABSTRACT: We analyze balanced scorecard data from a convenience store chain, Store24, during the implementation of an innovative, but ultimately unsuccessful strategy. Quarterly strategic reviews, based in part on the firm's balanced scorecard, led executives at Store24 to identify problems with, and eventually abandon, this strategy over a two year period. We find that formal statistical tests of the hypotheses underlying the firm's balanced scorecard and strategy map reveal problems with the strategy on a timelier basis. We also test alternative hypotheses to those underlying the firm's formal strategy map and scorecard that are consistent with concerns expressed by some of Store24's top executives during the initial stages of implementing the new strategy. Our analysis demonstrates that this firm's balanced scorecard contained useful and timely information for distinguishing between these alternatives. These results provide some of the first field-based evidence on the potential for a firm's balanced scorecard to provide useful information for detecting problems in its strategy.

I. Introduction This study investigates the role of the balanced scorecard in generating useful information for

testing and validating an organization's strategy. Numerous case studies of balanced scorecard

implementations document their use in translating organizational strategies to objectives and measures,

communicating strategic objectives to employees, evaluating the performance of business units, and

aligning the incentives of employees across business units and functions.1 Field-based and experimental

research in the accounting literature has also focused on these uses of balanced scorecards (Malina and

* The authors thank Store24 for use of its data. We thank Chris Ittner, Robert Kaplan, Ken Koga, Joan Luft, Michael Maher, Ella Mae Matsumura Tatiana Sandino, Philip Stocken, Dan Weiss, two anonymous referees, and seminar participants at the AAA Annual Meeting in Orlando, Boston University, the EIASM conference, Harvard University, Management Accounting Section Mid-year Meeting in San Diego, Michigan State University, Ohio State University, University of Arizona, UCLA, University of Michigan, University of Southern California, and the University of Wisconsin for their helpful comments and suggestions. 1 See Kaplan (1998), Campbell and Lane (2006), or many of the organizations documented in Kaplan and Norton (2006) for examples.

2

Selto 2001; Lipe and Salterio 2000: Ittner et. al. 2003; Banker et. al. 2004; Campbell 2008). In addition

to these uses, the literature on the balanced scorecard has long argued that these measurement systems can

play a role in facilitating feedback and learning through the testing, validation, and revision of the

underlying strategic assumptions embedded in the scorecard (Kaplan and Norton 1996, 2004, 2008).

Despite these long-standing claims, there has been comparatively little research on this potential learning

and feedback role of balanced scorecards.

The testing and validation of assumptions underlying balanced scorecards is an important topic

given the considerable uncertainty faced by decision-makers in designing these measurement systems.

The balanced scorecard framework advocates choosing performance metrics related to key financial and

customer objectives, the firm's internal processes for achieving these objectives, and organizational

capabilities necessary to execute its internal processes. Further, performance measures should be

explicitly linked via a "strategy-map" of hypothesized "cause-and-effect" relationships that depict the

firm's strategy (Kaplan and Norton 2000; 2004). Improvements in measures of organizational capabilities

are expected to drive improvements in the execution of internal processes which in turn lead to customer

and financial outcomes. In this way, the balanced scorecard framework explicitly recognizes

interrelationships between strategy-specific measures of financial and customer outcomes and input-

oriented "performance drivers" related to the firm's internal processes and organizational capabilities.

However, managers must formulate strategies, and select related objectives and measures, based on ex-

ante expectations about how the strategy will translate into financial performance. Thus, the strategic

objectives and performance measures chosen for an organization's scorecard are often based on uncertain

hypotheses about how measured performance against these objectives ultimately leads to financial

performance.

Proponents of the balanced scorecard concept have long recognized this uncertainty. To mitigate

against it, and as a mechanism for strategic feedback and learning, they have advocated for formal testing

of the hypothesized linkages among the performance measures included in an organization's scorecard

(Kaplan and Norton 1996; 2004, 2008). Theoretically, multiple performance measures selected based on

3

an organization's unique strategy, coupled with a set of hypothesized relationships among these

performance measures that describes the strategy (e.g. a "strategy map" or "business model"), should

provide the necessary data for strategy validation and testing via standard statistical analysis techniques.

Particularly in organizations with large numbers of similar operating units, statistical analysis can

potentially be used to explicitly test the hypothesized linkages among the performance measures in the

balanced scorecard and provide early feedback about the validity of the underlying strategic assumptions.

However, there are at least three reasons why such an approach may not provide useful

information even in a setting where improvements in nonfinancial performance measures are expected to

lead improvements in financial performance with a short time-lag. First, nonfinancial performance

measures related to a firm's unique strategy may be noisy indicators of true underlying strategic

performance limiting their usefulness for statistical testing purposes. Consistent with this notion, the

problem of "quantifying qualitative information" has frequently been noted by decision-makers as a

significant challenge in implementing balanced scorecards (Ittner and Larcker 1998). Second, an

organization's balanced scorecard and associated strategy map may not capture all dimensions necessary

for a strategy to succeed. Explicit (e.g. technology investment) and implicit (e.g. difficulty in

measurement) costs of information collection may limit the set of performance measures that are

ultimately included in an organization's measurement system to those related to dimensions of strategy

that are easiest to measure (Goold and Quinn 1990). Finally, statistical analysis of the hypothesized

linkages among performance measures in a balanced scorecard may not yield incremental information

relative to alternative mechanisms used to monitor performance within organizations. For example,

organizations frequently use formal strategy review meetings to assess whether strategy is progressing as

intended. Ongoing monitoring over time of measured performance against strategic objectives may help

decision-makers implicitly test the hypotheses underlying the organization's balanced scorecard even

absent formal statistical analysis. Given these considerations, the extent to which balanced scorecards

provide useful information for testing and validating an organization's strategy is an open empirical

question.

4

We investigate this issue by analyzing balanced scorecard data from a convenience store chain,

Store24, during the implementation of an innovative, but ultimately unsuccessful strategy. In FY 1998

Store24 initiated a new store-level strategy to differentiate itself by improving customer experiences.

There was, however, significant variation in how much and how well individual stores executed against

Store24's implementation plan, in how customers valued this strategy, and in financial performance across

stores. Though store24 monitored store performance via a set of performance measures formulated in a

balanced scorecard, they did not rely on, nor did they conduct, formal statistical tests of the hypothesized

relationships among the performance measures in the scorecard. Rather, quarterly strategic reviews,

based in part on the firm's balanced scorecard, led executives at Store24 to identify problems with, and

eventually abandon, this strategy over a two year period after which they reverted back to a traditional

strategy that emphasized speed of service and operational efficiency.

Our objectives in this paper are to explore whether, when, and how information about problems

with this strategy was captured in the firm's balanced scorecard. Doing so may, in turn, provide evidence

for or against claims in the balanced scorecard literature that these measurement systems provide useful

and timely information for testing the efficacy of an organization's strategy. Additionally, it may

stimulate new theories about the role of multidimensional performance measurement systems in the

strategic feedback and learning processes of organizations and the conditions under which formal analysis

of the data generated by these systems provides incremental learning relative to standard strategic review

practices.

To achieve these research objectives, we exploit a unique feature of our research setting.

Namely, Store24 as a research site offers three natural benchmarks against which we can gauge the

efficacy of the information in the firm's balanced scorecard in detecting problems in its strategy: (1) the

explicit hypotheses underlying the firm's balanced scorecard and strategy map; (2) implicit alternatives to

these hypotheses based on management concerns about the merits of the strategy; and (3) perceived

problems with the strategy revealed in the firm's formal strategy review processes.

5

We find that statistical tests of the hypotheses underlying the firm's balanced scorecard and

strategy map reveal problems with the strategy. Contrary to the explicit hypotheses underlying the firm's

balanced scorecard and strategy map, nonfinancial performance measures related to the strategy are not,

and in some cases are even negative, drivers of financial performance.

We further find evidence for and against several alternative hypotheses to those underlying the

firm's formal strategy map and scorecard that are consistent with concerns expressed by some of Store24's

top executives during the initial stages of implementing the new strategy. Field-evidence based on

interviews at our research site reveals that concerns about the strategy centered on issues of formulation,

implementation, and fit with the organization's existing level of employee capabilities.

As part of the customer perspective of its balanced scorecard, Store24 executives measured the

extent to which individual stores provided an entertaining experience (i.e., a strategy-specific customer

outcome measure). Concerns about whether the strategy was well formulated arose from disagreement

among executives about the merits of this choice of strategic objective for Store24. In particular, some

top executives where concerned that, even if the organization could achieve this strategic objective,

financial returns would not follow. We find that this is indeed the case. On average, store-level

performance on Store24's strategy-specific customer outcome metric is negatively related to store-level

financial performance even after controlling for a variety of location- and store-specific factors.

Concerns about whether the strategy was well implemented arose from disagreement among

executives about the merits of the operating standards chosen to implement the new strategy. Store24

executives developed a store-level action plan to implement the strategy, mapped the action plan into

operating standards, and measured store-level conformance with these standards as part of the internal

process perspective of its scorecard (i.e. a strategy-specific input measure). Thus, all stores worked on

executing against these operating standards to implement the new strategy. There was, however,

significant variation in how well the strategy was implemented in different stores and in how customers

experienced the implementation. Some top executives where concerned that, even if stores executed

well against these operating standards, this would not result in achievement of the strategic objective of

6

providing an entertaining in-store experience from the customer's perspective. We find that the firm's

scorecard metrics reveal evidence against this alternative hypothesis and in favor of the explicit

hypothesis underlying the firm's strategy map. On average, store-level performance on Store24's strategy-

specific input-metric is positively related to store-level performance on the firm's strategy-specific

customer outcome metric.

Concerns about the fit of the strategy with the organization's existing capabilities arose from

disagreement among executives about whether the skill levels of store-level employees were sufficient to

implement, and derive economic benefits from, the strategy. Consistent with these concerns, our results

indicate that cross-sectional differences in measures of employee capabilities in the firm's scorecard

account for differences in the success of Store24’s strategy. Low employee skill levels do not directly

affect strategy implementation. But in stores with low employee skills, even when outcome measures are

high, financial performance is poor. Conversely, in stores with high employee skills, when outcome

measures are high, financial performance is strong. These results are consistent with a "poor fit"

hypothesis in which regardless of how thoroughly Store24 implements its strategy, for the strategy to

succeed, store level employee capabilities need to be high.

Collectively, these results provide evidence that this firm's balanced scorecard contained relevant

information for detecting strategic problems and for distinguishing among implicit alternative hypotheses

(relative to those explicitly articulated in the scorecard) related to strategy formulation, implementation,

and fit problems. However, they do not provide evidence on whether formal analysis of the data

generated by Store24's balanced scorecard provides incremental learning relative to the firm's quarterly

strategic review process which did not rely on such analysis. Using a sub-sample of quarterly data from

almost one-year prior to the quarter in which Store24 executives decided to abandon the strategy, albeit

on a more limited set of performance measures due to data availability, we find results that are consistent

with those noted above. We view these results as providing evidence that formal analysis of the data

generated by Store24's balanced scorecard provides timely information about strategic problems relative

to the firm's quarterly strategy review process.

7

Our study makes three contributions to the accounting literature on performance measurement.

First, we provide some of the first field-based evidence on the potential for a firm's balanced scorecard to

provide useful information for strategy testing and validation. Prior research has largely ignored this

potential role of balanced scorecards and rather focused on their use in communicating strategic

objectives to employees, evaluating the performance of business units, and aligning the incentives of

employees across business units and functions (Malina and Selto 2001; Lipe and Salterio 2000: Ittner et.

al. 2003; Banker et. al. 2004; Campbell 2008). Related studies in management accounting demonstrate

relationships among financial performance metrics and non-financial measures such as product quality

and customer satisfaction (e.g., Banker, et. al. 2001; Ittner and Larcker 1998b; Nagar and Rajan 2001).

However, these studies do not explicitly analyze measures of a firm’s strategy and capabilities and,

consequently, the extent to which such measures provide information useful for timely detection of

strategic problems.

Second, despite the academic evidence that non-financial performance measures typically lead

financial performance, Ittner and Larcker (1998b) document that many executives do not tie together

firm-specific non-financial metrics with lagging accounting measures.2 Our paper shows that the

relationships between non-financial performance measures and financial performance depend on

characteristics of the strategy captured by those measures. A lack of a relationship between firm-specific

non-financial metrics and accounting returns may be informative about (1) the firm’s strategy

formulation, (2) its strategy implementation, and (3) the fit of the formulated strategy with the firm’s

internal capabilities. We provide some of the first field-based empirical evidence on the potential for a set

of strategically linked financial and non-financial performance measures to distinguish among these three

alternatives.

Third, we extend prior research on the relationships between non-financial performance measures

and financial performance by examining the potential moderating effect of employee capabilities. Prior

research suggests that business models are typically depicted by linear relationships between financial and

2 Consistent with this, Store24 management did not perform statistical analyses linking the performance measures together, although the metrics were consistently collected across stores and across time.

8

non-financial performance metrics (Rucci et al. 1998, Kaplan and Norton 1996; 2000). Except for Ittner

and Larcker (1998b), prior empirical work typically ignores potential nonlinearities in relationships

among performance measures. Moreover, these studies do not examine interactions among non-financial

performance measures as a source of nonlinearity that may moderate these relationships (Ittner and

Larcker 1998a).

The results in this paper are subject to the caveat that the field-based nature of our research limits

the generalizability of our findings. However, the unique nature of a firm’s strategy dictates that the

performance measures and links between these measures, articulated in the firm’s balanced scorecard, are

likely to be firm-specific. Future research should provide additional evidence from other settings of the

extent to which business model-based performance measurement systems such as the balanced scorecard

capture information useful for monitoring strategic progress.

The remainder of the paper proceeds as follows. In section II we present our research site and

describe the firm's strategy and related balanced scorecard implementation. Section III presents our

empirical research design and results. We conclude the paper in section IV.

II. Research Setting

Store24 is a privately held convenience store retailer in New England, the 4th largest in the region.

Its stores, located through Massachusetts, New Hampshire, Rhode Island, and Connecticut, are grouped

into nine geographic divisions, each with its own division manager. Stores are homogenous in many

aspects of their operations including compensation, technology, management structure, and product

pricing, but they vary in size, geographic location, market demographics, and product mix.

The company’s primary product categories include cigarettes, beverages, snacks, prepared foods,

and lottery tickets. Revenues totaled approximately $180 million in fiscal year 1998 (May 1, 1998 to

April 30, 1999). Store24 employed 800 people including 740 store managers and crew and 60 corporate

level employees. The skills and experience of these employees vary widely overall and across stores.

9

Store24 operates in a mature environment with competition from convenience stores, gasoline

retailers, and drug stores. Traditionally, convenience store retailing focused on short-term productivity

(e.g., inventory and cash control). As the convenience store industry matured and competition intensified,

marketing, customer service, and brand name emerged as differentiating factors. Before FY 1998 and

after FY 1999, Store24 did not differentiate itself; rather it focused on excelling at traditional service

quality metrics such as physical environment (cleanliness and store layout) and quality of the customer

experience (fast, friendly service) (Fitzsimmons and Fitzsimmons, 2001).

During FYs 1998 and 1999 (that is from May 1, 1998 to April 30, 2000), Store24 formulated a

strategy aimed at increasing same-store sales and margins because growing via new sites was difficult.

“Location is a primary driver of store performance. However, we are stymied on the growth front due to

a lack of acceptable new sites. This has led to a focus on optimizing our existing sites through an

increasing emphasis on store-level marketing and operations,” explained Store24’s CFO. To achieve its

goals, Store24 changed its strategy to creating entertaining in-store atmospheres that would differentiate

its stores from those of competitors.

The Differentiation Strategy

Store24 implemented this new, innovative store-level strategy during the first quarter of FY 1998

(i.e., beginning May 1, 1998). It aimed to differentiate its stores while maintaining performance on

traditional productivity measures. Successful retailers, such as Disney stores, offer “fun and interactive”

shopping experiences. Store24’s CEO believed that adopting a similar strategy would improve financial

performance. Store24 provided a fun in-store atmosphere by emphasizing specific themes.

Store-level strategy execution centered on a large display case (i.e., “endcap”) featuring theme-

oriented promotional items and store decorations that fostered employee interaction with customers. For

example, during the old movie theme stores featured life-size cutouts of movie stars, endcaps contained

high-margin videos of old movies, and old movies became a conversation piece. The themes sought to

attract urban adults between the ages of 14 and 29 years, a growing market segment and Store24’s target

market. A senior manager explained, “The [Differentiation] strategy was really playing off of the urban,

10

young adult market. Marketers know that this demographic gets bored easily and needs to be stimulated.

We wanted this group to always see new and different things in the store.”

In contrast to the basic service quality component, store-managers were accorded autonomy in

implementing the differentiation strategy. That is, although all stores were required to implement the new

strategy, how they implemented or how much they implemented varied across stores. Corporate defined

a theme and provided the endcaps, but store employees possessed considerable flexibility in strategy

execution. Thus, manager and crew skills were at least as important as theme choice to the strategy’s

success. Store24’s controller explained, “Our best managers really took the strategy to heart. The

strategy served as an outlet for manager and crew creativity. However, other managers put minimal effort

into this strategy and even stocked traditional items such as chips on the endcaps saying they needed the

product space.”

The differentiation strategy, as originally conceived, centered on the physical environment. But

the interaction between store employees and customers was crucial to the strategy’s success. Senior

management intended the themes and promotions to serve as points of interaction that would help Store24

establish relationships with customers and cross-sell high margin products. Explained a senior executive,

“The endcaps and displays under the [differentiation strategy] had the dual intention of building a rapport

with customers and bumping up the average sales per customer. We felt that store management and crew

could use the displays as “ice-breakers” in talking with customers. In addition, the margins on the

promotional items featured under the [differentiation] strategy were typically two to four times the

margins of our traditional products. When customers were browsing or “window shopping” we

encouraged store crew to direct the customer’s attention to these promotional items.” Store24 looked to

its differentiation strategy to attract new customers and increase store sales, specifically, sales of higher-

margin, strategy-specific products, and thereby boost store profits.

Balanced Scorecard Performance Measurement System

Store24 used a balanced scorecard-based performance measurement system. The company

collected information on a variety of performance measures at various levels of the organization and at

11

various frequencies. Store24's balanced scorecard along with its performance dimensions ("parameters"),

performance measures, organizational levels of measurement (e.g. store vs. corporate), and frequencies of

measurement (e.g. quarterly vs. annual) are provided in Figure 1.

Performance measures were organized around the four traditional balanced scorecard

perspectives (financial, customer, internal, and learning & growth). Store24 selected a variety of

traditional accounting based measures of performance for the financial perspective of its scorecard. In

addition to sales growth and asset utilization metrics, Store24 also monitored several different

profitability metrics including gross profit, controllable contribution, EBITDA, and return on capital

deployed. All financial performance metrics were collected and monitored on a quarterly basis. With the

exception of G&A overhead and return on capital deployed which were measured at the corporate level,

all other financial metrics were collected at the store-level but might also be aggregated at the regional or

corporate level for monitoring and review purposes.3

Performance measures in the customer perspective of the balanced scorecard were monitored

quarterly but, underscoring the difficulty of collecting customer information within stores, were measured

primarily at the corporate level. As Store24’s CFO explained “Our customers are informationally

anonymous. This is a high-transaction, low-ring environment. Our stores see an average of 8,000

customers per week and an average check-size of $5. The vast majority of transactions are cash-based.”

Store24 contracted with a third-party research firm which conducted quarterly telephone surveys of self-

identified convenience store customers in the company’s major markets to assess the likelihood of

customers shopping at Store24, name recognition of Store24, and, for self-identified Store24 customers,

the quality of merchandise, price, and store cleanliness. In order to measure the extent to which the

company was achieving its strategic objective of making its stores fun, entertaining places to shop,

Store24 also captured a strategy-specific customer outcome metric related to its differentiation strategy:

the proportion of self-identified Store24 customers that rated their shopping experiences at Store24 highly

3 Store24 eventually moved towards store-level allocation of portions of its capital investments, valued at historical cost, for measurement of return on invested capital at the store-level. However, this occurred later than the period we study in this paper, and we do not have access to data on store-level return-on-capital metrics.

12

on this dimension. While metrics for the customer-perspective were not routinely collected at the store-

level as part of the balanced scorecard, a feature we will exploit for our later empirical tests is that

Store24 did commission a one-time survey project to obtain store-level customer outcome metrics.

Between the 1st and 4th quarters of FY 1999, the same third-party research firm that conducted external

surveys for the company solicited feedback from customers at 65 stores about Store24, its product

selection, and other factors that would persuade them to shop at Store24 more often. Customers ranked

unique attributes related to the differentiation strategy that they found appealing; among these was “fun

place to shop,” “entertaining,” and “unexpected.” The latter measures capture whether customers observe

and value the new strategy. Throughout the remainder of the paper, we refer to these metrics as strategy-

specific customer outcome measures of the differentiation strategy.

The internal perspective of Store24’s balanced scorecard primarily contained measures of store-

level execution of the company’s unique operating standards. Store24 translated the components of its

strategy into a set of store-level operating standards and measured store-level conformance to these

standards via walk-through audits. During these announced visits, management evaluated store

performance on various dimensions including in-store image, in-stock position, and store appearance.

The walk-through audit score quantified the store-level implementation of Store24’s strategy. For FYs

1998 and 1999, the standards included for audit reflected both the differentiation strategy as well as

traditional dimensions of basic convenience store operations and service quality. A store’s differentiation

score referred to a separate measure of conformance to only standards related to the differentiation

strategy such as actions in terms of themes and products that would make Store24 a fun and entertaining

place to shop. This measure of strategy inputs captured the store-level activities that senior management

believed drove the success of the differentiation strategy. Senior and mid-level corporate management

measured performance by conducting walk-through audits twice per quarter.4 Points were awarded based

on compliance with 78 operating standards selected by senior management. A percentage score is

4 We omit the mystery shop scores due to their correlation with walk-through audit scores and data availability. We cannot disaggregate mystery shop scores into basic service quality and differentiation strategy implementation measures.

13

calculated by dividing total awarded points by total potential points. Store24 also measured conformance

to store-level operating standards through monthly surprise visits or “mystery shops.” The primary role

of the mystery shop review, which consisted of twenty high-level questions, was to ensure the validity of

the more detailed walk-through audit scores. Scores on the announced and unannounced visits are

significantly and positively correlated. Reflecting one focus of the differentiation strategy on driving

sales of higher-margin, non-traditional products, the company also monitored net gross profit from new

concepts at the corporate level as part of the internal perspective of its scorecard.

Senior and division management considered employee capabilities critical to consistent

implementation of the store-level strategy. Accordingly, Store24 selected various measures of employee

capabilities in the learning & growth perspective of the scorecard. Measures of the tenure of store

managers and crew were included in Store24’s scorecard consistent with management’s belief that the

retention of experienced employees was necessary for strong store-level operational performance.

Employee capabilities were also directly measured through bi-annual evaluations of manager and crew

skill levels. These evaluations were conducted during the 2nd and 4th quarters of each fiscal year.

Managers were rated, on a five-point scale, on many dimensions including ability to retain, train, and

interact with crew; customer service; merchandising; time and labor management; maintaining store

safety; and technology use. A store manager’s skill rating was the average score across all dimensions.

Crew skills were rated on a five-point scale along similar dimensions; all non-management employee

scores were averaged to devise a store’s crew skill rating. In addition to measures of employee

capabilities, the learning & growth perspective of Store24’s balanced scorecard contained corporate and

regional measures of employee satisfaction and information technology use.

Store manager and crew compensation was tied to, for example, store-level profit and strategy

implementation measures. To encourage implementation of the differentiation strategy specifically,

employee rewards were based on both the differentiation score and total walk-through audit score. As a

result of these measures and incentives, all stores implemented the new strategy. But, implementation of

the differentiation strategy was not straightforward. Beyond the physical environment and stocking of

14

new products, it required store staff to establish relationships with customers and sell high-margin

products. Implementation of the strategy varied significantly among stores. Even when stores

implemented the strategy well, there was variation in how customers experienced the new strategy. There

was also significant variation across stores in profitability. We exploit these variations in our empirical

tests to examine whether Store24’s balanced scorecard contained useful information for drawing

conclusions about both strategy formulation and implementation.

Strategy Map and Hypotheses Underlying the Balanced Scorecard

The performance measures included in Store24’s balanced scorecard were selected based on an

underlying “strategy map” which described senior management’s assumptions about cause-and-effect

relationships across the four perspectives of the scorecard. In particular, the strategy map detailed how

input metrics selected for the learning & growth and internal process perspectives of the scorecard were

linked to outcome metrics selected for the financial and customer perspectives of the scorecard via a

hypothesized set of cause-effect relationships. Store24’s strategy-map, including the objectives and

related performance measures in each perspective, is illustrated in Figure 2.

The strategy map at Store24 had a simple structure. Starting with the learning & growth

perspective, measures and objectives in each perspective were hypothesized to be drivers of those in the

next perspective. The strategy map did not capture all possible relationships among the performance

measures in the balanced scorecard but, rather, focused on management’s primary hypotheses about how

their chosen objectives ultimately led to financial performance.

The strategy map reflected several straightforward and specific hypotheses about how the

differentiation strategy would result in financial performance. First, improvements in measures of the

unique internal processes chosen by senior management to implement the strategy were expected to lead

to improved financial performance via a two-step process: improvements in measures of strategy-inputs

would lead to improvements in strategy-specific customer outcome measures which would lead, in turn,

to improved financial performance. As a starting point, it is worth considering the assumed link between

15

the internal process and financial perspectives of the scorecard (the dashed line in Figure 3) before

considering the intermediate performance measures in the customer perspective.

H1: Ceteris Paribus strategy inputs are positively related to financial performance.

Figure 3 illustrates this and subsequent hypotheses underlying Store24’s strategy map and balanced

scorecard. H1 could be rejected if the input metrics show no (or a negative) relationship with financial

performance. Underscoring the importance of the assumed linkages across the perspectives of Store24’s

balanced scorecard, this could occur if either of the following two hypotheses in the strategy map were

rejected.

H2: Ceteris Paribus strategy inputs are positively related to strategy-specific customer outcomes. H3: Ceteris Paribus strategy-specific customer outcomes are positively related to financial performance.

As discussed below, the extent to which these hypotheses are validated depends on whether the

differentiation strategy was well formulated and/or well implemented.

Finally, improvements in measures of organizational capabilities were expected to drive

improvements in measures of the unique internal processes chosen by senior management to implement

the strategy (e.g. strategy inputs).

H4: Ceteris Paribus measures of employee capabilities are positively related to measures of strategy inputs.

H1-H4 collectively represent the explicit hypotheses underlying Store24’s strategy map and balanced

scorecard.

Alternatives to the Hypotheses Underlying the Strategy Map and Balanced Scorecard

We interviewed several members of the top management team at Store24 including the CEO,

CFO, COO, Controller, V.P. of Marketing, and others to gauge the extent to which there was consensus

about the differentiation strategy at the time the strategy map was developed and/or in the early stages of

implementing the differentiation strategy. Our interviews suggest that views about the merits of the

differentiation strategy were not unanimous among top management even once the strategy map was

16

developed. The interviews reveal that concerns about the differentiation strategy centered on issues of its

formulation, implementation, and fit with Store24's then existing level of employee capabilities. These

concerns were discussed openly among the top management team early in the strategy development

process.

Concerns about whether the strategy was well implemented arose from disagreement among

executives about the merits of the operating standards chosen to implement the new strategy. In

particular, some executives at Store24 were uncertain early on about whether the operating standards for

the differentiation strategy were specific enough to deliver the differentiated customer experience

intended by the strategy. In a representative comment, Store24’s CFO noted “…there was a potential

disconnect between how we intended to execute the strategy at the operational level and understanding

how the strategy helped customers and sales.” This uncertainty among the top management team

suggests an implicit alternative to H2: strategy inputs are unrelated to strategy-specific customer

outcomes. Thus, rejection of H2 would provide evidence that the differentiation strategy was not well

implemented in the sense that the store-level execution of the action plan and internal processes selected

by senior management to implement the strategy, as captured by the strategy input measure, do not result

in the strategy-specific customer outcomes intended by the strategy.

In the customer perspective of its balanced scorecard, the performance metric that captured the

primary strategic objective of the differentiation strategy measured the extent to which individual stores

provided an entertaining experience (i.e., a strategy-specific customer outcome measure). Concerns about

whether the strategy was well formulated arose from disagreement among executives about the merits of

this choice of strategic objective for Store24. In a representative comment expressing the uncertainty that

surrounded this choice of strategic objective, one senior manager noted that “…we needed to differentiate

our stores, but some of us weren’t sure this is what our customers wanted. We were throwing a wild card

of entertainment into a business that is about fast-efficient service.” Reflecting this uncertainty, some top

executives where concerned that, even if the organization could achieve this strategic objective, financial

returns would not follow or might even decline. This suggests an implicit alternative to H3: strategy-

17

specific customer outcome measures are not or are negatively related to financial performance. Thus,

rejection of H3 would provide evidence that the differentiation strategy was not well formulated in the

sense that achieving the strategic objective of the differentiation strategy, as captured by the strategy-

specific outcome measure, does not result in the achievement of Store24’s financial objectives.

Questions about the fit of the strategy with the organization's existing capabilities arose from

uncertainty among executives about whether the skill levels of store-level employees were sufficient to

implement, and derive economic benefits from, the strategy. Senior management believed successful

store-level implementation of this strategy required performance in complementary, difficult to measure

activities. To leverage the environment into financial performance, skilled employees needed to establish

customer relationships. Senior management believed that high skill levels enhanced and low skill levels

limited, the relationship between implementation of the differentiation strategy and store performance.

Explained Store24’s CFO, “Managers and crew that were already skilled in our core [efficiency based]

strategy and other basic store operations such as cash, labor, and inventory control, were able to devote

considerably more time to implementing the [differentiation] strategy and to tailor this strategy based on

knowledge of their customers. These skills made it easier to build the [differentiation] strategy on top of

the basics.” The success of local strategy implementation relied on manager and crew interactions with

customers and local market knowledge. Absent these complementary activities, differentiation

implementation might not translate into improved store-level customer outcomes and financial

performance, and might, in fact, adversely affect performance. Interestingly, there appears to have been a

high degree of consensus early on about the importance of employee capabilities not just in ensuring

store-level execution of the strategy (e.g. H4), but also in ensuring that even if executed well at the local

level, the differentiation strategy ultimately translated into the desired customer and financial outcomes.

The following hypotheses capture these assumptions about the importance of employee capabilities.

H5: Ceteris Paribus the impact of increases in strategy inputs on strategy-specific customer outcomes is positively related to the level of employee capabilities. H6: Ceteris Paribus the impact of increases in strategy-specific customer outcomes on financial performance is positively related to the level of employee capabilities.

18

Despite the relative degree of consensus among top management about the role of employee capabilities

in the differentiation strategy, the strategy map did not directly reflect these hypotheses. Rather, these

hypotheses remained implicit and outside of the formal strategy map.

Strategy Change and the Quarterly Review Process

Store24 followed the differentiation strategy during FYs 1998 and 1999. During this time,

management monitored the scorecard. Formal monitoring of the scorecard metrics took place during

quarterly strategic review meetings which, in actuality, focused on both strategic and operational issues

within Store24. These meetings were attended by corporate and regional management with the intent of

monitoring strategic progress and identifying any operational issues that might hinder such progress. The

balanced scorecard was the primary, but not sole, source of performance information in these meetings.

During these meetings, the balanced scorecard served as a focus for identifying areas in need of

improvement. During each quarterly meeting, overall performance on each of the different perspectives

was given a grade of A+/-, B+/-, C+/-, D+/-, or F. The grades given for performance in each perspective

reflected the consensus judgment of management in attendance5. Once each perspective was graded,

reasons for abnormally high or low performance were discussed to identify whether the strategy

represented in the scorecard and strategy map were progressing as intended. For example, it was not

uncommon for financial performance to be strong while nonfinancial metrics in the customer, internal, or

learning & growth perspectives fell below targeted performance or vice versa.

During the two year period the strategy was being implemented patterns were observed in the

scorecard metrics that led top management to solicit additional customer feedback and, ultimately, to

question the validity of their strategic assumptions. Store-level execution of operating standards

(strategy-inputs) declined and then gradually increased over this period (Figure 4), and the strategy-

specific customer outcome measure followed the same pattern. In each quarter of FY 1999 Store24

posted a higher profit than in the corresponding quarter of FY 1998 (Figure 4). Store24 management,

5 We do not have historical information on the grade awarded for each perspective in each quarter. The grades were not formally recorded for use outside of the quarterly meetings. Rather, they primarily served as a way to focus management attention during the meetings on areas of needed improvement.

19

however, did not feel that they could attribute the strong financial performance to the new strategy for two

reasons. First, performance on the strategy input and outcome metrics showed a declining pattern during

most quarters suggesting a potential disconnect between the strategy and observed financial performance.

Second, growth in profits closely tracked industry averages suggesting the possibility that the strategy

was not differentiating Store24 as strongly as intended. In FY 2000, based on negative customer

feedback, Store24 concluded that the differentiation strategy had failed and refocused its strategy on

traditional service quality activities.6 See Figure 5 for a timeline of events related to Store24’s strategy

change. Based only on trends in the balanced scorecard metrics, it was difficult for management to

definitively disentangle problems with strategy formulation from those with strategy implementation.

That is, it wasn’t easy to pinpoint why the strategy failed.

III. Empirical Tests and Results

In this section, we test the explicit and implicit hypotheses underlying Store24’s balanced

scorecard to examine whether, when, and how its multiple performance measures captured information

about problems with the differentiation strategy. Our sample consists of financial, non-financial and

customer performance measures for 65 stores during fiscal years 1998 and 1999 (i.e., during

implementation of the differentiation strategy). To obtain scores on store-level differentiation, we

disaggregate the walk-through audit scores into their constituent components. We have data for store-

level implementation of the differentiation strategy for the fourth quarter of FY 1998 and the second and

third quarters of FY 1999.7 We supplement Store24’s balanced scorecard data with information on store

competition and demographics gathered during the same time period.

Empirical Variables

Financial Performance

6 Store24 received negative feedback from in-store comment cards, telephone surveys and focus groups. 7 Unfortunately, after abandoning the differentiation strategy, Store24 did not maintain consistent historical data on the performance measures related to this strategy. We were only able to obtain data on these performance measures for these quarters.

20

To improve its financial performance, Store24 can: i) increase customers; ii) increase spending

per customer; or iii) increase the efficiency and effectiveness of store personnel (decrease costs). The

measure of controllable contribution (Profit) from the financial perspective of Store24’s scorecard

summarizes these categories at the store level; it is defined as revenues (Sales) from general merchandise,

lottery tickets, money orders, and phone cards less cost-of-goods sold, utilities expense, and labor

expense. This measure reflects the financial components that Store24 believes store-level management

can influence and is the primary measure used by management to evaluate overall store financial

performance. We measure Profit as annual operating profit during FY 1999. This is the period we are

able to match with available strategy input measures, strategy outcome measures, and measures of

employee capabilities. FY 1999 is the second year of Store24's differentiation strategy, allowing enough

time for any start-up problems in implementation to be worked out. In all analyses, we scale Profit by

square feet of store selling space.8

Non-financial Performance Measures

Measure of Strategy Inputs. We disaggregate stores’ total operational audit scores into scores that

reflect the store’s compliance with operating standards (strategy input measures) for the differentiation

strategy.9 Input_Diff reflects a store’s percentage score on operating standards related to the

differentiation strategy; it reflects how well each store executed this strategy. We use the strategy input

measure taken at the beginning of FY 1999 in all our empirical analyses (Input_Diff). 10

Measure of Basic Operational Compliance. During the walk-through audit, Store24 management also

measures basic service quality items such as in-store image, fast service, and in-stock position.

Input_Basic is the average percentage score on operating standards related to basic service quality taken

over the same period as our measure of strategy-specific inputs.

8 We find similar results for all of our subsequent analyses when store-level EBITDA is used rather than controllable contribution. 9 Due to extra credit points for strong implementation of Differentiation, a store’s score on Input_Diff can reach 135%. Employees were compensated based on a separate measure of this strategy normalized by total available points. Thus, they were induced to invest in this implementation. 10 Mystery shop scores are positively and significantly correlated with walk-through audit scores and cannot be disaggregated. Adding mystery shop scores to the analyses does not change the results.

21

Measure of Strategy-Specific Customer Outcomes. A third-party research firm conducted in-store

customer interviews at a subset of stores throughout FY 1999.11 Customers rated the attributes they

“liked most about this particular Store24,” including whether Store24 was “entertaining,” “a fun place to

shop,” and “unexpected.” We collect the metrics specific to the differentiation strategy; these metrics

comprise a reliable set as evidenced by a Cronbach’s coefficient alpha of 0.9596. Each attribute is

measured as the proportion of surveyed customers who stated that they liked this characteristic about a

particular store; Outcome_Diff is the average of these measures. Outcome_Diff reflects whether

customers observe and value the new strategy; it represents a strategy-specific customer outcome measure

resulting from implementation of the differentiation strategy (strategy input measures).

Employee Capabilities. We take the measures of manager and crew skills (MgrSkill, CrewSkill) from

the learning & growth perspective of the scorecard as our primary measures of the firm’s employee

capabilities. The inclusion of tenure metrics in Store24’s scorecard reflects the idea that experience may

capture dimensions of employee capabilities that are not directly measured in the skill ratings. For this

reason, we also include Store24’s measures of manager and crew tenure (MgrTenure, CrewTenure) in our

analyses. In all subsequent empirical tests, we use the skill and tenure metrics taken in the beginning of

FY 1999.12

Were Store24’s senior management simply to infer skill ratings from actual store performance, a

store’s manager and crew skills ratings would reflect store performance rather than exogenous skill levels.

As shown in Table 2, neither manager nor crew skills exhibit significant univariate correlations with

Profit. Thus, on average, senior executives do not provide higher skill ratings to employees in better

performing stores. Data on individual employee skill ratings for a sample of 20 stores reveals variation in

skill ratings across individual employees within a particular store, reflecting senior management’s desire

to identify individual skills rather than infer skill-level from store performance.

11 Data was collected for approximately 15-20 stores per quarter. 12 Our results are invariant to the use of average skills and tenure throughout FY 1999 rather than taking the skill metrics at the beginning of FY 1999.

22

Control variables. Store24 collects demographic information for the half-mile radius around each store.

Many of these demographics relate to population and foot traffic in the trading area of a given store and

are highly correlated. Because many of these variables are correlated we use factor analysis to identify

the underlying constructs and find one population factor with an eigenvalue greater than one. Population

represents daily activity around the store location. It comprises primarily the student population (pre-high

school, high school, and college), pedestrian count rating, and population density. Income is an estimate

of the median level of annual disposable income available to a family for grocery and convenience store

purchases in the surrounding area which Store24 obtains from a third-party research firm. Because we

expect high income and/or large population areas to offer more sales potential, these variables should

relate positively to financial performance. Finally, having more competing stores in the area is expected

to be associated with lower financial performance. To control for this effect, we include Competition

which reflects the number of competing stores within a half-mile radius of each store.

We also control for unobservable location characteristics by including rent per square foot (Rent).

Store24 pays a premium to rent facilities in locations with, for example, high visibility. Cross-sectional

differences in Rent should capture store location differences which we do not directly control for in our

analyses. Finally, we include a measure of store size (SQFT), measured as square feet of retail selling

space, and a variable that indicates whether a store is open 24 hours per day (24Hours).

Methodology

We test the baseline hypothesis in Store24’s balanced scorecard, H1, by estimating the following

equation:

0 1 2 3 4 5 6

7 8 9 10 11 12

_ _

24 + + +

+ +

ti i i i i i i

i i i i i i i

PROFIT Input Diff MgrSkill CrewSkill MgrTenure CrewTenure Input Basic

Competition Population Income Hours SquareFeet Rent

α α α α α α α

α α α α α α ε

= + + + +

+ + ++ (1)

Where PROFITi denotes controllable contribution for store i during FY 1999. We estimate this equation

using OLS on a cross-sectional sample of 65 stores. To reduce collinearity due to the inclusion of the

interaction terms and to maintain interpretability of the coefficients, we mean center the interaction

variables prior to estimation (Aiken and West 1991).

23

If the strategy-input measure leads to improved financial performance, we expect α1 to be positive

and significant. Finding no (a negative) relationship implies that improved strategy implementation is not

(negatively) associated with improved performance, signaling potential problems with strategy

formulation, strategy implementation, or strategy fit.

Consistent with the explicit and implicit hypotheses underlying Store24’s strategy map and

balanced scorecard, we test for problems in strategy implementation (H2), strategy formulation (H3), and

strategy fit (H4, H5, and H6) by using OLS to estimate the following equations.

0 1 2 3 4

5 6 7 8 9

_ _ _ _

_i i i i i i i i

i i i i i

PROFIT Outcome Diff Outcome Diff MgrSkill Outcome Diff CrewSkill Outcome Diff MgrTenure

Outcome Diff CrewTenure MgrSkill CrewSkill MgrTenure Cr

γ γ γ γ γ

γ γ γ γ γ

= + + × + × + ×

+ × + + + + 10

11 12 13 14 15 16

_ 24 (2)

i i

i i i i i i i

ewTenure Input BasicCompetition Population Income Hours SquareFeet Rent

γγ γ γ γ γ γ η

+

+ + + + + + +

0 1 2 3

4 5

_ _ _ _

_ _

ti i i i i i

i i i i

O u tcom e D iff Inpu t D iff Inpu t D iff M grSkill Inpu t D iff C rew Skill

Inpu t D iff M grT enure Inpu t D iff C rew T enure

β β β β

β β

= + + +

+ +

× ×

× ×

6 7 8 9 (3 )

i i i i iM grSkill C rew Skill M grT enure C rew T enureβ β β β ε+ ++ + +

0 1 2 3 4_ (4 )i i i i i iI n p u t D if f M g r S k i l l C r e w S k i l l M g r T e n u r e C r e w T e n u r eα α α α α μ= + + + + +

Equation (2) is analogous to equation (1) where the outcome measure replaces Store24’s internal

strategy input measure13. Equation (3) tests the relationship between the outcome measure and Store24’s

input measure.14 A positive correlation, β1, indicates relatively good implementation of the differentiation

strategy because the outcome measure correlates with the input metrics. β1>0, 1γ ≤0 would provide

evidence in favor of H2 and against H3 implying a good implementation of a bad strategy. Conformance

to operating standards (strategy inputs) leads to the desired strategy-specific customer outcome

(customers view stores as “entertaining"), but the strategy-specific customer outcome does not translate

into improved store financial performance. β1≤0, 1γ >0 would provide evidence against H2 and in favor

13 In untabulated tests, we estimate equation 2 separately for stores where Outcome_Diff was measured during the first 6-months and second 6-months of FY 1999 respectively. In these tests, for stores measured in the first (second) 6-months, we measure manager and crew skills as the average of skills as measured during the end of the fourth quarter of FY 1998 (second quarter of FY 1999) and the second quarter of FY 1999 (fourth quarter of FY 1999). The results from estimation of equation 2 on each of these sub-samples are substantively similar to those reported in Table 5 on the full sample of stores. These results mitigate the potential that the findings in our paper are due to any mismatch in performance measurement periods within Store24. 14 Note that we do not include demographic and other store location characteristics as controls in Equation 3. There is no a priori reason to believe that strategy-specific outcomes should be driven by these factors. However, we have estimated Equation 3 using the same controls as in Equations 1 and 2 and results are substantively similar.

24

of H3; it is consistent with bad implementation of a good strategy. Strategy-specific customer outcomes

(more entertaining stores) are associated with higher financial performance; however the strategy input

measures do not lead to higher levels of strategy-specific customer outcomes.

To test the implicit assumptions underlying management’s beliefs about the complementary

impact of Store24’s employee capabilities on the relationships between input, outcome and financial

performance measures, we rely on the interaction terms between the strategy-specific measures and the

measures of employee capabilities (e.g. skill and tenure). Significant coefficients on these variables

indicate that the level of employee capabilities impacts the relationships among input measures, outcome

measures and financial performance (H5 and H6). Finally, we use equation (4) to investigate the final

explicit hypothesis in Store24’s balanced scorecard (H4) by examining the relationship between

performance on the strategy input metric (Input_Diff) and the level of employee capabilities (MgrSkill,

CrewSkill, MgrTenure, CrewTenure). We include MgrSkill, CrewSkill MgrTenure, and CrewTenure in

equations (2) and (3) to account for any main effects of employee capabilities on store financial

performance.15

Although scaling by store size (Square Feet) alleviates concerns with heteroskedasticity, we

calculate p-values based on both OLS standard errors and Mackinnon and White’s (1985)

heteroskedasticity consistent “HC3” standard errors with no substantive differences in results.16

RESULTS

Descriptive Statistics

Table 1 provides descriptive statistics and Table 2 presents the correlation matrix for the sample

of 65 stores. Note that the stores exhibit wide cross-sectional variability in both Store24’s input measure

(Input_Diff) and outcome measure (Outcome_Diff). The univariate correlations suggest that

Outcome_Diff is negatively related to Profit. Additionally, the outcome measure is significantly

15 Managers with high skills may, for example, more effectively manage labor and inventory costs which would have a direct effect on store-level financial performance. 16 White’s test for heteroskedasticity is not as reliable in small samples (Mackinnon and White 1985, Long and Ervin 2000). Long and Ervin (1997) suggest using the HC3 estimator for standard errors when heteroskedasticity is suspected. Although we have no a priori reason to suspect heteroskedasticity, we check p-values based on HC3 estimators for robustness (untabulated).

25

positively related to Store24’s input measure (Input_Diff). Together, this provides preliminary evidence

that the differentiation strategy was well implemented, as Store24’s view of good implementation

corresponds to the customer outcome, but possibly poorly formulated due to the negative relation of the

customer outcome with financial performance. Since stores vary on other factors that might affect

financial performance (e.g., location and skills) we refrain from making conclusions based on these

univariate tests. Competition, Population, Income, Sqft and Rent all exhibit significant correlations with

Profit. Thus, these seem to be powerful controls for unobserved location characteristics that might affect

store performance.

Tests of H1 (Linking Internal Processes to Financial Performance)

Table 3 reports the results of estimating the relationship between Profit and Store24’s assessment

of stores’ internal conformance with strategic operating standards. On average, the strategy input metric,

Input_Diff, is not associated with Profit. This suggests that store-level effort to implement the new

strategy was not translating into store-level profits. Manager skills and tenure significantly and positively

relate to profit as does population in the surrounding area; competition is negatively related to profit.

Compliance with basic operating standards (Input_Basic) is positively and significantly related to profit

with each 1% increase in this measure corresponding with a $2.16 increase in annual profit per square

foot all else equal.

These results highlight that the hypothesized link in the scorecard between internal

implementation of the action plans related to the new strategy and financial performance does not exist

(H1). However, it is unclear whether the strategy was poorly formulated or poorly implemented.

Tests of H2 and H3 (Distinguishing between Problems of Formulation vs. Implementation)

Table 4 contains results from estimation of equation (3). On average, Store24’s input metric

(Input_Diff) positively relates to the outcome measure (p<0.10). This result provides support for H2 and

suggests that the strategy of creating entertaining stores was well implemented. The operating standards

management selected for the differentiation strategy relate to customers’ views of stores being innovative

and entertaining (strategy-specific customer outcome). Estimation of equation (2) yields Panel A of

26

Table 5. On average, the outcome measure of the differentiation strategy implementation is negatively

related to Profit (p<0.10).

Overall, these results tend to support H2 and negate H3. Specifically, the relationship between

the way Store24 operationalized the differentiation strategy (input measure) and the way in which the

strategy was viewed by customers (outcome measure) is correlated; the implementation seems to be good.

However, the outcome measures of strategy implementation and financial performance are negatively

correlated. Thus, the evidence from the multivariate analysis provides only partial validation of the

assumed links underlying Store24’s scorecard and points to the potential validity of some of the concerns

expressed by top management about the objective of the strategy. In particular, the results imply that

although the strategy was well implemented, the strategy formulation may have been flawed.

Tests of H5 and H6 (Identifying Strategic Fit)

H5 and H6 focus on Store24’s employee capabilities and whether the fit between these

capabilities and the strategy influences its performance. In particular, employee skill levels might impact

the relationship between the input and outcome measures and/or the relationships between the outcome

measure and financial performance.

Table 4, presents the results for tests of the skill and tenure interaction (H5). The results imply

that the relationship between input and outcome measures is not contingent on the store-level capabilities

of Store24. Neither the interaction of the input measure and manager skills (tenure) nor the interaction of

the input measure with crew skills (tenure) is significant at conventional levels.

Panel A of Table 5 presents tests of H6.17 On average, crew skills appear to moderate the

relationship between the outcome measure and financial performance. The interaction between crew

skills and Outcome_Diff positively relates to Profit (p<0.001).18

Although Store24 predicted a positive relationship between the differentiation strategy and Profit,

the results imply that the benefits derived from the strategy depend on the level of crew skills in a store. 17 In untabulated results we interact skill metrics with Input_Basic. The interaction is not significant, and the results are substantively unchanged. 18 We also investigate, in untabulated results, whether the interaction of Input_Basic and skills and tenure as well as the interaction of Input_Diff with store location variables relates to performance. All results hold and the additional interactions are not significant.

27

We further examine the interaction between crew skills and the outcome measure using post-hoc probing

as suggested by Aiken and West (1991).19 Panel B of Table 5 illustrates the estimated relationship

between the outcome measure (Outcome_Diff ) and financial performance (Profit), conditional on high (1

point above the mean rating), mean, and low (1 point below the mean rating) crew skills, respectively.

We compute the standard errors for each estimated relationship in Panel A of Table 5 conditional on the

level of crew skills and adjust t-statistics accordingly prior to inference.

The outcome measure negatively impacts Profit in stores with low and average skills. However,

these negative impacts seem to be mitigated in stores with high crew skills where there is a positive

relationship between the outcome measure and Profit. Overall, the results suggest problems with the fit

of the differentiation strategy with Store24’s employee capabilities. Crew skills determine the magnitude

of the relationship between strategy outcomes and financial performance, but the relationship is only

greater than zero for high levels of crew skills.20

Results of H4 (Linking the Learning & Growth and Internal Process Perspectives)

The results of tests of the drivers of the input metrics are presented in Table 6. On average, crew

skills and tenure are not significantly related to strategy execution at the store-level; manager skills, but

not manager tenure, are positively and significantly related to store-level strategy execution (Input_Diff)

(p<0.10). Therefore, we find only mixed support for validation of the assumed link in Store24’s

scorecard between employee capabilities and strategy execution (H4). Contrary to the explicit hypotheses

underlying the scorecard, but consistent with the implicit hypotheses of the senior management team, the

information in the scorecard reveals that the primary role of employee capabilities is not necessarily in

ensuring store-level execution of the strategy (e.g. H4), but rather in ensuring that even if executed well at

the local level, the differentiation strategy ultimately translated into the desired financial outcomes.

19 Because we mean-center all variables prior to interaction, it is difficult to interpret the economic significance of the results directly from the tables. 20 The training costs associated with skill improvements should be considered before the long-run viability of differentiation is evaluated. Because data constraints preclude this analysis, this study indicates only the benefits of the strategy gross of training costs.

28

Did the Balanced Scorecard Contain Timely Information about Problems with the Strategy? Collectively, the results reported in Tables 2-6 provide evidence that this firm’s balanced

scorecard contained relevant information for detecting strategic problems and for distinguishing among

implicit alternative hypotheses (relative to those explicitly articulated in the scorecard) related to strategy

formulation, implementation, and fit problems. However, they do not provide evidence on whether

formal analysis of the data generated by Store24's balanced scorecard provides incremental learning

relative to the firm's quarterly strategic review process which did not rely on such analysis.

To shed light on this issue, we estimate a version of equation (1) using quarterly, rather than

annual, Profit measured during the first quarter of FY 1999 – almost a full year prior to the decision to

abandon the strategy. Strategy-specific customer outcomes were not available at the store level at this

point, so we focus on the direct link between the internal process and financial perspectives of Store24’s

scorecard. Although, it would have been possible for Store24 to perform a similar test even sooner, this is

the earliest quarter that we are able to line up Store24’s strategy-input, employee capability, and financial

performance metrics. We supplement the specification in equation (1) with interaction terms between the

strategy-input and employee capability metrics.21

As shown in Table 7, we find results that are consistent with those noted earlier. On average,

the input metric, Input_Diff, is not associated with Profit. However, the relationship between Profit and

Input_Diff is increasing in the level of crew skills. Panel B of Table 7 shows that the relationship

between Profit and Input_Diff is significant and positive in stores with high crew skills and significant

and negative in those with low crew skills. Despite the use of earlier quarterly data, these results are

largely consistent with those reported in Tables 3-5 offering evidence that formal analysis of the data

generated by Store24's balanced scorecard provides timely information about strategic problems relative

to the firm's quarterly strategy review process.

21 When we run the same analysis, but use the annual rather than quarterly data on Profit, we obtain qualitatively similar results. That is, if we repeat the analysis reported in Table 3 with the inclusion of interaction terms between Input_Diff and measures of employee capabilities, we find no relationship between Profit and Input_Diff on average and a negative (positive) relationship in stores with low (high) crew skills.

29

What did Store24 Learn about Performance Drivers in the Balanced Scorecard?

Executives at Store24 eventually refocused the company’s strategy on traditional convenience

store operating activities related to speed, efficiency, and service after abandoning the differentiation

strategy. While our analysis provides evidence that the company’s balanced scorecard contained useful

and timely information for detecting problems in its strategy, the results also suggest that Store24

executives eventually learned about problems with the strategy despite a lack of reliance on such formal

analysis. In particular, the results in Tables 3 and 7 show that store level execution of the operating

standards related to the differentiation strategy (Input_Diff) are not related to financial performance while

store level execution of basic operating standards (Input_Basic) are strongly and positively related to

financial performance. Thus, the operating standards which Store24 executives eventually abandoned

were not, while those they retained were, drivers of financial performance.

Similarly and perhaps unsurprisingly given the earlier results in the paper, in untabulated analyses

we find that once the differentiation strategy was abandoned, the updated internal process metric

capturing overall store compliance with operating standards (e.g. the walk-through audit) becomes a

stronger predictor of financial performance. Furthermore, this increase in predictive ability is

concentrated primarily in stores with low crew skills where the differentiation strategy was least effective.

Overall, these results provide evidence that Store24 executives learned about the underlying drivers of

store performance despite a lack of reliance on formal statistical analysis of the assumed relationships

underlying their scorecard. However, the earlier results in the paper show that such analysis could have

yielded more timely information as well as more detail on why the strategy was not working as planned.

IV. Discussion and Conclusion

Our research investigates whether, when, and how information about problems with a firm’s

strategy was captured in the multiple performance measures of its balanced scorecard. We analyze

balanced scorecard data from a convenience store chain, Store24, during the implementation of an

innovative, but ultimately unsuccessful strategy. Our results demonstrate that formal statistical tests of

30

the hypotheses underlying the firm's balanced scorecard and strategy map reveal problems with the

strategy on a timelier basis relative to the quarterly review process that eventually led management to

question and abandon the strategy. Our analysis also demonstrates that this firm's balanced scorecard

contained useful and timely information for distinguishing between several alternatives to the hypotheses

underlying the firm’s scorecard and strategy map that are consistent with concerns expressed by the top

management team early in the development and implementation of the strategy. These results provide

some of the first field-based evidence on the potential for a firm's balanced scorecard to provide useful

information for detecting problems in its strategy.

En route, we document that the extent to which non-financial performance measures predict

future financial performance depends on characteristics of the underlying strategy captured by those

measures. Little, or no, relationship between firm-specific non-financial metrics and accounting returns

may be informative about (1) the firm’s strategy formulation, (2) its strategy implementation, or (3) the

strategy’s fit with internal capabilities. We provide some of the first field-based empirical evidence on

the potential for the multiple measures in a balanced scorecard to distinguish between these three

alternatives.

Companies develop assumptions about the links in “business-model” based measurement systems

like the balanced scorecard based on ex ante expectations (Ittner and Larcker 1998). Our findings

indicate that non-financial and financial measures and the hypothesized links between them can be used

more extensively for continuous hypothesis testing ex post. Building on prior research illustrating the use

of balanced scorecards data to communicate strategy (Selto and Malina 2001; Banker et. al. 2004), we use

Store24’s balanced scorecard data to study how the system can be used to test strategy performance. Our

findings suggest that ongoing tests of these relationships are important to ensure that hypothesized links

are valid. Such investigation can potentially reveal specific aspects of a strategy’s merits as well as its

shortcomings; it can help distinguish between strategic problems related to formulation, implementation,

or fit of the strategy with the firm’s internal capabilities. If a company consistently applies its scorecard

31

across multiple units, these tests can be performed at an early stage, prior to collecting an extensive

longitudinal sample.

The results in this paper are subject to the caveat that the field-based nature of our research limits

the generalizability of our findings. However, the unique nature of any firm’s strategy dictates that the

performance measures and links between these measures, articulated in the firm’s business model, are

likely to be firm-specific. Future research should provide additional evidence from other settings of the

extent to which business model-based performance measurement systems, such as the balanced scorecard,

capture information useful for strategy testing and validation.

Throughout the paper, we rely on notions of strategy-formulation, strategy-implementation, and

fit in interpreting our results. We recognize that there is a strong interrelationship between these

concepts. We define tests related to strategy-formulation as analyzing whether, given the capabilities

available to Store24, their choice of strategy was sound. Similarly, our tests related to implementation

refer to the efficacy of Store24's unique internal processes in achieving its strategic objectives given its

available capabilities. Our point is not to belabor the distinction between formulation, implementation,

and fit, but rather to identify how multiple measures in a balanced scorecard might systematically be used

to test how well different drivers of performance are working to achieve strategic objectives and superior

financial performance.

References 1. Aiken, L.S. and S.G. West. 1991. Multiple Regression: Testing and Interpreting Interactions.

London: Sage Publications. 2. Banker, R. D., G. Potter, and D. Srinivasan. 2001. An Empirical Investigation of an Incentive Plan

that Includes Nonfinancial Performance Measures. The Accounting Review 75 (1). 3. Banker, R. D., H. Chang, and M. Pizzini. 2004. The Balanced Scorecard: Judgemental Effects of

Performance Measures Linked to Strategy. The Accounting Review 79 (1). 4. Campbell, D. and D. Lane. 2006. "China Resources Corporation (A): 6S Management."

Harvard Business School Case 107-013. 5. Campbell, D. 2008. Nonfinancial Performance Measures and Promotion-Based Incentives. Journal of

Accounting Research. Forthcoming. 6. Cstore News, 2000. 7. Fitzsimmons, J.A. and M.J. Fitzsimmons. 2001. Service Management: Operations, Strategy, and

Information Technology. New York: McGraw-Hill. 8. Goold, M. and Quinn, J. 1990. The Paradox of Strategic Controls. Strategic Management Journal, 11

(1), 43-57 9. Ittner, C.D. and D.F. Larcker. 1998a. Innovations in Performance Measurement: Trends and Research

Implications. Journal of Management Accounting Research 6: 205-238.

32

10. Ittner, C.D. and D.F. Larcker. 1998b. Are non-financial measures leading indicators of financial performance?: An analysis of customer satisfaction. Journal of Accounting Research 36: 1-35.

11. Ittner, C.D. and D.F. Larcker. 2001. Assessing Empirical Research in Managerial Accounting: A Value-Based Management Perspective. Journal of Accounting and Economics 32: 349-410.

12. Ittner, C., Larcker, D., and Meyer, M. 2003. "Subjectivity and the Weighting of Performance Measures: Evidence From a Balanced Scorecard." The Accounting Review. 78(3) : 725-758

13. Ittner, C.D. and D.F. Larcker 2005. Moving from Strategic Measurement to Strategic Data Analysis. Controlling Strategy: Management, Accounting, and Performance Measurement. Edited by C. Chapman. Oxford University Press.

14. Kaplan, R.S. and D.P. Norton. 1992. The Balanced Scorecard – Measures that drive performance. Harvard Business Review 70 (1): 71-79.

15. ___. 1996 The Balanced Scorecard: Translating Strategy into Action. Boston, MA: Harvard Business School Press.

16. Kaplan, R. S. 1998 "Mobil USM&R (A): Linking the Balanced Scorecard." Harvard Business School Case 197-025.

17. ___. 2000 The Strategy Focused Organization. Boston, MA: Harvard Business School Press. 18. ___. 2004 Strategy Maps: Converting Intangible Assets into Tangible Outcomes. Boston:

Harvard Business School Publishing 19. ___. 2006 Alignment: Using the Balanced Scorecard to Create Corporate Synergies. Boston:

Harvard Business School Press 20. ___. 2008 The Execution Premium: Linking Strategy to Operations for Competitive

Advantage. Harvard Business School Press 21. Nagar, V. and M. V. Rajan. 2001. The Revenue Implications of Financial and Operational Measures

of Product Quality. The Accounting Review 76 (4): 495-513. 22. Mackinnon, J.G. and White, H. 1985. Some Heteroskedasticity Consistent Covariance Matrix

Estimators with Improved Finite Sample Properties. Journal of Econometrics. 29: 53-57. 23. Selto, F. and M. Malina 2001. Communicating Strategy: An Empirical Study of the Effectiveness of

the Balanced Scorecard. Journal of Management Accounting Research. 13: 47-90

33

FIGURE 1 Store24 Balanced Scorecard

Financial Perspective

Parameter Measurement Level Frequency Return on Capital Deployed EBITDA divided by value of equipment and leaseholds Corporate Quarterly G&A Overhead Average G&A cost per store Corporate Quarterly EBITDA Controllable contribution less rental or lease cost Store Quarterly Controllable Contribution Gross profit less utilities and labor expense Store Quarterly Gross Profit Growth Growth in gross profit from same quarter in prior year Store Quarterly Sales Growth Growth in sales from same quarter in prior year Store Quarterly Inventory Turnover Days inventory for general merchandise and cigarettes Store Quarterly Customer Perspective

Parameter Measurement Level Frequency Loyalty - Recommend Store24

% would recommend Store24 and % will visit Store24 soon based on telephone survey

Corporate

Quarterly

Primary Convenience Store

% stating Store24 as their primary convenience store based on telephone survey

Corporate

Quarterly

Enjoyable Experience

% viewing Store24 as fun and/or entertaining place to shop based on telephone survey

Corporate

Quarterly

Internal Perspective

Parameter Measurement Level Frequency Concept Development Net gross profit $ from new concepts Corporate Quarterly Operational Excellence

Walk-through audit and mystery shopper ratings of compliance with basic operating standards

Store

Quarterly

Ban Boredom

Walk-through audit and mystery shopper ratings of compliance with Ban Boredom implementation standards

Store

Quarterly

Learning and Growth Perspective

Parameter Measurement Level Frequency Manager Skills Skill rating of store managers Store Every 6-months Crew Skills Average Skill rating of non-management store employees Store Every 6-months Manager Tenure Number of years manager has been with Store24 Store Quarterly Crew Tenure

Averge number of years with Store24 for non-management store employees

Store

Quarterly

Employee Satisfaction Gallup survey of employee satisfaction on 5-point scale Corporate Every 6-months Information System Use

Regional manager evaluation of store utilization of front and back-office technology

Regional

Every 6-months

34

FIGURE 2 Store24 Strategy Map

Financial Perspective

CustomerPerspective

InternalPerspective

Gross Profit Growth*

Return on Capital Deployed*

Learning & Growth Perspective

ROI

EBITDA*EBITDA

Gross Profit

Contribution AssetUtilization

Sales or Sales Growth*

Controllable Contribution*

Inventory turns*

EnjoyableExperience

Basic Requirements

Differentiate In-Store Experience

Create fun, entertaining in-store atmospheres

Increase Customer ValueEnhance the customer

experience with flawless operations

Ban Boredom Walk-Through Audits*Net Gross Profit from New Concepts*

Walk-Through Audits*Mystery shoppers*

CompetenciesRequired competencies arebuilt on capable employees

TechnologyFocus on technology is on information systems use

Climate for ActionAbility to implement

relies heavily on employee satisfaction

Skills evaluation*Employee Tenure*

Technology evaluation sheet* Gallup poll*

* Measures

Differentiators

Interesting Promotions

Quality, Value, Cleanliness, Selection Friendly Interactions

Increase Sales

Telephone Surveys*Telephone Surveys*

35

FIGURE 3 Summary of Hypotheses Underlying the Scorecard and Strategy Map

FIGURE 4 Store24’s Scorecard Metrics during Differentiation Period

Strategy-Specific Input Measure

120%

120%

121%

121%

122%122%

123%

123%

124%

124%

Q1 FY99 Q2 FY99 Q3 FY99 Q4 FY99

Quarter

Wal

k-Th

roug

h A

udit

Scor

e

Strategy-Spcific Customer Outcome Measure

5.60

5.70

5.80

5.90

6.00

6.10

6.20

Q1 FY99 Q2 FY99 Q3 FY99 Q4 FY99

Quarter

Enjo

yabl

e Ex

perie

nce

Rat

ing

Average Operating Profit

$4,000

$4,200

$4,400

$4,600

$4,800

$5,000

$5,200

$5,400

Q1 FY98 Q2 FY98 Q3 FY98 Q4 FY98 Q1 FY99 Q2 FY99 Q3 FY99 Q4 FY99

Quarter

Ope

ratin

g Pr

ofit

* Operating profit (e.g. controllable contribution) is scaled by the number of weeks in each respective quarter. ** Note that operating profit in convenience store retailing exhibits strong quarterly seasonality.

Strategy-Specific Customer Outcomes

Financial Perspective

Customer Perspective

Internal Process Perspective

Learning and growth Perspective

H5

H1

H6

H4

H2

H3

Financial Performance

Strategy Inputs

Employee Capabilities

36

•Store24 implements differentiation strategy. •Translates strategy to a set of operating standards and measures store-level implementation of these standards using walk-through audits.

•Customer focus groups confirm that differentiation strategy is not resonating with customers. • Store 24 refocuses on basic service operations.

•Customer feedback surveys suggest differentiation strategy is not resonating with customers.

•Store24 updates performance measures to only reflect basic service operations

Q1 FY 1998 Q3 FY 1999 Q4 FY 1999 Q1 FY 2000

•Monitors and enforces store-level strategy implementation using walk-through audits •Monitors customer feedback about strategy through in-store comment cards and telephone surveys.

FIGURE 5 Timeline of Events Related to Store24’s Strategy Change

TABLE 1 Descriptive Statistics for the Sample of 65 Stores Used in Empirical Analyses

Variable Mean SD Min Median Max Profit 133.93 54.88 51.88 121.63 349.49 Input_Diff 108.16 22.39 46.43 117.85 135.71 Input_Basic 89.89 5.58 71.21 89.60 99.26 Outcome_Diff 27.98 9.88 2.56 26.85 51.87 MgrSkill 3.27 0.63 1.21 3.27 4.38 CrewSkill 3.35 0.43 2.75 3.24 4.51 MgrTenure 45.2 57.95 0.20 22.51 265.76 CrewTenure 13.4 16.38 2.27 8.48 89.66 Competition 3.87 1.38 1.65 3.68 11.13 Population -0.06 0.90 -1.27 -0.28 3.06 Income 2,588.35 532.20 1,700.00 2,499.00 4,230.00 24hours 0.85 0.36 - 1.00 1.00 Sqft 2,139.05 374.78 1,333.00 2,133.00 2,919.00 Rent 23.73 15.90 4.76 19.02 85.71

Profit = Revenue from general merchandise, lottery tickets, money orders, and phone cards less expenses related to cost-of-goods sold, utilities, and labor, scaled by square feet of the store; Input_Diff = measure of store-level implementation of Differentiation strategy measured as percentage compliance with operating standards related to the differentiation strategy; Outcome_Diff = customer (Outcome) measure of Differentiation strategy; MgrSkill and CrewSkill =Average of bi-annual measures of the manager and crew skills in basic store operations, rated on a five-point scale; MgrTenure and CrewTenure = Months of tenure with Store24 for managers and average months of tenure with the company for non-management employees (crew) respectively; Input_Basic = measure of percentage compliance with operating standards related to standard convenience store operations; Competition = number of competitors within the trading area of a store; Population = store location factor score capturing items related to population density and foot traffic around the stores’ trading area; Income = Measure of median annual disposable income available for grocery and convenience store purchases in the stores trading area 24hours = 1 if store is open 24 hours per day, 0 otherwise; Sqft = square footage of the store; and Rent = monthly rent per square foot for store.

37

TABLE 2 Correlation Matrix for 65 Stores during FY 1999

Profit Outcome_Diff Input_Diff MgrSkill CrewSkill MgrTenure CrewTenure Input_Basic Competition Population Income 24hours Sqft Profit 1 Outcome_Diff -0.4125* 1 Input_Diff -0.1293 0.2433* 1 MgrSkill 0.0911 0.0682 0.183 1 CrewSkill 0.0949 -0.0401 0.0031 0.2551* 1 MgrTenure 0.3460* -0.1274 0.0575 0.2162* -0.1059 1 CrewTenure 0.0815 -0.0133 -0.0046 0.1847 0.12 0.3266* 1 Input_Basic 0.1022 0.1007 0.2612* 0.3514* 0.1446 0.0483 0.1386 1 Competition -0.3920* 0.2477* 0.0419 0.2896* -0.1647 0.1677 -0.0427 0.0138 1 Population 0.4452* -0.0848 -0.2750* -0.1605 0.0515 -0.0363 -0.0089 -0.0592 -0.1303 1 Income 0.2223* -0.4648* -0.1132 -0.2698* 0.0142 0.0323 0.0118 0.0434 -0.3821* 0.0281 1 24hours -0.096 0.0209 0.2731* 0.2067* 0.0612 -0.1923 -0.0949 0.2005 0.0998 -0.1776 -0.0095 1 Sqft -0.5790* 0.2280* 0.1885 0.1402 -0.0159 0.0266 0.063 0.1552 0.3145* -0.181 -0.0661 0.035 1 Rent 0.6526* -0.4003* -0.3130* -0.1444 0.0329 0.1956 0.172 -0.0654 -0.4161* 0.3904* 0.4206* -0.1363 -0.5487*

38

TABLE 3 Linking the Internal Process and Financial Perspectives

(Dependent Variable = Profit; Adjusted R2 = 0.72) Coefficient Standard Error Two-Sided p-Value Intercept 22.49 89.05 0.80 Input_Diff 0.02 0.19 0.92 MgrSkill 16.87 8.93 0.06 CrewSkill 2.02 7.36 0.79 MgrTenure 0.34 0.08 0.00 CrewTenure -0.41 0.25 0.11 Input_Basic 2.16 1.05 0.05 Competition -7.91 3.56 0.03 Population 22.63 4.68 0.00 Income 0.006 0.009 0.48 24hours -0.92 11.42 0.94 Square Feet (00's) -0.06 0.01 0.00 Rent per Square Foot 0.32 0.38 0.40

All bolded coefficients are significant at least at the 10% level using a two-tailed test.

TABLE 4

Linking Internal Processes to Customer Outcomes (Dependent Variable = Outcome_Diff; Adjusted R2 = 0.13)

Coefficient Standard Error Two-Sided p-Value Intercept 18.10 10.50 0.09 Input_Diff 0.11 0.06 0.07 Input_Diff x MgrSkill -0.11 0.13 0.39 Input_Diff x CrewSkill -0.02 0.13 0.86 Input_Diff x MgrTenure 0.002 0.001 0.184 Input_Diff x CrewTenure -0.001 0.004 0.834 MgrSkill 1.99 2.93 0.50 CrewSkill -2.03 2.38 0.40 MgrTenure -0.04 0.03 0.11 CrewTenure 0.05 0.06 0.42

All bolded coefficients are significant at least at the 10% level using a two-tailed test.

39

TABLE 5 Panel A: Linking Customer Outcomes to Financial Performance

(Dependent Variable = Profit; Adjusted R2 = 0.76) Coefficient Standard Error Two-Sided p-Value Intercept 37.12 88.53 0.68 Outcome_Diff -0.92 0.46 0.05 Outcome_Diff x MgrSkill 0.64 0.79 0.42 Outcome_Diff x CrewSkill 2.00 0.84 0.02 Outcome_Diff x MgrTenure 0.00 0.01 0.64 Outcome_Diff x CrewTenure -0.02 0.04 0.65 MgrSkill 21.00 8.55 0.02 CrewSkill 3.78 6.99 0.59 MgrTenure 0.28 0.07 0.00 CrewTenure -0.37 0.26 0.17 Input_Basic 2.47 1.00 0.02 Competition -7.51 3.35 0.03 Population 22.50 4.38 0.00 Income -0.01 0.009 0.93 24hours -5.28 11.58 0.65 Square Feet (00's) -0.07 0.01 0.00 Rent per Square Foot 0.06 0.38 0.88

All bolded coefficients are significant at least at the 10% level using a two-tailed test.

Panel B: Summary of Moderating Effect of Crew Skills Two-sided p-value for test of 1 3 0γ γ+ =

Coefficient Two-Sided p-Value Low Crew Skills -2.92 0.000 Mean Crew Skills -0.92 0.098 High Crew Skills 1.08 0.020

TABLE 6 Linking the Learning & Growth and Internal Process Perspectives

(Dependent Variable = Input_Diff; Adjusted R2 = 0.09) Coefficient Standard Error Two-Sided p-Value Intercept 55.90 34.13 0.11 MgrSkill 16.41 5.14 0.00 CrewSkill 0.14 9.12 0.99 MgrTenure -0.03 0.05 0.53 CrewTenure -0.12 0.18 0.49 All bolded coefficients are significant at least at the 10% level using a two-tailed test.

40

TABLE 7 Panel A: Did the Balanced Scorecard Contain Timely Information about

Problems with the Strategy? (Dependent Variable = FY 1998 fourth-quarter Profit; Adjusted R2 = 0.66)

Coefficient Standard Error Two-Sided p-Value Intercept 14.76 21.39 0.49 Input_Diff 0.002 0.05 0.98 Input_Diff x MgrSkill 0.09 0.07 0.26 Input_Diff x CrewSkill 0.17 0.09 0.05 Input_Diff x MgrTenure 0.00 0.00 0.73 Input_Diff x CrewTenure 0.00 0.00 0.91 MgrSkill 5.57 2.54 0.03 CrewSkill 0.11 2.18 0.96 MgrTenure 0.09 0.02 0.00 CrewTenure -0.10 0.06 0.13 Input_Basic 0.39 0.21 0.07 Competition -2.12 0.60 0.00 Population 4.27 1.28 0.00 Income 2.78 1.21 0.03 24hours -1.60 4.32 0.71 Square Feet (00's) -0.02 0.00 0.00 Rent per Square Foot 0.10 0.12 0.40

All bolded coefficients are significant at least at the 10% level using a two-tailed test.

Panel B: Summary of Moderating Effect of Crew Skills Coefficient Two-Sided p-Value Low Crew Skills -0.17 0.098 Mean Crew Skills 0.002 0.976 High Crew Skills 0.18 0.084