adoption and use of performance measures in medium-sized cities
TRANSCRIPT
The Adoption, Use and Impacts of Performance Measures in Medium-Size Cities: Progress Toward Performance Management
David H. Folz, Ph.D.Professor
The University of TennesseeDepartment of Political Science
1001 McClung TowerKnoxville, TN 37996-0410
Ms. Reem Abdelrazek, MPAResearch Associate
Tennessee Advisory Commissionon Intergovernmental Relations (TACIR)
Yeonsoo Chung, Ph.D.Managing Director
North American OperationsKnowledge Source, Inc.
Adoption, Use and Impacts of Performance Measures in Medium-Size Cities
Abstract
Based on a national mail survey of chief executives in mid-sized US cities (populations
between 25,000 and 250,000), this study examines the patterns of adoption, use and impacts of
performance measures for the purpose of advancing understanding of the challenges involved in
moving from performance measurement to performance management. This study identifies the
factors that distinguish cities that adopted and used performance measures and the results that
chief executives expected to derive from the use of performance measures. What chief
executives thought about the helpfulness of performance measures in making various types of
decisions and why they thought their use of performance measures met, fell short, or exceeded
their expectations are examined. The study finds that while most chief executives thought that
performance measures met or exceeded their expectations, several factors helped to explain why
the use performance measures fell short of leaders’ expectations. The single most important
factor that helped to explain the gap between expectations and actual experience was the extent
of “buy-in” of performance measurement by line managers and administrators. The level of
workforce unionization and the extent of municipal experience with performance measurement
also helped to explain whether or not performance management was perceived to be successful.
Keywords: performance measurement, municipal government, performance management
Performance measurement in the public sector has garnered a great deal of interest
among elected and appointed city officials as well as public administration scholars since at least
the early 1990s (Bouckaert 1992; Wechsler and Clary 2000). Scholars have described the
usefulness of tracking performance (Hatry et al 1990; Wholey 1999) and highlighted the many
obstacles and unintended consequences of implementing various kinds of performance measures
(Ammons 1992; Smith 1995). Several studies have examined the extent to which local officials
adopt various performance measures and use them for different decision applications (de Lancer
Julnes and Holzer 2001; Rivenbark and Kelly 2003; Poister and Streib, 1999, 2005; Melkers and
Willoughby 2005; Yang and Holzer 2006).
While the performance measurement literature is replete with descriptions of its potential
benefits, questions remain about whether the use of performance measures makes a difference in
local governance and policy making, particularly in budgeting and resource allocation decisions
(de Lancer Julnes and Holzer 2001; Hatry 2002; Ho 2005; Melkers and Willoughby 2005). As
Ammons and Rivenbark (2008, 304) observe, “local governments’ progress in using
performance measures to influence program decisions and service delivery has lagged behind
their pace in collecting and reporting basic measures.” In their survey of 277 city and county
administrators, Melkers and Willoughby (2005) found that almost half of the respondents in a
mixed sample of governments reported wide use of performance measures. However,
respondents were “much less enthusiastic about the effectiveness of using performance measures
to influence budgeting processes and outcomes in particular” (Melkers and Willoughby 2005,
188). Likewise, de Lancer Julnes and Holzer (2001) found that only a subset of local
governments that collect performance measures actually use them to improve program and
service decisions.
While feedback about municipal service performance has been found to be helpful in
informing the citizenry, Melkers and Willoughby (2005) found that few local officials expressed
strong views about the value of citizen involvement in the performance measurement process. In
his study of Midwestern mayors, Ho (2005) found that these officials considered performance
measurement to be an important tool for helping to enhance public accountability, but only 17
percent actually involved their citizens in the process of measuring service performance. Ho
(2005, 234) suggested that several political and organizational environment variables are helpful
2
for understanding how chief executives perceive the usefulness, sustainability and success of
performance measurement but concluded that “how performance measurement is integrated into
decision making remains a black box” and merits further study. Ammons and Rivenbark (2008)
examined fifteen North Carolina cities and concluded that the likelihood performance data will
influence operations is enhanced by the adoption of efficiency measures, the willingness of
officials to engage in performance benchmarking, and the incorporation of performance
measures into key management systems.
This study examines why city officials adopted performance measures, how they report
using them, what impacts and results municipal chief executives realized after adoption and
whether these impacts fell short or met/exceeded their expectations. We explore what city
leaders perceived to be the helpfulness of particular types of measures for specific types of
decisions and what these chief executives thought about the overall impact performance
information had on the quality of local decision making. We are particularly interested in
identifying the most salient reasons for why CEOs may perceive a gap to exist between their
expectations for and actual experiences with performance measures. In other words, an inquiry
that investigates why CEOs consider performance management to be successful or not offers the
prospect for identifying some of the factors within local control that may affect the extent to
which local officials can realize the benefits expected from performance measurement as a key
component of a performance management system.
Data and Methods
The data for this research were collected from a mail survey and from US census data. A
mail survey was sent to 670 chief executives in US municipalities with populations 25,000 to
250,000. The mayors or city managers of these mid-sized cities comprise the survey target
population. There are a total of 1,339 municipalities with populations in the 25,000 to 250,000
3
range. A stratified random sample of 670 cities (about 50% of the target population) and contact
data for their chief executives was obtained from the International City/County Management
Association (ICMA) in 2004. In addition to the names and addresses of chief executives, the
ICMA data file included information on population, region, metropolitan status, and form of
government for each city. The mail survey questions referenced in this study are included in the
appendix.
Cities with populations between the 25,000 and 250,000 were chosen because of the
availability of socioeconomic data for these cities, the desire to compare findings with previous
research on this population stratum (Streib and Poister 2002, 1998; Poister and Streib 1999), and
survey budget resource constraints. Cities with smaller populations were excluded because the
level of adoption and use of performance measures in these smaller cities is low (Rivenbark and
Kelly 2003).
The survey instrument was initially mailed in June 2004 followed by a subsequent second
mailing to non-respondents approximately two weeks later in early July 2004. A total of 280
completed surveys were returned for a response rate of about 42 percent. Table 1 shows that the
distribution of responses obtained are comparable to the distribution of cities in the sample.
With respect to population class, the survey response percentages are within a few percentage
points of target population. In terms of geographic region, municipalities from the northeast are
somewhat under represented (-6.7%). With respect to form of government, cities with the mayor-
council form of government are somewhat under-represented (-6.6%) while cities with a council-
manager form of government are somewhat over-represented by 7.4%. In most respects, the
profile of the cities that responded to the survey is comparable to that of all medium-sized US
cities, allowing generalization to this population.
Table 1 goes here
4
Adoption and Use of Performance Measures
Poister and Streib (1999) reported that larger cities were much more likely to adopt
performance measures. They found that only 30% of cities with populations between 25,000 and
50,000 use performance measures compared to more than half of all cities with populations
between 100,000 and 250,000 (Poister and Streib 1999). Our survey results also indicate a gap in
the adoption level among large and small cities but we also find that the adoption and use of
performance measures has grown in popularity among cities in all population ranges since the
Poister and Streib (1999) survey. As Table 2 indicates, more than two-thirds (68.0%) of cities
have adopted and use performance measures. Among those cities with populations between
25,000 and 50,000, 59% use performance measures while 83.7% of cities with populations
between 100,000 and 250,000 use them.
Table 2 goes here
That more than two-thirds of all medium-sized cities use performance measures supports the
conclusion reached by Melkers and Willoughby (2005, 188) that these metrics are now a “fairly
pervasive” feature of local governments. The 185 cities that reported both adoption and current
use of various performance measures are the main focus of this paper.
Poister and Streib (1999) found that those cities with a council-manager form of
government used performance measures more frequently than cities with a mayor-council form.
Among the jurisdictions in our survey, council-manager cities also report using performance
measures more frequently than mayor-council cities by a significant margin (72.6% v. 56.2%).
When using the conceptual definitions advanced by Frederickson, Johnson, and Wood (2004) to
classify municipal structures as either “political” (the traditional mayor-council form),
“administrative” (the traditional council-manager form) or “adaptive” (a combination of features
from the other two types), an interesting pattern emerges.
5
Table 3 shows the use performance measures across these three structural types. This
distribution indicates that cities with either an adapted or an administrative form are much more
likely than political cities to use performance measures. In fact, about 70% of those cities
Table 3 goes here
served by a professional chief executive or administrative officer use performance measures
compared to just 50% of those led by an elected CEO. This difference is not only statistically
significant but substantively important because it suggests that cities led by professionally
trained managers and administrators are much more likely to employ performance measures,
regardless of whether the city has a mayor-council form or not. Following Keene, O’Neil,
Portillo and Svara (2007), this finding underscores another way that professional managers add
value to the communities they serve.
How prevalent are particular types of performance measures among the 185 mid-sized
cities that report their adoption and use? Previous research by Poister and Streib (1999) found
that efficiency measures were the least frequently used type of measure and that workload or
output measures were the most frequently used. Ammons and Rivenbark (2008) also report that
many local governments measure performance but only with less sophisticated workload or
output measures that provide little in the way of diagnostic feedback compared to higher order
efficiency, effectiveness and quality of service measures. The findings in Table 4 confirm that
the largest proportion of cities report using workload or output measures, but a considerably
larger proportion of cities (about half) report using more sophisticated performance measures
that include indicators of citizen satisfaction with services, service quality, outcome
effectiveness, and efficiency. This pattern of use suggests an increasing level of sophistication in
the type of performance information being collected by mid-sized cities.
Table 4 goes here
6
Chief executives were asked to identify various features related to the organizational
culture of their cities. We examined several of these factors to determine whether any might be
associated with the use of particular types of performance measures, especially in light of the
previous findings by de Lancer Julnes and Holzer (2001) that variables such as management
attitudes and risk taking tolerance positively influenced the actual use of performance
information. Table 5 indicates that in those cities where the chief executive agreed that the
managers in their jurisdiction viewed performance measurement as an important basis for
making decisions, the information from outcome, efficiency, and service quality measures was
much more likely to be used. The strongest single bivariate association (as measured by gamma,
a frequently calculated PRE-based measure of association for ordinal data) occurred between the
use of outcome measures and having an organizational culture in which managers viewed
performance measurement as an important basis for making decisions. The use of workload
measures was strongly correlated with a greater perceived receptivity among non-management
employees to organizational change.
While the correlations are in the positive direction predicted by de Lancer Julnes and
Holzer (2001), we found no statistically significant connection between the use of the various
types of performance information and management’s willingness to implement organizational
change, the extent of support by elected officials or the presence of a system that encourages
risk-taking. These findings suggest that the actual use of more types of performance measures
occurs when managers understand the value of performance data for making decisions and when
non-management employees are receptive rather than fearful about possible organizational
change in the wake of the use of workload data in making management decisions.
Table 5 goes here
7
Reasons for Adoption and Expected Results
What reasons do chief executives identify for adopting performance measures? In other
words, what motivated local officials to invest the resources to measure and track the various
aspects of service performance? Figure 1 shows that a fairly strong consensus exists among chief
executives. By a large majorities, chief executives thought that adoption of performance
measures would improve management decisions (81.9%), support budget recommendations and
decisions (71.9%) and respond to citizen demands for greater accountability (68.6%). These
findings are consistent with previous research insofar as the desire to make better management
decisions also was the principal motivator reported by Poister and Streib (1999). Significant
proportions of chief executives in that study also reported that performance measures were
adopted in response to citizen demands for greater accountability and pressure from council
members.
As Figure 1 shows, the largest proportion of chief executives in our survey believed that
performance measures were adopted to help improve management decisions and support budget
recommendations and decisions. This suggests that local officials recognized the potential for
integrating performance measures in decisions about management and budgeting decisions to an
extent not reported previously.
Figure 1 goes here
However, what results did city officials really expect to see after using the performance
measures they adopted and to what extent do these expected results correspond to the most
prominent reasons why they adopted performance measures in the first place? In other words, to
what extent do the actual benefits of adopting performance measures correspond to what city
officials expected to achieve through their use? Within the limits of survey research, we attempt
to detect whether the adoption of various performance measures might just be symbolic or
8
“window dressing,” perhaps as a response to pressures by peers or other groups who do not wish
for their community to seem non-progressive since other cities measure performance.
Figure 2 suggests a fairly high level of correspondence among the three most frequently
cited reasons for adopting performance measures and the three most frequently reported expected
results. Sizeable proportions of chief executives expected that the use of performance measures
would result in stronger justification for management decisions and budget requests and also to
improve communication with citizens about the city service performance. Almost three of five
chief executives expected that the use of performance measures would enhance the level of
understanding among city council members.
These findings suggest that there is very little, if any, “cognitive dissonance” with respect
to the reasons offered for adopting performance measures and what local officials expected to
see as a result of their implementation. In fact, the consistency between the rationale for
adoption and the results expected suggests that chief executives appear to have a fairly mature,
outcome-focused view of performance measurement.
Figure 2 goes here
Applications of Service Performance Measures
Considering that chief executives expected that performance measures would help to
improve management decision making, justify resource allocation decisions and improve
communications with citizens about service performance, to what extent did cities actually use
the different types of performance measures to advance these objectives? Table 6 indicates that
for decisions related to managing/evaluating services and programs, the majority of cities used
outcome and efficiency measures and just under half of cities used service quality measures. For
resource allocation or budgeting decisions, the majority of cities used workload, efficiency and
9
outcome measures. To help improve communications with citizens about service performance,
most cities used the results from outcome measures and local surveys of citizen satisfaction.
Table 6 goes here
For each of the three types of decision applications, the majority of cities appear to use
the types of performance measures that are most appropriate to support decisions related to each
(Ammons 2001). That outcome measures are the most widely used for all three types of decision
applications suggests that local officials have recognized the limits of relying primarily on
workload or efficiency measures as reported earlier by Poister and Streib (1999) and Ammons
and Rivenbark (2008). Defining and collecting data for organizationally relevant outcome
measures and “integrating them into daily operations” is an expensive and time-consuming
process and it is “critical to compute the costs (direct, indirect and intangible) associated with …
PM implementation” (Frank and D’Souza 2004, 706-7). While a formal cost-benefit analysis is
beyond the scope of this paper, it is possible to ascertain what chief executives think about the
various impacts that these measures have had on specific types of decision outcomes, the overall
quality of decisions reached, and whether actual experience matched expectations. These
assessments are a useful beginning point to ascertain what chief executives think about the value
of the investment in performance measurement and to identify some of the reasons why use of
these measures either met, fell short or exceeded the expectations of chief executives.
Perceived Impacts of Performance Measures
What do chief executives think about the helpfulness of the performance measures with
respect to specific types of outcomes related to managing/evaluating, resource
allocation/budgeting and improving communication with citizens about service performance?
Table 7 summarizes the perceptions of chief executives with respect to the helpfulness of
performance measures in achieving specific types of impacts related to each function.
10
Table 7 goes here
Overall, most chief executives thought that their performance measures were either
somewhat or very helpful in achieving outcomes related to each of the three categories. For the
results related to managing/evaluating programs, many chief executives thought that
performance measures were not helpful in supporting personnel performance appraisals. This
finding is completely understandable since performance measures are targeted to program
outcomes rather than individual accomplishments.
For outcomes related to resource allocation and budgeting, chief executives were much
more likely to rate performance measures as being very helpful in focusing program priorities.
Almost half of all respondents perceived performance measures to be very helpful for this
purpose and similarly, almost four in ten chief executives thought they were very helpful for
making positive changes in program emphasis. In terms of the helpfulness of performance
measures in improving communications with citizens about service performance, more than 45%
of chief executives thought that they were very helpful in producing better communication
between citizens and administrators/elected officials. The largest proportion of chief executives
(41.4%) thought that performance measures were only somewhat helpful in improving relations
with community groups.
These patterns suggest that chief executives find performance measures to be most
helpful when performance information is used selectively to inform particular types of decisions.
Selective application of performance data that is most germane to particular types of decision
applications appears to be related to how chief executives rate the helpfulness of performance
information. In other words, chief executives of mid-sized cities have become more
discriminating and selective about the value of certain types of performance data and perhaps
less tolerant of an overload of performance information. At any rate, their collective experiences
11
suggest that they find performance measures to be very helpful for improving the quality of
decisions related to managing/evaluating programs, very helpful in helping to focus program
priorities in decisions related to resource allocation but only somewhat helpful for improving
communication with citizens.
Poister and Streib (1999) and Ammons and Rivenbark (2008) acknowledged the
possibility that favorable ratings of performance measures might outstrip their actual impacts. To
help check for that possibility, chief executives were asked what they thought about the overall
impact of performance information on the quality of decision making by the city officials that
use this information. In other words, what do they perceive as the magnitude of difference that
performance information has made in decisions by the city officials who use it? Table 8 shows
that about 60% of CEOs think it has had a slight positive impact while another 30% think
performance information has had a significant positive impact on the quality of decision making.
Less than eight percent think that performance information has had no impact while about four
percent are not sure about the impact of performance measures on overall decision quality. That
most CEOs consider performance information to have had only a slight positive impact on the
overall quality of decisions suggests that CEOs do in fact perceive a difference in the helpfulness
of measures for specific types of decisions and their impact on improving the overall quality of
decisions.
Table 8 goes here
What factors help to explain these differential perceptions about the impact of
performance information among chief executives? Since chief executives perceived variations in
the level of helpfulness of performance information, we hypothesized that these perceptions
would be related to how they perceived the impact of performance information on the overall
quality of decisions by city officials. Summary scores were computed for the helpfulness
12
indicators for decisions related to managing/evaluating programs, resource allocation/budgeting,
and improving communications with stakeholders. The gamma values for the crosstabulation
between these variables and the perceived impact of performance information on the overall
quality of decisions are shown in Table 9. These scores indicate that for each type of decision
application, there is, in fact, a strong positive association between the helpfulness rating of the
performance measure(s) used and the perceived impact that performance information has had on
the overall quality of decisions by local officials. So, when chief executives consider
performance measures to be helpful in making decisions about impacts related to each of the
three broad categories, they are much more likely to perceive that the use of performance
information has had a more significant positive overall impact on the quality of decisions by city
officials.
Table 9 goes here
While the strong connection between the perceived helpfulness of performance measures
and the overall positive impact of performance information on the quality of decisions is not
surprising, the question of whether the actual use of performance measures fell short, met or
exceeded expectations remains to be answered. If performance management is to become an
enduring feature of local government decision processes, part of the political calculus of making
that a reality may depend on whether chief executives think that the performance information
generated falls short, meets or exceeds their expectations. Consequently, it is important to
examine these perceptions and what factors may affect chief executives’ perceived success of
performance measures.
Table 10 indicates that most chief executives (68.7%) think that their community’s actual
experience in using performance information met or exceeded their expectations. This finding
augers well for the prospects of institutionalizing performance management in local governance.
13
Yet, almost one in five chief executives thought that performance information fell short of their
expectations and another 12.5% were unsure about it. What distinguishes those cities
whose chief executives thought that performance measures met or exceeded expectations
versus those whose experience fell short of expectations?
Table 10 goes here
Following previous research that illustrated the importance of political, cultural and
community features in measuring the effects of performance measurement (Melkers &
Willoughby 2005; Folz & French 2005), several municipal features were hypothesized to be
helpful for explaining the variation in chief executives’ perceptions about whether performance
measures either fell short or met/exceeded their expectations. Table 11 identifies four factors that
together explain over one-fourth of the variation in how chief executives evaluate the
“performance” of performance measures in terms of their expectations versus their actual
experiences with them. The dependent variable is a dichotomous measure of whether chief
executives think performance measures fell short of expectations or met/exceeded their
expectations. Each of the independent variables in the model attain statistical significance at
the .05 level except for the measure of employee receptivity to change which is just above this
threshold.
Table 11 goes here
In terms of the relative importance of the independent variables in the model, what most
affects whether chief executives think performance measurement has fallen short or
met/exceeded their expectations is whether they believe that the performance measures
employed by the city are in fact supported by the municipality’s administrative and supervisory
personnel. Clearly, the support of and commitment by the supervisors responsible for managing
the system of performance indicators is critical to how chief executives perceive the success of
14
the performance measurement. This finding is substantively important because it provides
empirical evidence that buy-in by line managers and supervisors is vital to the perceived success
of performance management. When most of the administrative staff support the use of the
adopted performance measures, chief executives’ expectations for performance measures are
much more likely to be met or exceeded.
The findings also indicate that in those cities where a larger proportion of municipal
employees are unionized, chief executives are more likely to think that performance measures
fell short of their expectations. We speculate that in those cities with a higher level of
unionization, representatives from employee unions may have some role in negotiating which
performance measures are used, the frequency of data collection, or perhaps even how
performance data may impact various programmatic decisions. At any rate, higher levels of
employee unionization adversely color how chief executives perceive the impacts of
performance measures.
There are few substitutes for experience in most local government activities and
performance measurement certainly numbers among them. We find that the longer a city has
used performance measures, the more likely the chief executive is to think that their city’s
performance measurement system meets or exceeds their expectations. Clearly, there is a
learning curve in performance measurement and a process of continual refinement in judgments
about which measures are most useful for different types of decision applications. Those chief
executives of cities that have more years of experience in the process are much more likely to
think that performance measures have met or exceeded their expectations.
Finally, when chief executives agree or strongly agree that the city’s non-managerial
employees are receptive to changes in organizational policies, chief executives are more likely to
think that the city’s performance measures have met or exceeded their expectations. After all,
15
change should occur when performance information suggests that policies need to be revised to
improve results and outcomes. A workforce that understands the need for such changes is more
likely to accept them, an impression not lost upon chief executives whose perceptions of the
impacts of performance measures are affected in a more positive way.
Summary and Conclusion
This study finds that among mid-sized US municipalities, the use of performance
measures is more pervasive than ever with more than two-thirds (68%) of these cities using one
or more types in 2004. Larger cities and those with appointed rather than elected chief executives
are the most likely to use performance measures. Performance measurement also is particularly
prevalent among cities with adapted or administrative forms of government. While workload or
output measures continue to be the most widely used type of performance indicator, this study
finds that outcome, service quality and citizen satisfaction measures are used more extensively
than reported in earlier studies. Furthermore, these measures are most likely to be adopted among
those cities where managers view performance measurement as an important basis for making
decisions.
Improving the decisions made by managers, supporting budget recommendations/
decisions and responding to citizen demands for greater accountability are the three reasons most
commonly cited by chief executives for adopting performance measures in the first place. The
high level of consistency between the reasons for adoption and the results expected from the use
of performance measures indicates that chief executives have a very outcome-focused view of
performance measurement and do not consider these measures to be merely symbolic.
The types of performance measures adopted and used by mid-sized cities appear to be the
ones most appropriate for improving various types of management decisions, supporting budget
recommendations and communicating performance results to citizens and council members. For
16
issues within each of these three areas, chief executives have definite views about the relative
helpfulness of performance measures. Most chief executives think that performance measures are
very helpful for improving the quality of decisions and decision capacity related to managing/
evaluating programs and for focusing program priorities in making resource allocations.
However, they found them to be only somewhat helpful for improving communications with
citizens.
Overall, most chief executives think that the use of performance information has had just
a slight positive impact on the overall quality of decisions by made by city officials, but almost
one-third believe that it has had a significant positive impact. Only about one in ten CEO’s are
not sure about or discern no impact on decision quality. In terms of whether these perceived
impacts fell short, met or exceeded their expectations, most CEOs thought that their experiences
with performance measures either met or exceeded their expectations, but almost one in five
indicated that their experience fell short of their expectations.
Regression analysis identified four factors that explained more than one-fourth of the
variation in these perceptions. Among these, the single most important factor that affected chief
executives’ perception of the success of performance management was whether or not they
believed that most line managers and supervisors in their city supported the use of performance
measures in their areas of service or program responsibility. The perceived success of
performance measurement appears to depend extensively on the level of commitment by
administrators and line managers to the objectives of performance measurement. While this
finding is intuitive, it underscores the importance of engaging these key stakeholders in local
decisions about what performance measures to use, how information for them should be
collected, computed and reported, and also how this information will be used to inform various
types of service and program decisions. Including line managers and administrators in these
17
decisions must play an important role in securing their support and buy-in. The ways and means
that localities accomplish this inclusive approach to the design and implementation of
performance management merits further investigation and then replication by communities,
especially since the findings from this study indicate that securing the support of these key
employees is vital to the perceived success of the performance management enterprise.
The success of performance measurement as a management tool also appears to be a
function of experience and a trial and error process that results in refinement of both the
measures used and the ways in which the information generated informs local government
decisions. While the extent of employee unionization appears to present special challenges for
realizing chief executives expectations for performance measurement, when both managers and
employees embrace the performance measures adopted and use the information generated from
these measures to improve program management, justify budget recommendations, and enhance
communications with council and citizens, performance measurement is apt to meet or exceed
expectations by chief executives. That the investment in and applications of performance
measures has met or exceeded the expectations of chief executives augers well for the continued
refinement of performance measures and indicates that chief executives recognize the specific
ways that performance measurement adds value to the process of governance.
18
Table 1. Characteristics of the Target Population and Mail Survey Responses 2004 Survey Target
populationSurvey responses Difference
Classification Number Percent Number Percent %Population group 100,000-249,999 88 13.1 44 15.7 2.6 50,000-99,999 197 29.4 75 26.8 -2.6 25,000-49,999 385 57.5 161 57.5 0Geographic region Northeast 164 24.5 50 17.8 -6.7 North Central 165 24.6 70 25.0 0.4 South 162 24.2 77 27.5 3.3 West 179 26.7 83 29.7 3.0Form of government Mayor-council 219 32.7 73 26.1 -6.6 Council-manager 422 63.0 197 70.4 7.4 Commission 11 1.6 5 1.8 0.2 Town meeting 5 .7 1 .4 -0.3 Representative town meeting
13 1.9 4 1.4 -0.5
Total Survey 670 100.0 280 100.0
Table 2. Adoption and Actual Use of Performance Measures by Population Size 25,000-29,999
30,000-39,999
40,000-49,999
50,000-99,999
100,000- 250,000 Totals
N % N % N % N % N % N %Not Adopted 26 53.1 26 39.4 13 29.5 15 21.4 7 16.3 87 32.0Adopted & Use 23 46.9 40 60.6 31 70.5 55 78.6 36 83.7 185 68.0Totals 49 100 66 100 44 100 70 100 43 100 272 100
Gamma = .404, Sig. = .000
Table 3. Use of Performance Measures by Form of Government, 2004
Political cities Adapted CitiesAdministrative
Cities TotalsN % N % N % N %
Not Adopted 11 50.0 58 29.9 16 30.8 85 31.7Adopted & Use 11 50.0 136 70.1 36 69.2 183 68.3Totals 22 100 194 100 52 100 268 100
19
Table 4. Types of Performance Measures Used
Type Measure Number Percent UsingWorkload or Output 156 57.4Efficiency or Unit Cost 108 40.4Outcome or Effectiveness 128 48.1Service Quality 135 50.6Citizen Satisfaction 136 50.9
Table 5. Use of Various Performance Measures and CEO Agreement with Selected Organizational Features
Organizational FeaturesManagement willing to implement organizational change
Management views performance measurement as an important basis for making decisions
Non-management employees are receptive to organizational change
Elected officials generally support innovative improvements
City has a reward/incentive system that encourages risk-taking
Workload .223 .361 .452* .138 .041Efficiency .141 .454* .066 .276 .155Outcome .259 .566* .255 .227 .197Service quality .266 .393* .293 .321 .259Citizen satisfaction
-.027 .139 .096 .114 .122
Notes: * Significant at the .05 level
20
Figure 1. Chief Executives’ Reasons for Adoption of Performance Measures
Figure 2. Chief Executives’ Expectations from Using Performance Measures
21
Table 6. Applications of Types of Performance Measures (N = 168)
Application
Type Measure Workload or Output
Efficiency/ Unit Cost
Outcomes/ Effectiveness
Service Quality
Citizen Satisfaction
N % N % N % N % N %Managing/ Evaluating Programs
75 44.6 87 51.8 96 57.1 82 48.8 68 40.5
Resource Allocations/Budgeting
99 58.9 89 53.0 92 54.8 55 32.7 43 25.6
Reports to Citizens/Media
55 32.7 45 26.8 87 51.8 60 35.7 77 45.8
Table 7. Chief Executives Perceptions of the Impacts of Performance Measures on Selected Dimensions
Helpfulness Level (in percents)
Possible impacts/results N Not helpful Somewhat helpful
Very helpful
Don’t know/ not
sureManaging/ Evaluating ProgramsImproved performance among employees
177 14.7 52.0 23.2 10.2
Supported personnel performance appraisals
175 28.0 33.1 24.0 14.9
Improved quality of decisions & decision capacity
176 5.1 33.5 53.4 8.0
Resource Allocations/ Budget RequestsMade positive changes in program emphasis
177 4.5 50.3 39.0 6.2
Realized some cost savings for city service(s)
177 15.8 50.8 24.9 8.5
Focused program priorities 176 13.6 31.8 49.4 5.1Increased service quality level 176 9.1 46.0 36.9 8.0Improved CommunicationImproved relations with community groups
174 17.8 41.4 25.3 15.5
Better communication between administrators & elected officials
177 6.2 39.5 45.2 9.0
22
Table 8. Perceived Impact of Performance Information on the Overall Quality of Decisions by City Officials
Frequency Valid PercentNo impact 12 7.4Slight positive impact 96 59.3
Significant positive impact 48 29.6
Don't know/not sure 6 3.7Total 162 100.0
Table 9. Associations Between Helpfulness Level of Performance Measures in DecisionApplications and Perceived Impact of Performance Measures on Overall Quality of Decisions
Helpfulness Level (none, somewhat, very) of Performance Measures for:
Resource Allocation and Budget Requests
(N =157)
Managing & Evaluating Programs
(N = 160)
Improved Communications
(N = 157)Perceived Impact on Quality of Decisions by Local Officials(none, slight, significant) .565* .457* .444*
* Significant at the .05 level
Table 10. Chief Executives Ratings of Performance Measures: Expectations v. Actual Experience
Rating N PercentFell short of expectations 33 18.8Met expectations 106 60.2Exceeded expectations 15 8.5Don’t know/ not sure 22 12.5Total 176 100.0
23
Table 11. Factors Influencing Chief Executive Perceptions of the Success of Performance Measures
Unstandardized
CoefficientsStandardized Coefficients t Sig.
BStd.
Error Beta (Constant) -.209 .194 -1.077 .284Years of experience with performance measurement .106 .035 .250 2.980 .004
Disagree (0) Agree (1) that non-managerial employees are receptive to changes in organizational policies
.107 .056 .163 1.914 .058
Disagree (0) Agree (1) that most administrators support the use of performance measures
.228 .058 .339 3.966 .000
Percent municipal workforce unionized -.003 .001 -.253 -3.028 .003
R SquareAdjusted R Square
Std. Error of the
Estimate.279 .252 .37167
24
AppendixSelected Questions from a National Survey of
Municipal Performance Measurement PracticesA. ADOPTION/ DEVELOPMENT OF PERFORMANCE MEASURES Cities may employ one or more of these types of measures:Workload or Output Measures - Amount of work or service provided or performed. Examples: tons of trash collected, number of calls answered.
Efficiency or unit cost Measures - Dollar cost per unit of output or workload. Examples: cost per police car dispatched, cost per refuse collection account served.
Outcome or Effectiveness Measures - Extent to which objectives, needs or desired impacts are achieved, met or produced. Examples: reduction in the number of commercial burglaries, reduction in substandard housing units.
Service Quality Measures - A value-based assessment of services. Examples: convenience level, response time, accuracy rate, safety level, turn-around time, courtesy rating.
Client or Citizen Satisfaction Measures - Extent to which clients think their needs are met; citizen ratings of programs. Examples: total complaints received, percent positive rating on a measure of service satisfaction; (information usually derived from surveys).
1. Considering these descriptions, please indicate whether your city has “Not adopted,” “Adopted but not used currently,” or “Currently use” each type of measure. (Please circle the number that applies to each type of measure).
Type of Measure Not adopted Adopted, not used Currently useWorkload or Output measures 1 2 3
Efficiency or Unit Cost measures 1 2 3
Outcome or Effectiveness measures 1 2 3
Service Quality measures 1 2 3
Client or Citizen Satisfaction measures 1 2 3
2. Cities adopt service measures for different reasons, some of which are listed below. In thinking about why your city adopted the measures you circled, please rank order the three most important reasons with “1” being most important.
Rank_____ To improve management decisions
_____ To respond to citizen demands for greater accountability
_____ To comply with wishes of elected city officials
_____ To respond to pressure from various community groups
_____ To support budget recommendations/decisions
_____ To comply with state or federal reporting requirements
_____ Other (please specify):
3. In your opinion, which results did city officials really expect to see after using the service or performance measures adopted by your city? (Please circle the numbers of all that apply).
1 Stronger justification for management decisions (e.g. personnel or resource deployment)
2 Improved communication with citizens about service performance
3 Enhanced understanding of service performance by council members
4 Stronger justification for budget requests
5 Improved employee morale
6 Improvement in employee performance
7 Other: (please specify):
4. In thinking about the above expectations city officials may have had for the impact of service performance measures, would you say your city’s actual experience with these measures generally “fell short,” “met,” or “exceeded” these expectations? (Please circle one number).
1 Fell short of the expectations 2 Met expectations3 Exceeded expectations4 Don’t know/ not sure
B. USE & APPLICATIONS OF SERVICE/ PERFORMANCE MEASURES7. Please circle the number of each type of measure city officials may use for each activity. Just skip any activity not
relevant to your city or that is not supported by any type of performance measure.
Type of MeasureActivity Workload Efficiency Outcomes Quality Citizen sat. surveys
Strategic Planning 1 2 3 4 5
Resource Allocation (Budgeting)
1 2 3 4 5
Managing/ Evaluating Programs
1 2 3 4 5
Internal Management Reports
1 2 3 4 5
Reports to Elected Officials
1 2 3 4 5
Reports to Citizens/ Media
1 2 3 4 5
2
C. IMPACTS OF PERFORMANCE MEASUREMENT9. How would you rate the overall helpfulness of the performance measures used in your city with respect to each of these possible impacts? (Please circle one number for each possible impact).
Possible Impact
Helpfulness Level
Not helpful
Somewhat Helpful
Very Helpful
Don’t know/
not sure
Made positive changes in program emphasis 1 2 3 4
Improved performance among employees 1 2 3 4
Improved quality of decisions & decision capacity 1 2 3 4
Facilitated program goal setting 1 2 3 4
Focused program priorities 1 2 3 4
Supported personnel performance appraisals 1 2 3 4
Increased service quality level 1 2 3 4
Enhanced employees’ understanding of goals 1 2 3 4
Improved relations with community groups 1 2 3 4
Realized some cost savings for city service(s) 1 2 3 4
Better communication between administrators & elected officials
1 2 3 4
Enhanced accountability of individual managers 1 2 3 4
11. Overall, what impact has the information derived from performance measures had on the quality of decision making by the city officials that use this information? (Please circle one).
1 No impact2 Slight positive impact3 Significant positive impact4 Don’t know/ not sure
13. What do city administrators think about the performance measures employed? (Circle the number that best fits your opinion).
City Administrators’ Stake Disagree Neither Agree nor Disagree
Agree Don’t know/ not
applicable
The CEO supports the use of performance measures
1 2 3 4
Most department heads support the use of performance measures
1 2 3 4
Most staff administrators support the use of performance measures
1 2 3 4
Most line supervisors support the use of performance measures
1 2 3 4
Most city employees support the use of performance measures
1 2 3 4
F. ORGANIZATIONAL FEATURES15. About how long has your city used performance measures? __________years
3
16. Please indicate the extent to which you disagree or agree with each of these statements.
Organizational FeatureStrongly Disagree
Disagree Agree Strongly Agree
Don’t Know
Management is willing to implement organizational change whenever appropriate.
1 2 3 4 5
Management views performance measurement as an important basis for making decisions.
1 2 3 4 5
Non-management employees generally are receptive to change in organizational policies.
1 2 3 4 5
Elected officials generally support innovative ideas for improvement. 1 2 3 4 5
We have a reward/incentive system that encourages risk-taking. 1 2 3 4 5
G. CITY CHARACTERISTICS17. Please indicate whether your city has any of the following features. Feature No YesMayor is directly elected by citizens 1 2
Mayor is selected by council 1 2
Most council members are elected by district 1 2
Most council members are elected at-large 1 2
Council members elected by a mixed district & at-large system 1 2
City has a Chief Administrative Office (CAO) position 1 2
Mayor presides over council meetings 1 2
Department heads report to the Mayor 1 2
Department heads report to a CAO 1 2
Mayor appoints and terminates CAO without consent of council 1 2
Mayor appoints and terminates CAO with consent of council 1 2
Council appoints and may terminate city manager 1 2
Statutory charter form is “Mayor-Council” form of government 1 2
Statutory charter form is “Council-Manger” form of government 1 2
Statutory charter form is “Commission” (without administrator) 1 2
18. What was your city’s total operating budget for FY 2004? $___________________________19. About how many full-time equivalent employees (FTEs) are employed in your city? _________ 20. About what percent, if any, of all FTEs are unionized? ___________%
4
References
Ammons, David N. 1992. Productivity Barriers in the Public Sector. In Public Productivity Handbook, edited by Marc Holzer, 117-36. New York, NY: Marcel Dekker, Inc.
Ammons, David N. 2001. Municipal Benchmarks: Assessing Local Performance and Establishing Community Standards. Thousand Oaks, CA: Sage Publications Inc.
Ammons, David N. and William C. Rivenbark. 2008. Factors Influencing the Use of Performance Data to Improve Municipal Services: Evidence from the North Carolina Benchmarking Project. 68(2): 304-318.
Bouckaert, Geert. 1992. Public Productivity in Retrospective. In The Public Productivity Handbook, ed. Mark Holzer, 5-46. New York: Marcel Dekker.
de Lancer Julnes, Patria and Marc Holzer. 2001. Promoting the Utilization of Performance Measures in Public Organizations: An Empirical Study of Factors Affecting Adoption and Implementation. Public Administration Review 6(6): 693-703.
Folz, David and P. Edward French. 2005. Managing America’s Small Communities: People, Politics and Performance. Lanham, MD: Rowman & Littlefield Publishers.
Frank, Howard. and Jayesh D’Souza. 2004. “Twelve Years in the Performance Measurement Revolution: Where We Need to Go in Implementation Research.” International Journal of Public Administration. 27(8&9): 701-718.
Frederickson, George H., Gary Alan Johnson, and Curtis H. Wood. 2004. The Changing Structure of American Cities: A Study of the Diffusion of Innovation. Public Administration Review 64(3): 320-30.
Hatry, Harry P. 2002. Performance Measurement: Fashions and Fallacies. Public Performance and Management Review 25(4): 353-358.
Hatry, Harry P., J. R. Fountain, J. M. Sullivan, L. Kremer. 1990. Service Efforts and Accomplishments Reporting: Its Time has Come. Norwalk, CT.: Governmental Accounting Standards Board.
Ho, Alfred Tat-Kei. 2005. Accounting for the Value of Performance Measurement from the Perspective of Midwestern Mayors. Journal of Public Administration Research and Theory 16: 217-237.
Keene, James; John Nalbandian, Robert O’Neill, Shannon Portillo, & James Svara. 2007. “How Professionals Can Add Value to Their Communities and Organizations,” Public Management 89, no. 2: 32-39.
Melkers, Julia and Katherine Willoughby. 2005. Models of Performance-Measurement Use in Local Governments: Understanding Budgeting, Communication, and Lasting Effects. Public Administration Review 65(2): 180-190.
Poister, Theodore H. and Gregory Streib. 1999. Performance Measurement in Municipal Government: Assessing the State of the Practice. Public Administration Review 59(4): 325-35.
Poister, Theodore H. and Gregory Streib. 2005. Elements of Strategic Planning and Management in Municipal Government: Status after Two Decades. Public Administration Review 65(1): 45-56.
Rivenbark, William C. and Janet M. Kelly. 2003. Management Innovation in Smaller Municipal Government. State and Local Government Review. 35, 3: 196-205.
Smith, Peter. 1995. On the Unintended Consequences of Publishing Performance Data in the Public Sector. International Journal of Public Administration. 18, 2/3: 277-310.
Streib, Gregory D. and Theodore H. Poister. 2002. The Use of Strategic Planning in Municipal Governments. In The Municipal Year Book, 2002, 18-25. Washington, DC: International City/County Management Association.
Wechsler, Barton and Bruce Clary. 2000. Implementing Performance Government: A Symposium Introduction. Public Productivity and Management Review 23(3): 264-66.
Wholey, Joseph S. 1999. Performance-Based Management: Responding to the Challenges. Public Productivity and Management Review 22(4): 288-307.
Yang, Kaifeng and Marc Holzer. 2006. The Performance-Trust Link: Implications for Performance Measurement. Public Administration Review 66(1): 114-126.