results based mgt

Upload: belmiro-merito-nhamithambo

Post on 07-Apr-2018

219 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/4/2019 Results Based Mgt

    1/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 1

    Promoting Results Based Management

    in the Public Sectors of Developing Countries

    Paper by Cedric Saldanha, Sr. Director for Governance and Regional Cooperation,

    Asian Development Bank,

    Presented at the Roundtable on Better Measuring, Monitoring, and Managing for Results5-6 June 2002, World Bank, Washington, D.C.

    1. Why Results Based Management?

    While traditionally the private sector is seen as the driver of economic development, it is the

    public sector that enables this to happen. A results-focused public sector is keenly aware of its role in

    delivering critical outputs and outcomes such as the management and regulation of policy, the

    establishment and enforcement of legal frameworks, the provision of essential public services, the

    catalyzing of private investment. Public sector management capacity has therefore become a central

    indicator of a countrys development prowess. (Edralin 1997).

    Public sector organizations are daily coming under ever-greater scrutiny. In developed countries,

    public demand has increased transparency of operations, greater accountability for the dollar spent, and

    better services. Politicians realize that the most effective strategy to re-election is the delivery of tangible

    results to the electorate. In developing countries, besides the politicians and the public, donors also are

    becoming more demanding on good governance frameworks, and public sector performance against

    benchmarked results as the price for continued development investment support. (Schiavo-Campo and

    Sundaram 2000). Private sector investors, particularly consequent to the Asian financial crisis of 1997,

    want tangible evidence of institutional and public sector management capacity as a precondition for

    investment in a country. Good governance checklists and ratings are now common and integrated into

    financial market risk rating systems of developing countries. (Asian Development Bank, 1999).

    Despite these pressures, it is not easy for public sectors in developing countries to quickly andeasily evolve into results-based institutions and organizations. Some typical factors, which constrain

    public sector performance in developing countries and the effective utilization of aid, are as follows:

    Policy frameworks, which influence the scope and nature of public sector results, are not easy to

    change.

    There is inadequate political discussion on strategic results for the state and how these should

    translate into ministry and departmental objectives and budget allocations.

    There is often a lack of clarity on the role of public sector organizations, exacerbated by the

    historical multiplication of such organizations with overlapping mandates. Organization mission

    statements are rarely available, and institutional objectives are often muddled, and in a confusing

    sequence of priority.

    Inadequate systems exist for performance specification, measurement and reporting. Outputs arenot adequately specified, monitored and reported on.

    Systems of public accountability are weak or non-existent, with an emphasis on input and activity

    management as against output and outcome management.

  • 8/4/2019 Results Based Mgt

    2/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 2

    Often, there is a lack of a work ethic and an institutional value system which focuses on service

    and results.

    Human resources selection, compensation, and career management systems do not reward for

    merit or encourage a focus on results and productivity.

    This paper examines how these constraints can be addressed through a results based management

    system. It also discusses issues related to the implementation of such a system in a developing country

    context. It cites examples of such applications in developing countries and lessons to be learned from

    these.

    2. Key Elements of RBM

    A results-based management system is, as the title suggests, an approach to public sector

    management with a focus on performance results. As compared to private sector for-profit organizations,

    this focus on performance results in public sector organizations continues to be a challenge. Private sector

    firms have clear and central profit making objectives, and relatively straightforward ancillary objectives,

    including those related to employees and society. The public sector organization, on the other hand, has to

    cope with deep complexity in objectives, widely differing stakeholder perspectives, and much less control

    over policy, strategy and incentive structures (Park 1997). The market and clients they service are alsodifferent. One is competitive, the other not inherently so. Human resource management systems also

    differ widely, with the public sector subject to more controls through civil service regulations. Public

    sector organizations having much less autonomy, particularly on financing, and internal structures are

    usually more hierarchical and rule bound.

    Proponents of RBM view a results based focus, measurement, evaluation and accountability as

    tools for achieving more efficient and effective resource utilization by the public sector. They see RBM asa mechanism for forcing politicians, bureaucrats and voting citizens to be clearer about the objectives of

    government programs and services. In contrast, opponents fear the consequences of performance

    measures that do not adequately reflect the complex intended purposes of public sector programs.

    (Nakamura et al 2000).

    Performance measurement enables a public sector organization to monitor its effectiveness in

    achieving its goals and objectives, managing its services, and ensuring customer/client satisfaction.

    Performance measurement systems have numerous uses: they help set and communicate priorities,

    formulate budgets, allocate resources, motivate employees, improve services, and facilitate information

    exchange between citizens and government. (Cook, Vansant, Stewart and Adrian 1995).

    In the UK, recent research which evaluated performance in 74 public sector organizations

    (Lawton et al 2000) indicates that performance measurement and reporting has often been imposed on

    these public organizations by external stakeholders. The Citizens Charter of the UK (OECD 1997) and

    the formal performance agreements between ministers and the chief executives of New Zealand publicsector organizations (Boston et al 1991) are examples of RBM and a performance culture spreading

    through the public sector in the developed world. In the United States, performance measurement ofpublic sector organizations has been enshrined in law in the Government Performance and Results Act

    of 1993 (Kravchuk and Schack 1996) as a basis for driving improvement efforts. In Australia, politicians

    who needed to demonstrate performance to their electorates could not wait for their public service topropose performance measurements. Instead, one reformist state government insisted on departments

  • 8/4/2019 Results Based Mgt

    3/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 3

    being judged in terms of outputs while at a community level, they also imposed a set of indicators on

    local governments that included citizen satisfaction ratings.

    2.1 Clarifies Customers and Mandate of Organization

    Public sector organizations sometimes lose sight of their purpose. They sometimes even forget theyexist to serve the public. RBM ensures that the client focus remains the key guiding force of operations.

    The clarification of customers and their relative hierarchy of priority is critical to public sector

    organizations, and particularly for those which do not have an evident service function. Central agencies

    such as ministries of finance and planning provide core control and support functions to the rest of

    government. They therefore serve a wide variety of clients the executive, parliament, service departments

    and agencies, the public. Clarification of mandates allows the agency to establish priorities and relationships

    among its clientele.

    Instituting a results based management system begins with an analysis and specification of the

    mandate of the organization, its customers, and the benefits and impacts it is expected to deliver for them.

    To assist this process the public sector organization undertakes some rather basic questioning. Key questions

    asked are:

    - Why does this organization exist?

    - What would be lost if it did not exist?

    - Who does it serve?

    - What is it supposed to deliver for them?

    In management terms, this questioning generally results in a succinct mission statement for the

    organization, and the identification of key result areas (KRAs). These are the operating spheres within

    which the organization will deliver concrete outputs and outcomes.

    2.2 Specifies Results and Performance Expectations of Clients

    RBM requires that once KRAs are established, these must be translated into targeted and

    benchmarked results. Not infrequently, it is difficult to obtain a clear and consolidated statement ofperformance results for a particular public sector organization. (Polidano 2000). On being asked for one,

    invariably a statement of intentions or activities or budgetary expenditures is produced to demonstrate the

    scope and extent of the institutions operations. Public sector organizations are notorious for their budget-

    focus and the supply-driven emphasis of their operations. This makes it extremely difficult to assess

    organizational performance, and introduce actions for performance improvement.

    2.2.1. Output-Outcome Indicators

    The school of new public management, born in the UK in the early 80s and then widely adopted inNew Zealand and Australia, first introduced in a systematic and cohesive manner, the application of theconcepts of outputs and outcomes to the public sector as an integral part of RBM. For the first time,

    public sector organizations were being held accountable for the delivery of tangible and measurable results,

    and budget approvals became contingent on costing outputs and justifying budgets on output delivery.

    (Boston et al 1996).

  • 8/4/2019 Results Based Mgt

    4/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 4

    The concepts of outputs and outcomes were defined in legislation in New Zealand since the

    distinction between them was made critical to financial appropriation and accountability. Outputs were

    defined as the goods and services produced by departments or third party suppliers. Outcomes were the

    consequences for the public resulting from the outputs and activities of government. The purpose ofgovernment is of course to produce desired outcomes. (Scott 1996).

    The linchpin of the UK, New Zealand and Australian public accountability regimes is outputs,

    which is a strong common interest of both ministers and managers. Ministers purchase outputs, while

    departments are paid a negotiated price (their budgets) to deliver those outputs. However, thisaccountability arrangement is most effective when the market for the services concerned is contestable

    and buyers and sellers can establish an arms-length relationship. This is not the case for the majority of

    government outputs, where for various reasons, government remains the sole supplier. However, the

    creation of output contracts for financial appropriation and performance monitoring has created better

    information on what the public service does than existed before. Previously, only financial informationwas subjected to strict standards of verification. (Matheson 1995). There are however, significant

    transaction costs incurred in maintaining an accountability arrangement based on detailed ex ante

    specifications. (Schick 1996)

    While outputs are important to plan for and monitor, they cannot, by definition, be used as thesole criteria for judging organizational effectiveness. An organization's performance must be judged

    based on "why" it was established. It must be evaluated for the outcomes or impacts (client-benefit and

    satisfaction) it achieves. An education agency should be evaluated not just by how many schools it builds,

    but by the resulting increase in literacy and school graduates; a health agency, not simply by the number

    of clinics built, but by improvements in key health indicators such as morbidity and mortality rates.(Saldanha and Whittle 1998). These are the raison dtre of the organization and government. They are

    the benefits the organization is expected to provide to its stakeholders. And if these impacts are not being

    achieved, there is no clearer sign that the organization concerned has become ineffective or irrelevant.

    A much debated issue is the extent of influence an organization can claim on impacts. (Schick

    1996) the issue of causation and attribution. After all, the health status of people is contingent on factorsthat go beyond the control and influence of a health service agency. Should then the agency be

    accountable for a performance indicator over which it does not have full control? In New Zealand, the

    government has taken the position that achievement of impacts are beyond the accountability of public

    sector delivery agencies; these remain accountable only for the delivery of outputs. The concernedminister, however, as a representative of the government in power, takes accountability (vis--vis the

    public) for the delivery of impacts/outcomes. (Matheson 1995). It is however arguable, whether the

    service agency should not also take some degree of accountability for impacts. While it is true that the

    health service agency does not have complete control over all of the external variables influencing public

    health, it is accountable however, for noting these other factors, attempting to influence them, and takingaccount of them in the delivery of its service

    The application of the concepts of outputs and outcomes has become widespread in developingcountries over the last few years, assisted by donor efforts to introduce RBM. Their effective and

    productive use however remains questionable. RBM systems using such performance indicators havebeen generally introduced with the assistance of developed country consultants more anxious to

    demonstrate to donors the potential of the system to be adapted to a developing country context, than to

    ensure the tailoring of the system to the skills and capacities of the client country.

  • 8/4/2019 Results Based Mgt

    5/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 5

    Figure 1: Typical Sectoral Outputs and 0utcomes in a Developing Country Context

    OUTPUTS OUTCOMES

    2.2.2. Efficiency Effectiveness Indicators

    In the USA, besides the use of descriptive, financial and input-output-outcome indicators, the

    Governmental Accounting Standards Board (GASB) issued, in 1992, guidelines of the measurement and

    reporting of service efforts and accomplishments (SEA) which cover measures of efficiency andeffectiveness as well. (James and Roob 1994).

    Efficiency indicators reflect the level of outputs (the actual products/services delivered) of the

    organization in the context of available resources. They therefore represent the ratio of inputs to

    Agriculture

    - area irrigated

    - irrig. systems operated &

    maintained

    - ag.extension services

    - research

    - farmer income

    - ag production/prodty

    - share of workforce

    - ag. Share of GDP

    Transport

    - roads constructed

    - roads operated &

    maintained

    - traffic management

    - driver education

    - health care services- clinics & hospitals

    - drug availability

    - health care education

    - education facilities

    - cirriculum: scope/quality

    - enrollments/graduations

    - teacher education/quality

    - installed capacity- distribution coverage

    - energy mix

    - revenues/surplus

    - road usage/density

    - journey times

    - safety

    - transport share of GDP

    - life expectancy- population control

    - health status

    - productivity

    - literacy

    - education level

    of workforce

    - labor absorption rate

    - productivity

    - capacity utilized- access to electricity

    - consumption

    - average cost to customer

    Health

    Education

    Energy

  • 8/4/2019 Results Based Mgt

    6/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 6

    outputs; for example, the extent of budget or staff resources required for every unit of electricity

    generated or every unit of health care service offered.

    Effectiveness indicators represent the expected impact of the organization on customers through its

    products and services. Thus, they are indicators which represent the ratio of inputs to outcomes such

    as the level of client coverage and client satisfaction. They can also represent the ratio of outputs to

    outcomes/impacts achieved; for example, the extent to which electricity generated is accessed andused to improve productivity and quality of life; or the extent to which improved or increased health

    services have improved the health of the target population.

    In addition to efficiency and effectiveness indicators of performance, the issue of equity and sustainability

    are also pertinent, and especially so in the context of developing countries where resources are usually

    severely constrained.

    Equity indicators are important for public sector organizations, since they reflect the distribution

    profile of outputs (public access to products and services) and related equity of impact; for example,

    the extent of rural versus urban areas which have access to electricity or to health services.

    Sustainability indicators reflect the ability of the organization to ensure self-sustaining and consistentperformance and, thus, the continuity of institutional effectiveness and efficiency; for example, the

    level of revenues and surplus generated by the power utility and therefore the financial capacity of the

    utility to sustain and expand service; or the ability of health care agencies to recover revenue from

    clients based on improvements in service, and thus depend less on government subsidy for continued

    operation.

    Box 1: Sample of Expected Results of a Power

    Utility(Outcomes / Outputs)

    KRA 1- Ability To Meet Demand (Effectiveness

    and Equity)- % of rural households covered

    - % of urban households covered

    - % of industrial requirements met

    - capacity utilization

    - matches between capacity and demand

    KRA 2 - Technical Performance (Efficiency)

    - installed capacity

    - capacity to maintain (breakdowns)

    - reliability (service interruptions)

    - response time to requests for new

    connections

    KRA 3 - Financial Performance (Efficiency and

    Sustainability)

    - revenues

    - MWh sales per employee

    - operating income/net income

    - rate of return on fixed assets

  • 8/4/2019 Results Based Mgt

    7/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 7

    The use of such performance measures indicates an increasing sophistication in the approach of

    government to making public agencies more accountable. However, their use assumes a high level of

    organization capacity with regard to management skills and the existence of suitable accountability

    mechanisms and incentives to support performance. Unless the latter are in place, such indicators of

    performance will remain difficult to use on a day to day basis, and will have little impact on performance.This has been demonstrated by Knapman and Saldanha (1999) in their study of the public sector reforms

    being implemented among Pacific countries with the assistance of the ADB and the World Bank.

    2.2.3 Client Satisfaction Surveys

    Members of the International City Management Association (ICMA) of the USA have recognized

    that cities, towns, counties and villages are government entities that have the greatest interface with

    people through the basic services they provide. Local government monitoring of people satisfaction with

    such services is not only a challenge, but increasingly a necessity. (Kopcynski and Lombardo, 1999).Thus, ICMA has developed the Comparative Performance Measurement Consortium which not only

    works on developing performance measures for such services, but also sharing information on results.

    A similar and increasingly popular approach to making public sector organizations moreaccountable for performance in developing countries is the introduction of the report card on publicservices. This has been attempted, for instance, in India and the Philippines. (Filipino Report Card on Pro-

    Poor Services, 2001). In the cities of Bangalore (Paul 1995), and Madras (Anjana 1996) in India, periodic

    surveys on customer satisfaction with public services began in the mid 1990s through the initiative of the

    Public Affairs Centers in these cities. This initiative has been expanded to cover five cities all over India.

    (Paul and Sekhar 1997). The assessment of public service performance covered essential services such astelephones, electricity, water, health services, postal services, bus transport, public sector banks and the

    police. Public satisfaction with these services was assessed using sample surveys. The dimensions of

    performance assessed included: staff helpfulness, quality and timeliness of service, and time taken to

    resolve problems. The report cards were not designed to provide an in depth assessment of what was

    wrong with public service agencies. They were intended, however, to put these agencies on call in terms

    of their accountability for better performance and service to the public.

    Box 2: Bangalore Garbage Disposal Program

    In 2000, the state government set up the Bangalore Agenda Task Force. It was established as a public/privatepartnership with the private sector bringing in capital, knowledge, materials and additional training capacity. It wasalso set up to be a kind of generator of ideas for the development of the city. The task force conducted a survey of

    citizens to establish what their major concerns were in a number of areas including telephones, security, water, andthe work of the Municipal Corporation. Garbage removal was apparently the most pressing concern of citizens.

    A telephone handling complaints center was established. The telephone number was widely publicized and people

    were asked to telephone in if their house was not visited on any particular day for garbage removal. Identifiedcitizens were nominated as on each street to inform the center if the garbage truck or the collectors did not come. A

    private/public task force was set up to supervise the program. It agreed that each garbage inspector would beprovided with a two way radio and the complaints center would inform him of a problem if any, and he would attend

    to it immediately. Inspectors were given uniforms which they wear proudly; it and the two way radio seemed tobecome a badge of office. Latterly, the inspectors have been given loans to purchase scooters which could also beused for personal work. An annual award is given to the most efficient inspector a gold watch with the corporation

    crest on it. The popular garbage disposal slogan is one time, on time, each

  • 8/4/2019 Results Based Mgt

    8/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 8

    An important caution from Hatry et al (1994) is what is measured and reported gets attention!

    Hence, misuse and counterproductive use can occur if the performance indicators are not chosen and

    implemented with care. For example, solely focusing on the time it takes to provide a service is likely to

    cause employees to push for speed of service delivery at the expense of the quality of service.

    2.2.4 Balanced Scorecard

    In the 1980s, the multiple-constituency approach was widely espoused for measuring

    organizational performance particularly for not-for-profit-organizations. (A S. Tsui 1990; Connolly et al1980). The approach suggested that the interests of all stakeholders in an organization must betaken account of in assessing its effectiveness. This approach contributed to organization management

    literature by broadening measure of organizational performance and was in many ways the predecessor of

    the currently popular Balanced Scorecard approach.

    The Balanced Scorecard is a comprehensive approach to measuring performance and building

    capacity in both the private and public sector. First proposed in 1992 (Kaplan and Norton 1992), it

    proposes the use of a holistic group of measures, not just to assess performance, but to drive it. It

    recognizes the power of concrete goals on performance, and seeks to enlarge organization goals to coverareas critical to sustained organization performance. It underscores the issue that an organization responds

    essentially to the goals it focuses on. If these are short term, the organizations bias will be similar.

    The balanced scorecard supplements traditional private sectorfinancial measures with criteria that

    measure organization performance from three additional perspectives those of customers, internal

    business processes, and learning and growth. It therefore enables companies to track financial results while

    simultaneously monitoring progress in building the capabilities and acquiring the intangible assets they

    would need for future growth. A predominant focus on financial results motivates short-term behavior often

    at the expense of long term behavior and benefits. Thus, companies need to broaden their performance focus

    and benchmarks from short-term measures such as returns on investment and revenue growth, to longer

    term ones like market share, customer relations and satisfaction, revenues from new products and employee

    satisfaction and growth. These latter measures ensure the more holistic and longer term development of theorganization.

    There has not as yet been any widespread application of the balance scorecard to public sector

    organizations. However, as focus on output and outcome performance in the public sector matures, quite

    naturally managers begin broadening out performance measures to ensure sustainability of efficient and

    effective performance of the public sector organization. The use of the balanced scorecard then becomes

    pertinent.

    2.3 Links Budget allocation to Output Delivery

    New public management promotes a direct link between results based public sector management

    and the budgetary process. Agency budgets are assessed and allocated based on the cost of deliveringoutputs, a process known as output-based budgeting. The process would usually include performance

    agreements between a central coordinating department and the recipient department. These agreements

    record expected performance (outputs) for the budget allocated. This budgeting system quite rightly

    assumes that budgets cannot be realistically based on the delivery of outcomes. These are often medium

  • 8/4/2019 Results Based Mgt

    9/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 9

    term objectives; they are influenced by a number of variables, some not within the control of the agency

    concerned; and their monitoring is a very complex task.

    Output based budgeting is a logical consequence of the application of results based management.

    However, it has generally been a failure in its application in developing countries for some very goodreasons. Output based budgeting assumes an operational and financial management infrastructure of a fair

    degree of sophistication and this is often not feasible in a developing country context. It requires accrualaccounting systems often unrealistic when governments still struggle with maintaining line budgeting

    systems. The degree of skills required for establishing outputs, costing them, verifying the

    realism/accuracy of the costing, monitoring and reporting implementation, assessing the need to adjustbudgets based on past performance etc are simply often not available in developing countries. If available,

    they have often already been captured by the private sector who can afford to pay more attractive salaries.

    In the context of the above, it is not recommended to intrinsically link results based management

    with output based budgeting. An adequately effective results based management system with aperformance-related budget can be put in place using the conventional line budgeting system.

    Budgets can still be linked to the delivery of a program of outputs, and should be justified on this

    basis. Selective input-output ratios are feasible for certain services and can appropriately become the basis

    for controlling inputs.

    2.4 Requires Performance Reporting

    Effective and results-focused public sector organizations typically adopt a performance reporting

    system as part of larger public accountability responsibilities. In countries which have adopted the outputbased budgeting system, reporting on achievements of outputs becomes an essential basis for appropriating

    the next years budget. Periodic performance reporting provides the essential data on which performance

    diagnosis is based. The performance reportessentially provides period information on organizational

    performance against a set of performance indicators which represent outputs delivered by the organization

    and the outcomes/impacts these have achieved. Outputs can and should be reported annually.

    Outcomes/impacts sometimes can only be assessed in a longer time frame, perhaps of up to three years.Notwithstanding this, it is essential that outcomes are monitored and reported.

    Organizational performance measured in terms of outputs and outcomes is normally assessed using

    threeparameters:

    (1) targets that have been set by management or stakeholders,

    (2) past performance, using a trend analysis to review whether performance is on an upward trend

    or otherwise, and

    (3) comparators or industry benchmarks (in the public sector, comparators could be similarinstitutions in comparable countries).

    Often, targets set by management for a specific performance period already take account of pastperformance and comparator performance levels. This comparative performance analysis is essential to

    assess the quality of organizational performance and identify related variances, either positive or negative,from expected or required levels of performance. The variances or gaps between present and desired levels

    of performance thus provide the basis and a trigger for initiating performance diagnostic analysis.

  • 8/4/2019 Results Based Mgt

    10/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 10

    A thorough performance report and its analysis is of course contingent on performance information.

    Many public sector institutions are known to be particularly deficient in having information readily available

    on performance, more specifically on outputs and impacts. This is a reflection of public sector institutions

    being typically budget and supply driven rather than output and impact focused. The introduction of RBM

    must be accompanied with the parallel establishment of a management information system to provide timelyfeedback on output performance.

    2.5 Promotes Performance Analysis and Continuous Improvement

    Another integral element of RBM is performance analysis and continuous improvement.

    Performance reporting leads to the identification of performance problems (negative variances) or

    performance improvement opportunities (potentially positive variances). The greater the clarity and

    preciseness with which these are defined, the easier will be the subsequent diagnostic processes which

    isolate causal factors and consequently lead to institutional strengthening actions. (See Box 3). If we are toimprove performance, this is the necessary starting point.

    Identifying performance variances in tangible, verifiable and timely terms is contingent on the

    quality of the performance indicators used. This is a particular challenge when dealing with those publicsector organizations where the outputs cannot be quantified. An example would be a policy making andregulation body. While quantification is in such cases not possible, documenting outputs in tangible and

    monitorable terms is usually feasible.

    Performance variance or improvement identification also assume assumes that past and

    comparator performance information is accessible for reference.

    Box 3. Examples of Performance Variances

    in Developing Country Public Sector Organizations

    Investment applications have fallen over the last 12 months compared

    to a neighboring country with similar socio-economic characteristics

    Production and exports of specific crops have remained stagnant over

    the last 24 months despite consistent weather patterns

    Quality of road maintenance has fallen over the last year as evidenced

    by increased potholes and related traffic congestion

    The female entrants for primary education remains at 30% of male

    entrants despite a specific education program for parents in this regard

    over the last two years

    Shortage of critical drugs in medical clinics in specified rural areas City cleanliness continues to be below specified standards with regard

    to littering and garbage clearance

    Population growth rates continue to be above 2%

  • 8/4/2019 Results Based Mgt

    11/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 11

    Performance analysis essentially seeks causes in order to rectify and improve performance.

    Causes generally fall within two broad categories internal and external. The temptation is to first seek

    for external causes so blame can be assigned to factors beyond the organization. However, numerous

    internal factors also do influence performance resources, systems, structure, strategy, policy, culture,

    and leadership. This performance analysis process must consciously assess the influence of these internalvariables first thus focusing on causes within the control of the organization.

    The performance analysis process is in some ways a difficult process since it deals with non-

    quantitative relationships, subject to varying interpretations. The search is for root causes, and therefore

    the process requires to keep asking why as one goes deeper into cascading levels of causes.

    For instance, a pubic works agency is analyzing the problem of traffic management and its

    primary contributor viz continuing potholes in the road (the performance variance). A first level cause is

    identified as inadequate staffing of maintenance gangs. This in turn is caused by inadequate budgetary

    outlays for maintenance (resources), which is due to the priorities of the government in the allocation of

    resources (strategy). If the resource issue is therefore a given, the analysis asks - how can constrained

    resources be used more effectively to at least minimize the problem (systems and structure)? Is the

    information system on the status of various roads working effectively to allow for prioritization of road

    maintenance programs? Are first line supervisors being held accountable for best use of availableresources? Is there an incentive in place to motivate them to better performance? This process of

    questioning and searching for root causes must continue till key causative variables are identified.

    Figure 2

    A Simplified Cause-Effect Analysis of the Public Works Agency in a Developing Country

    Performance Variance:Road maintenance deteriorating

    Shortage of staffdue to PublicServiceretrenchment

    (Resource Issue)

    Information onroad conditionsnot received asper schedule

    (System Issue)

    Accountability forroad conditions notclarified andenforced

    Quality of firstline supervisionhas deterioratedwith staff

    attrition

    Govt. policy

    on which

    services getpriority(Strategy/Pol

    icy Issue)

    Inability of

    Mgt. to

    persuadeFinance onbudget

    allocation

    Client

    Orientation notemphasized byManagement

    (Value Issue)

    Salary scales

    unattractive to

    qualifiedengineers(HR System

    Issue)

  • 8/4/2019 Results Based Mgt

    12/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 12

    It is sometimes difficult for an outsider, without an understanding of the inner dynamics of the

    organization, to undertake this causal analysis in any useful degree of depth and accuracy. On the other

    hand, process consultants do have special skills to assist managers and staff within the organization move

    systematically through this causal analysis. The most practical and effective approach appears to be

    group-based cause-effect tree analysis. This implies establishing the variance and then seeking causesand causes of causes, in a cascading process as demonstrated below.

    It is important to identify which category variable is influencing the performance variance

    because the appropriate corrective or improvement action will differ depending on the variable. For

    instance, addressing a system or process inefficiency will require a very different approach to thatrequired to address an issue related to leadership and values of the organization.

    2.6 Assumes Meritocracy in Managing HR

    RBM can only work if staff selection, compensation and career management are professionally

    managed and based on merit. Like budgets, human resources are a key resource to the delivery of results.

    A department head cannot be held accountable for results if she/he is not given discretion on the quality

    of staff provided to the department. Accountability for results demands that such resources be of thehighest possible quality and with a repertoire of competences responsive to the results expected. Thus,

    selection must be professionally managed, open and competitive, eschewing political pressures and

    influence. Promotions and career management must be guided by HR best practice, and staff must feel

    that principles of fair competition are transparently applied.

    Compensation in the public sector is often not able to be competitive with the private sector.

    However, it is not necessary that it be competitive to attract the talent and skills needed by RBM. Often

    professionals join the public service for reasons other than financial reward. It remains essential though,

    that a fair wage is provided based on comparative salary studies of the local market. Performance pay and

    bonuses are far more difficult to apply and manage than in the private sector and are usually avoided even

    in developed countries, except in situations where the organization, though within the public sector,

    retains financial independence in terms of revenue, costs and surpluses. Issues such as complexity ofresults to be delivered, and difficulties in quantification, monitorability, and attribution usually make

    performance pay impractical in the public sector.

    Professional HR management has been one of the major constraints to a results-oriented public

    sector in developing countries. It is not uncommon to encounter a public sector with personnel

    appointments substantially influenced by political forces, or departments with bloated and underpaid staff

    complements because regulations or absence of political will do not allow for related rationalization.

    However, there is no easy way out. Professionally managed human resources is a sine qua non for

    effective RBM.

    3. Triggers for RBM

    Ideally, RBM can most effectively be introduced when it is part of a larger public sector reform

    program, initiated by the government for a number of possible reasons. In such a situation, RBM is

    usually one of many reform initiatives in a reform program that usually addresses the role of government

    and the public sector in a multi-faceted fashion. Thus, the package may include economic policy reforms,

    the privatization or contracting out of certain services, reforms in the management of the budget and

  • 8/4/2019 Results Based Mgt

    13/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 13

    human resources, and RBM for a variety of public sector agencies or perhaps across the whole public

    sector. In this scenario, the introduction of RBM is made easier given the support of the highest levels

    within the government (it becomes a mandatory exercise with no option given to the concerned public

    sector agencies) , and the reinforcement provided by parallel reforms.

    However, the above is not a typical or common situation. Usually, government or donors are

    called upon to persuade rather than coerce the concerned public sector agency to adopt RBM. Selling theconcept of RBM to public sector agencies in developing countries is not an easy task unless the concerned

    agency is led by a reformist management. RBM calls for greater accountability on the part of the agencies

    for concrete results. It makes their budgets subject to performance evaluation. It often requires hardhuman resource decisions to ensure required levels of productivity. Later in this paper are described

    typical incentives available to promote the use of RBM to such agencies. At this point however, it is

    useful to note that there are three often-encountered entry points for donors, who seek to catalyze the

    adoption of RBM by public sector agencies in developing countries.

    3.1 Capacity BuildingDonors are often approached by public sector agencies of developing countries for capacity

    building assistance. Every year millions of dollars are invested by donors in this type of assistance.Unfortunately, much of this investment goes to waste, spent on interventions that have limited sustained

    impact. Interventions are usually not anchored in any disciplined institutional diagnostic analysis. The

    typical capacity building intervention is training. Skills are imparted but often without parallel

    strengthening of the institutional environment within which they can be put to productive use. It is

    therefore necessary for donors to move to an institutional and more holistic approach to capacity buildingto ensure sustained results. One such approach is RBM.

    RBM focuses on results, which must also be the focus and guiding parameter for any capacity

    building exercise. Thus, when requested for capacity building assistance, it is wise for donors, as a first

    step, to require the requesting agency to begin with a specification current performance shortfalls,

    described in terms of outputs and outcomes. The identified performance variance is then used to workdiagnostically to the identification of institutional causes which can then be addressed through capacity

    building. The success of the capacity building investment will be tangible and monitored improvement in

    the identified performance shortfall, viz outputs and outcomes.

    The above process becomes a staring point to focus the public sector agency on tangible and

    monitorable results. When aid dollars are used to demonstrate tangible improvement in public sector

    performance, it becomes a powerful motivator for both donor and recipient to program future aid for

    capacity building along similar processes, thus gradually promoting the move to results based

    management. Donors should therefore use the opportunity of request for capacity building to graduallymove the agency to adopting a full fledged RBM system.

    3.2 RBM Supports the Sector Wide Approach

    The sector wide approach (SWAp) is another potential entry point for the application of RBM in

    developing countries. The SWAp approach to the delivery of public services assumes that the central

    service agency, be it the ministry of health, education or transport, assumes lead responsibilities for the

  • 8/4/2019 Results Based Mgt

    14/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 14

    delivery of specified outcomes for the sector over a medium term period (5 to 10 years). The focus is very

    much on outcomes. A SWAp discourages the project-based approach of some donors under the principle

    that all project investment should contribute to key sectoral outcomes. Therefore, it is logical to begin

    with medium term outcomes and then work back to developing a program of outputs, reforms,

    investments and recurrent activity which will ensure outcome achievement. In this connection,performance analyses such as described above, are undertaken to identify what policy, institutional and

    financial constraints hinder the achievement of eventual outcomes. The sector program of investment andreform is built around addressing these constraints. Hence the term programmatic approach. The sector

    program is led by the lead sectoral agency, and supported by colleague agencies and donors. The lead

    agency takes accountability for achievement of stated outcomes.

    Figure 3. Elements of a Sector Wide Approach

    SWAps assume that a results based management system is in place for the concerned sector

    agency. In fact a SWAp is not feasible unless: (i) clear and strategic outcomes and outputs have been

    identified by the sectoral agency, (ii) there is in place an information system which monitors and reportson the achievements of these, and (iii) there is continuing analysis of the variables affecting the

    achievements of related targets.

    Policy &Legal

    Framework

    Sector Inputs

    Institutional

    Capacity

    Sector

    Outputs

    Sector

    Impacts

    - Economic

    Growth andPovertyReductionIndicators

    - HumanDevelopmentIndicators

    - EnvironmentIndicators

  • 8/4/2019 Results Based Mgt

    15/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 15

    Box 4. The Cambodian Priority Action Program

    Ministry of Education

    Key Sectoral Outcome: In furtherance of its contribution to poverty reduction the Ministry of Education

    established in 2000 the overall policy objective of achieving universal education by the year 2010

    Constraints Identified through Problem Analysis:

    There was a need to reduce the educational cost burden on poor households so as to increase

    participation of their children in basic education;

    Delivery of education to the primary level was constrained by high levels of repetition (on

    average, about 40% of students repeated grades I and II). It was estimated that the annual cost

    of repetition was Riels15-20 billion in the education budget;

    Teachers were underpaid and many supplemented their salaries by demanding additional

    payments from students and by absenting themselves from teaching to take on second jobs;

    Program Actions Developed:

    Provide a discretionary budget for each school consisting of a flat amount topped up by a pro

    rata amount for each student registered. The money was to be used for operations and

    maintenance. The requirement for parents to pay a registration fee for each student was

    discontinued and School Support Committees were expected to ensure that such payments were

    not demanded at the start of each school year.;

    Deliver remedial lessons during the long vacation for all grades I and II students who did not

    meet the required standards during the school year. The lessons would be for half a day for two

    months of the vacation. The program was extended to years I-VI in 2001;

    Pay teachers additionally to provide the remedial classes;

    Outcomes /Impacts: In the first year the program, despite some hiccoughs, had the following outcomes:

    Not all schools were able to implement the requirement for parents not to pay the registration

    fee, partly as a result of tardy disbursements on the part of the Ministry of Finance. The

    reported Provincial Treasury requirement that, in some provinces, it be paid up to a 10%

    commission for funds released, also reduced the amount of funds received by each participating

    school;

    An additional 86,000 students gained access to the next grade. In the pilot provinces, promotion

    rates from Grade I to Grade II increased from 54% in 1999 to 82% in 2000 and from Grade II to

    Grade III from 67% to 83% respectively, approximately halving the rate of repetition and,

    consequently, saving the education budget in the region of Riels 10 billion in costs;

    4,500 teachers in 10 provinces were trained by UNICEF and the NGO KAPE in remedial

    teaching and gained additional wages for 120 hours of teaching;

    The role of the School Support Committee as an active representative of the community in

    school affairs was reinforced by giving the Committees something positive to do by ensuring

    that payments were not demanded by schools for pupils to register;

    An IEC program supporting PAP resulted in improved parent support for schooling which

    produced an increase in Grade I entries in 2000 of about 41%;

    64 District Budget Management Centres had their first experience of managing priorities

    between schools and disbursing funds which was increased to 183 in 2001.

  • 8/4/2019 Results Based Mgt

    16/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 16

    3.3 Decentralization

    Many developing countries are in the process of decentralization. Strong political motivations are

    usually behind this trend and often governments have little choice but to move in this direction. The

    decentralization process provides an excellent opportunity for developing country governments anddonors to promote and institutionalize RBM as part of this process. The objective of decentralization is to

    delegate and empower local agencies and governments with authorities and accountabilities for a varietyof purposes more responsiveness to local needs, relieving central fiscal pressures, giving a greater voice

    to local authorities in managing government.

    Decentralization provides an ideal opportunity to institutionalize RBM. Local government is

    usually required to substantially revamp itself in terms of mandate, focus, structure, systems, resources,

    and staffing in this process. Such a dramatic reconstitution of any organization is a perfect opportunity to

    ask all the basic questions that RBM requires such as what is the fundamental purpose of this

    organization? Who are its clients? What are the results it should deliver to them? How can it be beststructured and managed to do so?

    RBM is in fact the only reasonable and practical way to approach organization restructuring as

    part of a decentralization process. It ensures that the expectations and objectives of the decentralizationprocess have indeed some probability of being achieved. It also ensures that all the relevant issues relatedto ensuring effective decentralization are put on the table and examined by decision makers in a

    responsible and comprehensive manner.

    4. Potential Coverage of RBM within Government

    Theoretically, RBM can and should be applied to every government department or agency. In New

    Zealand, RBM was applied equally to service delivery agencies such as health and education services, as

    to central and core agencies such as Treasury, the Public Service Commission and the Central Bank. For

    instance, as part of this application, the Reserve Bank Act of New Zealand provides for the government to

    enter into a contract with the Reserve Bank to maintain inflation within agreed parameters.

    In general, the application of RBM probably benefits support, advisory and regulatory agencies

    more than service delivery agencies, though it is more easily applied in the latter. The results of service

    delivery agencies are fairly obvious and are usually the focus of day-to-day activities and decisions(though they may not be regularly reported on). In the case of advisory, regulatory and support agencies,

    such results are not easily apparent. They need to be fleshed out through disciplined analysis, asking basic

    questions such as what is the purpose of this organization? Who are its clients and in what way does it

    serve them? If it disappears, what will be lost and who will suffer?

    In developing countries, public sector agencies that have most easily accepted and adopted RBM are

    municipalities and essential service agencies such as health and education. The common characteristic of

    these agencies is that they deliver critical services to the public, their performance on these services iseasily translated into monitorable performance indicators, and elections are often won or lost based on the

    quality of or improvements in their services.

    The fact that implementing RBM across whole of Government may be difficult does not preclude its

    application to specific agencies whose functions lend themselves more easily to such a system such as

  • 8/4/2019 Results Based Mgt

    17/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 17

    public works, transport, education or health. RBM can be applied across the public sector, or limited to

    selected agencies, or even to specific units within an agency. Sometimes, it is strategically wise to begin

    with small discrete organization units who become in turn, the beacons and torch bearers for the adoption

    of the system within the broader organization. The discipline of focusing on results, monitoring and

    reporting on performance, analyzing shortfalls and improving has a powerful positive influence throughthe organization, and makes work more meaningful and rewarding.

    5. Principles for Implementing RBM

    Implementing RBM is not an inherently expensive exercise. There are numerous examples, such

    as the Bangalore City Revival Program, where with very modest resources, but with strong high level

    commitment, the program was initiated and implemented.

    Sometimes, when donors enter the picture, they visualize the introduction of RBM as an end initself. Thus, it is projectized, substantial money is thrown into the exercise, enthusiastic consultants are

    given design and implementation responsibilities, conditionalities are imposed to milestone the

    implementation, and a detailed schedule is developed. However, to be successful the implementation of

    RBM must be a process, often using trail and error in a slow and gradual process of adjustment. Findingthe best possible fit between the system on the one hand, and the nature and constraints of the concernedorganization on the other, is a major challenge. Conditions are never ever perfect.

    5.1 Incentives are Essential

    Like any new system, implementation is contingent on the strength of incentives. Practicing RBM

    is a challenge, and particularly for public sector organizations that are traditionally activity and

    disbursement oriented. Taking on accountability for clearly defined outputs, and making budget

    allocations conditional on achievement of these can be threatening to say the least. The system therefore

    must be made attractive enough for the leadership to pursue it determinedly. What are some of the

    possible incentives to promote institutionalization?

    One option is to make contestable the services to be provided by the public agency. This is not

    always feasible since many public functions, particularly those related to policy making and regulation,

    are intrinsic to the role of government. However, it is often possible to make the delivery of publicservices contestable. When a service departments existence depends on competing with private sector

    suppliers, performance very soon improves dramatically. Such services can range from provision of

    agricultural extension services, to health and education, and even weather forecasting. In PNG, more than

    half the annual recurrent budget for health is allocated to NGOs to manage rural health clinics. And, at

    the encouragement of donors, this practice is now spreading to other services as well such as educationand agricultural extension.

    A second option is the use of the public report card. As indicated earlier, the experiments withpublic report cards in Bangalore, Madras and the Philippines have been hugely successful in enhancing

    the service performance of municipalities and central agencies. The issue of course is who takesaccountability for the financing and management of this audit process? In the above examples, it has been

    NGOs, supported the government. This is the preferred option since it ensures that the exercise remains

    cost effective and transparent.

  • 8/4/2019 Results Based Mgt

    18/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 18

    Political re-election always remains a fairly strong incentive for politically elected heads of

    agencies, such as municipalities, to push for more effective performance. Tangible improvements in the

    delivery of essential services are usually the most dependable strategy for re-election. However, while re-

    election is an incentive for the elected, it is not for the administration. Thus, it then falls on the electedhead of the municipality to find ways to motivate the administration to improve services.

    The most effective incentive has been linking annual budget allocations to the delivery of outputs.

    At one extreme, this is in the form of output based budgeting. However, one need not go all this way.

    Even simply requiring a disciplined public report on output achievement based on the budget provided tothe agency, is often incentive enough to make the administration feel accountable and therefore motivated

    enough to meet assigned targets.

    5.2 Internal Ownership and Commitment is Necessary

    RBM cannot be effectively implemented and institutionalized unless it is internally led. It needs

    the commitment and support of the highest level of management of the organization. While this may be

    saying the obvious, it is repeatedly apparent that donors do not always practice this principle. There havebeen numerous cases of SWAps where donors, unhappy with the performance of the sectoral agency,have themselves taken over and developed the sector program for the developing agency. Their

    motivation has usually been the need to commit funds to the program and this commitment is often

    contingent on the quality and acceptability of the program to their own home offices. Needless to say, the

    program has failed.

    RBM is more than a system or a program. It envisages a set of managerial attitudes and an

    organizational culture that give pre-eminence to results in the management of operations. This implies

    substantial changes in the nature of daily managerial transactions ranging from decision-making, financial

    management, and HR management. It implies a different attitude to the public and relations with major

    stakeholders. In essence, effective RBM means a whole new perspective to agency operations, and one

    which cannot be imposed from outside. External incentives can of course be created to encourage thischange in mind set. But finally, it is necessary for internal leadership to accept and lead the initiative.

    A related issue is the lack of attention paid by donors to management and absorptive capacity

    when deciding on supporting a sectoral or municipal program. It is typical that the merits and urgency ofthe intended outcomes and impacts become the overarching criterion for decision making on levels of

    donor aid to a particular program. While this is indeed important, program dimensions and funding must

    finally be to internal leadership capacity and commitment. If not, then program failure should come as no

    surprise.

    5.3 Keep it Simple

    This implementation principle parallels the principle of internal leadership. In effect it states thatRBM should be implemented by and within the capacities of the concerned organization. Too often

    donors contribute consultants who usually take over leadership of the exercise, and devote much effort totransplanting a system that has worked in their own developed country but is quite inappropriate in the

    conditions prevailing in the developing country. There is reluctance to compromise on the sophistication

    of the system even though local capacities very apparently cannot cope with it. The classic example is the

  • 8/4/2019 Results Based Mgt

    19/22

  • 8/4/2019 Results Based Mgt

    20/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 20

    The opportunity should be used to also introduce RBM as a system to support the sustained management

    and effectiveness of a SWAp.

    It would be helpful if success stories in the design and implementation of RBM in developing

    countries are documented and shared more widely. There is a wealth of experience in the application ofRBM. And while results have been mixed, the lessons of the success stories can have an important impact

    on encouraging wider application.

    Finally, there is no easy fix. The task of shifting to results based management is a long term

    proposition. It can only be addressed in gradualist fashion through carefully developed and integratedcapacity building initiatives that address the typical constraints and choose the right types of indicators.

    Donors have an important role to play in providing judicious though sustained support to get the desired

    outcome better use of the developing countrys limited resources for the benefits of its citizens.

  • 8/4/2019 Results Based Mgt

    21/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 21

    References

    Anjana, I, Report Card on Public Services in Madras, 1996.

    Boston J, Martin J, Pallot J, and Walsh P, Public Management: The New Zealand Mode, Oxford University Press, Auckland,New Zealand, 1996.

    Bower, J.L, The Two Faces of Management, Houghton Mifflin, Boston, 1993.

    Cook T. J, Vansant J, Stewart L, Adrian J, Performance measurement: lessons learned for development management, World

    Development 23(8): 1303-15. August 1995.

    Connolly T, Conlon E.J, and Deutsh S.J, Organizational Effectiveness: A Multiple Constituency Approach, Academy ofManagement Review, 1980, S, 211217.

    Garrity R.B, Total Quality Management: An Opportunity for High Performance in Federal Organizations , PublicAdministration Quarterly, Vol 16, No 4, Winter, 1993, pp. 430-459.

    Edralin J, New Local Governance and Capacity Building, Regional Development Studies, Vol. 3 1997.

    Hatry H, Gerhart C, Marshall M, Eleven Ways to Make Performance Measurement More Useful to Public Managers, PublicManagement, S15, September 1994.

    James Jr, Roob F, Service Efforts and Accomplishment Measures, Article 11, Public Management, March 1994.

    Kaplan R.S, and Norton D.P, Using the Balanced Scorecard as a Strategic Management System, Prepared for the World Bank,Washington D.C., 1996.

    Knapman B, Saldanha C, Reforms in the Pacific, Asian Development Bank, 1999.

    Kopcynski M, Lombardo M, Comparative Performance Measurement: Insights and Lessons Learned from a Consortium

    Effort, Public Administration Review, March/April 1999, p. 124.

    Kravchuk R. S, Schack R.W, Designing effective performance-measurement systems under the Government Performance and

    Results Act of 1993, Public Administration Review, July 1996, pp. 348-358.

    La Porta R, The quality of government, National Bureau of Economic Research Working Paper No. 6727. 1998.

    Lawton A, McKevitt D, Millar M, Coping with ambiguity: Reconciling external legitimacy and organizational implementationin performance measurement, Public Money & Management, July 2000, pp.13-19.

    Matheson A, Impact of Contracts on Public Management in New Zealand, Government Paper, 1995.

    Nakamura A, et. al., Public Sector Performance Measurement, Evaluation and Accountability, Department of HumanResources Development, A PEER Group Working Paper, Canada, 2000.

    OHara K, Measuring Outcomes in the Social Union, A CPRN Working Paper, 1997.

    OECD, Administration as Service, the Public as Client, Report of the Organizations Technical Cooperation Committee, 1987.

    OECD, Benchmarking, Evaluation and Strategic Management in the Public Sector, OECDs Public Management Service,

    Paris, 1996.

    Osborne D, Gaebler T, Reinventing Government, Reading, Mass.: Addison-Wesley, 1992, pp. 156176.

    Paul S, Making Voice Work: Some Lessons from Bangalore, 1995.

  • 8/4/2019 Results Based Mgt

    22/22

    C. Saldanha June 2002

    Promoting Results Based Management Page 22

    Paul S, Sekhar S, A Report Card on Public Services, Public Affairs Center, Bangalore, 1997.

    Park B, Total quality Management (TQM) Operation in Public Organizations, University of Nebraska, 1997.

    Polidano C, Measuring Public Sector Capacity, World Development, Vol.28, No.5 (2000), pp. 808-822.

    Ridley C, Simon H, Measuring Municipal Activities, International City Managers Association, Chicago, 1943.

    Saldanha C, Using the Logical Framework for Sector Analysis and Project Design, ADB, 1998.

    Scott G, Government Reform in New Zealand, IMF Paper, 1996.

    Schick A, The Spirit of Reform: Managing the New Zealand State Sector in a Time of Change, State Services Commission,New Zealand, 1996.

    Schiavo-Campo S, Sundaram P, To Serve and To Preserve: Improving Public Administration in the Competitive World, ADB,2000.

    Theurer J, Pitfalls to Avoid when Establishing Performance Measures, Public Management, July 1998, p. 21.

    Tsui A.S, A Multiple-Constituency Model of Effectiveness: An Empirical Examination at the HR Subunit Level, Administrative

    Science Quarterly, 458, September 1990.

    Tuck N, Zaleski G, Criteria for Developing Performance Measures in the Public Sector, International Journal of PublicAdministration, Vol. 19, 1996, pp. 1945-1978.

    U.S Congress, Performance Management, Benchmarking, and Reengineering within Government, (Washington: USGPO,1996)

    World Bank, Filipino Report card on Pro Poor Services, 2001.

    OECD, Benchmarking, Evaluation and Strategic Management in the Public Sector, 1997.