performance assessment – programmes and institutions 绩效评估 – 项目和机构 dieter...

Performance Assessment – Programmes and Institutions

绩效评估 – 项目和机构

Dieter Schimanke迪特施曼克

Professor, State Secretary (retired)教授，州务卿（已退休）

Hamburg/Germany汉堡 / 德国

Changchun/Jilin, December 5-6, 2012长春 / 吉林，

Case 1: Youth education

Emloyment policy:Indicator (definition) Youth education attainment level (total, females, males):‘The percentage of young people aged 20-24 years having attained at least upper secondary education attainment level as a proportion of the total population of the same age group‘.

Source: Eurostat Unit Labour Market

Changchun, December 2012 2

Case 2: Lisbon Strategy

• The Lisbon Strategy 2000 (updated 2005, 2008, 2010): From a complex system of general objectives to measurable targets

• New strategic goal: ‚to become the most competitive and dynamic knowledge-based economy in the world, capable of sustainable economic growth with more and better jobs and greater social cohesion.’


Case 2: Lisbon Strategy - Targets with indicators

The Lisbon Strategy sets specific targets:An overall employment rate of 70% by 2010An employment rate for women of over 60%An employment rate of 50% among older workersAnnual economic growth around 3%


I. What is Performance Assessment?绩效评估是什么？

(1) Definition 定义(2) Develpoments(3) Functions 功能(4) Areas 领域


(1) Definition of Performance Assessment 绩效评估的定义

Definition: Measuring the outputs and outcome of the activities of public institutions or public servants on the basis of a value-/quality-orientated system of criteria. 定义：在价值和 / 或质量标准体系基础上，对公共机构或公务员活动的产出和结果进行评测。

Performance Assessment as a concept.绩效评估被当作一个概念应用时

Performance Assessment as an agenda.绩效评估被当作一个议事日程应用时


Survey on the 27 EU-member states 对欧盟 27 个成员国的调查

‘The concept of performance is very complex. The only points on which there are consensus are the factors that influence individual and organisational performance. Since performance as “input” is very difficult to measure, the member states are increasingly measuring target achievement, results or outputs.’

“绩效这个概念非常复杂。人们只是在那些对个人和组织绩效存在影响力的因素上意见一致。由于绩效作为一项“投入”评测起来非常困难，各成员国正越来越多地采用目标成就、结果或产出作为评测对象。“

Demmke 2007b


(2) Developments

System-analysis-approaches in the 1970‘ies: Implementation and evaluation as a new perspective

NPM-approaches in the 1990‘ies: efficiency and effectiveness as core catchwords (‚everything must be calculated‘)

Post-NPM-period: good governance; better service delivery (quality orientation) ; citizen orientation


(3) Function of Performance Assessment 绩效评估的功能

• Analyses of the output and outcome of actions of public administration (and/or the single public servants) with the aim to improve these outputs and outcomes对公共行政行为（和 / 或单个公务员）的产出和成果进行分析，以提高其产出和成果

• In general, to improve the efficiency and effectiveness and especially the quality of the public administration and its services 一般而言，是提高效率和效能，特别是公共管理的质量及其服务


(4) Areas of Performance Assessment 绩效评估领域

A: Performance Assessment of Personnel 人员绩效评估 established forms of PA (following the Weber-model)

绩效评估的固定格式（依据韦伯模型） New appraoches in the field of PA (the NPM-appraoch)

绩效评估领域的新方法（新公共管理方法）B: Performance Assessment of Programmes and

Institutionsnumber of approaches since the 1970‘s (from system analysis until result-orientation) 项目和机构的绩效评估1970 年以来的方法（从系统分析到结果导向）


II. The process of public programmes 公共项目流程

The Programme Cycle: 项目周期：(1) Objectives 目标(2) Indicator selection 指标选取(3) Data collection 数据采集(4) Analysis 分析(5) Reporting 报告

Each step of this heuristic model: quality assurance 该启发式模型的每个步骤：质量保证


III. Programmes and objectives 项目和目标

• From tasks and responsibilities to objectives: a fundamental change of perspective 从任务和职责到目标：一个根本性的观念改变

• Definition of objective: A result (change of situation) that shall be reached in a certain time in the future 目标的定义：需在未来一定时间内达到的结果（形势变化）

• Demands for proper objectives 对恰当目标的需求• The number of objectives 目标数目• The system (hierarchy) of objectives 目标体系（分级系

统）


IV. Indicators 指标 Indicators 指标 Information on indicators 指标信息 Source of information 信息来源 Process and responsibilities 流程和职

责


What is a good indicator?好的指标是什么？

• Clear theoretical framework 明晰的理论框架• Selection on the basis of quality and relevance 在质量和相关性基

础上来选取• Choice in weighting and aggregation explained 解释（指标的）权

重和加总选择• Indicators normalized to make them comparable 指标的标准化，

令其具备可比性• Assessment of the robustness of the composite indicator made 对

综合指标的健壮性进行评估 • Underlying indicators or values readily available 隐含指标或随时

可用的数值 • ….Source: OECD , Measuring Government Activities来源：经合组织，政府活动评测


Information on indicators指标信息

• Quality and quantity 质和量• Comparable information 可比信息• Old statistics and new sources of

information 过往数据和新的数据来源


Information on indicators指标信息

• Discussion on information on indicators, methods and assumptions

• The case of a system of ‚operating figures‘


Information on economic indicators

Economic backgroundGross Net Income per person ProductivityPercentage of employment (persons between 18 and 65)Percentage of employment of young people (between 18 and 24)

Percentage of employment of elderly people (between 50 and 65)


Case: operating figures

(1) Financial operating figures: input (budget figure; personnel etc.) per defined reference parameter (citizen/area/time etc.)

(2) Economic operating figures: input per certain service (resources per case or per sqare meter)

(3) Objective oriented operating figures: measuring following a certain target value (cases per employee; ambulatory treatment instead of inpatient treatment)

(4) Structural operating figures: describing the socio-economic framework


V. Analysis and results分析和结果

• The process of analysis 分析流程• The methods of analysis 分析方法• The criteria of analysis 分析标准• New Public Management: effectiveness,

efficiency, quality 新公告管理：效能、效率、质量


Benchmarking 基准分析• Benchmarking – concepts in Sweden, Germany and Great

Britain • 基准分析 – 瑞典、德国和英国的概念• Benchmarking on a voluntary basis: the case of performance

of programmes of Social Affairs • 自主基准分析：社会事务项目绩效实例• Benchmarking on the level of cities and communes: the case

of the Agency for Auditing the Communes (Northrhine-Westfalia/Germany)

• 城市和社区层面的基准分析：社区审计机构实例（北莱茵 - 威斯特法伦 / 德国）


Benchmarking 基准分析

UK: Compulsory benchmarking

The case: Benchmarking Innovation centres and Communities of interests for public sector (reform) learning


UK: Innovation centres and Communities of interests for public sector (reform) learning

Good Hope Hospital (West Midlands)

Wirral Hospital (NW England)

15-months inpatient waits 20 patients waiting over 15 months (significantly underachieved)

0 patients waiting (achieved)

Deaths within 30 days of surgery (current national figure 2.922 per 100.000 patients)

3.322 (average) 1.967 (significantly below average)

Inpatient survey: respect and dignity (current national figure: 82.3%)

78.2% 85.8%

Table: Service Quality Indicators: UK NHS (see also http://www.dh.gov.uk/PolicyAndGuidance/Performance/fs/en)

http://www.dh.gov.uk/PolicyAndGuidance/Performance/fs/en

VI. Examples I 实例 I

(1) EU: the Lisbon strategy 欧盟：里斯本战略(2) Programmes on social affairs 社会事务项目(3) Strategic programmes on regional policy 区

域政策战略项目(4) Sustainable Policy Development(5) Health Care


Example 1: EU _ Lisbon concept实例 1 ：欧盟 _里斯本概念

The system of objectives 目标体系 The sources of information 信息来源 The instruments for improvement 改进工具 The process of information and decision

making 信息和决策流程


Information on economic indicators

Economic backgroundGross Net Income per person ProductivityPercentage of employment (persons between 18 and 65)Percentage of employment of young people (between 18 and 24)

Percentage of employment of elderly people (between 50 and 65)


Information on indicator ‚Social cohesion‘

Social CohesionPercentage of persons unemployed for long-term (>12 months)Regional difference of employmentRegional difference of incomePercentage of people below the poverty linePercentage of children under 7 below the poverty line


Example 2: Programmes on social affairs实例 2 ：社会事务项目

New definition of objectives: 目标的新定义：Reforms of the laws on social allowances 社会津贴法改革Optimizing the service delivery 优化服务交付Higher efficiency of the services 提高服务效率Contribution to fiscal shortcuts 增进财政便捷性 Changes of the processes inside public administration 改进公

共管理内部流程 Changes of methods 改进方法 Criteria and indicators of new performance measurement 新

绩效评价的标准和指标


Example 3: Strategic programme on regional policy

实例 3 ：区域政策战略项目


Effiziente Organi-sation

Effiziente Organi-sation

Verkehr Verkehr

Inno-vation

Inno-vation

Europäi-sche Union

Europäi-sche Union

Regionale Entwicklung

Regionale Entwicklung

Selbstän-digkeit

Selbstän-digkeit

Internet-wirtschaft

Internet-wirtschaft

Dienst-Leistungs-wirtschaft

Dienst-Leistungs-wirtschaft

MWMW

Example 3: Strategic programme on regional policy

实例 3 ：区域政策战略项目


Ministerium für Wirtschaft, Technologie und Verkehr

Starker Standort für Dienstleister

Wachstumsstandort für Internet-

Unternehmen und Unternehmen

im Internet

Niedersachen Partnerland der Grün-

der: überdurch-schnittlich hohe

Selbständigenquote

Niedersachsens Regionen Konzentra-

tion auf eigene Stärken

EU: Unternehmen profitieren vom

Binnenmarkt und der Osterweiterung

Innovation in Niedersachsen –

vorhandene Potenziale stärken

Verkehr in Niedersachsen –

schnell und sicher zum Erfolg

MW in Niedersachsen Dienstleister

für die Wirtschaft

Example 3: Strategic programme on regional policy 实例 3 ：区域政策战略项目


Ministry: Main targets of regional policy: 部门：主要区域政策目标

Service delivery for the economic sector (private enterprises) 经济部门（私营企业）服务交付

Strong location for service delivery 重要服务交付项目 Location of growth for IT-industries IT 行业的增长点 Location for self-employed persons 自雇人士交付项目 Support innovations in the potentials of the region 支持增强

本区域发展潜能的革新项目 Fast and safe traffic system 快捷安全的交通系统

Example 4: Strategy on Sustainable Development (GE)

4 Areas of the Strategy on Sustainable Development: Fairness between the generations; quality of life; social cohesion; international responsibility. 21 policy fields 35 key indicators


Example 4: Strategy on Sustainable Development (GE)

Fairness between generations

Area Indicator Expected Result

Saving natural resources Productivity of energy(Productivity of using raw materials)

Doubled from 1990 until 2020 (1994 until 2020)

Protection of climate Emission of CO₂ Reduction of 21% from 1990 until 2012 and 40 % until 2020

Renewable energy Proportion of renewable ernergy of the whole comsumption of ernergy

Increase from 4,2% (2010) to 10 % (2020)

Use of soil (spatial policy)

Consumption of soil for housing and infrastructure

Reduction of the daily use down to x ha (2020)

Public debts Deficits of public budgets Balanced budget (no new credits) from 2011 onwardsChangchun, December 2012 33

Example 5: Targets of Health Care (1)

Targets and indicators in the Health Care Sector (State LSA) <selection>:•Vaccination rate (related to useful vaccination for the different age-groups) increased to 90%•Proportion of smokers and of deseases and deaths caused by abuse of alcohol is reduced to the average on the national level•Dental Health is imroved up to the average on the national level


Example 5: Targets of Health Care (2)

The Process of Evaluation and Quantification:•Working groups on results and reviewing the targets (every year)•Adjustment of structures and responsible institutions (every year)•Survey of strategies and methods (Conference, all 3 years)•Consistent operationalizing of targets and model projects (on- going) •Sectoral reports, documentations, General Report on Health Policy


Examples II

(6) Sustainable City development(7) Programmes of Youth Policy(8) Benchmarking (GPA)(9) Education


Example 6: City Development 5 slides

City of Ludwigsburg (South-West Germany):12 fields of action and objectiveswith66 indicators


12 fields of city development

0 General indicators1 Attractive Housing2 Cultural life3 Economy and labour4 Vital city districts5 Lively city center


12 fields of city development

6 Living together of generations and nations7 Green City8 Mobility9 Education and social services10 Sport11 Energy


4 Lively city center (4 Indicators)

• Visitors satisfaction• Customers satisfaction• Numbers of customers• Percentage of vacantness of shops


8 Mobility (9 Indicators)

• Density of traffic• Level of motorization• Traffic safety• Parking places in the city• Environment-friendly mobility • Quality and amount of bicycle lanes• Access of the city by public transport• Quality of public transport in the housing areas• Effectivity of the bus services


Example 7: Youth policy (‚Support to breeding‘)6 slides

5 groups of indicators:(1) Results and effects(2) Programmes and products(3) Structures and processes(4) Ressources(5) Structural figures


(1) Results and effects

• Coached children per number of all children• …• Development of cases• Quota of girls• Age at the start and at the end of the coaching

activity


(2) Programmes and products

• …• Relationsship between ambulant and inhouse

measures• Percentage of cases for young adults (over 18

years old)• …..


(3) Structures and processes

• Duration of the support• Intensity of the support• …• Quota of breaking off• Number of measures of financed education


(4) Ressources

• Costs per case• …• Costs per all young inhabitants • Costs per hour/day• Cases per engaged public employee full-time

employed


(5) Structural figures

• Structure of the young population • Quantity and quality of school exams• Percentage of young people under 18 that live

under the poverty line• Percentage of chidren effected by divorce• Structural factors of the city


Example 8: Benchmarking (GPA)


Benchmark: Definition (GPA) We define benchmark in principle as data that are

– reached by a certain number of communes in reality,– in a process of fulfilling the tasks complete and obeying

the legal regulations– the result of a steering approach that is target-oriented

and worth to be copied (processes, structures, methods). Remark: analysing the possibility to match the benchmark the

conditions which cannot be influenced should be considered. This shall be the result of an empirical approach.


Benchmark (GPA)


Benchmark: Example (GPA)

Task Indicator Benchmark RemarksManagement of personnel

Cases per full-time-employee

1900 6 cities match the benchmark

Support to breeding Proportion of ambulatory measures

60 % 2 cities below the line of 60%

Long term care Proportion of persons living outside of homes for long term care

65 % 3 cities match the benchmark

Consumption of electricity and heating

Consumption of kilowatt-hour per square meter

Heating: 90Electricity: 20

Several cities are below this line

Expenditures for adult evening class, music, public library

X Euro/citizen 1.50 €3.00 €5.50 €

High differences between the cities


Example 9: Education: Relationship between teachers and students (HE)

Klassenhöchstgrenzen für das Schuljahr 2011/2012

Schulform Jahrgangsstufe

KlassenhöchstgrenzebereitsohneSternchenregelung

Klassenhöchstgrenze mit Sternchenregelung

Grundschule

1-3 25

4 28

Hauptschule

5-8 25

9-10 28

Realschule 5-7 30

8-10 33

IGS 5-7 27

8-10 30

Gymnasium

5-7 30

8-9 (10) 33


Example 9: Education: abilities and skills learned

Abilities of students aged 10 (extract):•Reading skills of new and superior texts•Writing skills of drafting logic and comprehensive reports•Skills in multiplication tables•Knowledge in basics of the environment•Skills in oral communication (ability to express precisely, properly; level of active wording and of grammar; ability to differentiate)


VII. Consequences 影响

(1) The feed-back-loop in the programme cycle 项目周期中的反馈圈

(2) Changes of programmes 项目变更(3) Performance assessment and evalution:

Always connected to the political system 绩效评估和评价：总是与政治体系挂钩


VIII. Further approaches进一步的办法

(1) Contracting targets between institutions (the cases of universities and Ministry of Interior)缔结机构间目标（大学和内政部的案例）

(2) Result-orientated management (Quality management, including CAF/Common Assessment Framework) 结果导向管理（质量管理，包括 CFA/ 共同评估框架）


Many thanks for your attention and discussions!


performance assessment – programmes and institutions 绩效评估 – 项目和机构 dieter...

Documents