11

Actionable Governance Indicators: Actionable Governance Indicators: Progress and Prospects Progress and Prospects

DEC Seminar on DEC Seminar on

Measuring Governance Measuring Governance

May 1-2, 2008May 1-2, 2008

22

Purposes of broadly-defined indicatorsPurposes of broadly-defined indicators

Research on development impact of civil Research on development impact of civil liberties, corruption, rule of law, etc.liberties, corruption, rule of law, etc.

Research on determinants/correlates of Research on determinants/correlates of “good governance” “good governance”

Tool for reformers in governmentTool for reformers in government Country-level diagnosis: more sensitive to Country-level diagnosis: more sensitive to

random measurement error random measurement error

33

Purposes of narrowly-defined indicatorsPurposes of narrowly-defined indicators

Monitoring progress of public sector reform Monitoring progress of public sector reform programs: are reforms being implemented? (e.g. programs: are reforms being implemented? (e.g. pay de-compression)pay de-compression)

Measuring results: are intermediate-level Measuring results: are intermediate-level outcomes improving? (e.g. number of qualified outcomes improving? (e.g. number of qualified applicants for high-skill positions increasing)applicants for high-skill positions increasing)

Research on impacts of specific reforms on Research on impacts of specific reforms on performance of governance sub-systems (budget performance of governance sub-systems (budget management, civil service, tax administration)management, civil service, tax administration)

Input into “actionable” parts of CPIA criteria Input into “actionable” parts of CPIA criteria

44

Actionable Governance Indicators Actionable Governance Indicators are the are the “Missing Middle”“Missing Middle”

Inputs and outputsInputs and outputs Bank operations consistently trackBank operations consistently track

project inputsproject inputs and and outputsoutputs A growing number of Bank operations A growing number of Bank operations

and client countries have begun to and client countries have begun to tracktrack program and policy outputs program and policy outputs

OutcomesOutcomes Broad governance Broad governance outcomes outcomes are are

measured bymeasured by Freedom House, Freedom House, ICRG, etc. and by TI, WGI ICRG, etc. and by TI, WGI aggregate indexesaggregate indexes

The Missing Middle:The Missing Middle: Direct impacts of institutional Direct impacts of institutional

reform efforts on how reform efforts on how particular governance sub-particular governance sub-sub-systems function: sub-systems function: institutional arrangements, institutional arrangements, procedures, and practicesprocedures, and practices

Examples include PEFA, Examples include PEFA, Doing Business, Global Doing Business, Global Integrity, Open Budget Index, Integrity, Open Budget Index, Statistical Capacity IndexStatistical Capacity Index,, OECD procurement indexOECD procurement index

55

"Effective implementation, including further development of actionable and disaggregated indicators, will now be critical to achieving the GAC strategy's desired results."

Development Committee Communiqué, Washington DC, 15 April, 2007

66

ActionabilityActionability

implies greater clarity regarding the steps implies greater clarity regarding the steps governments can take to improve their score on governments can take to improve their score on an indicatoran indicator

is reduced to the degree extraneous factors can is reduced to the degree extraneous factors can affect indicator scores affect indicator scores

in general requires that indicators have in general requires that indicators have narrownarrow and and explicitexplicit definitions definitions

does not preclude subjective or perceptions-based does not preclude subjective or perceptions-based indicators (if they have explicit and narrow indicators (if they have explicit and narrow definitions) definitions)

77

Specific indicators enhance: Specific indicators enhance: • Actionability: clear link between actions Actionability: clear link between actions

and indicator values and indicator values • Replicability: independent informed Replicability: independent informed

observers should arrive at same scoreobservers should arrive at same score• Contestability: debates over narrower Contestability: debates over narrower

concepts are more tractable and concepts are more tractable and meaningful (DB)meaningful (DB)

88

Some specific indicators may also be Some specific indicators may also be • Manipulable (design reforms narrowly Manipulable (design reforms narrowly

for sole purpose of improving score) for sole purpose of improving score) • Irrelevant (rule of law matters for Irrelevant (rule of law matters for

development, but some “best development, but some “best practice” advice on how to improve practice” advice on how to improve rule of law may be wrong)rule of law may be wrong)

99

examples of narrow indicatorsexamples of narrow indicators

PEFAPEFA Open Budget Index Open Budget Index OECD Procurement AssessmentOECD Procurement Assessment Doing BusinessDoing Business Global Integrity Global Integrity Statistical Capacity Index Statistical Capacity Index Human Resource Management Human Resource Management Enterprise surveysEnterprise surveys Household surveys Household surveys

1010

PEFAPEFA

• 28 high-level indicators measuring performance 28 high-level indicators measuring performance of PFM systems (3 more on donor practices)of PFM systems (3 more on donor practices)

• a performance report based on the indicatorsa performance report based on the indicators

• applied in approximately 80 countries and by 19 applied in approximately 80 countries and by 19 donors (mostly WB and EC)donors (mostly WB and EC)

• common pool of information to promote country common pool of information to promote country leadership, coordinated and aligned donor leadership, coordinated and aligned donor support (“the strengthened approach”)support (“the strengthened approach”)

• about 30 assessments available at pefa.orgabout 30 assessments available at pefa.org

1111

Open Budget IndexOpen Budget Index

Using questionnaire with 122 items, Using questionnaire with 122 items, the OBI assesses: the OBI assesses:

public availability of key budget public availability of key budget documents documents

quality of information they providequality of information they provide timeliness of their dissemination timeliness of their dissemination

59 countries assessed in 2006 study; 59 countries assessed in 2006 study; goal of 80 or more in 2008goal of 80 or more in 2008

1212

OECD Assessment Methodology OECD Assessment Methodology for Public Procurement Systems for Public Procurement Systems

22 pilot countries22 pilot countriesBaseline indicatorsBaseline indicators 12 indicators12 indicators 54 sub-indicators54 sub-indicators Based on review of regulatory framework Based on review of regulatory framework

and practicesand practicesCompliance and performance indicatorsCompliance and performance indicators• Based on sample of procurement Based on sample of procurement

transactions and other performance transactions and other performance informationinformation

Assessment reportAssessment report

1313

Doing BusinessDoing Business

Contract enforceabilityContract enforceability Registering propertyRegistering property Paying taxesPaying taxes

Number of proceduresNumber of procedures TimeTime CostCost

1414

statistical capacity indicator (DECDG)statistical capacity indicator (DECDG)

Countries scored against specific criteria, Countries scored against specific criteria, using information available from the World using information available from the World Bank, IMF, UN, UNESCO, and WHO. Bank, IMF, UN, UNESCO, and WHO.

Overall score based on 3 dimensions:Overall score based on 3 dimensions: statistical practicestatistical practice: ability to adhere to : ability to adhere to

internationally recommended standards and internationally recommended standards and methodsmethods

data collection:data collection: frequency of frequency of censuses/surveys and completeness of vital censuses/surveys and completeness of vital registrationregistration

indicator availability: indicator availability: availability and availability and frequency of key socioeconomic indicators frequency of key socioeconomic indicators 

1515

Enterprise Surveys Enterprise Surveys

107 countries, 75000+ firms107 countries, 75000+ firms ““unofficial payments” as share of firm unofficial payments” as share of firm

salessales time required to resolve dispute in courtstime required to resolve dispute in courts costs of providing security against crimecosts of providing security against crime time spent meeting with tax officialstime spent meeting with tax officials percentage of firm sales reported for tax percentage of firm sales reported for tax

purposespurposes major constraints to doing business major constraints to doing business

1616

Household surveysHousehold surveys

Gallup World Poll Gallup World Poll TI/Gallup International “Global TI/Gallup International “Global

Corruption Barometer”Corruption Barometer” Regional “barometer” surveys Regional “barometer” surveys World Values SurveyWorld Values Survey ““is corruption widespread” is corruption widespread” paid bribes to police, tax officials, paid bribes to police, tax officials,

etc. etc.

1717

in-country diagnostic toolsin-country diagnostic tools PER, CFAA, CPAR, fiscal ROSC PER, CFAA, CPAR, fiscal ROSC WBI GAC diagnostic surveyWBI GAC diagnostic survey PETS (Uganda & others) PETS (Uganda & others) Citizen scorecards (Bangalore)Citizen scorecards (Bangalore) Road composition (Indonesia)Road composition (Indonesia) Municipal audits (Brazil)Municipal audits (Brazil) Pharmaceuticals (Argentina)Pharmaceuticals (Argentina) HRM (Albania, Macedonia, Romania)HRM (Albania, Macedonia, Romania)

1818

Global IntegrityGlobal Integrity Assesses existence of laws and institutions Assesses existence of laws and institutions

designed to curb corruption, and their designed to curb corruption, and their implementationimplementation

Covers most UNCaC provisionsCovers most UNCaC provisions Assessment by lead researcher in country; Assessment by lead researcher in country;

two peer reviewers per countrytwo peer reviewers per country Six broad categories, 23 sub-categories, Six broad categories, 23 sub-categories,

81 questions and numerous sub-questions81 questions and numerous sub-questions Assessments in 2007 for 55 countries; 43 Assessments in 2007 for 55 countries; 43

countries in 2006countries in 2006

1919

Egypt 2007Egypt 2007

strengthsstrengths•procurementprocurement•political financingpolitical financing

weaknessesweaknesses•public access to public access to informationinformation•election integrityelection integrity•accountability of accountability of public officials public officials •external auditexternal audit

2020

Hierarchy of Global Integrity IndexHierarchy of Global Integrity Index

Overall indexOverall index

Government Accountability (1 of 6 major categories)Government Accountability (1 of 6 major categories)

Executive Accountability (1 of 23 total sub-categories)Executive Accountability (1 of 23 total sub-categories)

Regulations governing conflict of interest by executive branch Regulations governing conflict of interest by executive branch (1 of 81 questions)(1 of 81 questions)

Are heads of state/government required Are heads of state/government required to file regular asset disclosure forms? Can public access them?to file regular asset disclosure forms? Can public access them?

2121

not all categories are highly inter-correlatednot all categories are highly inter-correlated

Procurement rules

100806040200

Pu

blic

acc

ess

to

info

rma

tion

100

80

60

40

20

0 Rsq = 0.0806

2222

implementation not as easy as passing lawsimplementation not as easy as passing laws

"in law" sub-index

1009080706050403020100

"in

pra

ctic

e"

ind

ex

100

90

80

70

60

50

40

30

20

10

0

2323

enforcement: a function of income per capitaenforcement: a function of income per capita

GDP per capita

45000

40000

35000

30000

25000

20000

15000

10000

5000

0

law

s-p

ract

ice

s g

ap

60

50

40

30

20

10

0 Rsq = 0.1783

2424

How can ratings on broad How can ratings on broad indicators be improved? indicators be improved?

little understanding of what actions most little understanding of what actions most effective in reduce incidence of corruptioneffective in reduce incidence of corruption

actionable indicators may improve this actionable indicators may improve this understandingunderstanding

criteria in broad expert ratings often not criteria in broad expert ratings often not detailed or even availabledetailed or even available

unclear how different aspects (e.g. grand unclear how different aspects (e.g. grand v. petty corruption) are weightedv. petty corruption) are weighted

unclear what information sources are used unclear what information sources are used by expertsby experts

ratings may lag actions or events ratings may lag actions or events

2525

ICRG corruption ratings for IrelandICRG corruption ratings for Ireland

date

OCT 2007

JUL 2006

APR 2005

JAN 2004

OCT 2002

JUL 2001

APR 2000

JAN 1999

OCT 1997

JUL 1996

APR 1995

JAN 1994

OCT 1992

JUL 1991

APR 1990

JAN 1989

OCT 1987

JUL 1986

APR 1985

JAN 1984

corr

up

tion

ra

ting

5.5

5.0

4.5

4.0

3.5

3.0

2.5

2.0

1.5

2626

timelinetimeline

Dec. 1996: story breaks Dec. 1996: story breaks 5 to 4 5 to 4

Oct. 1997: tribunal convenes 4 to 3Oct. 1997: tribunal convenes 4 to 3

Jan. 1999: hears 1Jan. 1999: hears 1stst witnesses 3 to 2 witnesses 3 to 2

Feb. 1992: Haughey resigns as PMFeb. 1992: Haughey resigns as PMNov. 1992: retires from politicsNov. 1992: retires from politics

2727

some hints from the datasome hints from the data

Examine correlations of 23 anti-corruption Examine correlations of 23 anti-corruption policy areas from Global Integrity with policy areas from Global Integrity with outcomes (TI, WEF) for up to 66 countriesoutcomes (TI, WEF) for up to 66 countries

High: conflict of interest & asset disclosure High: conflict of interest & asset disclosure rules for executive branch, public access rules for executive branch, public access to informationto information

Low: conflict of interest & asset disclosure Low: conflict of interest & asset disclosure rules for judiciary, A-C laws & agencies, rules for judiciary, A-C laws & agencies, ombudsmanombudsman

2828

TI weakly related to GI “in law”TI weakly related to GI “in law”

LAWS

1009080706050

CP

I9

8

7

6

5

4

3

2

1 Rsq = 0.0749

ZWEZAR

ZAF

YEM VUTVNM

USA

UKR UGATZA

TMPTJK

THA

SRB

SLE

SEN

SDN

RUS

ROM

PHL

PER

PAKNPLNIC

NGA

MWIMOZ

MEXMDA

LVA

LKA

LBR

LBN

KGZKENKAZ

JPN

JORITA

ISR

IND

IDNGTM

GHA

GEO

FRA

ETH

ESP

EGY

ECU

DZA

CRICOL

CMR

CHN

CAN

BRABIH

BGR

BGD

BENBDI AZE

ARM ARG

2929

TI strongly related to GI “in practice”TI strongly related to GI “in practice”

IMPLEM

9080706050403020

CP

I9

8

7

6

5

4

3

2

1 Rsq = 0.4167

ZWEZAR

ZAF

YEM VUTVNM

USA

UKRUGATZA

TMPTJK

THA

SRB

SLE

SEN

SDN

RUS

ROM

PHL

PER

PAKNPLNIC

NGA

MWIMOZ

MEXMDA

LVA

LKA

LBR

LBN

KGZ KENKAZ

JPN

JORITA

ISR

IND

IDNGTM

GHA

GEO

FRA

ETH

ESP

EGY

ECU

DZA

CRICOL

CMR

CHN

CAN

BRABIH

BGR

BGD

BENBDI AZE

ARM ARG

3030

Global Integrity: Global Integrity: not all categories are highly inter-correlatednot all categories are highly inter-correlated

Procurement rules

100806040200

Pu

blic

acc

ess

to

info

rma

tion

100

80

60

40

20

0 Rsq = 0.0806