bridging research, policy and practice john young: odi, london [email protected] sudarno sumarto:...

Bridging Research, Policy and PracticeJohn Young: ODI, [email protected]

Sudarno Sumarto: SMERU, Indonesia [email protected]

Addressing Poverty: Pro-Poor Growth and Financial Inclusion in Asia Pacific

Overview• Introduction to ODI, RAPID & SMERU

• Group work: Policy influence stories

• Some theory

• How SMERU does it

• Some tools for researchers

• Group work: Using the tools

• Tools for organisations

• How SMERU does it

• Sources of further information.

ODI, UK• Development Think Tank

• 60 researchers

• Research / Advice / Public Debate

• Rural / Humanitarian / Poverty & Aid / Economics / Policy Processes

• DFID, Parliament, WB, EC

• Civil Society

For more information see: www.odi.org.uk

RAPID Group• Promoting the use of

research-based evidence in development policy

• Research / Advice / Public Affairs & Capacity-building

• Programmes:– Research for Policy– Progressive Policymakers– Parliamentarians– Southern Think Tanks

for further information see: www.odi.org.uk/rapid

http://www.odi.org.uk/RAPID/Projects/P0166/deza_product_en_847.pdf

http://www.odi.org.uk/RAPID/Meetings/Evidence/Evidence_Series.html

SMERU, Indonesia• Independent research and

policy studies on– the impact of government

programs and policies,– poverty, health, education, gender,

labour, fuel costs labour market conditions etc– decentralization and impact on services etc.

• Publications and public affairs.• NGO unit:

– provides research-based evidence to and facilitates NGO interaction

– capacity building: research methodology.

Civil Society Programme• More use of research-

based evidence for policy and practice through:– A Network of Think Tanks.– Capacity-development for the network

and other CSOs.– Research and learning from practical

experience.– Global and national action-research

projects.

for further information see: www.odi.org.uk/cspp

Group work: Stories of evidence and Policy

Change

End of slide show, click to exit

Stories of evidence and policy• Refresh your memory about your story• Find someone you don’t know and tell each other

your stories– Don’t write anything down!– 2½ minutes each!

• Use the story templates to interview and write each others story down– Make sure your images and messages are clear. – 5 minutes each!

• Tell your partner’s story to everyone else at your table– 2½ minutes each!

• Discuss among yourselves and identify 5 concrete actions emerging from these stories.– How might you apply these?– 10 minutes!

The theory and practice of bridging

research and policy

Definitions• Research: “any systematic effort to

increase the stock of knowledge”

• Policy: a “purposive course of action followed by an actor or set of actors”

• Evidence: “the available information supporting or otherwise a belief or proposition”

• Evidence-based Policy: “public policy informed by rigorously established evidence”.

Policy Processes

Identify the problem

Commission research

Analyse the results

Choose the best option

Establish the policy

Evaluation

Implement the policy

Monitoring and Evaluation

Agenda Setting

DecisionMaking

Policy Implementation

Policy Formulation

Policy Processes

Civil Society

DonorsCabinet

Parliament

Ministries

Private Sector

Chronic Poverty in Uganda

Kate Bird et al, Fracture Points in Social Policies for Chronic Poverty Reduction, ODI WP242, 2004 (http://www.odi.org.uk/publications/working_papers/wp242.pdf)

…in reality…• “The whole life of policy is a chaos of

purposes and accidents. It is not at all a matter of the rational implementation of the so-called decisions through selected strategies 1”

• “Most policy research on African agriculture is irrelevant to agricultural and overall economic policy in Africa2”

• “Research is more often regarded as the opposite of action rather than a response to ignorance”3

1 Clay & Schaffer (1984), Room for Manoeuvre; An Exploration of Public Policy in Agricultural and Rural Development, Heineman Educational Books, London2 Omamo (2003), Policy Research on African Agriculture: Trends, Gaps, and Challenges, International Service for National Agricultural Research (ISNAR) Research Report No 213 Surr (2003), DFID Research Review

But Policy makers are…

• Speed

• Superficiality

• Spin

• Secrecy

• Scientific Ignorance

Vincent Cable – Lib. Democrat MP & Shadow Minister of FinanceMore at: www.odi.org.uk/RAPID/Meetings/Evidence

…practically incapable of using research-based evidence because of the 5 Ss…

Evidence

Experience & Expertise

Judgement

Resources

Values and Policy

Context

Habits & Tradition

Lobbyists & Pressure Groups

Pragmatics & Contingencies

Factors influencing policy making

Source: Phil Davies Impact to Insight Meeting, ODI, 2005

Different Notions of Evidence

• Colloquial (Contextual)

• Anything that seems reasonable

• Policy relevant

• Timely

• Clear Message

Policy Makers’ Evidence

Source: Phil Davies Impact to Insight Meeting, ODI, 2005

• ‘Scientific’ (Context free)

• Proven empirically

• Theoretically driven

• As long as it takes

• Caveats and qualifications

Researchers’ Evidence

Existing theory1. Linear model2. Percolation model, Weiss3. Tipping point model, Gladwell4. ‘Context, evidence, links’

framework, ODI5. Policy narratives, Roe6. Systems model (NSI)7. External forces, Lindquist8. ‘Room for manoeuvre’, Clay &

Schaffer9. ‘Street level bureaucrats’, Lipsky10. Policy as social experiments,

Rondinelli11. Policy Streams & Windows,

Kingdon12. Disjointed incrementalism,

Lindquist13. The ‘tipping point’, Gladwell14. Crisis model, Kuhn15. ‘Framework of possible thought’,

Chomsky16. Variables for Credibility, Beach

17. The source is as important as content, Gladwell

18. Linear model of communication, Shannon

19. Interactive model, 20. Simple and surprising stories,

Communication Theory21. Provide solutions, Marketing I22. Find the right packaging, Marketing

II23. Elicit a response, Kottler24. Translation of technology, Volkow25. Epistemic communities26. Policy communities27. Advocacy coalitions etc, Pross28. Negotiation through networks,

Sebattier29. Shadow networks, Klickert30. Chains of accountability, Fine31. Communication for social change,

Rockefeller32. Wheels and webs, Chapman &

Fisher

X

• The RAPID Framework

Existing theory – a short list• Policy narratives, Roe• Systems of Innovation Model, (NSI)• ‘Room for manoeuvre’, Clay & Schaffer• ‘Street level bureaucrats’, Lipsky• Policy as social experiments, Rondene• Policy streams and policy windows,

Kingdon• Disjointed Incrementalism, Lindblom• Social Epidemics, Gladwell

An Analytical Framework

The political context – political and economic structures and processes, culture, institutional pressures, incremental vs radical change etc.

The evidence – credibility, the degree it challenges received wisdom, research approaches and methodology, simplicity of the message, how it is packaged etc

External Influences Socio-economic and cultural influences, donor policies etc

The links between policyand research communities – networks, relationships, power, competing discourses, trust, knowledge etc.

Case Studies• Detailed:

– Sustainable Livelihoods– Poverty Reductions Strategy

Processes– Ethical Principles in

Humanitarian Aid– Animal Health Care in Kenya– Dairy Policy in Kenya– Plant Genetic Resources

• Summary– GDN x 50– CSPP x 20– Good news case studies x 5– Mental health in the UK

Professionalisation of Public Services.

Structural Adjustment → collapse of services.

Paravet projects emerge.

ITDG projects.

Privatisation.

ITDG Paravet network.

Rapid spread in North.

KVB letter (January 1998).

Multistakeholder WSs → new policies.

Still not approved / passed!

1970s

1980s

1990s

2000s

Professionalisation of Public Services.

Structural Adjustment

Privatisation

ITDG Paravet network and change of DVS.

KVB letter (January 1998).

Multistakeholder WSs → new policies.

ITDG projects – collaborative research.

The Hubl StudyDr Kajume

Animal Healthcare in Kenya

International Research

A Practical Framework

External Influences political context

evidencelinks

Politics and Policymaking

Media, Advocacy, Networking Research,

learning & thinking

Scientific information exchange & validation

Policy analysis, & research

Campaigning, Lobbying

What you need to knowThe external environment:

• Who are the key actors?

• What is their agenda?

• How do they influence the political context?

Links:

• Who are the key actors?

• Are there existing networks?

• How best to transfer the information?

• The media?

• Campaigns?

The evidence:

• Is it there?

• Is it relevant?

• Is it practically useful?

• Are the concepts new?

• Does it need re-packaging?

The political context:

• Is there political interest in change?

• Is there room for manoeuvre?

• How do they perceive the problem?

What you need to doWhat need to know What need to do How to do it

Political Context:

Evidence

Links

• Who are the policymakers?• Is there demand for ideas?• What is the policy process?

• What is the current theory?• What are the narratives?• How divergent is it?

• Who are the stakeholders?• What networks exist?• Who are the connectors,

mavens and salesmen?

• Get to know the policymakers.• Identify friends and foes.• Prepare for policy

opportunities. • Look out for policy windows.

• Work with them – seek commissions

• Strategic opportunism – prepare for known events + resources for others

• Establish credibility• Provide practical solutions• Establish legitimacy.• Present clear options• Use familiar narratives.

• Build a reputation• Action-research• Pilot projects to generate

legitimacy• Good communication

• Get to know the others• Work through existing

networks.• Build coalitions.• Build new policy networks.

• Build partnerships.• Identify key networkers,

mavens and salesmen.• Use informal contacts

Policy entrepreneurs

Storytellers

Engineers

Networkers

Fixers

Conclusions

• Clear intent

• A thorough understanding of the context

• A strategic approach

• The right incentives / culture

• The right skills in the team

• To engage, engage, engage

To influence policy you need:


How SMERU influences policy and

practice

SUDARNO SUMARTOThe SMERU Research Institute

Jakarta, Indonesia

CSPP Partner’s Workshop 2006Canberra, Australia7 December 2006

BRIDGING RESEARCH POLICY AND PRACTICE: SMERU’S EXPERIENCE

A Case Study of Indonesia’s Unconditional Cash Transfer

The SMERU Research Institute www.smeru.or.id

Background Since the New Order era, domestic retail fuel price in

Indonesia has been subsidized. The fuel subsidy is regressive as consumption of each

energy category increases with income. The subsidy was not pro-poor because the poor benefited

less from it, including the kerosene subsidy. Very high world price of oil in 2005 has made the subsidy

balloon out of proportion.


To ease the pressure on the state budget from the ballooning subsidy and to improve spending allocation, GOI has increased the fuel price twice in 2005:

• 1 March, by an average of 30% “compensation” program: free education & healthcare, rice for the poor, rural infrastructure.

• 1 October, by an average of 120% “compensation” program, among others, unconditional cash transfer (UCT) to poor households.

Background


Brief Description of the UCT Program The largest cash transfer program in the world, targeting

more than 15 million households starting in October 2005. Transfer of Rp 300,000 per beneficiary household per

quarter for 1 year. Eligibility was determined by a household census

conducted by BPS using district-specific “proxy means testing” based on 14 indicators of poverty.

Delivery of the UCT is conducted through the post office by way of direct payment.

An additional 2.5 million households were added as eligible recipients after the first tranche.

SMERU's Involvement in Evaluating UCT

SMERU carried out simulations on the likely impact of fuel price increase on poverty (based on Susenas 2004 data):

Baseline: poverty head count (P0) = 16.66% After oil price increase: P0 = 22.05% Compensation through cash transfer:

• 100% correct targeting: P0 = 17.87%




SMERU's Involvement in Evaluating UCT

• SMERU conducted a rapid appraisal in Jakarta at the end of October 2005.

• The results were then disseminated to policymakers through consultations and presentations of research findings.

• Bappenas commissioned SMERU to conduct a larger evaluation, funded by the World Bank.

• The larger evaluation was conducted in 5 districts across Indonesia.



SMERU’s Policy Inputs

Convert the UCT to conditional cash transfers (CCT) on education, health, and nutrition

Improve targeting by reformulating criteria, strengthening local cadre, and empowering locally-specific poverty assessments

Develop cash for work or guaranteed employment scheme to improve rural infrastructure

Strengthen the supply sides of social services (health, nutrition, and education facilities)


Results from SMERU's Inputs

The government will pilot test CCT in 2007 The government intensified its effort to increase the budget

allocation for education and health


What Worked?

Stakeholders’ trusted SMERU’s credibility because of its evidence-based recommendations

SMERU’s approach with gov’t and top level officials

Effective linkages and networking with stakeholders

Maintenance of quality of research

Impartiality

Successful presentation of findings through discussions


Constraints Encountered

Political leverage Lack of tools to understand the political context of policy

change Lack of lobbying skills & opportunities Lack of resources (human and financial) and time to monitor

and follow-up the policy decision-making process SMERU being associated with donors

Tools for bridging research and policy


Introducing some tools• Some simple tools for researchers

– RAPID Framework– Outcome Mapping– Problem Situation Analysis (Tree Analysis) – Policy Process Mapping– Stakeholder Analysis– Force field analysis– SWOT analysis

• Some examples• Some more complex tools (for donors)• Some tools for policymakers

A Practical Framework

External Influences political context

evidencelinks

Politics and Policymaking

Media, Advocacy, Networking Research,

learning & thinking

Scientific information exchange & validation

Policy analysis, & research

Campaigning, Lobbying

Policy entrepreneurs

Storytellers

Engineers

Networkers

Fixers

Practical Tools

Overarching Tools - The RAPID Framework - Using the Framework - The Entrepreneurship

Questionnaire

Context Assessment Tools- Stakeholder Analysis - Forcefield Analysis - Writeshops - Policy Mapping - Political Context Mapping

Communication Tools - Communications Strategy- SWOT analysis - Message Design - Making use of the media

Research Tools - Case Studies - Episode Studies - Surveys - Bibliometric Analysis- Focus Group Discussion

Policy Influence Tools- Influence Mapping & Power Mapping - Lobbying and Advocacy - Campaigning: A Simple Guide - Competency self-assessment

RAPID Framework

Outcome Mapping

OUTCOME MAPPING:Building Learning and Reflection into Development ProgramsSarah Earl, Fred Carden, and Terry Smutylo

http://www.idrc.ca/en/ev-9330-201-1-DO_TOPIC.html

Problem Tree Analysis1. Discuss and agree

the problem or issueto be analysed.

2. Identify the causes of the focal problem (roots) and then the consequences(branches)

NB: The discussion is the most important thing: as issues are debated factors are arranged and re-arranged, often forming sub-dividing roots and branches

Policy Process Mapping• General Context issues – domestic and

international.• Specific Policy Issues (i.e. the policy cycle)• Stakeholder analysis

– Arena: government, parliament, civil society, judiciary, private sector.

– Level: local, national, international • What is their Interest and Influence?• Process matrix + political matrix• Political and administrative feasibility assessment

[Sources: M. Grindle / J. Court ]

Policy Process MappingFormulation Implementation

Politicians

Cabinet

Government

Bureaucrats

Civil Society

International

Stakeholder Analysis

Why:• To understand who

gains or lose from a policy or project.

• To help Build Consensus.

Steps:1. Identify Stakeholders

2. Analysis Workshop

3. Develop Strategies

Keep Satisfied

Engage Closely

Monitor (minimum effort)

Keep Informed

High

Power

Low

Low HighInterest

Forcefield Analysis

• Identify what you want to achieve

• Identify forces for and against change

• Identify which are most important

• Develop strategies to reinforce those for and overcome those against

SWOT Analysis• What type of policy

influencing skills and capacities do we have?

• In what areas have our staff used them more effectively?

• Who are our strongest allies?

• When have they worked with us?

• Are there any windows of opportunity?

• What can affect our ability to influence policy?

Strengths Weaknesses

Opportunities Threats

Skills and abilitiesFunding lines

Commitment to positionsContacts and Partners

Existing activities

Other orgs relevant to the issue

Resources: financial, technical, human

Political and policy spaceOther groups or forces

Communications strategy

• Identify the audience(s)• Identify the message(s) • Promotion• Evaluate impact and

change as necessary• Clear Strategy

– Interactive – Multiple formats

How?

Who?What?

Writing Policy PapersProviding a solution to a policy

problem

• The policy community• The policy process• Structural elements of a paper

– Problem description– Policy options– Conclusion

• Key issues: Problem oriented, targeted, multidisciplinary, applied, clear, jargon-free.

[Source: Young and Quinn, 2002]

Groundwater in India• to maximise impact of DFID forest/

ground water research project in India• Researchers, policy makers and activists• Used framework to analyse factors in

water sector in India• Developed strategy for final phase:

– Less research– More communication– Developing champions in regional

and national government – Local, Regional & National advocacy

campaign

SMEPOL Project Egypt• Policy Process Mapping

• RAPID Framework

• Stakeholder Analysis

• Force-Field Analysis

• SWOT

• Action Planning

Political Context Tools• Civil Society Index (CIVICUS) • Country Policy & Institutional

Assessment (World Bank)• Democracy and Governance

Assessment (USAID) • Drivers of Change (DFID) • Governance Questionnaire (GTZ) • Governance Matters (World Bank Institute) • Power Analysis (Sida) • World Governance Assessment

• Increasing the pull for evidence– Require the publication of the evidence base– Require spending bids to provide evidence base– Submit government analysis to external expert scrutiny– Provide open access to information

• Facilitating better evidence use– Encourage better collaboration across analytical services– Co-locate policy makers and internal analysts– Integrate analytical staff at all stages– Link R&D strategies to departmental business plans– Cast external researchers more as partners than as

contractors– Second more university staff into government– Train staff in evidence use

Demand-side Tools

Source: Abstracted from PIU 2000, Bullock et al (2001)

UK Government “Tools”Overview and Checklist 1. Impact Assessment and Appraisal: guidance checklist for policy makers.

Strategy and Policy Evaluation 2. Strategy Survival Guide 3. Magenta Book: Guidance notes on Policy Evaluation 4. Green Book: Appraisal and evaluation in Central Government 5. Regulatory Impact Assessment (RIA)

Ensuring Key Perspectives are Included 6. Incorporating regional perspectives into policy making toolkit (Subnational) 7. International Comparisons in Policy Making Toolkit 8. Gender Impact Assessment: a framework for gender mainstreaming 9. Managing risks to the public: Appraisal Guidance

Testing Policy Ideas 10. Policy Pilots

Public-orientated Tools 11. Concern Assessment Tool 12. Community Engagement How to Guide

13. Connecting with Users and Citizens

Getting Better Advice and Evidence 14. Expert Advisory Bodies for Policymakers

15. Improving Standards of Qualitative Research

Group work: Trying out some of the tools


Trying the tools

• RAPID Framework (Page 12)

• Stakeholder analysis (Page 24)

• Force-field analysis (page 20)

• Problem Tree (Page 22)

• SWOT (Page 28)

Working on one of the stories, try out one (or more) of the tools:

Building effective organisations

The (changing) role of CSOs• Is huge: Worth $12bn globally, reach 20%

of world’s poor, provide 40% health & education services in Ghana, Zimbabwe & Kenya.

• Is changing: service provision → policy engagement.

• Can be very effective: Globally, eg Jubilee 2000; locally eg animal health in Kenya,

• Is uncomfortable:– with governments: lack of trust– with donors: emphasis on GBS & policy – with academics/policy advisers: weak evidence

Some examples• AFREPREN: a network to promote pro-

poor energy policies.

• CIPPEC: works on Education, Fiscal Policy, Health, Transparency and Justice an Argentina.

• Unnayan Onneshan: works on pro-poor agricultural and trade policies in Bangladesh.

• International Budget Project: works to promote budget transparency and accountability

• SMERU: provides reseach-based evidence to NGOs and other actora and promotes public debate

How CSOs influence Policy

Obstacles to CSO Engagement

Obstacles for Research

Key problems and solutionsExternalDifficult Political Contexts • Campaigns

• Boomerangs• Policy Pilots

InternalWeak understanding of political contexts

• Rigorous context assessments

Weak engagement • Better strategies for engagement at all parts of the policy cycle

Inadequate use of evidence

• Collecting the right evidence for each situation (qualitative vs quantitative etc)

Weak communication • Better communication: publications, events, face-to-face

Isolation • Collaboration with other CSOs, donors and government agencies: Networks

Capacity constraints • “Systemic” capacity-building: of organisations and networks within their contexts

Organisational Capacity• Knowledge, Attitudes & Practice

– Policy entrepreneurs– Training & mentoring etc

• Organisational development

– Finance, admin & personnel systems

– Strategic (action & business) planning

– Fundraising & reporting

• Building an organisational profile– Communications, Public Affairs and the Media

Struyk, 2002, Local Governance Institute, Open Society Network, Budapest

Learning & KM• Learning before, during

& after– Peer Assist– Reflective Inquiry– After-Action Review

• Collaboration Tools– E-discussions– Shared workspaces

• Information management tools• Intelligent search engines• Incentives

Policy Processes in DFID• Looking at internal policy

processes – what works in DFID. • Small, informal workshop with 7

staff.• Participatory pair-wise ranking of

factors influencing the success of 8 policy processes.

• Worked quite well.• In DFID - agendas and processes

rather than documents are key

How SMERU does it


The SMERU Research Institute

SUDARNO SUMARTOThe SMERU Research Institute

Jakarta, Indonesia

CSPP Partner’s Workshop 2006Canberra, Australia7 December 2006


What is SMERU?

SMERU is an independent research institute that undertakes research and policy studies on socio- economic, poverty, decentralization, and vulnerability issues considered most urgent and relevant for the welfare of the people of Indonesia

SMERU adopts both quantitative and qualitative research methods to carry out these studies


What is SMERU?

SMERU engages in policy dialogues and provides inputs SMERU enhances the capacity of NGOs through training

and discussions Organizes workshops, seminars, and discussions as part

of the sharing and networking process Regularly publishes papers, reports, and newsletters and

maintains a website to disseminate research findings


How and Why SMERU Was Set Up? In October 1998, SMERU (previously known as the Social

Monitoring & Early Response Unit) was established with a mandate to carry out independent, reliable, real-time monitoring of the social impact of the crisis unfolding in Indonesia

It was funded by AusAID, ASEM, and USAID, and with logistical and administrative support from the World Bank.

SMERU's mandate from its stakeholders came to an end by December 2000

The core group of staff and researchers continued SMERU's existence as an independent institution for research and policy studies under the name of The SMERU Research Institute


Initial Sources of SMERU’s Core Funding (Year 2001) AusAID The Ford Foundation DFID (ended in 2005)

Commissioned research projects and training

Other Sources of Funding


How Does SMERU Operate?

Planning • Based on strategic planning and Vision 2010 • Administrative and financial matters: in consultation with its

management and staff, and occasionally with the Board of Governors

• Research matters: in consultation with designated research teams

Reporting• Project Consultative Committee Meetings and BOG

meetings; quarterly and biannual reports


Maintenance of Academic Credibility Peer review/refereed journals Linkage with universities, research institutes, development

organizations Participation in local and international seminars,

conferences, and workshops through presentations of papers and attendance as participants


Maintenance of Independence SMERU does not belong to any entity No kickback – remains impartial, transparent and

accountable Third party agreement – SMERU has the right to co-own the

data and publish the research findings Research outcome is not influenced by political or religious

and other entities, and is not being controlled by any entity Not driven by donors’ agenda


Engagement with Policy Processes

Interacting, organizing seminars and workshops, disseminating SMERU’s publications, research findings and recommendations, sending memorandum/policy briefs to relevant high-ranking government officials, donors, and other stakeholders.

Establishing networks and collaboration/coalition with CSOs, and lobbying with MPs


Engagement with Policy Processes

Disseminating information/research findings towards forming better-informed CSOs

Providing free access to all SMERU’s publications, reports, working papers, and NGO Database through the SMERU Website: www.smeru.or.id

Linking SMERU’s website to others


CHALLENGES …….. Developing trust and credibility through better research

quality Understanding and dealing with other influencing

contextual factors in bridging research to policy Striving to be financially sustainable by maintaining the

research quality to win the trust of stakeholders and donors

Balancing consultancy work and SMERU’s own research agenda

Widening networks and linkages Facilitating linkages and cooperation among CSOs,

enhancing capacity and promoting public accountability

Conclusions and sources of more

information


Conclusions• It is possible to improve the impact of

research on policy if you:– really, really want to do it– understand the (esp. political) context– use a strategic approach– establish the right incentives / culture– develop the right skills in the team– engage, engage, engage

• There are some good tools• There are a growing number of

organisations who can help

Civil Society Programme

Further InformationODI – www.odi.org.uk

RAPID - www.odi.org.uk/rapid

CSPP – www.odi.org.uk/cspp

or contact: John Young [email protected]

SMERU: www.smeru.or.id

or contact: Sudarno Sumarto

[email protected]

bridging research, policy and practice john young: odi, london [email protected] sudarno sumarto:...

Documents

policy slide

policy evaluation

policy studies

policy narratives

economic policy

life of policy

researchbased evidence

exploration of public