comparative study for the report on the status of pastoralism project

8/7/2019 Comparative Study for the Report on the Status of Pastoralism project

1/52

COMPARATIVE STUDY FOR THE REPORT ON THE STATUS OF PASTORALISMPROJECT

COMMISSIONED BY:

OXFAM GB

CONSULTANT:Amrik Heyer

2006


2/52

2

AcknowledgementsI would particularly like to thank Sarah Gibbons, Kemal Mustapha, Professor Vigdis Broch-Dueand Tom Wolf for their useful insights into this report.


3/52

3

Glossary

ADB The African Development BankAHDR Arctic Human Development ReportCBS Central Bureau of Statistics (Kenya)CDD-Ghana Center for Democratic Development, Ghana

CEE Central and Eastern EuropeanCGB Calouste Gulbenkian FoundationCR Comic ReliefDANIDA Royal Danish Ministry of Foreign AffairsDFID Department For International DevelopmentEA East AfricaEPAG Emergency Pastoralists Assistance GroupFGD Focus Group DiscussionGDG Gudigga Dhaganka Gurtida (Somali Council of Elders)GoK Government of KenyaHDI Human Development IndexHDP Human Development ParadigmHDR Human Development ReportHH Household

IDASA The Institute for Democracy in South AfricaIDS Institute of Development StudiesIDT International Development TargetsILO International Labour OfficeILRI International Livestock Research InstituteINGO International Non Governmental OrganisationITDG International Technology Development GroupM&E Monitoring and EvaluationMoP&ND Ministry of Planning and National Development (Kenya)MSU Michigan State UniversityDutch The Netherlands Ministry of Foreign AffairsNorway Norwegian Agency for International DevelopmentNGO Non Governmental Organisation

NSF The National Science Foundation, US0PDC Oromia Pastoralist Development CommissionOIDC Oxford International Development CentreOWDA Ogaden Welfare Development AssociationPA Participatory AssessmentPARU Poverty Analysis Research UnitPEC Poverty Eradication CommissionPSI Pastoralists Special InitiativeRBA Rights Based ApproachesRPK Resource Projects KenyaSDWG Sustainable Development Working Group (Arctic Council)SID Society for International DevelopmentSIDA Swedish International Development Agency

SL Sustainable LivelihoodsUNDP United Nations Development ProgrammeUSAID The United States Agency for International DevelopmentWB The World Bank


4/52

4

CONTENTS

AcknowledgementsGlossaryContentsExecutive Summary1. Introduction

1.1 Background1.2 Purpose of the Assignment1.3 Report Summaries1.3 Structure of the Report

2. Research Context: Aims, Agendas, Concepts and definitions2.1 Introduction2.2 Aims and Agendas2.3 Defining the area of research2.4 Framing Research2.5 Conclusions

3. Implementing Policy-relevant Research3.1 Introduction3.2 Addressing Knowledge gaps

3.3 Comparability: breadth vs depth3.4 Understanding long-term trends3.5 Conclusions

4. Institutionalising Evidence-based Policymaking4.1 Introduction4.2 Ownership4.3 Sustainability4.4 Credibility4.5 Accessibility4.6 Conclusions

5. Strengthening Policy Influences5.1 Introduction5.2 Stakeholder Sensitization

5.3 Local Voice5.4 Conclusions

6. Lessons Learned for ROSPAnnexesAnnex 1: BibliographyAnnex 2: Report SummariesAnnex 3: Qualifying comments on quantitative surveys

TablesTable 1: Report SummariesTable 2: Aims and AgendasTable 3: Organisational ProfileTable 4: Dissemination Avenues

Table 5: Lessons learned


5/52

5

Executive Summary

1. IntroductionThe Report on the Status of Pastoralism (ROSP) is an Oxfam GB initiative, designed to improvethe quality and accessibility of information on pastoral communities and their livelihoods in theHECA region. The current study has been commissioned to develop a series of lessons

learned from key research initiatives on poverty and/or marginalized communities, to inform thenext stage of the ROSP process. The report begins by locating each reports research aims andagendas in relation to those of the ROSP, and looking at the ways in which these have informedconceptual frameworks and the defining of research areas. Sections 3, 4 & 5 discuss theresearch initiatives in relation to three primary agendas: implementing policy relevant research,institutionalising evidence-based policymaking and strengthening policy influence. The reportconcludes by summarising the discussion and presenting lessons learned for ROSP.

The following reports and projects were analysed in this study:

Young Lives Project ; DFID/Save the Children, 2001 (ongoing research project)Avoiding the Dependency Trap: Roma in Central and Eastern Europe, 2002 (report)

Arctic Human Development Report, 2002-4 (report)Kenya Human Development Report, 2001 (report)Geographic Dimensions of Well-being in Kenya: Where are the poor? From districtsto locations, 2003; Who and where are the poor? A constituency level profile, 2005(Poverty mapping)Afrobarometer, 1999 (ongoing research project)Vulnerable Livelihoods in Somali Region, Ethiopia; IDS Research Report 57, 2006(report)Pastoralists Special Initiative Research Project; Comic Relief & Partners, 2005 (researchproject )

2. Research context: aims, agendas, concepts and definitions

Unlike some of the other research initiatives examined, the ROSP does not aim to focus oncommissioning primary research from the onset. Instead it aims to play a facilitating role,strengthening direct and indirect links between research and policy, and ultimately enhancingpolicy effectiveness. In other respects, the overall aims and agendas of the various reportsanalysed have much in common with those of the ROSP and can thus usefully inform the ROSPproject.In defining the research area, most reports were concerned to highlight dynamics ofmarginalisation for specific groups or areas. Whereas population-based definitions can havehigh advocacy potential, geographic definitions may enable researchers to better capture thedynamics leading to marginalisation, which crucially include relationships between groups.All the reports (with the possible exception of the KHDRs) have, in various ways, expandedinternational conceptual frameworks such as the Human Development Framework and

Sustainable Livelihoods, to better capture dynamics of marginalization, vulnerability and povertyincluding:

redefining international targets to highlight particular areas of marginalization asopposed to general poverty (Roma)

interrogating international targets and definitions to make these more appropriate fornon-conventional dynamics and value systems (AHDR)

broadening definitions of poverty in relation to specific population groups

developing frameworks for analysing the affects of policy on poverty dynamics (YL)


6/52

6

understanding political and social roots of vulnerability as well as economic causes (SR,Afrobarometer)

providing a geographically disaggregated analysis of poverty trends (PSI, PovertyMaps).

3. Implementing policy relevant research

The section examines the uselfulness of different research methodologies for achieving theoverall aims of the projects. These aims include addressing knowledge gaps, producingcomparable statistics for targeting policy and trend analysis.In relation to addressing knowledge gaps, benchmark studies are useful for:

Profiling a particular area, issue or population group.

Consolidating existing research or undertaking new research to identify policy issues.

Assessing the extent and quality of available information and highlighting knowledgegaps to inform future research.

Comparative studies are helpful for mapping broad trends and highlighting particular areas ofvulnerability in relation to normative standards. This is useful for policy targeting and advocacyas it highlights difference between various groups or areas. However they may not be so usefulfor the substance of policy formation, where in-depth context-specific research is more valuable.In relation to the latter, qualitative methods may be more appropriate than quantitative methods,for illuminating policy-relevant dynamics.

For trend analysis longitudinal studies fulfil two main agendas:

monitoring and tracking the effects of policy change

analyses of long-term trends and the evaluation of policy (and other) influences on localdynamics.

While the latter may be better undertaken through qualitative research, quantitative surveymethodologies are often more suitable for monitoring the effects of policy change.

4. Institutionalising Evidence-based PolicymakingTo help generate substantial policy impacts from research, institutional roles need toestablished so as to:

Foster user-ownership, through user-involvement in design, set-up, coordination,implementation and funding.

Foster sustainability through long-term links between research institutions and policyforums, which attract financial resources for policy-relevant research withoutcompromising the primacy of local policy agendas.

Ensure credibility by grounding the implementation of research in high-quality academicinstitutions, while continuing to sustain links between these and policy forums

Strengthen accessibility of research through institutionalising user-friendly dissemination

channels which ensure messages are delivered to policy-users in an appropriate andaccessible mannerA lesson which emerged from this analysis is the need to balance institutional involvement sothat the strengths of different types of institutions are maximized. Of particular importance islinking the intellectual contribution of high quality research institutions, with the political clout offormal policy actors and the advocacy and dissemination role of civil society organisations.

5. Strengthening policy influences


7/52

7

Where formal policy arenas are weak or distorted policy impacts will be weakened. It istherefore necessary to involve a wide range of policy influencers in research exercises(including civil society organizations, the general public, and research populations themselves).Actions which interest and involve wider stakeholders include advocacy, networking andconsultation forums as well as using imaginative and accessible dissemination avenues. Inrelation to the latter, interactive websites have many advantages, including being able to

disseminate a range of materials and products in a manner which is easily accessible to a rangeof audiences.Setting up research in such a way as to enable research populations to control and useinformation about them, can be significant to policy impacts. This can be fostered throughconsultation and dissemination, but also needs to include a more proactive dimension involvingresearch subjects in the production of research.

6. Lessons Learned for the ROSP

1. Defining the area of study geographically could, have similar advocacy mileage to apopulation definition. This is particularly the case in Northern Kenya.

2. A benchmark study consolidating existing material, is a useful way to set-up policy-

relevant research processes, through, identifying knowledge gaps, and policy issues.3. Developing links between policy forums, research institutions and donors should focus

on sensitising them to appropriate institutional roles, as well as identifyappropriatemethods and frameworks which will produce policy-relevant research.

4. Creating a well-marketed, interactive product such as a website, as well as having ahigh-profile research survey, would be a good way to involve a wide-range ofstakeholders in research on pastoralism.

5. Exploring creative ways to engage local populations in the production and disseminationof knowledge would strengthen local voice and could enhance policy impacts on povertyand marginalisation.


8/52

8

INTRODUCTION

1.1 BackgroundThe Report on the Status of Pastoralism (ROSP) is an Oxfam GB initiative, designed to improvethe quality and accessibility of information on pastoral communities and their livelihoods in theHorn and East Africa region. Current barriers to the accessibility and quality of information

include: Data Production: inappropriate methodologies and frameworks for capturing the

specificities of pastoralist dynamics in policy-oriented information systems. Dissemination: unimaginative dissemination channels which fail to target relevant

user groups and a lack of communication between policy forums and researchinitiatives.

Institutionalisation: lack of ownership and involvement in research exercises byusers and subjects of research, leading to unsustainabe data collection processesand limited relevance of information produced.

ROSP seeks to address these shortcomings by making available time-series, comparative dataacross four countries in the region (Ethiopia, Kenya, Tanzania and Uganda), to provide anauthoritative analysis of the trends and processes affecting pastoral livelihood systems. It will

also work to bring together key lessons and best practice examples from currently disparateresearch processes. It is hoped that a combination of quantitative and qualitative data willprovide a detailed analysis of the changes affecting pastoral systems and the processesunderlying these trends.

ROSP seeks to be a collaborative venture, working with a range of governmental and non-governmental partners, building networks and encouraging active sharing of information andlearning. Within this it aims to build upon the processes and systems currently in place for datacollection, analysis and research, providing institutional support and capacity building wherenecessary.

1.2 Purpose of the Assignment

The current study has been commissioned to develop a series of lessons learned from keyreports/data collection processes on poverty and/or marginalized communities such aspastoralists, to inform the next stage of the ROSP process. The reports were examined inrelation to four main questions:

How were the conceptual and analytical frameworks, and the data productionmethods chosen and how effective were these in capturing the information required?

To what extent did the institutionalization of the research process achievesustainability and local ownership?

How effective were dissemination channels in reaching target audiences andachieving desired impacts?

What are the lessons learned for the ROSP process, in relation to the production,

dissemination and institutionalization of high quality data on pastoralism? Inparticular, what lessons do the reports offer for impacting on policy and povertyreduction processes?

The reports/surveys selected for the assignment were chosen due to their aims and objectivesand their similarities with ROSP agendas. With this in mind the following reports and studieswere chosen:

Young Lives; DFID/Save the Children, 2001 (ongoing)


9/52

9

Avoiding the Dependency Trap: Roma in Central and Eastern Europe, 2002Arctic Human Development Report, 2002-4Kenya Human Development Report, 2001Geographic Dimensions of Well-being in Kenya: Where are the poor? From districts tolocations, 2003; Who and where are the poor? A constituency level profile, 2005Afrobarometer, 1999 (ongoing)

Vulnerable Livelihoods in Somali Region, Ethiopia; IDS Research Report 57, 2006Pastoralists Special Initiative Research Project; Comic Relief & Partners, 2005

The reports were then examined to determine the ways in which the aims and objectives haveinformed research frameworks and processes, including:

Data collection processes: ways of selecting and defining population groups;development of appropriate conceptual frameworks; duration and scope of researchprocess (longitudinal vs status reports); data collection methodologies (quantitativevs qualitative),

Dissemination processes: ways to capture the interest of key stakeholdersincluding the general public

Institutionalisation of research process: ways to forge links between research

and policy, enable long-term sustainability, and foster local ownership & capacitybuilding

1.3 Report SummariesThe reports/surveys were analysed to determine the ways in which their initial aims andcontexts shaped processes of data collection, dissemination and institutionalisation. Specificcharacteristics of the reports are summarized in Table 1 below, and are discussed further insubsequent sections. For a fuller summary, see Annex 2.


10/52

10

Table 1: Summary of Research Processes

Core Instit. Aims Objectives ConceptualFramework

Methods TargetAudiences

Dissemination Institutional framework

Young Lives(Status:longitudinal:15 years on-going)

AreaGlobal (4country sites)Population:children under18

SC-UK

Local RIs

DFID

OIDC

MonitorprogresstowardsachievingIDTs among

children

Understand how keypolicies affect childpoverty Influence policy Produce long-term,

comparative data totrack effects of policychange Create replicable,low cost methodologyfor comparativeresearch Sensitisestakeholders to childpoverty issues

Poverty &Livelihoodsanalysis

Policy

Analysis

PrimarySurvey

Communitycontext

analysis

Thematicstudies

PolicyMakers(Nat. &Internat.)

GeneralPublic

Researchers

Website

Reports

Publicity

(leaflet, photoexhibition,newsletter etc.)

Consultationforums

Public Archive

Funds: DFID

Coord & Design: DFID,SC-UK, OIDC

Research/Analysis:National ResearchInstitutions, and local SCoffices

Advisory: Country andInternational AdvisoryPanels of key stakeholde

Roma HDR(Status:2001-2002One-off)AreaCEEPopulationRoma

UNDP

ILO

Improveintegration ofvulnerablegroups,especiallyRomapeoples.

Provide regionallycomparative statisticaldata to inform policy Create a replicableset of guidelines tofacilitate integration ofvulnerable groups

HDIRBA

PrimarySurvey

Documentreview

PolicyMakers(Nat. &Internat.)

Academics

Civil Society

RegionalReport

Nationalreports

Website

Funds, design &Research/Analysis:UNDP/ILO

Advisory: consultation wkey stakeholders includinpolicy makers

Arctic HDR

(Status: 2002-2004 One-off)AreaArctic RegionPopulationTotal indefined area

Arctic

Council(govts)

Inform the

SDWG onhuman dpt inthe Arctic

Provide a

comprehensiveknowledge base forthe Arctic CouncilsSustainableDevelopmentProgramme

Modified -

HDI

Secondary

documentAnalysis(reports books,articles)

Policy

Makers

Regional

Report

Funds, Design & Coord

- Arctic Council (Govs ofIceland, Sweden, FinlandCanada, Norway)-Civil Society (SaamiCouncil)- Universities & ResearchinstitutesResearch/Analysis:Academics


11/52

11

Table 1: Summary of Research Processes cont/.

CoreInstit.

Aims Objectives ConceptFramework


Dissemination Institutional framewor

Kenya HDRs(Status:longitudinal:2001 ongoing)Area:National(Kenya)Population:Total Adult

UNDP

IDS Uniof NBO

Promote HD &people-centredapproach tonational policymaking inKenya

Provide informationfor policy planningand programming. Institutionalise HD-centred information

systems

HDI SecondaryAnalysis ofCBS Census& Surveys

Primaryparticipatoryresearch

PolicyMakers

YearlyReports

Funds, Design & CoorUNDP

Research/Analysis:UNDP, IDS, CBS(MoP&ND, GoK)

KenyaPovertyMaps (Status:longitudinal:2003 ongoing)Area:National(Kenya)Population:Total Adult

WB

ILRI

CBS

Institut-ionalisean M&E systemfor effectivemonitoring &targeting ofpovertyprogrammesIn Kenya

Build time seriesbenchmarks forpovertymeasurement Inform a pro-pooreconomic recoverypolicy agenda Enable effectivetargeting & M&E ofdecentralized budgetallocation

IncomePoverty linemeasures:

Disaggreg(provincial,district,location &constituencylevels)

SecondaryAnalysis ofCBS Census& Surveys

PolicyMakers

Poverty Maps/Reports

Funds: World Bank, ILR(SIDA, DFID, GTZ,Rockefeller)

Design and Coord:WB, CBS, ILRI

Research/AnalysisCBS (MoP&ND, GoK),ILRI, University of NairoPEC, PARU, SID

Afro-barometer(Status:

longitudinal:1999 ongoing)Area:Continental(Africa, 18countries)Population:Total Adult

MSU,

IDASA,

CDD

Improvegovernancestructures and

processes inAfrica

Collect views ofcitizens ongovernance and

economy Provide a baselinefor comparisonovertime Feed into policyprocess Inform voting-ageadults in Africa

GovernanceandDemocracy

Frame-works

PrimarySurvey

Voting ageadults inAfrica

PolicyMakers

Researchers

Website

Press releases

& BriefingPapers (


12/52

12

Table 1: Summary of Research Processes cont/.

CoreInstit.

Aims Objectives ConceptFramework


Dissemination Institutional framew

VulnerableLivelihoods inSomaliRegion(Status: 2004-2006 One-off)Area:State (SomaliRegion,Ethiopia)Population:Total: RandomSample

IDS -Sussex

UNOCHAPCI

Contribute tomore informeddebate andimprovedpolicy-makingfor SomaliRegion

Investigate causesand consequencesof livelihoodvulnerability in SR Improve

understandings oflivelihoodvulnerability amonglocal populations,federal government& internationalagencies

Livelihoods /Vulnerability

Socio-politicalchange

Primary HHSurvey

Primaryqualitative

research

PrimaryTrade &MarketingSurvey

Localpopulations

PolicyMakers

NGOs &Donors

Researchers

(Ongoing)ResearchReport

Workshops ofinitial findings(Local, National& International)

Funds: DFID

Design and Coord,research and analysUNOCHA-PCI IDS:Sussex

Advisory: GoE, GDGDistrict & MunicipalAdministrations, OPDSC UK, Hope for theHorn, OWDA

PSI(Status:2005:One-off )Area: Country(Kenya)Population:Pastoralists

ComicRelief

OxfamGB

To improvetargeting offunds toensuremaximumpoverty impactfor pastoralists

Enhanceunderstanding ofkey trends likely toaffect pastoralistways of life over thenext 10 15 yrs Enable pastoralistcommunities tomake betterinformed decisions Feed into GoK

and DevelopmentPartners (Donorsand NGOs) policyand planning

AdaptedLivelihoods

Systemssustainability& trends

Voices ofpastoralists

Primaryqualitativeresearch

Stakeholderinterviews(national)

Documentreview(general)

NGOs

PolicyMakers

LocalPopulations

Donors

ResearchReport

Workshops(National &District)

Funding: Comic RelCoordination: ComiRelief, Oxfam GB, ITEA, RPK, EPAG

Design & Analysis:Acacia Consultants

Research: Oxfam GBITDG-EA, RPK, EPA


13/52

13

1.4 Structure of the ReportThe report begins by locating the research aims and agendas in relation to those of ROSP, andlooking at the ways in which these have informed conceptual frameworks and definitions ofresearch areas. Sections 3, 4 & 5 then discuss the research initiatives in relation to threeprimary agendas: implementing policy relevant research, institutionalizing evidence-basedpolicymaking and strengthening policy influences. Lastly, the document summarises the

discussion in the form of a series of lessons learned for ROSP arising from the analysis.

2. RESEARCH CONTEXT: AIMS, AGENDAS, CONCEPTS & DEFINITIONS

2.1 IntroductionThis section describes the aims of the different research initiatives in relation to those of theROSP.. It goes on to analyse different ways of framing research and defining research areas, inthe light of these overall aims and objectives.

2.2 Aims and AgendasThe ROSP project was developed through Oxfam GBs concerns to improve policyeffectiveness for poverty reduction in pastoralist areas. To this end, ROSP aims to enhance the

quality and accessibility of policy relevant research through the following objectives. To: Produce and disseminate a benchmark assessment of existing research to identify key

issues and knowledge gaps, and assess the extent of current data and researchavailability

Encourage policy-relevant research that illuminates long-term dynamics in a andregionally comparative manner; to identify trends, track changes, monitor progress,establish targets and better inform policy substance

Contribute to the institutionalisation of evidence-based policy-making in a manner whichstrengthens user-ownership, ensures credibility and accessibility of research and fosterssustainability

Strengthen policy influences through enhancing local voice, increasing stakeholdersensitivity and raising public awareness

Like all the research initiatives examined for the present assignment, the ROSP ultimately aimsto improve the well being of its target group by improving policy. Unlike other initiatives, ROSPdoes not seek to achieve this solely through undertaking or commissioning primary research.Instead it aims to play a facilitating role, strengthening local research and analysis capacity andimproving the links between research and policy, thus ultimately enhancing policy effectiveness.The shared aim of improving the policy impacts of research, has led to many commonalitiesbetween the reports here analysed and the aims and agendas of ROSP.

The Young Lives project arose from DFID and Save the Childrens concerns to monitorprogress towards achieving IDTs, focusing on improved child well being. The project identifiedseveral shortcomings in current research on policy effects on child poverty (which are also of

relevance to ROSP), as follows1

- Research is segmented (e.g. focused on health or education):

- Research is not well disaggregated (e.g. by gender, age, disability etc.)- Changes in the situation of children are not well documented; instead we gain a

snapshot view which does not capture improvements or deteriorations over time- Quantitative and qualitative research are not well integrated (e.g. documenting school

attendance, but not the extent to which children feel valued by their families issues

1Young Lives Conceptual Framework


14/52

14

which children themselves say are important to them. Equally, small-scale studies,which capture this information, are too small scale to be a good basis for policymaking.)

- Most research does not link the situation of children with broad national/internationalpolicies: recommendations often stop at development projects/programmes

- Research is not well disseminated; it often sits on library or office shelves but is not

used

In tackling these issues, Young Lives was designed to address knowledge gaps on child povertythrough undertaking primary, comparative, longitudinal research. It was also concerned toimprove the policy relevance of research to enable national policy makers to better addressinternational targets, including developing a low cost survey methodology, which would bereplicable elsewhere. The project seeks to inform policy through extensive consultation anddissemination processes, as well as through raising stakeholder and public awareness on childpoverty issues, thus also influencing policy indirectly.

The Roma research2

was initiated by the UNDP and ILO in the context of EU accession,where significant resources were suddenly made available to bring vulnerable groups suffering

from poverty and marginalization, into the mainstream. The Roma report identifies similar policy-research deficits to those of Young Lives. Despite the existence of extensive qualitative data onthe Roma therefore, the policy aims of the project led researchers to develop a quantitativesurvey to obtain policy-relevant comparablestatistics on Roma populations. It also aimed tocontribute to the effectiveness of IDTs through developing a set of core principles, which canact as a practical guide for the assessment of vulnerable groups more generally.

The Arctic Human Development Report was commissioned by the Arctic Council3

to informits newly established Sustainable Development Programme, which extends a previous focus onenvironmental concerns, to embrace social and livelihoods imperatives. Unlike Young Lives andthe Roma reports, the research was not so much concerned with providing a comparativeunderstanding to address international development agendas; rather, it was intended to inform a

regional policy forum.. Instead of investing in an expensive quantitative survey therefore,leading academic experts were commissioned to undertake a scientific assessment of existing(mainly qualitative) research, which would:

provide an accessible overview of the state of human development in the Arcticthat can be used as a benchmark for assessing progress in the future; identifycritical gaps in knowledge and; provide a framework to help establish priorities forthe Sustainable Development Working Group (SDWG).4

Kenya HDRs and Poverty Maps are funded and designed by the UNDP and WB/ILRIrespectively, in collaboration with the Central Bureau of Statistics and local research institutions(mainly government but also academic). While the Kenya HDRs are viewed as being important

tools for the promotion of the cause of HD and people-centred approach to nationalpolicymaking the Poverty Maps have been initiated to build:

2Avoiding the Dependency Trap: Roma in Central and Eastern Europe

3The Arctic Council is made up of the governments of Iceland, Sweden, Finland, Canada, Norway)

4Arctic HDR


15/52

15

sustained time series benchmarks for poverty measurement in Kenya necessaryfor institutionalising an effective monitoring and evaluation system for theeffective implementation and targeting of poverty programmes.5

Both are concerned to provide comparative data (in the case of the poverty maps, alsolongitudinal data), which will monitor and inform national policy in relation to internationaltargets. Both are also centrally concerned to institutionalise policy-relevant research through

capacity building and linking research institutions and policy forums.

The Afrobarometer project aims to consolidate and understand democratic processes in Africathrough conducting a longitudinal, comparative public opinion survey on perceptions ofgovernance reforms and outcomes, and assessments of poverty and economic conditionswhich will feed directly into the policy process.

We seek to reach diverse audiences: decision-makers in government, policyadvocates, donor agencies, journalists and academic researchers, as well asvoting-age adults in Africa who wish to become informed and active citizens.6

Unlike other reports discussed so far, the research inspiration, design and implementation isgrounded in academic institutions, most of which are local to the research area. The research isparticularly concerned with enhancing the political capacity of electorates through improved

information. .

The Vulnerable Livelihoods in Somali Region report was initiated from IDS Sussex, as aresponse to a dearth of policy-relevant data on a particular area of Ethiopia, which has beenmarginalized by the state, mainly on ethnic grounds.

In this highly politicised context, this research study is an attempt to gatherinformation and give voice to the people of Somali Region themselves, and topresent this evidence to policy makers (community leaders, the regional andfederal governments, international donors, and international and local NGOs) asan input to their strategizing and decision-making. 7

Primary research in this case is geared to addressing national data gaps as well as providing abaseline through which to assess current and future policy impacts. Like the Afrobarometer, the

research is concerned not only to inform formal policy, but also to enhance the political capacityof local populations through improved access to policy relevant information.

The Pastoralists Special Initiative was commissionedto enhance Comic Reliefs and partners knowledge and understanding of keytrends and factors currently affecting and likely to affect the pastoralist way of lifeover the next 10 15 years, so that they are better able to target funding toensure favourable outcomes and maximum poverty impact for pastoralists. 8

It is thus primarily concerned with informing internal institutional policies, through primaryresearch centred around the identification of long-term trends and local perceptions. Like thetwo projects described above, the PSI is also concerned to enhance local voice through the use

of participatory methods and local dissemination.

5Central Bureau of Statistics (2005) Geographic Dimensions of Well-being in Kenya: Who and where are the poor? A constituency

level profile.6

www.afrobarometer.com7

Devereux, 20068

Pastoralists Special Initiative, 2005


16/52

16

Table 2: Comparison of research initiatives in relation to ROSP aims

Aim ApproachesIMPACT ON POLICY

Implement policy-relevant research Institute evidence-based policy making Strengthen policy influences

Bench-mark

9Longitud-inal

10Comparative

11Userown-ership

Sustain-ability

Access-ibility

Cred-ibility

LocalVoice

Stakeholdersensitisation

Publicawareness

ROSP Improve policies andpractice in pastoralistareas

YL Monitor progresstowards achievingIDTs among children

X X ()

Roma Improve integration ofvulnerable groups,especially the Roma

X X X X X X X X X

AHDR Inform developmentpriorities for theSDWG

X X X X X

KHDR Promote HD approachto national policymaking

X X X X X X

Pmaps Institutionalise M&Esystem for effectivetargeting of nationalpoverty programmes

X X X X X

Afrob Improve governancestructures andprocesses in Africa

X X X () X

SR Contribute to moreinformed policy-making for SR

X X

X (

)

(

) X

PSI Improve efficacy ofpastoralist programsfor CR & partners

X X () X X X () X

9Benchmark study provides a basis for future research and analysis.

10Longitudinal studies are those which seeks to monitor or track changes over time. They often use time series data or select indicators which can be measured at periodic stages

through the research11

Comparative studies seek to show difference between groups/areas within a society or country, or in some cases across a region.. They use common sets of indicators andanalyses to allow comparison.


17/52

17

2.3 Defining the research area.Like the ROSP, many of the reports analysed were concerned with collecting information onspecific population groups, often neglected or not well understood in policymaking circles. Forexample, Young Lives set out to collect data on children partly because children are particularlyvulnerable to poverty, and partly because little is known about childhood poverty, especiallyabout children aged 5 up to adolescence. The primary emphasis in such reports is on making

more visible particular arenas of impoverishment for policy-making purposes. In this sense theyaddress issues of rights and exclusion, as well as a more general poverty discourse.

Reports that have opted to study particular population groups are able to highlight issues ofmarginalization for policy and advocacy purposes, but they have also had to grapple with thefact that population definitions evolve in the context of social and political processes, and do nothave an objective basis outside the agendas and perceptions of those who use them (includingresearchers). These reports underscore the necessity of locating chosen population definitionsin relation to local socio-political dynamics as well as research agendas, so that research resultscan be interpreted accordingly. (Despite the fact that it uses a geographic area definition, theArctic report, for example, contains a lengthy section deconstructing an idea-scape about theArctic and its peoples, which usefully delineates the social demography of the area.)

The reports under discussion divide roughly between those in which the research is targeted onparticular population group and those that target a specific geographic area. 12

Whereas theformer may have more impact in advocacy terms and in highlighting the visibility of particulargroups, the latter poses less definitional challenges and allows the research to capture a widerrange of inter-group dynamics, which are germane to marginalization and impoverishment.Geographical definitions may also yield more information about the kinds of dynamics, whichlead to the marginalization of particular populations on the basis of cultural identity (such asrelationships between different groups living in the same area). Both have important policyimpacts. Whilst the former may improve the visibility of a particular population in policy, the lattermay better inform policy substance on issues affecting the well-being of the populationconcerned.

For Young Lives, population definition was relatively unproblematic, as a juridical definition ofchildren in terms of those who have not attained voting majority, was sufficiently able to capturethe target population. The PSI report was concerned to identify changes and trends inpastoralist livelihoods among those who were no longer pursuing a pastoral livelihood as well asthose who were still nomadic pastoralists. In this research context, a continued valuation of thepastoralist way of life by many pastoralist drop outs (rich and poor), as well as a politicalassociation between pastoralism and specific ethnic identities, led the researchers to use acombination of self-identification in cultural terms, and geographic location (broadly reflectingethnic boundaries). As opposed to relying on a livelihood definition therefore, the combination ofself-identification and geographic targeting was thought adequate to capture the populationgroup they wanted to understand, and neither include or exclude any group strictly on the basis

of livelihood practice.

The Roma report was concerned with the socio-political exclusion of a particular group and itsconsequent impoverishment. In this case, researchers grappled with a conundrum wherebyRoma has in many ways become synonymous with poverty and vice versa. People who mightbe considered to be ethnically Roma therefore may well not identify themselves as Roma in

12For example the Roma and the Young Lives reports target particular segments of the population, whilst the Arctic report and the

Vulnerable Livelihoods in Somali Region report chose geographic delineations of the their research areas.


18/52

18

order to avoid stigma, particularly when they are moving into higher income brackets. Equally,those who are not ethnically Roma may define themselves as such, in order, for example, tohave easier access to state transfers associated with this group. Therefore, as the reportacknowledges, any population sampling of Roma is likely to be biased towards the poorermembers of the group and of the population at large.

In order to deal with the influence of social stigmatization of this population group, researchersused a mixture of self-identification and objective cultural traits 13

identified by localadministrators and organizations, i.e. by others. (Fourteen percent of those interviewed did notidentify themselves as Roma, and in Hungary, researchers deliberately did not concludeinterviews with this group, opting instead for self-identification alone.) Whereas thecombination of self-identification and identification by others would have been useful revealingthe ways in which perceptions about who is and is not Roma highlight the aims and agendas ofspecific interest groups, the research does not sufficiently interrogate this dynamic. Instead, itappears to collude with the idea that objective cultural traits do indeed have some basis infact.

The Arctic report was framed around a contested regional geographic definition that was partly

defined ecologically and partly justified on the basis of being an increasingly important politicaland policy making arena. Like the Somali Region research, the geographic definition of theresearch area allows the Arctic report to explore the complexity of relations between and withingroups, which may otherwise appear similar in relation to broad demographic, social andeconomic features14

. For example the report captures important relationships between recentmigrants into the region and so-called indigenous groups (these relationships are also highlyrelevant to studies of pastoralist communities). According to the report, these relationships arecrucial to a human development understanding of socio-economic and political dynamics in theregion, as well as providing a more accurate picture of its overall demography.

2.4 Framing researchThis section examines the conceptual framing of research aims, to illuminate ways in which

these have influenced subsequent research agendas. With the influenceof InternationalDevelopment Targets, policy making has shifted from assumption based to evidence based.This has spawned a range of poverty assessment initiatives, together with poverty reductiontargets, which have gained acceptance among policy makers, researchers and developmentpractitioners. Many of the reports discussed can be seen in this light. Conceptual frameworkssuch as the Human Development Paradigm are now geared to the realization of Internationaldevelopment Targets (such as MDGs), and the resulting need to generate comparable data inorder to inform policy towards realizing these. Striking a balance between framing research inrelation to international frameworks and ideas, and the interrogation of these in relation to localconcepts and agendas, is a challenge, which reports have addressed with different levels ofsuccess.

Young Lives is primarily concerned with improving child well being, with relation to meetingIDTs. It is also strongly influenced by DFIDs strategic shift towards channelling developmentfunding through budget support to governments, rather than through project-basedinterventions. The projects concepts and frameworks are thus based around improvingunderstandings of policy processes, and developing appropriate ways of measuring andanalysing poverty dynamics in relation to its target group. For example:

13These are certain traits thought to be associated with Roma cultures, such as certain traditional practices and beliefs.

14Arctic HDR


19/52

19

Developing frameworks for analysing policy effects, including:- an eclectic understanding of policy contexts, which captures the multi-layered

and non-linear dynamics influencing policy processes- a rigoros framework for understanding the relationships between policy outputs,

outcomes and impacts, as well as modifying and mediating influences on these

Broadening poverty definitions to better encapsulate indicators relevant to child well

being:- broadening an income-based concept of poverty to embrace a more holistic

definition incorporating a range of livelihoods assets and processes- viewing children as important contributors to household livelihoods- incorporating subjective as well as objective indicators

Enhancing the sensitivity of poverty measurements in relation to the effects of poverty onchildren

- Children are particularly susceptible to poverty in terms of being subject todecisions taken by others, and more exposed to hazards and risks; somechildren are more susceptible than others and children as a group must bedisaggregated in relation to gender, age, culture and so forth.

- Poverty effects on children must be understood developmentally across

childhood and monitored at different stages of the maturation process;- Poverty impacts directly and indirectly on children (eg through disruption of

services).- One of the most serious threats to the formation of childrens identity and pro-

social skills and sense of self efficacy is relative as opposed to absolute poverty:(this relates to relationships of power)

The Roma research is primarily inspired by an integrationist policy agenda, which aims tolessen the burden on national and regional economies, of populations permanently dependenton state transfers. This has been framed through modifying a more conventional rights-basedapproach, which has frequently been used to promote the cause of marginalized groups suchas the Roma. As well as a rights-based approach therefore, the project adopts a human

development framework focusing on choices, opportunities, participation and responsibility,maintaining that legal rights are necessary but insufficient for increasing integration.15

Thefocus on participation and responsibility in particular, is meant to inform sustainable solutionsthat are affordable and will gain the support of majority populations.

The conceptual framework is therefore primarily inspired by a specific policy agenda, as well asa concern to improve the achievement of IDTs with respect to general issues of vulnerability andmarginalization. With respect to the latter, the Roma report draws on its research experience tousefully interrogate the framing of Human Development Targets:

Universal targets, while useful in advocacy terms, have limited relevance as policy tools.The data suggest that national-based measurements of poverty should be emphasizedover universal thresholds. The latter may be appealing for their simplicity and high

advocacy potential, but they are not very useful as policy targets. National surveys relating to IDTs often fail to highlight specific locations and distributions

of poverty: Although marginalized communities (such as Roma) face severe problems


20/52

20

regarding all aspects of the MDGs, national averages cloak the severity of theseproblems and deprive them of policy attention.

The report therefore begs the question: In relation to progress in achieving MDGs, should wetalk of halving national poverty, or halving the poverty of marginalized groups?Like the previous two research initiatives, the AHDR was also concerned to modify universalconcepts in relation to the specificities of its target area. The regional focus of the research has

therefore resulted in a thought-provoking interrogation of human development concepts inrelation to local concerns. Although not central to the project aims these insights are alsovaluable for similar resource-rich but socially and politically peripheral regions. In particular, thereport:

Interrogates the universal validity of Human Development indicators:- GDP as a measure of standard of living: it is difficult to use GDP to measure the

well being of subsistence systems or mixed economies more generally. Instead,a good life may be defined by other parameters such as the maintenance oftraditional hunting

- Schooling and literacy as a measure of knowledge: arctic residents have ahighly sophisticated grasp of matters important to their well being, but this

doesnt translate into school enrollments or literacy- Life expectancy as a measure of well being: longevity by itself is not a

paramount goal, and well being instead has more to do with qualityof life

Identifies additional parameters of significance to well being in the arctic context,including:

- fate control (guiding ones own destiny)- cultural integrity (belonging to a viable local culture)- contact with nature (interacting closely with the natural world).

Focuses on communities, not individuals

Identifies special characteristics of Human Development that are relevant to the Arcticcontext:

- capturing economic rent associated with extraction of natural resources

- extent of devolution of authority to regional or local decision-makers- measures to empower men and women where traditional gender roles are

changing fast

Captures relationships between local and global, including the effects of internationaleconomic actions on fragile local economies16

The AHDR thus succeeds in setting the arctic region in a global context through recourse to aninternationally relevant conceptual framework, as well as developing a locally derived set ofvalues and concerns with which to inform regional policy 17

The primary aim of the Kenya HDRs is to promotea particular conceptual framework (HumanDevelopment) and enable a more people-centred approach to national policy making. Thesereports are thus firmly centred within a Human Development conceptual paradigm. The Kenya

Poverty Maps aim to increase the effectiveness of national poverty targeting and monitoring. Inparticular, the poverty maps have been instituted in a context of increasing decentralization of

.

16These kinds of links are muted in the Human Development Paradigm, which tends to analyse economic and political outcomes in

relation to national variables, as opposed to their global context where dynamics of power are more clearly exposed.17

Its particular success in balancing the imperatives of international and local conceptual relevance, has been partly due to itsdependence on the type of in-depth qualitative research which the Roma report disregards.


21/52

21

political resources, in which context locally-sensitive policy targeting is more significant. Thereports are framed around a conventional income-based measurement of poverty, but onewhich is geographically disaggregated in relation to administrative and political boundaries(locality, district, province and constituency). In this way, the maps have been framed toincrease our spatial awareness of poverty determinants, and to highlight national inequalities.

The Afrobarometer research is particularly concerned to consolidate and understand relativelyrecent governance reforms in Africa towards greater democratization. The research is framedaround identifying new indicators that are particularly relevant to developing country contexts,through which to illuminate popular perceptions of governance and economy. These include:

- Democracy: Popular understanding of, support for, and satisfaction with democracy, aswell as any desire to return to, or experiment with, authoritarian alternatives.

- Governance: The demand for, and satisfaction with, effective, accountable and cleangovernment; judgments of overall governance perfomance and social service delivery.

- Livelihoods: How do African families survive? What variety of formal and informalmeans do they use to gain access to food, shelter, water, health, employment andmoney?

- Macro-economics and markets: Citizen understandings of market principles and

market reforms and their assessments of economic conditions and governmentperformance at economic management

- Social Capital: Whom do people trust? To what extent do they rely on informalnetworks and associations? What are their evaluations of the trustworthiness of variousinstitutions?

- Conflict and Crime: How safe do people feel? What has been their experience withcrime and violence? Which mechanisms do they prefer for the resolution of violentdisputes?

- Participation: The extent to which ordinary folks join in development efforts, complywith the laws of the land, vote in elections, contact elected representatives, and engagein protest. The quality of electoral representation.

- National Identity: How do people see themselves in relation to ethnic and class

identities? Does a shared sense of national identity exist

The Somali Region research aims to contribute to better informed policymaking for an areawhich has been marginalized, mainly on ethnic grounds, and which suffers from high levels ofvulnerability. It is primarily framed around understanding the causes and consequences oflivelihood vulnerability. In particular, the project uses qualitative research to expand a livelihoodsbased understanding in relation to the analysis of social relations, political dynamics, genderand conflict, thus foregrounding socio-political as opposed to economic determinants ofvulnerability.

The PSI report aims to improve the efficacy of pastoralist programs through identification oflong-term trends. The research is framed around a livelihoods analysis which enables the

identification of 3 major livelihood trends, and a prediction of their expected trajectories over thenext 15 years. The framework thus adapts a livelihoods approach to enable the identification oflong-term trends, as well as disaggregating the livelihood profile of its target group to facilitatemore appropriate and nuanced policy formation.

2.5 ConclusionsThe overall aims and agendas of various reports have much in common with those of theROSP. The ways in which these have translated into particular conceptual frameworks includingdefinition of research area, can thus usefully inform the ROSP project.


22/52

22

In defining the research area, most reports were concerned to highlight dynamics ofmarginalization for specific groups or areas. The implications of geographic vs population-baseddefinitions however pose significant differences with respect to policy impact. Whereaspopulation-based definitions can have high advocacy potential, geographic definitions mayenable researchers to better capture the dynamics leading to marginalization, which crucially

include relationships between groups.

In relation to the ROSP, a geographic definition of its area of interest would broadly capture thedemographic group it is concerned to understand (namely pastoralists), due to theirconcentration in particular regions. It would also enable the ROSP to highlight relationshipsbetween different groups within the area, as well as and the diversity of livelihood trends. At thesame time, a geographic definition could lose some of the political mileage of a population-based definition, which has the advantage of fore-grounding the rights and interests of a specificpopulation group. In the case of Kenya, however, the North East as a region has much thesame connotations of marginalization and impoverishment as do pastoralists as a group. Usinga geographic definition may thus have similar levels of advocacy potential as would apopulation-based definition, although this may vary from country to country. Lessons may be

drawn here from the PSI report which used a combination of geographic targeting and self-identification to capture the variable dynamics that characterize pastoralist livelihood trends,without restricting the research to a narrowly defined pastoralist scenario.

Most of the reports here analysed have been framed within international developmentparadigms such as Sustainable Livelihoods (DFID), Human Development Paradigm (UN) andGovernance agendas, which have all been geared to expanding an income-based definition ofpoverty to include other dynamics, such as social and human capital. While basing research onthese conceptual paradigms has the advantage of setting research projects in relation tointernational frameworks and thereby increasing their policy relevance, particularly in thecontext of IDTs, the extent to which they can be adapted to capture the specificity of localrealities has been a significant issue for many of the studies.

All the reports (with the possible exception of the KHDRs) have, in various ways, expandedthese frameworks to better capture dynamics of marginalization, vulnerability as well as povertyincluding:

redefining international targets to emphasise particular areas of marginalization asopposed to general poverty (Roma)

interrogating international targets and definitions to make these more appropriate fornon-conventional dynamics and value systems (AHDR)

Broadening definitions of poverty in relation to specific population groups

Developing frameworks for analyzing policy effects on poverty dynamics (YL)

addressing political and social roots of vulnerability as well as economic causes (SR,Afrobarometer)

providing a disaggregated analysis of poverty trends (PSI, Poverty Maps).

The reports therefore have valuable lessons for the ROSP, which seeks to improve analyticalframeworks to better capture and target poverty issues for a marginalized population group. Asthese reports demonstrate, it is important to use the research context to meaningfullyinterrogate universal concepts and emphasise the significance of local experiences not only toenable more effective policy targeting, but also to expand general understandings of povertyand well being, and make these more sensitive to marginalized groups.


23/52

23

3. IMPLEMENTING POLICY-RELEVANT RESEARCH

3.1 IntroductionThis section discusses various ways of implementing policy-relevant research, to try andimprove policy effectiveness. It examines the appropriateness of different methodologies forachieving the various research aims, which include:

- addressing knowledge gaps- producing comparable statistics for policy targeting- understanding changes over time to track and monitor policy effectiveness and

identify trendsIt analyses the use of benchmark (or status) reports, longitudinal research and comparativestudies as well as the relative merits of qualitative and quantitative methods, and use primary orsecondary data. Although ROSP does not aim to focus on conducting primary research, thevarious methodologies are significant to its aims, which include facilitating and encouragingresearch, which is more relevant to policy.

3.2 Addressing knowledge gapsLike the ROSP many of the research initiatives were specifically concerned with addressing a

knowledge gap in relation to marginalized communities/areas, so as to make them more visiblein policy terms (Young Lives, Roma, AHDR, PSI, Somali Region). A major issue in relation tothis aim is the choice between conducting primary research or secondary investigation. Primaryresearch is considerably more expensive than secondary analysis, and needs to be stronglyjustified in relation to its potential to add substantially to our understandings of particular issues,or to contribute to their policy effectiveness.

The Afrobarometer, for example, engaged in primary research to capture relatively newprocesses of governance reform in its context setting. In the case of the Somali Region report, amajor aim of the project was to address the lack of research on a particular area, as well as lackof policy visibility for an ethnically marginalized population. Young Lives chose to conductprimary research not only to fill a perceived knowledge gap, but also as an advocacy tool

through which to profile issues relating to marginalized and vulnerable population group.Primary research in this case has been used as a context through which to increaseinvolvement of key actors in issues relating to its target group, as much as a means of gatheringknowledge.

The AHDR on the other hand approached the issue of addressing a knowledge gap throughconsolidating secondary sources. In this case the team chose to consolidate existing researchin the form of a benchmark study to inform the analysis of general trends and issues of interestto policy makers, as well as isolating knowledge gaps for future research. This had theadvantage of drawing on extensive in-depth research stemming from a variety of sources(academic and policy), much of which had been developed over extended periods and whichrepresented considerable resources of money, time and technical expertise. It would have been

nearly impossible to commission new research, which approached this level. The report servesas a useful baseline from which to monitor future progress, develop policy indicators andcommission shorter more targeted studies on themes of particular interest.

3.3 Comparability: breadth vs depthCentral to the selection of research methodologies has been the aim of producing comparabledata in order to improve policy targeting, and inform policy change to bring populationgroups/areas into line with normative standards (e.g. MDGs). Quantitative survey methodsbased on standardized indicators and replicable questionnaires, carry with them connotations of


24/52

24

accuracy, generalisability, comparability and truth, attractive to policy making contexts.However, such studies tend to emphasise breadth rather than depth, lending themselves moreto targeting, monitoring (and advocacy), rather than the substance of policy formation whichrequires in-depth analysis. In addition, the samples on which such studies are based are oftentoo small to meaningfully reflect generalized traits, or too large to be feasible in research terms.

The Afrobarometer, Young Lives, and Roma reports have all developed a standardizedquestionnaire to facilitate cross-country comparisons, but with some room for countryspecificities. (Interestingly, the Roma research acknowledges that country specific questionshave been the more useful in policy terms). The Roma research aims to provide national andinternational policy makers and other stakeholders with accurate, reliable, and comparativestatistical data which it deems necessary to design and implement sound policy. Despite theexistence of extensive qualitative data on the Roma therefore, this particular view of the policycontext has led researchers to implement a core quantitative survey in order to obtaincomparable statistics on Roma populations in Central and Eastern Europe. This is a differentapproach to that of the AHDR for example, where the research team opted for a scientificassessment of existing mainly qualitative research as the best means to inform policy.

Like Young Lives, the Roma study also aims to standardize methods and principles to enablewider comparability and to harmonise policy agendas. Whereas Young Lives develops a lowcost standard questionnaire on child poverty replicable in other research settings, the Romaresearch develops a set of core principles to standardize assessments of, and interventionsrelating to vulnerable groups more generally.

The Afrobarometer is also based on a regionally comparative questionnaire. Unlike the HumanDevelopment and Roma surveys however, the Afrobarometer study is not intended to besufficient unto itself in analytical or policy terms. Rather it is intended to be complemented bymore in-depth studies of deeper research issues. It also includes a full discussion of thelimitations of the survey technique (see annex 2). Young Lives also addresses the limitations ofquantitative surveys through accompanying its core questionnaire with a community context

analysis to set the results in contextual perspective, and by commissioning thematic studies onareas of interest identified through the main survey, which need to be looked at in more depth.The core questionnaire acts partly as a map around which to identify deeper research issues.

The Roma report on the other hand, is very much dominated by its survey results, with little inthe way of qualifying information, especially including the voices and viewpoints of researchsubjects.

In countries like Kenya, distinguished by substantial inequalities between groups andlocalities, poverty maps based on census and survey data provide the comparable statisticalcoverage that policy making requires, along with a disaggregated analysis of this data rightdown to the local level which deepens the accuracy and policy relevance of the research. For

example, the reports discover that only two thirds of the rural population live in locations thatexhibit the district average in terms of poverty. When complemented with other data, the reportswill enable a better understanding of the geographic determinants of poverty, such as distancefrom market places and roads, service centres, relationship to biophysical, environmental andagro-climatic patterns, useful for targeting interventions.

At the same time, as the introductions to poverty map reports acknowledge, the maps arelimited to indications of poverty, and do not address its root causes. Therefore, in relation to thesubstance of poverty reduction policy formation, their use is limited. The danger is that, the


25/52

25

attractive simplicity of their presentation, and the perceived levels of certainty that oftenaccompany arrays of quantitative statistics will lead to large investments in surveys andquantitative exercises to inform future reports of this nature, at the expense of in-depth(qualitative) research. Despite extensive qualifying comments in the report introductions, thereis also a danger that the simplicity and accessibility of these maps will mean that their policyrelevance is over-rated.

The Somali Region report gives equal weight to both quantitative and qualitative methodologies,combining the merits of both. Due to the fact that little rigorous fieldwork has been conducted inthe Somali Region, a quantitative survey estimating incomes, livestock ownership, mortality andother quantifiable indicators of household well being was an important objective of the study. Atthe same time, the centrality of conflict, politics, gender and social relations as determinants ofwell-being outcomes necessitated a qualitative and contextual analysis as well as anunderstanding of trends in vulnerability over time. A household survey conducted in differentlivelihood zones (including urban areas and a refugee camp), was therefore complemented byqualitative fieldwork 18

. In addition, partly as a reflection of the primacy of livestock in thelivelihoods of the region, a trade and marketing survey was conducted. The result has been areport rich in textured analysis of key issues, that complements local observations with

objective indicators such as mortality rates, and in-depth socio-economic analysis. This type ofin-depth qualitative research has enabled a deeper understanding of local dynamics, enhancingthe sensitivity of national policy to local issues and highlighting the aspirations and agendas of(marginalized) local agents (the Poverty Maps to an extent straddle both these camps). Theextent to which this type of study is replicable is uncertain however. The level of detailedanalysis would be difficult and costly to reproduce through regular surveys, and at present theresearch remains as a one-off.

Young Lives has attempted to bring this kind of qualitative dimension to its research throughinnovative data collection methodologies, including the collection of subjective as well asobjective data such as feelings and perceptions, which are regarded both as indicators anddeterminants of poverty processes. However, large-scale quantitative survey methodologies do

not generally foster participation or local ownership of the research process. Instead, moreparticipatory qualitative approaches which allow the world views and values of informants to beexpressed, and which invest in longer-term relationships between respondents and researchers,are likely to produce more contextually informed analysis.

In the case of Afrobarometer, Young Lives and Roma studies, standardized quantitative surveyshave enabled research initiatives to measure local dynamics in relation to trans-localcomparable standards, thus setting national policy imperatives in relation to wider targets.These surveys have also been specially tailored to capture appropriate indicators onmarginalized groups in a form which is accessible to, and usable by policy makers. For the PSI,and Somali Region studies, in-depth qualitative research has enabled a deeper understandingof local dynamics, to enhance the sensitivity of national policy to local issues and to highlight the

aspirations and agendas of (marginalized) local agents. As these studies demonstrate, whereasquantitative research is useful for policy targeting and monitoring, qualitative research is likely tobe of more relevance to the substance of policy formation.

18This involved semi-structured interviews, focus group discussions, key informant interviews, individual case studies and life

histories, with several methods being used to facilitate discussion (community mapping and wealth-ranking, seasonal calendars,historical timelines etc.).


26/52

26

3.4 Understanding long-term trendsFor some of the research projects capturing dynamics and trends over time, rather thanproducing a snapshot of conditions at one time was one of the main aims of the research.Young Lives accorded the longitudinal aspects of the study the highest priority: Without this

emphasis it becomes just another study of children. Young Lives implements a standardquestionnaire at 3-4 yearly intervals on a core sample of children to gain a dynamic picture ofpoverty, as well as including questions on how poverty indicators for children play out in adultlives. This enables the study both to generate an understanding of the long-term effects ofchildhood poverty, and to influence and monitor the extent to which international (and national)policy targets are being met.

In the case of the Afrobarometer, a standardized public opinion questionnaire implemented atregular intervals in different political settings is intended to provide comparative comment onprocesses of liberalization and governance, as well as being a baseline to monitor thesedynamics in relation to future surveys (3 rounds have so far been conducted covering 18countries). Similarly, the Human Development Reports and Poverty Maps are based on national

surveys and censuses, which are repeated at regular intervals (although not using astandardized format). Future reports will make use of new data, thus reflecting changes overtime, useful for monitoring policy impacts.

Standardised longitudinal surveys are useful from a monitoring perspective in measuring policyimpacts, and understanding the effects of policy change. Where such research is based onrapid questionnaires however, the effect may be more of a series of snapshots rather than agenuine picture of long-term dynamics. The PSI report was also primarily concerned to identifylong-term trends and use this analysis for policy purposes. In this case however, researchersconducted a one-off study based on qualitative participatory methods (including timelines andmemory) to collect data on changes over time. Whereas the research had limited value as amonitoring tool, it was useful for informing policy substance through the identification of long-

term trends, and analysis of the effects of change on local populations. Had the researchcontextualized its analysis in relation to quantitative indicators, its policy-relevance would havebeen greater. In this sense it suffers from a similar lack of all-roundedness as the Roma report,but from the opposite perspective: over-reliance on qualitative data.

3.5 ConclusionsAs this section has demonstrated, research methodologies are often shaped by the overall aimsof the research process, whether it is to fill gaps, identify trends or to enable comparison acrossand within countries.There are policy benefits to all the different types of research examined: benchmark studies,comparative surveys and longitudinal research.

Benchmark studies are useful for

Profiling a particular area, issue or population group

Consolidating existing research or undertaking new research to identify policy issues Assessing the extent and quality of information availability and knowledge gaps to inform

future research

In relation to the ROSP commissioning a benchmark study of existing research would be auseful way to identify future research needs and relevant policy issues, as well as profiling its


27/52

27

target population group. In addition, such as study could help to identify the kinds of researchmethods which are most usefully applicable to policy formation, targeting, and monitoring, andwhich could inform institutional capacity building. For example, in relation to pastoralism inKenya, there exists substantial in-depth research quantitative and qualitative research stemmingfrom high quality research institutions, which would be extremely useful in policy terms.However, this has yet to be collated and organised into a policy format, including a benchmark

study from which to identify gaps, or policy lessons.

Subsequent to identifying knowledge/policy gaps through a thorough assessment of existingresearch, ROSP may want to encourage a new research, partly to address these gaps andpartly as an advocacy tool to profile its target population. In this respect, it is important to beaware of the pros and cons of various research methods.

Longitudinal studies fulfil two main purposes: on the one hand they enable the monitoringand tracking of the effects of policy change; on the other hand they enable the analysis of long-term trends and evaluation of policy (and other) influences on local dynamics. While the lattermay be better addressed through qualitative research (either one off studies which use historicalmethodologies, or repeated studies where a particular area or theme is periodically revisited),

studies which aim to monitor the effects of policy change lend themselves more easily toquantitative survey methodologies. Longitudinal studies are definitely a priority for ROSP, bothfrom the point of view of monitoring the effects of policy change, as well as identifying long-termtrends and dynamics. The ROSP may therefore wish to encourage more longitudinal research(quantitative and qualitative), as well as facilitating the dissemination of existing long-termstudies.

Comparative studies are helpful for mapping broad trends and highlighting particular areas ofvulnerability in relation to normative standards. They are therefore useful for policy targeting andadvocacy (the poverty maps are a particularly good example of this). However, as the Romareport recognizes, they may not be so useful for the substance of policy formation, where in-depth context-specific research is more valuable. In relation to the latter, qualitative methods

may be more appropriate than quantitative methods, for illuminating policy-relevant dynamics.

Both of these agendas are relevant to ROSP, which aims to provide a regionally comparativeframework through which to measure and assess relative changes in pastoralist povertyindicators, and at the same time to enhance the sensitivity of national policies in relation to theparticular dynamics of marginalization and vulnerability germane to its target group.The aim of providing regionally comparative data to inform policy should be carefully consideredin view of the fact that comparative studies are more useful for addressing internationaldevelopment targets, than the substance of national policy formation. Policies on pastoralistareas suffer more from lack of in-depth knowledge of relevant dynamics, than broadassessments of poverty indicators, in which context in-depth localized studies may be moreappropriate than large-scale quantitative surveys using comparable indicators. At the same

time a high profile quantitative survey such as that of the Afrobarometer or Young Lives wouldbe useful as an advocacy tool and a means through which to institutionalise policy-relevantresearch processes.


28/52

28

4. INSTITUTIONALISING EVIDENCE-BASED POLICYMAKING

4.1 IntroductionThis section addresses the capacity of research to impact on policy through institutionalisationand dissemination. It assumes that, no matter how relevant is the research produced for policypurposes, impacts will be limited unless research is disseminated in such a way as to be

accessible to policy makers, and institutionalised in such a way as to foster user-ownership andsustainability while maintaining sufficient levels of quality and rigour. The following sectionlocates reports in relation to their organisational setting, and examine the ways in whichresearch initiatives have reached out beyond their institutional roots to foster the ownership andinvolvement of potential users.

4.1 OwnershipThe aim of influencing national policy agendas has been achieved in various ways by thedifferent studies under consideration. In most cases, consultation with governmentrepresentatives prior to, and during the research process, as well as stakeholder workshops todisseminate research findings, have been the main methods to involve national policy makers.Young Lives has given policy makers a formal role on advisory boards, and has also partnered

with local research institutes, which have government credibility. The Kenya HDRs and PovertyMaps have institutionalised the research process through capacity building of governmentresearch offices, as well as extensive technical workshops and consultation processes, whichare expected to enhance national ownership of the research, as well as its policy relevance andlong-term sustainability.

Table 3 below depicts the organisational profiles of research processes, disaggregatinginstitutions into academic, NGO and government. The agendas of each of these broadcategories influence research projects in similar sorts of ways. For example, research that isdominated by government institutions will probably be oriented to formal policy-makingagendas, whilst research grounded in academic institutions is likely to be oriented to increasinggeneral understandings of particular issues/populations. Many of the reports span various

institutional contexts. The table therefore prioritises certain types of involvement over others inrelation to their levels of influence over the research process. For example setup, fundingdesign and coordination are likely to have more influence over reports and their outcomesthan fieldwork and consultation.

The table also shows whether research is dominated by international or local institutions. Whereresearch exercises aim to impact on national agendas, a dominance of international institutionsover key processes may have negative effects on sustainability and user-ownership, and theconsequent capacity to instigate meaningful change. It is significant, for example, that the tophalf of each research process (indicating more substantial involvement in research from set-upto design) is often dominated by international as opposed to local institutions, whilst the bottomhalf which indicates less substantial involvement and more of an advisory role, is dominated

local institutions.

Despite being geared primarily to influencing national policy arenas, a glance at the table showsthat most of the research exercises here examined were dominated by internationalresearch/policy agendas outside target user-groups. In particular, the table shows that donoragencies have had a strong involvement in these studies. Due to their large resources, it is tobe expected that these organisations will have a substantial influence over policy-orientedresearch projects. At the same time, however, where donors limit their involvement to funding(as is the case with the Afrobarometer and the AHDR) a greater role for research-users in key


29/52

29

processes of design, coordination and instigation would deepen user-ownership andconsequent impacts of research. The challenge is to build sufficient awareness and capacityamong user groups and donors to facilitate appropriate organisational roles.


30/52

30

Table 3: Organisational Profile

IRI INGO Donor Academics LRI LNGO Govt Res PopYoung Lives

Set up Funds

Design/co-ord Research

Advisory/Consultation Dissemination

ROMA HDRSet up Funds


Advisory/Consultation Dissemination

Arctic HDRSet up Funds


Advisory/consultation Dissemination

KHDRSet up Funds



Ken Poverty MapSet up Funds

Design/co-ord

Research

Advisory/consultation

Dissemination

Vul Livelihoods SRSet up Funds



PSISet up Funds

Design/co-ord

Research Advisory/consultation Dissemination

AfrobarometerSet up Funds




31/52

31

In relation to the research exercises here considered, the Afrobarometer is an example wherelocal (if not user) involvement in the research has been substantial, despite reliance on donorsources of finance. Whileh funded by donors, the setup, coordination, design andimplementation rest with local and international research bodies. The report which particularlystands out relation to user-ownership is the AHDR, where the users of the research, i.e. thepolicy makers have had a strong stake in the research process through funding as well as

instigation and coordination. This deepens the potential policy impacts of the AHDR research,also lending it greater sustainability and long-term relevance. The PSI report on the other hand,while institutionalised within its user-group, fell victim to issues of inefficiency due to attempts tofoster ownership through joint coordination, compromising both the research process andoutcomes. This is an important lesson for ROSP in that coordination of the research processneeds to balance imperatives of user-ownership with efficiency.

In the case of Young Lives, institutionalisation of the study at the national level through inputsinto implementation and national-level coordination by national research institutes and NGOs, islikely to contribute to long-term sustainability, but has not necessarily fostered ownership bypotential end user. It appears as though national research institutions and policy makers havebeen consulted on the core questionnaire and enabled to make limited modifications in relation

to country contexts, but have not impacted substantially on the overall set-up and design of theresearch. The major processes of set-up, research design, funding and coordination thereforecome across as being primarily dominated from outside, with implications for long-term impacts.This is mitigated to some extent by a rigorous dissemination strategy, including involving policy-makers at an institutional level as project advisors in order to encourage dissemination of theresearch in policy circles. Their role however has not been substantiated beyond being usefulagents for dissemination and publicity.

4.3 SustainabilityWhereas ownership of research by policy makers is significant for the force and extent of policyimpact, capacity to influence long-term policy change will depend on the sustainability ofresearch exercises. Some of the studies in question, such as the KHDRs and Poverty Maps as

well as Young Lives, have attempted to foster sustainability through capacity building andforging institutional links between policy and research forums. The KHDRs and Poverty Mapinitiatives, although instigated by international donors (UNDP and WB respectively) havedeliberately sought to inculcate national ownership and increase the long-term sustainability, bysetting up and funding a long-term research initiative through local policy and research bodies.The poverty maps in particular, have engaged in technical capacity building of governmentresearch staff, through extensive workshops and trainings.

A worry is that, with their large budgets and resources, the donor agencies in question havepushed their own research agendas at the expense of locally initiated research concerns, andhave also diverted scarce technical resources away from national research agendas 19

19

For example, there is little sense of the ways in which IDS or CBSs research agendas have informed these projects.

. Thestrong outside pressure on these research exercises backed by considerable resource outlays,

may thus achieve conditional involvement at the user level, but are ultimately likely tocompromise ownership and long-term policy impacts. Had resource inputs remained separatefrom research design and management, ownership may have been deepened alongsidesustainability.


32/52

32

Afrobarometer and Young Lives have also addressed sustainability issues in terms of financialresources, through developing internationally relevant research agendas, which are attractive todonor audiences. However, as the KHDR/Poverty Maps cases demonstrate, the necessity ofattracting donor finance must be balanced by the need to ensure that local policy and researchagendas dominate processes of design and coordination.

4.4 CredibilityPolicy-relevance of research also depends on credibility and quality of data collection methodsand processes. In this respect, achieving a degree of separation between researchdesign/implementation and set-up/funding is important. In the case of Somali Region, YoungLives, Afrobarometer, and AHDR studies, implementation of the research by high qualityacademic institutions is likely to produce results that are both credible due to their degree ofobjectivity, and of high quality due to the technical expertise and resource backing involved inresearch implementation.The AHDR has tempered its institutional grounding in a formal policy arena throughcommissioning implementation by academic experts. It therefore maintains a level of objecti

comparative study for the report on the status of pastoralism project

Documents