fighting hunger worldwide measuring results, sharing lessons · 2017. 1. 18. · flexibility to...

Annual Evaluation Report Office of Evaluation May 2016

Figh

ting

Hun

ger

Wor

ldw

ide

2015 mea

surin

g res

ults,

sh

arin

g les

sons

Executive Summary

Part 1. Evaluation Findings Introduction

Emergency preparedness and response (EPR)

Nutrition

Country-specificevaluations

Part 2. WFP’s Evaluation Function NewEvaluationPolicy OfficeofEvaluation(OEV)performancetoplanin2015

Annex 1: Evaluation Charter

Acronyms

1

3

3

3

8

12

1616

16

25

26

Contents

1WFP Annual Evaluation Report 2015

ExecutiveSummary

2015Part1ofthisannualevaluationreportpresentssynthesizedfindings,lessonsandrecommendationsfromOfficeofEvaluationreports.Whilethemajorityareforevaluationscompletedin2015,twosynthesisreportsincludesomeevaluationscompletedearlier.

RelevanttothecurrentglobalcontextandforthcomingWorldHumanitarianSummit,Section1.2presentsasynthesisofevaluationfindingsonWFP’sstrategicand operational emergency preparedness and responseefforts.ReflectingthegrowingprominenceofnutritionissuesinglobaldevelopmentdialoguearoundtheSustainableDevelopmentGoalsandZeroHungerChallenge,Section1.3highlightsfindingsfromevaluationsconcerningWFP’spolicy,partnershipsandprogrammeactivitiesrelatedtonutrition.Thesynthesisofotherfindingsfromcountry-specificevaluationspresentedinSection1.4reflectsonWFP’sworkindifferentcontextsanditsshiftsfromfoodaidtofoodassistance,andfromimplementertoenabler.

Part2reportsondevelopmentsinWFP’sevaluationfunction,notablyitsnewevaluationpolicy,whichwasapprovedinlate2015,andontheOfficeofEvaluation’sperformanceagainstits2015workplan.Thenewpolicyissupportedbyanevaluationcharter,whichestablishesthe mandate, authorities and institutional arrangements forthenewevaluationfunctionandisannexedtothisreport.Alongsideitscontinuedprogrammeofcentralizedevaluations,theoperationalizationoftheaugmentedfunctionenvisagedinthepolicyisamajorfocusfortheOfficeofEvaluation’seffortsin2016.

BuildingonWFP’sstrengthsisevermoreimportantinthe2030Agendaera.DrawingonthesynthesizedevaluationspresentedinPart1andfocusingonthetopicsthatfeaturedmostfrequentlyintherecommendationsofindividualevaluationsandthetwosynthesisreports,thissummaryidentifiesthefollowingissuesforseniormanagement’sconsideration.

Overarching Lessons andRecommendations

WFP’scontinuedshiftsfromfood aid to food assistanceandfromimplementer to enabler, asheraldedbythe2014–2017StrategicPlan,wereconfirmedbymanyevaluationsashighlyrelevantandappropriateforsustainablehungersolutionsindiversecontexts.WFP’sstrategicreorientationappearstobegatheringpaceonthegroundwhereverthecontextpermits,andincreasinglyrecognizedbystaff.

TheevaluationsconfirmWFP’sstrengthsinemergency response–ofteninextremelychallengingcontexts–whichcontinuedtoformthebulkofitsprogrammeexpenditures.WFP’sinvestmentsinstrengtheningits response capacity and the coordination and qualityofitsresponseshaveenhanceditsreputationinthehumanitariansystem,withpositiveresults,especiallyinlarge-scale,sudden-onsetemergencies.However,thehighvisibilityofLevel3emergencies,andtheglobaldemandsofmanagingseveralsuchemergenciesconcurrentlyhaveattimeshadunintendedconsequencesforchronic,underfundedandlower-levelemergencies.

TheincreasingambitionandrangeofWFP’sworkrequireaknowledge-driven organization to: manage thecontinuousinnovationdemandedbytoday’scomplexcontext; support its partnerships; and underpin its comparativeadvantage,especiallyinrapidlyevolvingfieldssuchasnutrition,resilienceandassistancemodalities.Reflectedinmultiplefindingsconcerningdata and analysis to underpin programme design, modality selection, targeting and prioritization, outcome monitoringandcost-effectiveness,theevaluationsprovidedampleevidenceofpositiveefforts,althoughthesearenotyetfullyreflectedontheground.ThemostfrequentcategoryofrecommendationsrelatedtofurtherstrengtheningofWFP’s evidence and knowledge culture,byapplyingstrongerincentivestoincrease

2 WFP Annual Evaluation Report 2015

attentiontomonitoring,evidencegenerationandanalysisinallcontexts;andbyincreasingthesharinganduseoflessonsinprogrammedesignanddecision-making.

SuccessinallareasofWFP’swork–fromemergencyresponsetocapacitydevelopment–dependsoneffectivepartnerships.Theevaluationsrevealedamixedpictureregardingcollaborationandsynergyamong United Nations agencies, and relationships withcooperatingpartners.ThesecondmostfrequentgroupofrecommendationsurgedWFPtoapplyits strong corporate commitment to partnerships moreconsistently,supportedbyclearanalysisofcomplementaritiesandaddedvalue.

Concerningpartnershipswithgovernments,asimilarnumberofreports,includingbothsyntheses,recommendedbuildingonidentifiedpositiveresultswith more systematic and systems-oriented approaches to national capacity developmentinWFP’sareasofprovenexpertise,suchasemergencypreparedness,foodsecurityandvulnerabilityanalysis,socialprotection,andschoolfeeding.ThisstrategicreorientationtowardssupportingnationalsystemstailoredtothediversecontextsandsectorsinwhichWFPworksentails:i)incontexts other than emergency response, positioning WFP’sprogrammingwithinnationalsocialprotectionandotherframeworks,suchasfornutrition,andacceleratingtheshifttoenablerbymakingtheprovisionoftechnicaladvicethedefaultstandardapproach;ii)inprotractedsituations,developinglong-termtransitionplansthatprepareWFPtohandoverresponsibility

to national partners; and iii) in emergency contexts, strengtheningtheengagementandcapacityofnationalcounterpartsinpreparednessandresponse.

Severalevaluationsrecommendedgreaterfinancing flexibilitytosupportthehumanitarian–developmentnexusandthelong-termperspectiveofthe2030Agenda,anticipatingWFP’songoingFinancialFrameworkReviewandassociatedprocessestobetterenablecountry-levelresults-basedmanagement.Inthisregard,findingsconcerningdevelopments–forexampleincash-basedtransfer1 programming and in monitoring systems–suggestthatthetimelagbetweensystemimprovementsandoperationalresultsshouldnotbeunderestimated.

Mixedfindingsongenderledtorecommendationsinmanyreports–includingthetwosyntheses–onsystematicallyincludingplansformeetinggenderpolicyobjectivesinallprojectsandcountryandcorporatestrategies,movingbeyondthe“inclusionofwomen”approachinwaysthatareappropriatetoeachcontext.UnderpinningmanyoftheseissuesisWFP’sinvestmentinitsstaffthroughthePeopleStrategy.Thereweremultiplerecommendationson:i)ensuringthatrelevantstaffprofilesanddeploymentscombinethenecessaryoperational competence with strategic, partnership andanalyticalskills;andii)providingstaffwithaccessibleguidanceandskillsdevelopment,notablyinaccountabilitytoaffectedpopulations,gender,equity-focusedprogrammedesignandmonitoring,andinrapidlyevolvingareasofknowledgesuchascash-basedtransfersandnutritionprogramming.

1 WFPnowusestheterm“cash-basedtransfers”torefertobothcashandvouchertransfermodalities,whileevaluationreportsreflectthepriorseparationofcashandvouchers.


Part1.EvaluationFindings

Introduction

Atthistimeofunprecedentedhumanitarianneedandcomplexity,amidtheglobaldialogueonstrengtheningtheeffectivenessoftheglobalhumanitariansystemanditsconnectionswithdevelopment,andnotingtheglobalSustainableDevelopmentGoal(SDG)commitments to ensuring that through the 2030 Agenda“nooneisleftbehind”,thisyear’ssynthesisofcommonfindingsandlessonsfromevaluationshighlightsissuesthatareparticularlyrelevanttoWFP’snextStrategicPlan.

Reflectingthisglobaldialogueandtherangeofevaluationscompletedin2015,thisyear’ssynthesizedfindingsarepresentedinthreesections:i)emergencypreparedness and response; ii) nutrition; and iii) country-specificevaluations.

WiththeresourcesavailabletoWFP’sOfficeofEvaluation(OEV),differenttypesofcentralizedevaluationareconductedtoassesssystematicallytherelevance,alignment,coherenceandcoordinationofWFP’spolicies,strategies,countryportfoliosandoperations,andtheirresultsintermsofeffectiveness,efficiency,impactandsustainability.Whilethe32evaluationscoveredinPart1(Table1),werenotselectedforstatisticalrepresentitiveness,theyspanallWFPactivitytypesacrossawiderangeofenvironmental,politicalandsocio-economicsettings,andcovertwoofitsmostimportantthematicareas–emergencyresponseandnutrition.

Eachsynthesisusedananalyticalframeworkbasedontheevaluationquestionsandtheprominentthemesoffindings,systematicallyextractedfromtheevaluationreportsandanalysedforcommonpatternsandnotabledivergencesrelatedtostrengths,weaknesses,challenges,strategicconclusionsandlessons.

Emergency Preparedness andResponse

WFPisaleadingactorintheinternationalhumanitariansystem.In20142WFPwasthesinglelargestrecipientofoverallhumanitarianfundingandofpooledfunds;emergencypreparednessandresponse(EPR)activitiestotalledUSD3.65billionor86percentofWFP’sprogrammeexpenditures,directlyassisting70percentofitsbeneficiaries.Itleadsorco-leadsthreeoftheglobalclustersintroducedin2005:Logistics,FoodSecurity,andEmergencyTelecommunications.

Acknowledgedweaknessesinthehumanitariansystem’scollectiveresponsetothreelarge-scaleemergenciesin2010promptedtheInter-AgencyStandingCommittee(IASC)toadopttheTransformativeAgendain2011tostrengthenleadership,coordinationandaccountabilityinmajorhumanitarianresponses.Aswellasparticipatinginthisglobalreformagenda,WFPalsointroducedaseriesofinternalstrengtheninginitiatives,whichwereconsolidatedintoamajororganizationalredesignin2012and2013.

Sincethenhowever,thenumber,scaleandcomplexityofemergencieshaveincreased,furtherstretchingthecapacitiesofdonorsandhumanitarianorganizations.In2015,WFPandtheIASC-ledhumanitariansystemrespondedtosixconcurrentLevel3emergencies3 and sixLevel2emergencies,4themajorityofwhichhadbeenongoingforlongerthanayear.InMay2016,theUnitedNationsSecretary-GeneralconvenesthefirstWorldHumanitarianSummitto“generategreatergloballeadershipandpoliticalwilltoendconflict,alleviatesufferingandreducerisk”.

2 Thelatestyearforwhichpublisheddatawereavailable.

3 CentralAfricanRepublic,Ebolavirus,Iraq,SouthSudan,SyrianArabRepublic,Yemen.

4DemocraticRepublicoftheCongo,Libya,Mali,Nepal,Somalia,Ukraine.


Su

bje

ctR

efer

ence

Per

iod

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

12

20

13

20

14

20

15

20

16

20

17

Syn

thes

is 1

- E

mer

gen

cy P

rep

ared

nes

s an

d R

esp

onse

Syn

thes

is

of W

FP’s

Em

erg

ency

P

rep

ared

nes

s an

d

Res

pon

se

WFP

’s P

rep

ared

nes

s an

d Re

spon

se E

nhan

cem

ent

Prog

ram

me

(PR

EP)

Glo

bal

log

isti

cs c

lust

er**

FAO

/WFP

Joi

nt E

valu

atio

n of

Foo

d S

ecu

rity

Clu

ster

Coo

rdin

atio

n in

Hum

anita

rian

Act

ion*

WFP

’s u

se o

f p

oole

d f

un

ds

for

hum

anita

rian

pre

pare

dnes

s an

d re

spon

se*

WFP

’s R

egio

nal R

espo

nse

to t

he S

yria

n Crisi

s In

ter-

Age

ncy

Hum

anita

rian

Eva

luat

ion

of t

he R

espo

nse

to C

onfli

ct in

Sou

th S

ud

anIn

ter-

agen

cy E

valu

atio

n of

the

Typ

hoo

n H

aiya

n h

uman

itarian

res

pons

e*

Moz

amb

iqu

e: P

RRO

200

355

(201

2-20

14)*

Mal

i: E

MO

P 20

0525

(20

13-2

014)

*

Eth

iop

ia:

PRRO

200

290

(201

2-20

13)

mid

-ter

m e

valu

atio

n*Ta

jiki

stan

: PR

RO

200

122

(201

0-20

14)*

Mad

agas

car:

PRRO

200

065

(201

0-20

13)*

Mal

i, M

auri

tan

ia,

Bu

rkin

a Fa

so,

Nig

er:

Regi

onal

EM

OP

2004

38 (

2012

-201

3)*

CP

EU

nite

d Re

publ

ic o

f Ta

nzan

ia C

PESta

te o

f Pa

lest

ine

CPE

Op

EvS

omal

ia:

PRRO

200

443

(201

3-20

15)

mid

-ter

m e

valu

atio

nIr

an:

PRRO

200

310

(201

3-20

15)

mid

-ter

m e

valu

atio

n

Syn

thes

is 2

- N

utr

itio

n a

nd

Syn

thes

is 3

- C

oun

try-

Sp

ecifi

c

Pol

icy*

**

Nut

ritio

n Po

licy

Str

ateg

ic*

**

REA

CH

- J

oint

Eva

luat

ion

of R

enew

ed E

ffor

t Aga

inst

Chi

ld H

unge

r an

d U

nder

-nut

ritio

n (F

AO

/WFP

/UN

ICEF

/WFP

/WH

O/D

FATD

Can

ada)

CP

EU

nite

d Re

publ

ic o

f Ta

nzan

ia C

PESta

te o

f Pa

lest

ine

CPE

Leve

l 3W

FP’s

Reg

iona

l Res

pons

e to

the

Syr

ian

Crisi

s

Syn

thes

is

Yea

r 2

of

Op

erat

ion

s Ev

alu

atio

ns

Arm

enia

: D

EV 2

0012

8 (2

010-

2016

) m

id-t

erm

eva

luat

ion

Ban

gla

des

h:

CP

2002

43 (

2012

-201

6) m

id-t

erm

eva

luat

ion

Ecu

ador

: PR

RO

200

275

(201

1-20

14)

Gh

ana:

CP

2002

47 (

2012

-201

6) m

id-t

erm

eva

luat

ion

Gu

inea

Bis

sau

: PR

RO

200

526

(201

3-20

15)*

Hai

ti:

DEV

200

150

(201

2-20

14)*

Hon

du

ras:

CP

2002

40 (

2012

-201

6) m

id-t

erm

eva

luat

ion*

Iran

: PR

RO

200

310

(201

3-20

15)

mid

-ter

m e

valu

atio

nM

alaw

i: C

P 20

0287

(20

12-2

016)

mid

-ter

m e

valu

atio

n*M

ali:

EM

OP

2005

25 (

2013

-201

4)*

Moz

amb

iqu

e: C

P 20

0286

(20

12-2

015)

mid

-ter

m e

valu

atio

nP

akis

tan

: PR

RO

200

250

(201

3-20

15)*

Som

alia

: PR

RO

200

443

(201

3-20

15)

mid

-ter

m e

valu

atio

nTu

nis

ia:

DEV

200

493

(201

2-20

15)

Zam

bia

: CP

2001

57 (

2011

-201

5) m

id-t

erm

eva

luat

ion*

Zim

bab

we:

PRRO

200

453

(201

2-20

15)*

CP =

co

untr

y pr

ogra

mm

e

DEV

=

developm

entproject

EMO

P =

emer

genc

y op

erat

ion

PRR

O =

protractedreliefand

recoveryoperation

*

Evaluationcompleted

in2014

**

Evaluationcompleted

in 2

012

***

Evaluationonly

incl

uded

in th

esy

nthe

sis 2

on

nutr

ition

Sour

ce:

OEV

dat

abas

e.

Ref

eren

ce p

erio

d:

For

Ope

vals

, it r

efer

s to

the

dura

tion

of

the

Ope

ratio

n an

d th

e sc

ope

of th

e ev

alua

tion.

Tab

le 1

: Ev

alu

atio

ns

syn

thes

ized

in P

art

1 o

f th

e 2

01

5 A

nn

ual

Eva

luat

ion

Rep

ort

(AER

)


Reflectingthiscontext,emergencyresponseandpreparednesscontinuedtoformamajorfocusofWFP’sevaluationsduring2015.Inadditiontoparticipatingininter-agencyevaluationsofIASC’scollectiveresponsestoTyphoonHaiyaninthePhilippinesandthecrisisinSouthSudan,in2015OEVcompletedevaluationsofWFP’sresponsetotheSyriancrisisandofitsPreparednessandResponseEnhancementProgramme(PREP).ThelatterevaluationformedthelastinOEV’sseriesofstrategicevaluationsonEPR,complementingthosecompletedin2014onWFP’suseofpooledfundsandofthefoodsecuritycluster.ThissectionoftheAERconsidersthesynthesisevaluationreportofthisseries,5 together withEPR-relatedfindingsfromfiveotherevaluationscompletedin2015.6

Together,thesixbaseevaluationreportscoveredWFP’sEPRactivitiesin30countries,includingthetwocollectiveresponsesinthePhilippinesandSouthSudan.Theanalysisrevealedahighdegreeofconsistencyinfindingsandrecommendationsacrossthereports.

WFP’s EPR capacity and position in the humanitarian system

AmajorfindingfromtheevaluationsisthatWFPstrengthened its capacity to implement EPR programmesinlinewithglobalreformeffortssuchastheTransformativeAgenda.Alongwithitsactivecontributionstointer-agencyreformsandprocesses,thiscapacitystrengthenedWFP’spositionintheevolvinghumanitariansystem.

WFP’sinvestmentsinEPRthroughPREPandtheclustersitleadsorco-leadswerehighlyrelevant.Theycontributedtopositiveresults,especiallyinresponsestoLevel3emergencies.Newsystem-wideandcorporateprocessespromotedbyglobalreformssatisfieddemandsfromHeadquarters,donorsandinternationalhumanitarianpartners.StrengthenedcoordinationcapacitiesandWFP’sinvestmentsinoperationalinformationmanagementresultedinmoretimely,consistentanduser-friendlyproductsforcorporateandexternalaudiences.ThereformeffortsalsoenabledWFPtoharnessmorepredictablecontributionsfromtheCentralEmergencyResponseFund.

WFPusedsystem-wideinstruments,suchasglobalandcountry-basedpooledfunds,withpositiveresults.Althoughtheyaccountedforonly4percentofWFP’sdonorcontributionsbetween2009and2013,pooledfundssignificantlycontributedtoWFP’soperations,complementinginternaladvancefinancingmechanisms.TheywerealsoimportantforfundingcommonservicesprovidedbyWFP.

Inaddition,WFPsupportedglobalreformprocessesbysharingexperiences,goodpracticesandtools.ItsEPRpackagefacilitatedthedevelopmentofaglobalprotocol on emergency response preparedness, ledbytheIASCPreparednessandResilienceTaskTeamaspartoftheTransformativeAgenda.WFP’sinnovativeinternaladvancefinancingmechanismswerealsoidentifiedasvaluableexamplesforotherhumanitarianactors.

However,theevaluationsalsoidentifiedsomeshortcomings.Forexample,theEPRsynthesisfoundthatwhilethefocusonLevel3emergenciesimprovedWFP’sresponsetocorporateemergencies,thereweresometimesunintendedconsequencesforchronic,underfundedandlower-levelemergencies.

WFP’sstrongengagementininter-agencyresponseplanning increased coherence, trust and ownership atthecountrylevel,buttheseprocesseswerehighlyresource-intensiveanddidnotresultinsignificantchangestothestrategicapproachorcontentofoperations.Overlydemandingprocessesandlimitedorinconsistentfield-levelacceptanceofreformswerewidelyreportedintheevaluations.

Gender and accountability to affected populations

Thesynthesisreporthighlightedthattheclearcorporatecommitmenttocross-cuttingissuesexpressedbyWFPandthehumanitariansystemwasoftenincorporatedtoonlyalimiteddegreeinoperations.Forexample,althoughpooledfundshadfacilitateduseoftheIASCgendermarker,andgender-disaggregated data were collected during needs assessmentsandmonitoring,thesedevelopmentshadlimitedinfluenceonWFP’sprogramming.Limitedfundingforgender-relatedprogrammingconstrainedWFP’sabilitytoreducegendergaps.

5 See“SynthesisReportoftheEvaluationSeriesofWFP’sEmergencyPreparednessandResponse(2012–2015)”,coveringevaluationsofPREP,thefoodsecuritycluster,thegloballogisticscluster,theuseofpooledfunds,thePhilippines(interagency),emergencyoperationsinMaliandtheSyrianregion,andprotractedreliefandrecoveryoperationsinEthiopia,Madagascar,MozambiqueandTajikistan.

6 CountryportfolioevaluationsintheStateofPalestineandtheUnitedRepublicofTanzania;inter-agencyevaluationinSouthSudan;andoperationevaluationsintheIslamicRepublicofIranandSomalia.


Theextenttowhichbeneficiarieswereconsultedandtheirconcernsaddressedinprogrammingvaried.Forexample,althoughWFPsetupabeneficiaryhotlineinSomaliain2010,itwasnotwidelyknowntolocalsatthetimeoftheevaluation.Ontheotherhand,WFP’sapproachtomainstreamingbeneficiaryconsiderationsintoprogrammesinSouthSudanwascitedasapositiveexample.AnotherpositiveexamplewasnotedintheStateofPalestine,whereWFPlearnedfromandactedonbeneficiaryfeedback.

Programmes using cash-based transfers

Thehumanitariansystemisundergoingadynamicshiftintransfermodalitiesfromin-kindassistancetocash-basedtransfers.7Thistrendwasalsovisiblein–andinpartledby–WFP,whoseuseofcash-basedtransfersinallcontextsrosefrom1percentofbeneficiariesin2009to11percentin2014,representing21percentofWFP’s2014operationalcosts.8

Echoingthe2014AER,whichcoveredtheevaluationofthecashandvoucherspolicy,the2015evaluationsgenerallycommendedWFPforitsincreaseduseofcash-basedtransfers,highlightingthebenefitsofthesemodalitiesandtheirpositivereceptionbybeneficiaries.IntheStateofPalestineforexample,WFP’sinnovativeelectronicvouchersystemservedasamodelforoperationselsewhere.However,thereweregapsinWFP’scapacityforrapidimplementationofcash-basedtransfersinemergencysettings.Severalevaluations–includingthoseintheStateofPalestineandoftheregionalresponsetotheSyriancrisis–identifiedweaknessesintheanalysissupportingtheselectionofmodalities,particularlybetweenvouchersandcash.

Followingthepolicyevaluation,in2015,changesweremadetocorporatesystemsandguidancetoenablemoreprecisecostandoutcomeanalysisofrestrictedandunrestricted,conditionalandunconditionaltransfers;thesechangeswillbeassessedinfutureevaluations.

Non-governmental partners

TheevaluationsfoundthatWFP’srelationshipswithnon-governmentalpartnersvariedwidely.ExamplesofeffectivepartnershipswerenotedforexampleinEthiopiaandtheUnitedRepublicofTanzania;however,inJordan,LebanonandthePhilippines,NGOsperceivedtheirrelationshipswithWFP–andotherUnitedNationsagencies–ascontractualratherthantruepartnerships.

Limitedrisksharing,gapsinpartners’capacityandinsufficientconsultationwithnon-governmentalpartnerscompromised programme implementation in some cases, suchasinIraqandMadagascar,andnecessitatedashifttodirectimplementationbyWFPinSouthSudan.PooledfundsalsodidnotresultinanysignificantchangeinWFP’srelationshipswithitspartners.Incontrast,someclustersledorco-ledbyWFPwerefoundtohavebuilttrustandimprovedrelationshipswithpartners,suchasinBangladeshandMali.Overall,measuresforaddressingtheseshortcomingsthroughPREPwerefoundtobeinadequatefortheimportanceoftheissue,andthereremainsscopeforensuringthatclusterssystematicallyinvolveallparticipantsasequalpartners.

Capacity development of government partners

WFPmadeimportantinvestmentsindevelopingtheEPRcapacitiesofgovernmentagencies,particularlyincountriesenduringfrequentnaturalshocks.Forexample, the logistics cluster supported national disaster managementagenciesinimprovingwarehousingandcontingencyplanninginHaiti,Mozambique,PakistanandseveralPacificislandcountries.WFPalsoachievedpositiveresultsinbuildinglocalcapacitiesthroughvulnerabilityanalysisandmappingactivitiesintheStateofPalestineandtheUnitedRepublicofTanzania,andthroughcontingencyplanningandfood-managementsupportinEthiopia,MozambiqueandTajikistan.

Overallhowever,thestrengtheningofgovernmentpartners’capacitiesdidnotreceivesufficientemphasisandlackedcoherence.Only11percentofPREPfundingwasallocatedtostrengtheningthecapacitiesofnationalauthorities.Despitetheavailabilityofinter-agencyguidance,evaluationsoffoodsecurityandlogisticsclustersreportedthattheclusters’rolesinpreparednessandcapacitydevelopmentwereunclear.Pooledfundswerefoundtohavelittlecomparativeadvantageinfinancingcapacity-developmentactivities.

Human resources

In addition to relationships with partners and the oftenchallengingoperatingcontexts,otherfactorsalsoaffectedWFP’sEPRperformance.WhileallevaluationsemphasizedWFP’sexperiencedandpragmaticstaff,whosecapabilitiesearnedWFPcredibilitywithpartners,humanresourcesremainedamajorconcern.

7SeealsoDoing Cash Differently: How Cash Transfers Can Transform Humanitarian Aid.London:OverseasDevelopmentInstitute,2015.

8 WFPAnnualPerformanceReport2014,latestavailabledata.


InitiativesintroducedbyPREP,suchastheemergencyresponseroster,addressedonlysomeaspectsofreportedchallenges,andsystemicstaffinggapssignificantlyconstrainedtheperformanceofoperations.Globalclustersledorco-ledbyWFPdeployedtheirownstafffromsupportteamsassurgecapacitytofillgapsinthefield,butthisreducedstaff’scapacitytoundertakeimportantcoretasksatthegloballevel.Stafftrainingcourseswereofhighquality,buttheirtargetingandlinkstodeploymentswereidentifiedasareasforimprovement.

Funding and flexibility

MostevaluationsidentifiedfundinggapsaslimitingWFP’sperformance,causingdelays,pipelinebreaks,reduceddeliveryvolumesandrationcuts.InconsistentresourcingforstrategicactivitiessuchasthoseintroducedbyPREPcreatedconcernsaboutsustainability.Dedicatedfundingforthegloballogisticsclusterbroughtbenefitsinpredictabilityandtimeliness,butfundingshortfallsatthecountrylevelledtoinefficiencyandthesuspensionofpartners’programmes,asinSouthSudan.Theeffectivenessofthefoodsecurityclusterwaslimitedbyinconsistentfundingattheglobalandcountrylevels,despitewiderecognition–confirmedbyevaluations–thatinvestmentsincoordinationareworthwhile.

WFP’stwoadvance-financingmechanisms–theImmediateResponseAccountandtheWorkingCapitalFinancingFacility–allowedittorespondandscaleupquickly.Pooledfundswereoftenusedascollateralfor,orrepaymentsof,internaladvances,forwhichceilingsnearlydoubledbetween2012and2014,significantlyimprovingtheavailabilityofrapidandflexiblefunding.In2014,WFPsetupajointrapidresponsemechanismwiththeUnitedNationsChildren’sFund(UNICEF),whichwasactivatedinSouthSudanthefollowingyear,enablingthetwoorganizationstoscaleupquicklytoreachmorethan1millionpeopleinremoteareas.

Lessons from EPR evaluations

i) Withsomeshiftsinemphasis,thefindingsfromtheevaluationscompletedin2015largelyconfirmedthelessonsonEPRprovidedinthe2014AER.First,thepositivebenefitsofinnovationsintroducedthroughtheglobalhumanitarianreformprocess,includingclusters,pooledfundsandadvancefinancingmechanisms,werereaffirmedbythisbroader2015analysis.WFP’sinvestmentsinEPR,includingstrengthenedtechnicalcapacityandadvancefinancingmechanisms,enabledittobuildonitsstrongreputationwithmoreeffectiveandpredictableresponses,especiallytolarge-scale,sudden-onsetemergencies.

ii) OfimportancetotheWorldHumanitarianSummit(WHS),thesecondemerginglessonfrom2014–whichidentifiedtheriskofsystem-wideprocessescrowdingoutoperationallyrelevantactivities–isconfirmed.The2015evaluationsfoundthattheincreasingdemandsofglobalprocessesarelimitingtheimplementationofcorporatereformsandhaveunintendedeffects,leadingtorecurringcallsforscalingbacktheseprocessestoreducedemandsonfieldstaff.

iii)Despitetheconfirmedcontributionofclusterstohumanitarianresponse,thechallengeofadequatelyresourcing cluster coordination remained in 2015: severalevaluationscalledformoreresourcesandleadershipofhumanresourcesforemergencyresponse.WFP’sdedicatedandcompetentstaffdeliveredinever-moredemandingcontexts,butthe2015evaluationsfoundthat,despitetheprogressmade,challengesremain,particularlyregardinglackofcapacityinnewerareassuchastheuseofcash-basedtransfersinemergencies.ConsideringtheimportanceofsuchtransfersforWFP’seffectiveness,allevaluationsrecommendedfurtherinvestmentandattentioninthisrapidlychangingarea.

iv) Buildingonthefourthemerginglessonfromthe2014evaluations,theanalysisforthisreportconfirmedtheneedforcontinuedeffortstoshiftrelationshipswithnon-governmentalpartnersfromcontractualtogenuinepartnerships.Evaluationsin2015identifiedseveralgoodpracticesinWFP’sengagement with national counterparts, particularly in strengthening EPR, contingency planning and foodmanagement.Theseexamplesillustratetheimportanceofstrengtheningnationalsystemsandcapacitiesforemergencypreparedness,tomovebeyondimmediateresponsetowardsdisasterriskreductionandresilience.Manyofthe2015evaluationsrecommendedanincreasedfocusoncapacitydevelopmentinrelationshipswithgovernmentpartners,andamoreconsistentapproachtorelationshipswithnon-governmentalpartners.

v) Echoingthe2014AER,the2015evaluationsfoundthatthereisstillroomforimprovingWFP’sconsiderationofgenderandaccountabilitytoaffectedpeopleinEPR.Thisreportanditsconstituentevaluationshighlightseveralexamplesonwhichtobuild.

vi)The2015synthesisrevealedrecurringcallsformore rigorous analysis and its greater use in decision-making,especiallyrelatedtogender,andaccountabilitytoandfeedbackfrom.affectedpeople.


Nutrition

Internationalinterestinandunderstandingofnutritioncontinuetogrow.NutritionfeaturesintheSDGs,andtheannualGlobalNutritionReport9 promotesglobalunderstandingandaccountabilityonnutrition.NutritionfeaturedmoreprominentlyinWFP’sStrategicPlan2014–2017thaninitspredecessor,buildingonthe2012nutritionpolicy.

Reflectingthisincreasedattention,in2015nutritionwasprominentontheevaluationagendabothwithinWFPandbeyond.AnindependentcomprehensiveevaluationoftheScalingUpNutrition(SUN)movement10 confirmedtherapidmobilizationofabroadbaseofsupportersandrecommendedthatSUNcontinuetobuildonitsstrengths,addressfundamentaldesignweaknessesandstrengthenimplementation.

In2015,OEVcompletedanevaluationofWFP’s2012nutritionpolicyandledajointevaluationofthemulti-agencyRenewedEffortsAgainstChildHungerandUndernutrition(REACH)initiative,whoseSecretariatisinWFP.Inaddition,11ofthe16WFPoperationsandoneofthecountryportfoliosevaluatedin2015includednutritionactivities.Thissectionsynthesizesallevaluationfindingsrelevanttonutrition.

Evaluation of the 2012 nutrition policy

WFP’snutritionpolicyseekstoinfluencealmostallofWFP’soperationswhileadvocatingforasubstantialexpansionofnutritionprogrammes.Itsconceptual

frameworkdistinguishesbetweennutrition-specificinterventions,whichaddressmalnutritiondirectly,andnutrition-sensitiveinterventions,whichaddressitsunderlyingcauses(Figure1).Fivedistinctpolicyareasreflectagrowingawarenessoftheneedtoaddressstuntingaswellasacutemalnutrition,andfocusonpreventionaswellastreatment.ThepolicyadvocatesforworkingthroughpartnershipsandachievinggreatercoherenceamongUnitedNationsagencies.

Theevaluationfoundthepolicyoverallbothrelevantand timely: it was clearly written and well understood acrossWFP,anditsconceptualframeworkwasrelevantanddurable.ItwasconsistentwithWFP’smandateandcoherent with its strategic plans, rightly emphasizing theimportanceofmulti-sectorandmulti-stakeholderapproaches and partnerships to address chronic and acutemalnutrition,andmakingcapacitydevelopmentofgovernmentsandpartnersaspecificobjective.

Thepolicydrewonemergingevidenceonundernutrition,includingthesignificanceofstunting.However,theevaluationconcludedthatadequatelysupportingsomeofthepolicy’sprescriptionsandrecommendationswithastrongevidencebaseremainsachallenge.Thepolicyomittedimportantemergingissuessuchasovernutrition,anditstreatmentofgenderwassuperficial.Ithadapracticalorientation,butitsimpliedtargetsforexpandingWFP’snutritionprogrammingwereoverambitious.Itsemphasisonsupplementaryfeeding,alongwithinsufficientattentiontothecomplementaryfactorsrecognizedinitsconceptualframework,reinforcedperceptionsofthepolicy’sexcessivefocusonproduct-basedsolutions.

9InternationalFoodPolicyResearchInstitute.2015.Global Nutrition Report 2015: Actions and accountability to advance nutrition and sustainable development.Washington,DC.

10 MokoroLimited.2015.IndependentComprehensiveEvaluationoftheScalingUpNutritionMovement:FinalReport–MainReportandAnnexes.Oxford,UK.

Nutrition activitiesfocused on vulnerable

groups includingyoung children,

pregnant and lactatingwomen,

and people livingand HIV

Treatment of moderate acute malnutrition

WFP Nutrition Strategy

Enabling environment: Technical assistance and advocacy with governments and other stakeholders

Food-assistance-for-workFood-assistance-for-training Others

Ensure other programmes contribute to improved nutrition outcomes

General food distribution School feeding

Prevention of acute malnutrition

Prevention of chronic malnutrition

1

4

5

2 3

Addressing micronutrient deficiencies: i) among vulnerable groups, especially to save lives in emergencies;

ii) for the general population through food fortification

Figure 1. Nutrition Policy Conceptual Framework


Initial results included upgraded nutrition specificationsforthecommoditiesWFPprocures;however,nutritionprogrammeswerenotscaleduptotheextentenvisagedbythepolicy(Figure2).FoodremainedthedominantmodalityforWFP’snutritioninterventions,withlimiteduseofcash-basedtransfersreflectingthelimitedglobalevidencebasefornutritionoutcomesrelatedtothismodality.Stunting

preventionprogrammesgrewrapidlyfromalowbase,butbeneficiarynumbersinareassuchastreatmentandpreventionofacutemalnutritiondidnotincrease.

WFPhasbeenproactiveintheSUNmovementandotherpartnerships,butprogresstowardsgreatercoherenceamong United Nations agencies, although not entirely withinWFP’scontrol,hasbeenregrettablyslow.

Theevaluationrecommended:i)revising,updatingandfurtherdevelopingthenutritionpolicy,andlinkingittoWFP’snextStrategicPlan;ii)improvingpolicy guidance and dissemination, particularly in criticalareasidentifiedintheevaluation,includinggenderandnutrition-sensitiveprogramming;iii)improvingmonitoringandoperationalresearch;iv)developingcapacitywithinWFP;v)continuingWFP’scommitmenttomulti-sectorpartnerships;andvi)addressing systemic issues that constrain resource availability.

Evaluation of REACH

TheUnitedNationsREACHinitiativeaimstosupportthegovernanceofcountry-levelnutritionresponses.ItbringstogetherWFP,theFoodandAgricultureOrganizationoftheUnitedNations(FAO),theWorldHealthOrganization(WHO),UNICEFandtheInternationalFundforAgriculturalDevelopment(IFAD)tostrengthencountry-levelnutritionplanningandpromote stronger nutrition coordination among these UnitedNationsagencies.REACHbeganin2008andwasexpandedin2010.Thisevaluation,covering2011–2015,focusedontheroleoftheREACHSecretariat,hostedbyWFP,anditsresultsinBangladesh,Ghana,Mali,Mozambique,Nepal,Rwanda,UgandaandtheUnitedRepublicofTanzania.

Figure 2. Actual beneficiaries in nutrition policy areas 1–3, 2010–2014

Area 1Treatment of MAM

Area 2Prevention of acute malnutrition

Area 3Prevention of stunting

Areas 2 & 3

14

12

10

8

6

4

2

02010 20122011 2013 2014

Ben

efici

arie

s ch

ildre

n un

der

5, m

illio

ns)

Source:Nutritionpolicyevaluationreport.MAM=moderateacutemalnutrition.Theanalysiswaslimitedtochildrenunder5.Area4beneficiariesarecapturedinAreas1–3.ForArea5,theevaluationwasunabletodistinguishbetweenpotentialandactualnutrition-sensitiveinterventions.


Figure 3. REACH theory of change

Source:REACHevaluationreport.

1

2

3

4

If we address these issues …

Little consensus on the causes of undernutrition

Little transparency about multitude of activities

Limited political commitment

Weak coordination of gov'ts with UN agencies and other stakeholders

Nutrition is not seen as a multi-sectoral issue

Poor capacity development

Accountability and responsibility are undervalued

with these strategies …

then we can achieve … this impact

REACH outcomes

Increase awareness and consensus of stakeholders

Strengthened national policies and programmes

Increased human and institutional capacity

Increased effectiveness and accountability

Nutritional impact and coverage

Governance impact

Better management and governance of a multi-sectoral approach to nutrition

Political support to fund programs

Improved nutrition

for women and children

TheevaluationfoundthatREACHfittedwellwiththeprioritiesoftheinternationalnutritionagendaandofpartneragencies,includinggenderandequityobjectives.Atthecountrylevel,itcomplementedtheSUNmovement.However,consideringthecomplexityoftheinstitutionalenvironment,itstimeframeswereoverambitious;itstheoryofchangeunderestimatedtheimportanceofpoliticalcommitmentandthepoliticaleconomyofinter-agencycooperation;anditsdesignwasunder-resourcedforthescaleofintendedchange.

REACHmadesubstantialprogressintwoofitsfouroutcomeareas:Outcome1–Increasedawarenessandcommitment;andOutcome2–Strengtheningnationalnutritionpoliciesandprogrammes.TherewaslessprogresstowardsOutcome3–CapacitydevelopmentandOutcome4–Increasingeffectivenessandaccountability.Theinitiative’shigh-levelplanning,toolsandanalysishighlightedequityandgenderissuesinnutrition,butimplementationintheseareaswasslow.Country-levelresultswereachievedatlowercostthanbudgeted,allowingtimelinestobeextended.

Evenbeforetheevaluationwascomplete,ithadbeendecidedthatREACHwouldbecomethecoordinatingbodyfortheUnitedNationsSUNNetwork.Takingthisdecisionintoaccount,theevaluation’srecommendationscentredon:i)maintainingthefocusofREACHasaneutralfacilitatorofcountry-levelnutritiongovernance;ii)expandingtimeframesforengagement;iii)strengtheningincentivesforagencies’contributionstotheinitiative;iv)redesigningthetheoryofchange;v)aligningREACHwithothertechnicalsupportinitiativesinnutrition;andvi)strengtheningsupportforgenderandequityinnutrition.

Nutrition findings from other evaluations

Nutrition-relatedfindingsfrom2015operationevaluationscorroboratedthenutritionpolicyevaluationfindingsregardingtheunevenpaceofroll-outofthepolicy’sconceptsandapproaches.Thereweredelaysinimplementingnutrition-sensitiveprogrammingandbuildingtherequiredevidencebasethroughaccuratemonitoringandimpactmeasurement.Inafewcases–mostnotablyinBangladesh–countryofficesclearlyappliedthepolicyinanalysisandprogrammedesign.Inothers,theevaluators’analysisdrewexplicitlyonthepolicy’sconcepts,butinseveralcases,thenutritionpolicyhadlittleapparentinfluenceonprogrammesorontheapproachtotheirevaluation.

SeveralfindingsfromoperationevaluationsconcerningWFP’soveralleffectivenessarealsorelevanttoitsnutritionefforts.Forexample,operationalambitionsfrequentlyoutstrippedWFP’shumanresourcecapacity;theskillsandapproachesrequiredwerenotalwaysmatchedbyWFP’stechnicalandhumancapacities;and,whenfundingwasconstrained,WFPcouldbemoreeffectivebyconcentratingitsactivitiesinfewerareasoverasustainedperiod.

Reflectingprogrammedesign,nutritionwasnotasignificantthemeintheevaluationoftheSyrianresponse.TheTanzaniancountryprogrammeevaluation(CPE)foundthatWFPhadfollowedpolicyguidelines in addressing stunting and micronutrient deficienciesinrefugeecamps,andhadengagedconstructivelyinnationalnutritionpolicydiscussions,butcouldhavebeenmoreproactive.Consideringthe


11 UnitedNationsGlobalNutritionAgendaversion1.0,2015.TheUnitedNationsNutritionNetworkcomprisesWFP,IFAD,FAO,UNICEFandWHO.

relativelylowlevelsofundernutrition,theincreasingobesityandthemicronutrientdeficienciesintheStateofPalestine,itsCPEaffirmedWFP’sdecisionstoadjustthecompositionoffoodrationsandworkwithgovernmentandotherpartnersinsteadofprovidingspecialnutritionproducts.

Lessons from nutrition evaluations

i) WFP’s role and comparative advantage in nutrition. Evaluationfindingssuggesttheneedforamorerealisticviewofavailableresourcesfornutritionprogrammes,particularlypreventionactivities,reflectingtheneedforclearevidenceofWFP’scomparativeadvantage.Incoordinationwithpartners,betterarticulationoftheroleofproduct-basedapproachesisneeded,alongwithevidenceofWFP’saddedvalueineachnutritionfocusarea,inbothemergencyanddevelopmentcontexts.

ii) United Nations collaboration.TheevaluationsofWFP’snutritionpolicyandREACHechoedfindingsregardingUnitedNationscollaborationfromtheindependentevaluationofSUN.Theyallunderscoredtheneedfor:i)commitmenttotheUnitedNationsGlobalNutritionAgenda11andSUNNetworkfromthehighestlevelsofUnitedNationsagencies;andii)aclearmandateandstrongeraccountabilitymechanismssetbythegoverningbodiesofUnitedNationsagenciestostrengthenincentivesforinter-agencycooperationandcoordination.

iii) Operationalizing the nutrition policy. WFP’snutritionpolicyisastrongplatform,butshouldbestrengthenedinseveralareas:

a)WFPisappropriatelymakingnutritionahighercorporatepriority,whichneedstobereflectedinitsnextStrategicPlan,infullcollaborationwith other agencies and in line with the United NationsGlobalNutritionAgenda.

b)ThereisscopeforexpandingWFP’snutritioncapacityfurtherbyequippingcountryoffices

andregionalbureauxwithappropriatestaffandskillstodeliverhigh-qualityprogrammemanagement,effectivenationalpolicyadvocacyandsupportfornationalnutritionstrategyandplanning.

c)High-qualityguidanceshouldbemademoreaccessibletostaff.Significantgapsinnutritionguidanceshouldbefilled,especiallyrelatedtonutrition-sensitiveapproaches,the“doubleburden”,outcomemonitoring,andcoherenceamongnutrition,genderandcash-basedtransferactivities.

iv) Strengthening the evidence base for WFP’s work in nutrition:

a)Providingevidenceoftheeffectiveness-in-practiceofsomeofWFP’snutritionapproachesremainsachallenge.Thereisscopefordevelopingacomprehensiveoperationalresearch strategy in partnership with international and national nutrition research institutions.

b)ProgressonutilizingthenutritionoutcomeindicatorsincludedintheStrategicResultsFrameworkisatanearlystage.Asinotherareas,decision-makersarecalledontoprioritizeconsistentlytheneedforevidenceofimpact.ThedevelopmentofguidanceappropriateforWFPprogrammesandnationalmonitoringandevaluationsystemswillhelp.

v) Addressing systemic resource constraints. Inadditiontostrengtheningtheevidencebaseforcredibleadvocacy–particularlyonpreventionofmalnutrition–andtheinternalandexternalcoherenceofnutritioninterventions,othereffortstoaddressWFP’sresourceconstraintsthatarerelevantfornutritioninclude:i)theFinancialFrameworkReviewandrelatedmeasurestoincreasetheflexibilityandpredictabilityoffunding;andii)improvedfinancialandoutcomemonitoringsystemstoenablebetteranalysisofcost-effectivenessandresults-basedbudgetingandreporting.


Country-Specific Evaluations

Context

Thissectionidentifiesadditionalfindingsandlessonssynthesizedfromcountry-specificevaluationscompletedin2015.12TheevaluationsreflectedthediversityofWFP’soperations,spanningallitsactivitytypesimplementedinarangeofenvironmental,political and economic contexts, including escalating conflicts,suddenshocks,protractedcrises,andrelativelystableenvironments.ThecontextsdescribedinthetwoCPEs–theStateofPalestineandtheUnitedRepublicofTanzania–werenotablydissimilar,exceptforthelongevityofWFP’sengagement.

Aboveall,theoperationsandportfoliosevaluatedwereexceptionallydiverseinscale.Forexample,WFP’sregionalresponsetotheSyriancrisishadabudgetofUSD4.7billionfor2011–2014,comparedwithabudget

ofUSD293millionfortheUnitedRepublicofTanzaniaportfolioforthesameperiod.Ofthe16individualoperationsevaluated,25percenthadbudgetsoflessthanUSD17million.13Figure4illustratestherangeofcontextsandscalesofWFP’sresponses.Thisdiversitymakesthecommonqualitativefindingsandlessonsemergingfromthesynthesisallthemoresignificant.

Aswellasthesheernumberandscaleofcrisesin2015(seeIntroduction),thechallengesWFPfacedhavearguablybecomeincreasinglycomplex,concerningforinstance:i)understandingbeneficiaryneeds;ii)seekingcoherenceamonganincreasinglywiderangeofactors;iii)assuringthatshort-termemergencyactivitiestakelonger-termandinterconnectedproblemsintoaccount–“connectedness”;iv)managingtransitionsbetweenimmediateemergencyresponse, protracted crises and long-term social protection;andv)thedifferentapproachesappropriateinlow-andmiddle-incomecontexts.

12 TheSynthesisReportofOperationsEvaluationswaspresentedtotheBoardinNovember2015andincludesoperationevaluationsconductedbetweenJuly2014andJune2015.

13 Ofthe16operationsevaluationsincludedinthesynthesis,9aremid-termevaluations.

14Sourceofdefinition:OrganizationforEconomicCo-operationandDevelopment(OECD)2015(2014WorldBank,AfricanDevelopmentBank,AsianDevelopmentBankharmonizedlist);2014FragileStatesIndex,FundforPeace.

15 Source:WorldBank.

16 Source:WorldRiskIndex2013.

Figure 4. Contextual characteristics of complex evaluations and operations evaluations

Sta

te o

f Pa

lest

ine

CPE

Uni

ted

Rep

ublic

of

Tanz

ania

CPE

Syr

ian

resp

onse

Arm

enia

Ban

glad

esh

Ecua

dor

Gha

na

Gui

nea-

Biss

au

Hai

ti

Hon

dura

s

Iran

, Isl

amic

Rep

ublic

of

Mal

awi

Mal

i

Moz

ambi

que

Paki

stan

Som

alia

Tuni

sia

Zam

bia

Zim

babw

e

Fragilestate14

Middle-income country15

Affected by recurrent natural disasters16

WFP office size

Partial Country Office key: • = Small; •• = Medium; ••• = Large

Complex evaluations Operation Evaluations

••• •• ••• • ••• • • • ••• •• • •• ••• •• ••• ••• • • •••

“Large”combinesWFP’slargeandverylargeoffices;“small”combinessmallandverysmalloffices.


Strategic positioning and partnerships

Whilecontinuingtoprogressfromfoodaidtofoodassistance as well as responding to escalating and multipleprotractedcrises,WFPalsohadtotakeaccountof:i)thechanginginternationallandscapeandobjectivesreflectedintheglobalhumanitariananddevelopmentdialogue;ii)globalhumanitarianreforms;iii)demandsforgreatertransparency,accountabilitytoaffectedpopulationsandevidenceofeffectivenessatgoodvalue;iv)emergingpracticesinhumanitarianaction,influencedbyrapidtechnologicaladvances;andv)evolvingknowledgeinfieldssuchasnutrition.

BroadlytheevaluationsconfirmedthatWFPis continuing its adaptation to these changing internationalneedsandexpectations.Generally,operationswerefoundappropriatetobeneficiaries’ needs,butwithsomequalifications.Inseveralcases,WFP’sprogrammeswerenotaswellalignedwithbeneficiaryneedsduringimplementationastheywereatthedesignstage.Thisreducedalignmentreflectsthechangingcontexts,over-optimisticinitialassumptionsaboutfundingand/orfailuretousemonitoringtorefinetargeting.

Whereassessed,WFP’sworkwasbroadlyalignedwithhumanitarianprinciples,butsomedifficultcompromiseswerenoted.Forexample,whileWFPreportedthatitsapproachmaximizedaccesstoaffectedpopulations,somebeneficiariesandpartnersperceivedthatWFP’srelationshipwiththeSyrianGovernmentundermineditsreputationforimpartialityandneutrality.IntheStateofPalestine,debateovertheperceiveddichotomybetweenhumanitarianresponseanddevelopmentreflectedwiderinstitutionalchallengesforWFPasitseekstoplayamoreenablingrole.

ParticularlyasWFPcontinuesitsshiftfromimplementertoenabler,itschoiceof,andrelationshipswith,partnersarecriticaltomeetingitsStrategicObjectives.The2015evaluationsshowedthatWFPrecognizesthis,althoughgreaterdistinctionisrequiredin its partnering approach, appropriate to its many differentrelationshipswithimplementingagencies,state actors, United Nations and other international agencies,andtheprivatesector.

Forexample,mostevaluationscommentedpositivelyonthebroadcoherencewithnationalpoliciesandframeworks.Regardingpartnershipswithgovernments,theevaluationsfoundoverallstrong

intentwhereopportunitiesexisted,albeitwithsomeover-optimisticassumptionsaboutpartners’technicalandfinancialcapacities,whichledtoover-ambitiousprogramming.Severalevaluationsfoundthatmoreprogresscouldhavebeenmadetowardsalignment with existing national social protection systems–notingthepotentialforsustainablecapacitydevelopmentofnationalinstitutionsandforWFP’sinfluenceonnationalpolicies.

Regarding inter-agency partnerships, alignment with countries’UnitedNationsDevelopmentAssistanceFrameworks(UNDAFs)wasgenerallystrong.However,inthecountryportfoliooperatingwithinaDeliveringasOneframework,theeffortsinvolvedwerefrequentlyreportedtooutweighthebenefits.

Somegoodexamplesofprivate-sectorpartnershipswerehighlighted,includingtheinnovativepartnershipforelectronicvouchersintheSyrianregionalresponse.InboththisresponseandtheStateofPalestineportfolio,therewasincreasedattentiontothesecondaryeconomicbenefitsarisingfromprivate-sectorinvolvement.

Analysis and design

Substantialanalyticalworkfedintothedesignofoperationsandportfolios,with12ofthe16operationsevaluatedshowingastronganalyticalbase.However,evaluationsalsohighlightedtheneedforcontinuedanalysistosupporttherefinementandadaptationofactivitiesovertime,andprogrammedesignsfrequentlyfailedtoaddressthetrade-offsnecessitatedbyunderfunding,whichcouldhavebeenanticipated.Forinstance,whileWFP’sunderstandingofhumanitarianneedsenabledrapidscaleupinresponsetotheSyriancrisis–withbroadlyappropriateprogrammesattheoutset–analysesofmarkets,gender,foodinsecurity,conflictdynamicsandcost-effectivenesswereinsufficienttoinformdecisionsregardingtargetingandtransfermodalities.

Ofthesixcasesassessedinthecontextofcountrystrategies,onlytwousedtheseanalysestoinformoperation design; in some cases and sectors, implementation did not match strategic design, orstrategicre-positioningwasslow,forreasonsthatwerenotalwaysevident.Selectionofthebesttransfermodalityisincreasinglyimportant,andrapidlyevolvingtechnologyhasbroadenedtheoptions.Echoingfindingsfrompreviousyears,manyevaluationsfoundtheanalysisunderpinning


theselectionoftransfermodalitiesunsatisfactory–particularlyfordecidingbetweencashandvouchers.Recentlyupdatedguidanceandsupportforcash-basedtransfersareexpectedtobereflectedinfutureevaluationfindings.

Performance and results

The2015evaluationsfoundgenerallystrongtechnicalperformance,withrenewedacknowledgementofWFP’sagilityandstrengthinlogisticsandofitsleadingroleininternationalhumanitarianresponse.TheyalsofoundinnovationandadaptationinresponsetoevolvingcontextsandtothedirectionofWFP’ssuccessivestrategicplans,buttheyrecognizedthattheseadaptations–fromfoodaidtofoodassistanceandfromimplementertoenabler–areworksinprogress.

AlthoughWFP’sresponsetotheSyriancrisiscouldhavebeenstronger,theevaluationfoundthatoverall,WFP’ssupportwastimelyandresponsiveinanoperationthatwasmassivelyscaledupasthecrisisevolved.WFPwaseffectiveinprotectingrefugees’foodsecurity,anddeliveredassistanceinwaysthathadcollateraleconomicbenefits.IntheStateofPalestine,theinnovativeelectronicvouchermodalitywasrapidlyscaledup,enablingmorebeneficiariestobereached than originally targeted, and with greater cost-effectivenessthanin-kindassistance.

Attheoutputlevel,theoperationandcountryportfolioevaluationsillustratedthatthelargestshortfallsincoverageachievedagainstthatplannedwereinnutritionandfoodassistance-for-assetsactivities,mainlybecauseoffundingshortfalls.Althoughgeneraldistributionwasmorelikelytoreachtargetnumbersofbeneficiaries,fundingshortfallsandpipelinebreaksmeantthatbeneficiariesoftendidnotreceiveasmuchassistanceasplanned.Evenwhenbeneficiarynumbersmetorexceededtargets,transferswereoftenreducedinfrequency,quantityorboth.

Theevaluationsfoundmixedperformanceincapacitydevelopment.PositiveexamplesconsistentwithWFP’sshiftfromimplementertoenablerwereidentifiedinschoolfeeding,disasterriskreduction,nationalcapacityforsocialprotectionintheStateofPalestine,andvulnerabilityanalysisandmappingintheUnitedRepublicofTanzania.However,5ofthe16operationevaluationsidentifiedpiecemealapproachesratherthan systems-oriented models, and some opportunities

weremissedforengaginginandinfluencingnationalpoliciesand/orintegratingWFP’sportfoliointothese.Confusionbetweencapacitydevelopmentandcapacityaugmentationcontinued,andpossibilitiesforhand-overwerefrequentlyconstrainedbygovernments’limitedtechnicalandfinancialcapacitiesorthenationalcontext.Under-reportingandunder-representationcontinuedtobesignificantissues.

Theoperationsandcountryportfoliosevaluatedin2015alsoyieldedmixedresultsongender.Fourofthe16operationshadgender-sensitivedesigns,whilegenderanalysisandsubsequentmonitoringinotherswereonlysuperficiallyconsidered.However,13oftheoperationevaluationsandboththeCPEsfoundevidenceofcountryofficesmakingeffortstotacklegenderbarriersandempowerwomen,whichwerenotalwayscapturedinreportingsystems.

Lessons from country-specific evaluations

i) On innovation.AmongWFP’smoststrikinginnovationsisitsuseofelectronicvouchers,especiallyinemergencies,includingitsfacilitationofplatformsforjointusebymultipleagenciestomeetarangeofbeneficiaryneeds.ThedevelopmentanduseofelectronicvouchersintheStateofPalestinewasseenasoneoftheportfolio’sgreatestachievements,andbecameamodelforWFPoperationselsewhere,includingintheSyrianresponse.Echoingsimilarlessonsfromlastyear’sAERconcerninginnovationmanagement,however,innovationmustbeunderpinnedbystronganalysisandmonitoring.StrengtheningWFP’smonitoringwillenablebetterunderstandingandcrediblecommunicationofthecostsandbenefitsofdifferentdesigns,approachesandmodalities.

ii) Analysis and monitoring.AlmostalltheevaluationsstressedtheneedforWFPto strengthen monitoring and analysis to improvemodalitydesignandtargetingduringimplementationaswellastoinforminitialchoices.Boththe2015synthesisofoperationevaluationsandlastyear’sAERnotedthatinWFP’sincreasingly complex and crowded operating environment,thecapacitytoprovideevidenceofoutcomesandcost-effectivenessanalysisroutinelyisofever-greaterimportanceforsecuringstakeholders’confidence.


iii) Stronger links to national social protection systems.WFPactivitiesincreasinglyresonatewithandformpartofwidernationalsocialprotectionsystems.WFPshouldproactivelyandmoreconsistentlyengageinthedevelopmentofthesesystems,inlinewithitscontinuingshiftfromfoodassistanceimplementertoenablerofsustainablehungersolutions.

iv) Funding constraints. WFPisoftenconstrainedbythetypeandvolumeoffundingitreceives.Inmanycases,transfermodalitieswereboundedbydonorpreferences.17Theshifttomorestrategicandsustainablehungersolutionsrequirescontinuedefforttoachievemoreflexibleandpredictablefundingandfinancialallocations.Atthesametime,fundingconstraintscanoftenbeanticipatedandshouldbetakenintoaccountmoreexplicitlywhenplanningthescaleandtargetingofWFP’soperations.

v) From implementer to enabler. Theevaluationsfoundthatwhileprogressinthistransitioncontinued,limitationspersisted.The2015evaluationsindicatedthatsuccessliesinsmart,strategicchoicesofnationalpartnersforprogrammedeliveryandlong-termcommitment,linkedtomorecomprehensiveassessment and systematic approaches to capacity developmentbuiltonsynergies.

vi) Corporate systems and support.WFPhasdemonstrated its agility in scaling up during rapid-onsetstagesofcrises.FurtherdevelopmentofWFP’scorporatesystems,alongwithgreatersupport and guidance, would enhance country offices’managementoftransitioninprotractedand rapidly altering contexts, and engagement innationalcapacitydevelopmentinmorestablecontexts.Thisdevelopmentrequires:i)moreflexiblefunding,asimpliedbytheongoingFinancialFrameworkReview;ii)skillsforundertakingWFP’snewroles;andiii)moresystematic guidance and support in areas where WFPisseekingtostrengthenorinnovate,particularlymonitoringandanalysisofcost-effectiveness,gender,socialprotectionandcapacitydevelopment,and“enabling”approachesmoregenerally.

17Theoperationevaluationsynthesisnotedthat:“Dependentonvoluntarycontributions,WFPfounditselfvulnerabletodonorpreferencesinthese16operations.Earmarkingoffunds,short-termcommitmentsandfragmentedcontributionsrestricteditsroomtomanoeuvreandlimiteditsscopetostrategizeforthefuture.Country-levelreorientationhasoccurreddespite,ratherthanbecauseof,resourceflows.”


New Evaluation Policy DevelopmentofWFP’sevaluationfunctionin2015wasframedbythefindingsandrecommendationsoftwosignificantreviewsintheprecedingyear:thepeerreviewofWFP’sevaluationfunctioncarriedoutbytheOrganisationforEconomicCo-operationandDevelopment’sDevelopmentAssistanceCommittee(OECD-DAC);18andtheJointInspectionUnit’sanalysisofevaluationfunctionsacrosstheUnitedNationssystem.FollowingtheBoard’sendorsementofmanagement’sresponsetothepeerreviewrecommendationsinNovember2014,anewevaluationpolicywasapprovedbytheBoardinNovember2015.

Coincidingwiththe2015InternationalYearofEvaluation,progressinUnitedNationsandIASCsystem-widearrangementsforevaluation,andpreparationsforthe2030Agenda,thenewpolicyplacesevaluationatthecoreofWFP’scontinuedorganizationalstrengtheningeffortstoachieveitsStrategicObjectivesandmaximizeitscontributiontotheSDGs.

The2016–2021evaluationpolicy19 aims to strengthen WFP’scontributiontoendingglobalhungerbyembeddingevaluationthinkinhaviourandsystemsintoWFP’scultureofaccountabilityandlearning,throughensuringthatevaluationresultsareconsistentlyandcomprehensivelyincorporatedintoWFP’spolicies,strategiesandprogrammes.

Reflectingthe2030Agenda’semphasisonglobalandnationalpartnerships,thenewpolicyestablishesWFP’sevaluationfunctionasacombinedcentralizedanddemand-leddecentralizedevaluationmodel–asignificantdeparturefromthepreviouspolicy.Itaimsto:i)respondtorisingstakeholderdemandforevidenceofandaccountabilityforresultsatthecountrylevel;andii)underpinWFP’spartnershipsandevidence-basedcontributionstonationalpolicies,systemsandcapacitiesforachievingtheSDGs.

Thepolicyincludesacomprehensivenormativeandaccountabilityframeworkcomprisingcoveragenorms,principles,standards,rolesandresponsibilitiesforevaluationacrossWFP.AlongsideitstargetsforresourcingthesubstantialincreasesrequiredinthenumberofWFP’sevaluations,thepolicyalsoenvisages:augmentationandcapacitydevelopmentofWFPstaff;advisorysupporttoenhancethequalityofevaluations;knowledgemanagement;andreportingsystems.TheDirectorofEvaluationprovidesgloballeadershipof,setsstandardsfor,overseesandreportsontheentireevaluationfunction.

RecognizingthescaleofchangeimpliedbythisaugmentationofWFP’sevaluationfunction,thepolicyadopts a phased approach to organizational change, foreseeinggradualimplementationbetween2016and2021.Thepolicyissupportedby:i)anEvaluationCharter(Annex1),whichelaboratestheevaluationfunction’smandate, authorities and institutional arrangements; and ii)aninternalevaluationstrategytoguidethepolicy’sphasedimplementation.Together,theevaluationpolicy,charterandstrategyformthebasisforembeddingevaluationacrossWFPoverthecomingyears.Asafirststep,theExecutiveDirectoropenedWFP’sfirstglobalevaluationmeeting,heldinlate2015todisseminatethepolicyandlaunchWFP’sglobalevaluationnetwork.

OEV Performance to plan in 2015

ThissectionreportsonOEV’sperformancetoplan,aspresentedinWFP’sManagementPlan2015–2017.Itoutlinesperformanceon:i)theconductandcoverageoftheplannedprogrammeofcomplexevaluationsandthetemporaryseriesofoperationevaluationsmanagedbyOEV;ii)establishmentofWFP’sdecentralizedevaluationfunction;iii)evaluationdisseminationanduse;iv)engagementintheinternationalevaluationsystem;andv)theuseofhumanandfinancialresourcesfortheyear,toconcludereportingonOEV’smanagementresults.

Part2.WFP’sEvaluationFunction

18 Availableat:http://documents.wfp.org/stellent/groups/public/documents/reports/wfp264679.pdf.

19WFP/EB.2/2015/4-A/Rev.1


Evaluations and coverage

The2015programmeofevaluationscompletedbyOEVsustainedthesignificantadvancesinevaluationcoveragebegunin2014(Figure5).In2015,27countrieswerecoveredbyOEVevaluations–

fewerthanthepeakof33coveredin2014,butstillrepresentingahealthyincreasesincelaunchoftheoperationevaluationseriesin2013,with20in2013and21in2012.

20 Evaluationteamsofcentralizedevaluationscompletedin2015.

Figure 5. 2015 Countries covered by centralized evaluations completed in 2015, by region

RBB - Bangkok Regional BureauRBC - Cairo Regional BureauRBD - Dakar Regional Bureau RBJ - Johannesburg Regional Bureau RBN - Nairobi Regional Bureau RBP - Panama Regional Bureau

0 5 10 15 20

Number of countries

Countries covered by evaluation

Countries in the regionRBP

RBN

RBJ

RBD

RBC

RBB Sources:2015OEVdatabaseand2015programmeofworkasof24January2016.

Figure6showsthatasinpreviousyears,thereweresignificantdisparitiesintheregionaldistributionofevaluations.Thisispartlybecause,whenselectingcountries,greaterweightwasgiventotherelevanceof

thetopic,forglobalevaluations,andtothetimelinessofevaluationtoinformdecision-making,forcountry-specificevaluations,thantogeographicaldistribution.

WFPcorporateemergencyresponseevaluation=Syrianregionalcrisis;IAHEcorporateemergencyresponseevaluation=SouthSudan.

Figure 6. Countries visited by evaluation teams20


Figure7illustratesthecoverageofoperationevaluationscompletedin2015.Althoughtheselectionoftheseevaluationstookintoaccountthedistribution

ofWFP’sprogrammeofwork,heretoocoveragein2015alonewasnotevenlyrepresentativeofprogrammecategoriesorregions.21

Figure 7. Completed operation evaluations and WFP operations by programme category and region, 2015

a. Operation evaluations by programme category b. WFP operations by programme category

CP/DEV 70%

PRRO 30%

EMOP 0% SO 0%

c. Operation evaluations by regional bureau d. WFP operations by regional bureau

SO 18%

CP/DEV 38%EMOP 6%

PRRO 38%

RBP 10% RBB 10%

RBC 30%

RBN 10%

RBJ 20%

RBD 20%

RBP 8% RBB 17%

RBN 17%

RBD 26%

RBC 15%

RBJ 17%

Calculatedintermsofnumberofoperations.Datacoverthetenoperationevaluationscompletedin2015,andWFP’s2015programmeofwork,excludingLevel3emergencies.Sources:OEVinternaldatabaseandWFPProgrammeofWorkasofFebruary2016.RBB–BangkokRegionalBureau;RBC–CairoRegionalBureau;RBD–DakarRegionalBureau;RBJ–JohannesburgRegionalBureau;RBN–NairobiRegionalBureau;RBP–PanamaRegionalBureau

WithUSD2.4millioninnon-staffProgrammeSupportandAdministrative(PSA)funding,tencomplexevaluationsofmultipleoperations,policiesandstrategies were completed and 12 new ones started in 2015.Maintainingthislevelofoutputwithareducedinvestmentwasmadepossiblebyefficiencygainsfromconductingmoreevaluationsthroughpartnerships(seeparagraphs75and76).ThefundsthusliberatedwerededicatedtoenhancingotheraspectsofWFP’sevaluationfunction.22Theseriesofoperationevaluations,whichwerefundedlargelyfromprojectresources,continued:10werecompletedand15started.

Table2showsperformanceratesagainstplansforthevariousevaluationtypesmanagedbyOEV.Inall,20evaluationswerecompleted;atten,thenumberofcomplexevaluationscompletedwas91percentofthoseplanned,whilethetenoperationevaluationscompletedwerethreemorethanplanned,bringingtheoverallcompletion23rateto111percent.WithonemoreCPEbegunin2015thanoriginallyplanned,theoverallstart24ratewas104percent.

21 Coverageoverthelifeofthewholeseriesofoperationevaluationsismorerepresentativeofgeographicdistributionandprogrammecategories.

22 AccordingtoWFP’s2008EvaluationPolicy(applicablethrough2015),paragraph.25,“theDirectoroftheOfficeofEvaluationhasfulldiscretioninestablishingtheevaluationworkprogrammeinlinewiththeEvaluationPolicyandfullauthorityoverthemanagementofhumanandfinancialresourcesforevaluation”.

23 “Completion”occurswhenthefinalevaluationreportisapprovedbytheDirectorofOEV.ReportsapprovedattheendofthecalendaryearareusuallypresentedatthefirstBoardsessionofthefollowingyear.

24Anevaluationstartswhenbudgetexpenditurecommences.


Tota

l ev

alua

tions

Sing

le

oper

atio

n ev

alua

tions

(tem

pora

ry)

Sub-

Tota

l (c

ore

prog

ram

me)

Hum

anita

rian

emer

genc

y Le

vel 3

Synt

hese

s

Glob

al

eval

uatio

ns(p

olicy

and

st

rate

gic)

Impa

ct

eval

uatio

ns

Coun

try

portf

olio

ev

alua

tions

Planned to be completed in 2015

Actual completions 2015

Completion rate

Planned to start 2015

Unplanned new actual starts

Total actual starts 2015

Start rate

Star

ts

2

2

100%

3

1

4

133%

0

0

-

4

-

4

100%

3

3

100%

1

-

1

100%

3

2

67%

0

-

0

0%

3

3

100%

3

-

3

100%

11

10

91%

11

-

12

109%

7

10

143%

15

-

15

100%

18

20

111%

26

-

27

104%

Table 2. Implementation of Evaluation Work Plan 2015Co

mpl

etio

ns

25 Evaluabilityassessmentsassesswhetherobjectivesareadequatelydefinedandresultssufficientlyverifiabletoenablecredibleandreliableevaluation.

26 JointevaluationscoordinatedbyOCHAarenotpresentedtotheBoard.

Inaddition,preparationswerebegunforanevaluationofWFP’sresponsetotheEbolacrisisandanInter-AgencyHumanitarianEvaluation(IAHE)ofthecorporateemergencyresponseinIraq.AnevaluabilityadvisoryassessmentofWFP’sStrategicPlan25 was also conducted(reportingin2016).Thisassessmenthadbeenpostponedto2015totakeaccountofadjustmentofthenewStrategicPlantotheSDGsandZeroHungerChallenge.

Pursuingthedevelopmentandmodellingofbestpractices in partnerships with other actors in internationalhumanitariananddevelopmentevaluation–outcome4ofWFP’s2016–2021evaluationpolicy-–OEVcontinueditsparticipationinIAHEsofLevel3emergencyresponsesinSouthSudan(completed)andtheCentralAfricanRepublic(nearcompletion),withevaluationmanagementcoordinatedbytheOfficefortheCoordinatonofHumanitarianAffairs(OCHA).26Initiatedin2014,thisnewtypeofevaluationispartofthehumanitarianprogrammecycleoftheIASCTransformativeAgenda,andprovidessharedanalysisofandlearningfromthecollectivehumanitarianresponse.IncludingtheOEV-managedevaluationofWFP’sEbolaandSyrianresponses,fourofthesevenLevel3emergenciesongoingin2014andthesixongoingin2015areorhavebeenunderevaluation.

TheseriesoffourimpactevaluationscoveringWFP’sworkonmoderateandacutemalnutritionin humanitarian contexts was started as planned in partnershipwiththeInternationalInitiativeforImpactEvaluation.Partofalargerthematiceffortonthistopic,theevaluationswereplannedforeightcountriestogeneratelearningfromandformanyactors.OEVhostedaninceptionworkshopinSeptember2015with30practitionerandacademicparticipants.Thisserieswillbecompletedin2017.

Bothofthesepartnershipsinvolvecostsharing,enablingincreasedcoverageandenhancedlearningwiththeavailableresources.

Strengthening decentralized evaluation

In2015,activitiestostrengthenthedecentralizedevaluationfunctionfocusedprimarilyonitsnormativeframework.AllelementsofthefunctionhavebeenembeddedinthenewevaluationpolicybysettingnormsandstandardsfordecentralizedevaluationsandclearrolesandaccountabilitiesforthevariousstakeholderswithinWFPintermsofplanning,resourcing,capacitydevelopment,qualityassurance,reporting,use,managementresponseanddissemination.Theframeworkalsoincludesclearprovisionsforsafeguardingtheimpartialityofdecentralizedevaluations.


OEVprioritizedinitiativesfordevelopingWFP’scapacitytoundertakehigh-qualitydecentralizedevaluations:

i) Aprocessguidewasdraftedfordecentralizedevaluation.FollowingthestandardsoftheEvaluationQualityAssuranceSystem,itclarifiesthemanagementprocessfordecentralizedevaluationsandtherolesandresponsibilitiesofkeystakeholdersatvariousstages.Italsoincludestemplates,checklistsandtechnicalnotes.Thefirstdraftoftheguidewasvalidatedbycountryofficesandregionalbureaux,andwillbepilotedin2016.

Thisprocessguideispartofacomprehensivedecentralizedevaluationqualityassurancesystem,whichappliesthesameinternational,professionalevaluationstandardsasthoseinOEV’swell-establishedEvaluationQualityAssuranceSystemforcentralizedevaluations.

ii) Initiatedin2014,thedecentralizedevaluationhelpdeskbecamefullyoperationalin2015,supportingregionalbureauxandcountryofficesin20exerciseswithinitsfirstyear.Themajorityofrequestscameattheplanning,inceptionandpreparationstagesofdecentralizedevaluations(Figure8).

Dissemination 6%

Planning 12%

Data collection 6%

Reporting 6%

Inception 18%

Preparation 53%

Figure 8. Percentages of requests for support at different evaluation phases, 2015

iii)EmbeddingofevaluationinWFP’sMonitoringandEvaluationLearningProgramme.OEVledthedesignandimplementationofsessionsonevaluationandinitiateddevelopmentofafurthermodulededicatedtoevaluationandreviewaspartofthisinitiative,ledbyWFP’sPerformanceManagementandMonitoringDivision.

iv) ToincreaseWFPstaff’sawarenessofevaluation-relatedissues,OEVorganizedevaluationsessionsintworegionalmonitoringandevaluationnetworkmeetings.

v) Anintranetpageondecentralizedevaluationwasestablished,providingcolleaguesworldwidewith immediate access to guidance and other informationondecentralizedevaluations.

Inparallel,OEVengagedwithotherdivisionstostrengthenplanningforevaluationanddiscussanddevelopsustainablefundingmechanismsfordecentralizedevaluations.


Learning from and use of evaluations

TostrengthentheuseofevaluationevidenceinWFP’spolicyandprogrammeplanningprocesses,OEVprovidedevaluationevidencethroughthestrategicprogrammereviewprocesstoinformthedevelopmentofcountrystrategiesandprojects.OEVprovidedcommentson94percentofstrategicprogrammereviewdocumentsandattendedtwothirdsofreviewmeetings.OEValsosystematicallyreviewedpilotCountryStrategicPlansandadvisedonassociatedtemplates.

CPEsaredesignedtoprovideevidenceofWFP’scurrentstrategicpositioningandresultsasastartingpointforfuturecountrystrategiesandCountryStrategicPlans,whileoperationevaluationsaredesignedtoinformprojectplanning.Sothenatureandtimingofcountryofficeplanningdecisionscontinuedtobeastrongcriterionintheselectionprocessforcountry-levelevaluations–whetherofsingleoperations,corporate

emergencyresponsesorcountryportfolios–toensurethemostappropriatetypeofevaluationforimpendingdecision-makingneeds.

OEVisalsoanobserverinthePolicyandProgrammeAdvisoryGroup,whichincludescolleaguesfromHeadquarters,regionalbureauxandcountryoffices.OEVcontributeslessonsfrompastevaluationstoinformreflectiononcorporatedevelopmentsandengagementinevaluation-relatedissues.Aspecialconsultation with this group was held in 2015 on the demand-led,decentralizedfunctionandprovisionsforsafeguardingimpartiality.

Continuingeffortstobuildtailor-madelearningopportunitiesintotheevaluationprocessincludeddedicatedeventsforstakeholderlearningfromevaluations(Table3),supplementedbybriefings,consultationsandpresentationstostakeholdersandWFPseniormanagement.

Figure 9. Core elements of the decentralized evaluation function

Planning• Embedding evaluation in programme cycle and in national processes

Resourcing • Sustainable financing • Human resources • Access to expertise

Capacity Development • Guidance and tools • Technical advice/support• Learning journey

Quality Assurance• Post evaluation quality assessment • Audit trail

Reporting• Key Performance Indicators • Annual Evaluation Report

Normative framework • Evaluation policy / strategy• Norms and standards• Coverage norms• Roles and accountabilities

Utilization/communication• Publication • Management response• Learning lessons• Support to decision making• Support national processes

Decentralised evaluation framework

CPE United Republic of Tanzania

Final Evaluation of the Purchase for Progress (P4P) Pilot

Nutrition Policy

Strategic Evaluation of WFP’s Use of Pooled Funds

Evaluation

External partners and WFP staff

Participants at the annual P4P Global Consultation (including staff, partners, donors and academia)

WFP technical and management staff

IASC Humanitarian Financing Task Force

OCHA

Stakeholders

Dar-es-Salaam

Rome

Rome

Geneva

New York

Location

Table 3. Evaluation-related stakeholder learning events in 2015


SynthesesofevaluationsofWFP’sworkhavealsobeenwellreceivedastoolstoenablelearning.Asynthesisoffindingsfromtheseriesoffourrecentstrategicevaluationsonaspectsofemergencypreparednessandresponse27wassupplementedby:i)cross-referencingoffindingswithrelatedfindingsfromseveralotherevaluations,includingthatofWFP’sresponsetotheSyriancrisis;andii)interviewswithseniorEPRstakeholdersconcerningprogressonfollow-upactionstotheevaluations.

TheevaluabilityassessmentofWFP’sStrategicPlanadoptedaninnovativestrategicapproachinresponsetothedecisiontobringforwarddesignofthenewStrategicPlantoalignitwiththe2030Agenda,andtoadaptWFP’splanningandfinancingsystemsthroughtheRoadMaptoZeroHunger.TheevaluabilityassessmentadvisedWFP’smanagementontheextenttowhichtheperformancemanagementarchitectureenablesassessmentofprogressonWFP’sStrategicObjectives,andonwhatimprovementscouldbemadeinthenextStrategicPlan.

Staffingfactorsnecessitatedashiftinprioritiesofplannedactivitiesinthisarea,includingthepostponementofupgradestoOEV’sintranetandinternetsites.However,allevaluationreportscontinuedtobepublicallyavailableonline,andcontributionstoWFP’songoingworkoncorporateknowledgemanagementweremaintained.

UniquepageviewsofOEV’sintranetsiteincreasedby68percent,withonly7percentbeingnewvisitors.Intranetactivityincreasedsignificantlyinthelastquarterof2015,whenthenewevaluationpolicywasapprovedandnewmaterialsfordecentralizedevaluationwerepublished.UniquepageviewsofOEV’sinternetsitedroppedby15percent,witha25percentdecreaseinvisitors,80percentofwhomwerenew.

Evaluation function reporting

EvaluationformspartofWFP’sperformancemanagementarchitectureandthequalityofevaluationsisgreatlyenhancedbyhigh-qualitymonitoringdata.Therefore,OEVengagedwiththePerformanceManagementandMonitoringDivisiontoformulateWFP’smonitoringstrategy.

OEV’ssystemsforreportingonthecentralizedevaluationfunctionwereanalysedinrelationtothenewevaluationpolicyandstepsweretakentodesignasystemforoversightofthepolicy’simplementation.Thissystem,tobefurtherdevelopedin2016,willmeasurethequalityandextentofevaluationactivitiesacrossWFP.

Developmentofapost-evaluationqualityassessmentsystemcoveringbothcentralizedanddecentralizedevaluationswaspostponedto2016–thefirstyearofthenewpolicy’simplementation.

Anindependentmeta-assessmentof21OEV-managedevaluationsfoundthatOEVmettherequirementsforgenderintegrationsetbytheUnitedNationsSystem-WideActionPlan,andimprovedonits2014score.ThiswasachievedbyinvestinginthetechnicalcapacityandawarenessofgenderrequirementsofOEVandevaluationteams.

Engagement with the international evaluation system

Throughitsinter-agencycollaborationandpartnershipswithUnitedNationssystem-wideevaluationnetworks,OEVcontinueditsengagementinandsupporttotheIASC’sIAHEarrangementforgreaterlearningandaccountabilityundertheTransformativeAgenda.OEVparticipatedinevaluationsofthecollectiveresponsesintheCentralAfricanRepublicandSouthSudan,andintheCoordinatedAccountabilityandLessonsLearning(CALL)initiativefortheSyrianresponse.Asynthesisreportoffindingsandlessonsfromthealmost1,000entriesintheCALLinformationsystemisbeingpreparedasacontributiontopreparationsfortheWHS.

Inlinewiththeir2014jointstatementofintent,theevaluationofficesofthefourRome-basedagencieshostedatechnicalseminarinNovember2015ontheevaluabilityofSDG2–End hunger, achieve food security and nutrition, and promote sustainable agriculture.Attendeesfrom38countriesincludedevaluators,academics,governmentrepresentatives,UnitedNationsandotherinternationalagencystaff,withanadditional1,000virtualparticipants.TheseminarsetthebasisforfuturedevelopmentofasharedSDG2evaluationagenda.

27 PREP;thegloballogisticscluster;theWFP/FAOjointevaluationoffoodsecurityclustercoordinationinhumanitarianaction;andWFP’suseofpooledfundsforhumanitarianpreparednessandresponse.


Reflectingincreasedglobalattentiontohumanitarianissues,OEVlaunchedahumanitarianevaluationinterestgroupintheUnitedNationsEvaluationGroup(UNEG).Itsfirststudy,startedin2015,willallowabetterunderstandingofhowapplicationofthehumanitarianprinciplesofhumanity,impartiality, neutrality and independence are evaluated–highlightingbestpractices,challengesandopportunities.

OEVcontinueditsleadroleinUNEG,withtheDirectorofOEVservingasvice-chairforsystem-wideevaluationissuesduringtheveryactiveInternationalYearofEvaluation.In2015,OEV:i)convenedagrouptoconsidertheimplicationsofthe2030AgendaandtheSDGsforUnitedNationsevaluationfunctions;ii)continueditsengagementintheIndependentSystem-WideEvaluationInterimCoordinationMechanismanditspilotevaluationsofUNDAFsandcapacitybuildinginstatistics;andiii)assistedthereviewandupdatingofUNEG’sfoundationalnormsandstandards,alsotakingaccountofhowspecificaspectsofevaluationinhumanitariancontextsarerecognized.

Inaddition,OEVcontinuedtosupporttheworkofUNEGonprofessionalizationofevaluation,decentralizedevaluationandtheQuadrennialComprehensivePolicyReviewofevaluationissuesaffectingWFP.TheDirectoralsocontinuedasasteeringgroupmemberoftheActiveLearningNetworkforAccountabilityandPerformance(ALNAP);twoevaluationsmanagedorco-managedbyOEVwereselectedasgoodpracticeexamplesinALNAP’sGuideonEvaluationofHumanitarianAction.

Resources for evaluation

ThissectionreportsonlyonresourcesavailabletoOEVforevaluation.InlinewithWFP’sEvaluationPolicy(2016–2021),overthecomingyearsWFP’smanagementinformationsystemwillbeexpandedtoenableaggregatedreportingonresourcesdedicatedtoWFP’sevaluationfunctionasawhole.

Thetotalbudgetforevaluationin2015wasUSD9million–8percentmorethanin2014.Thisrepresented0.18percentofWFP’stotalprojected2015contributionsincome.28

WFPallottedUSD5.5millionfromthe2015PSAbudgettoOEV’sworkprogramme:USD2.5millionforstaffandUSD3millionfornon-staffexpenditures,approximatelyinlinewiththe2014allotment.AfurtherUSD600,000wasallottedtothecriticalcorporateinitiativeofcontinuingtodevelopthedecentralizedevaluationfunctionandotherenhancementsinlinewithWFP’sresponsetotheDAC/UNEGpeerreview.Thesetwocomponentsrepresenteda13percentincreaseover2014.OEVexpended99percentofallfundswithintheyear.

Thespecialaccountfromwhichoperationevaluationsarefunded–largelyfromprojectsources–totalledUSD2.59million,similartothe2014level,butwithasmallincreaseinstandardWFPstaffrates.

OEV’sestablishedstaffcomprisedtheDirector,nineprofessionalofficersandthreegeneralservicestaff.Furtherimprovementsweremadeingeographicalandgenderdiversity,althoughtherearestillmorewomenthanmen.The50:50balancebetweenWFPstaffonrotationandexternallyrecruitedexpertswasmaintained.

Overtheyear,thepositionoccupancyratewas89percent,comparedwith81percentin2014;temporarystafffilledgaps.Professionalstaffturnoverduringtheyearwas17percent,comparedwith33percentin2014.

Justoverthetargetof2percentofprofessionalstaffworktimewasspentinprofessionaldevelopment–slightlylessthan2percentforestablishedprofessionalstaffandslightlymoreforshort-termpersonnel.

28 Source:WFPMPEB.2/2015/5_A/1/Rev.1(pg.6)

Figure 10. OEV budget, 2015

6

5

4

3

2

1

0PSA Project

sources

USD

Mill

ion


Currentsystemsdonotadequatelycapturevirtualmodesoflearningsuchaswebinarsandpeer-to-peerexchange,whichareincreasinglycommon.

OEVmaintained12long-termagreements(LTAs)withconsultancyfirmsandresearchinstitutionsprovidingevaluationservicesinthetechnicalandgeographicalareasrequiredfortheprogrammeofcomplexevaluations.SevenLTAsprovidedservicesfortheoperationevaluations.29Asin2014,allevaluationteamsin2015werecontractedthroughLTAs.

ForevaluationsmanagedbyOEVin2015,81consultantswerehired,comparedwith106in2014.Fifty-sevenpercentofconsultantsforcomplexevaluationswerecontractedforthefirsttimebringingfreshexpertisetocomplementthatofconsultantswithWFPexperience,(comparedwith41percentin2014).Theaverageevaluationteamforcomplexevaluationswas6.1consultants,upfrom4.8in2014;operationevaluationteamsaveraged3.6consultants.

Thecompositionofevaluationteamshadareasonablegenderbalance,with57percentmenand43percentwomencomparedwith46percentmenand54percentwomenin2014.Theproportionofprofessionalsfromdevelopingcountriesclimbedto36percent,30 comparedwithalowof25percentin2014,with58percentfromdevelopedcountriesand6percentofdualnationality.Figure11showsthegenderandgeographicaldiversitycombined.

29 ThreeorganizationshaveLTAsforbothtypesofservice.

30 Thisfiguredoesnotincludelocalresearcherswhoaresubcontractedatthecountrylevel.

Figure 11. Composition of evaluation teams, 2015

45

40

35

30

25

20

15

10

5

0

Developing countries

N.

of c

onsu

ltant

s hi

red

Developed countries

Developing and developed countries

Women

Men


Availableatthefollowinglink: http://docustore.wfp.org/stellent/groups/public/documents/cd/wfp283812.pdf

Annex1.EvaluationCharter


AER AnnualEvaluationReport

ALNAP ActiveLearningNetworkforAccountabilityandPerformance

CALL SyriaCoordinatedAccountabilityandLessonsLearninginitiative

CERF CentralEmergencyResponseFund

CP country programme

CPE countryportfolioevaluation

DAC DevelopmentAssistanceCommittee

DEV DevelopmentProgramme

EMOP emergency operation

EPR emergency preparedness and response

FAO FoodandAgricultureOrganizationoftheUnitedNations

IAHE Inter-AgencyHumanitarianEvaluation

IASC Inter-AgencyStandingCommittee

IFAD InternationalFundforAgriculturalDevelopment

LTA long-termagreement

MAM moderateacutemalnutrition

OCHA OfficefortheCoordinationofHumanitarianAffairs

OECD OrganisationforEconomicCo-operationandDevelopment

OEV OfficeofEvaluation

P4P PurchaseforProgress

PREP Preparedness and Response Enhancement Programme

PRRO protractedreliefandrecoveryoperation

PSA ProgrammeSupportandAdministrative(budget)

REACH RenewedEffortsAgainstChildHungerandundernutrition

SDGs SustainableDevelopmentGoals

SO SpecialOperation

SUN ScalingUpNutrition

UNDAF UnitedNationsDevelopmentAssistanceFramework

UNEG UnitedNationsEvaluationGroup

UNICEF UnitedNationsChildren’sFund

WHO WorldHealthOrganization

Acronyms

Produced by the Office of Evaluationwww.wfp.org/evaluation

Printed: May 2016D

esignbyCPPublicationsUnit

World Food Programme Via C.G. Viola, 68/70 - 00148 Rome, Italy - Tel: +39 0665131

fighting hunger worldwide measuring results, sharing lessons · 2017. 1. 18. · flexibility to...

Documents