WFP Evaluations concerning Safety Nets

IntroductionSocial protection and safety nets are important components of national systems to promotedevelopment and within which other assistance can and should be aligned. WFP’s strategicplan and the 2004 safety net policy seek to position WFP as an integrated part of socialprotection and safety net systems. The Top 10 Lessons here are synthesized from 18 WFPevaluations since 2006 of WFP operations across the globe that relate to safety nets and 9thematic or strategic evaluations covering key cross-cutting program developmentelements. In addition, a number of external evaluations of food-based safety nets werereviewed to help identify lessons.

The 'Top 10 Lessons' series is intended to be of practical value, primarily to field staff, in planning andimplementing WFP operations. Drawn from evaluations of past operations, they summarise ways to tackle 10key challenges that have been encountered by others before on a selected topic. They are not policy directives,but have been compiled within the current policy framework and are in line with WFP's mission and mandate.Background

WFP’s work in safety nets is not a new phenomenon, however focus on such work has beengaining attention within WFP in recent years in response to changing internal and externalfactors.

There are a range of definitions for safety nets among different organizations and experts.In WFP, safety nets are defined as : “formal or informal non-contributory transfers providedto people vulnerable to or living in poverty, malnutrition and other forms of deprivation”.Safety net transfers should be non-contributory, i.e. requiring no payment by beneficiaries(e.g. such as premiums when purchasing insurance), and be provided publicly. Safety netstransfers can include conditional transfers, unconditional transfers and public andcommunity works. Safety nets are only a component of broader social protection systemsthat include insurance, and social services.

The 2011 strategic evaluation of WFP’s Role in Social Protection and Safety Nets consideredfour primary quality criteria, or principles of good practice, which are:

Adequate: sufficient to meet people’s needs Timely: both on time and at the right timePredictable: regular, so people know it is coming and can plan accordinglySustainable: both financially and politically

20

11

As a subset of social protection systems, safety nets may have a variety of purposes. In theshort term, they may focus on protecting people and enabling them to survive periods ofstress and shock. Longer-term objectives include mitigating shocks or reducing the risk ofcrises occurring. Increasingly safety nets are also being designed with the aim ofstrengthening livelihoods and reducing chronic poverty.

WFP workshops in 2009 and 2011 brought together senior staff to discuss ways for WFP tobe more effective when working with safety nets, and where efforts to integrate intonational social protection and safety net systems are appropriate.

The Strategic Evaluation of WFP’s Role in Social Protection and Safety Nets commissionedby the WFP Office of Evaluation (OE) contributed to internal consideration of these issuesand has helped to increase understanding of this area and its relevance for WFP. However,WFP staff have requested more detailed and practical-level lessons and information thanwas reported in the evaluation summary report.

This Top 10 Lessons document seeks to provide some of this additional practical informationand is organized based on three categories – lessons regarding results, lessons regardingexternal factors affecting performance, and lessons regarding internal factors affectingperformance.

Lesson 1: Many WFP operations have a time-bound safety net effect, but in order to be an effective safety

net, support needs to be adequate, timely and predictable.

Summary: WFP programmes often have the intended effects of alleviating hunger andsaving lives, which serves protective safety net purposes. Different instruments requiredifferent considerations to effectively meet good practice standards of being adequate,timely and predictable. Some instruments tend to more frequently serve as safety nets,which promote livelihoods in addition to protecting lives.

Experiences from the field: WFPoperations have been found to have life-saving safety net effects by allowingpeople to survive periods of stress orshock. The 2010 evaluation of theEthiopia Protracted Relief and RecoveryOperation (PRRO) found that relief,Government of Ethiopia ProductiveSafety Net Programme (PSNP) andTargeted School Feeding (TSF)components of the programme deliveredtransfers to millions of people whichsaved lives, prevented acute hunger,

reduced the risk of chronic hunger andaddressed under nutrition.

The 2010 evaluation of the OccupiedPalestinian Territory PRRO noted thatWFP food assistance through generalfood distribution (GFD), food for work(FFW) and food for training (FFT) helpedto mitigate the worst effects of theeconomic crisis on food security.

Experiences with different instrumentsshows the ways in which quality criteria

2

(e.g. adequate, timely, predictable andsustainable) interact with programdesign and implementation decisions toincrease the safety net effects of WFPprogrammes.

School feeding programmes have beenshown to enable people to surviveshocks and also to strengthen livelihoodswhen implemented predictably andwithout interruption. The 2007evaluation of WFP School Feeding inEmergency Situations, the 2010evaluations of WFP School Feeding inGambia and Kenya, and the 2011evaluations of WFP School Feeding inCambodia and Cote d’Ivoire all foundthat school feeding has a significantvalue transfer effect when programmesprovide predictable and reliable deliveryof food, positively increasing householdincome and freeing up time forproductive activities.

The value of the transfer from schoolfeeding appears to be highest for themost vulnerable groups of beneficiaries.School feeding linked to take homerations (THR) appears to have thegreatest value transfer effects and alsoshows evidence of enabling beneficiariesto be more resilient during lean periods.The 2011 evaluation of school feeding inCambodia specifically noted that THR isan adequate approach for targetingspecifically vulnerable people and hasthe potential to function as a safety netmechanism.

General Food Distribution (GFD),especially in emergency operations, isaimed primarily at protecting lives ofpeople but can also help free up incomeand enable household investments inother non-food areas. When GFD iscontinued as part of a recovery operation

to fill a gap in national food safety nets(often in a PRRO or Country Programme(CP) it is more likely to have asustainable protective effect and maycontribute to promoting livelihoods whenthe amount of assistance is sufficient.

The 2009 strategic evaluation of theEffectiveness of WFP LivelihoodsRecovery Interventions noted that thegreatest impact of WFP assistance issometimes achieved simply bycontinuing relief. This evaluation foundthat in certain contexts where people’slivelihoods are recovering, WFP GFDassistance is likely to have recoveryimpacts by helping people to meet basicneeds, which frees up income andenables people to make investments intheir own recovery.

FFW, FFT and food for assets (FFA) aregenerally designed to protect fromshocks and strengthen livelihoods,however their scale, duration, value oftransfers, challenges in targeting andfrequent interruptions limit their impactas seen in the 2009 strategic evaluationof the Effectiveness of WFP LivelihoodsRecovery Interventions, the 2009 and2010 evaluations of the Malawi andEgypt Country Programmes, the 2006and 2010 evaluations of the CentralAmerica, OPT and Timor-Lest PRROs.The 2009 strategic evaluation of theEffectiveness of WFP LivelihoodsRecovery Interventions specificallyexpressed concerns that FFA activitiesseem to frequently be of smaller scalewith lower coverage, which can limitimpact.

FFA, FFW and FFT programmes provideopportunities for WFP to model safetynets, and demonstrate their potentialimpact. The 2010 evaluation of the

3

Egypt CP noted that FFA programmeshave led to increased governmentcommitment to pursue safety netsdesigned to reduce food insecurity incertain areas.

FFW and FFA programmes can havepositive effects on improving livelihoodsby increasing income, creatingcommunity assets and the environment.Positive examples of longer-termimprovements in household assets andlivelihoods were identified in the 2010evaluation of the Rwanda CountryPortfolio. The 2010 evaluation of theNepal Country Portfolio found that WFPhas been effective in building communityassets through FFW, FFA and cash forassets (CFA) projects which reachedlarge numbers of people, reducedimmediate food shortages and protectedassets and livelihoods in the short-term.However this evaluation found theevidence less clear that theseprogrammes have created longer-termhousehold assets or improved long-termlivelihoods conditions. Countryprogramme and portfolio evaluations in2009 and 2010 in Rwanda, Nepal, Malawiand Egypt found mixed evidence oflonger-term improvements in householdassets and livelihoods conditions fromFFW and FFA programmes.

FFT can promote livelihoods outcomes aspart of a safety net if skills beingdeveloped are linked to demand in thejob market and other services which

provide job placement. The 2010evaluation of the OPT PRRO noted thatmore attention should be given toproviding training related to marketableskills and to developing partnerships thatwould support graduates of such trainingto be employed using the skills they hadacquired.

Maternal and Child Nutrition and Health(MCNH) programmes can provide aplatform for serving as a safety netduring crises while also promoting healthoutcomes which can improve livelihoods.The 2006 thematic evaluation of WFPSupported Mother and Child NutritionInterventions found that in all of itscountry studies MCNH programmesappeared to function as a food safety netduring crises because they provide anexisting food-distribution mechanism forreaching the most vulnerable groupswhen a crisis occurs.

Vulnerable Group Development andFeeding (VGD/VGF) programmes wereshown to be effective in increasingproductive assets of beneficiaries in the2010 evaluation of the BangladeshCountry Programme. However, the 2009strategic evaluation of the Effectivenessof WFP Livelihoods RecoveryInterventions evaluation found thatVGD/VGF often suffers fromunpredictability and therefore has limitedits use as a livelihood protectioninstrument in some context

Lesson 2: Sustainability of WFP safety net programs and outcomes requires adequate funding,government capacity building, and long-term assistance to beneficiaries that is adequate andpredictable. Safety nets which are manageable at scale, over longer periods of time and developed andimplemented in partnership with governments are most likely to become sustainable.

Summary: Long-term sustainability of safety nets implies that they become incorporated

4

into national policies and systems which requires strategic, political and financialcommitments from the government as well as technical and programmatic capacities.Building government capacity and commitment requires gradual efforts to encourage andenable transition to ownership, rather than abrupt handover. When initiating programs andoperations WFP should balance the need for expediency with the need to develop partnercapacities.

Once programs are transitioned to government ownership WFP can transition from anoperational role towards advisory and technical support roles, which can help thegovernment by providing added legitimacy and independent feedback on potential gapsand programme improvements.

Experiences from the field: The 2009strategic evaluation of the Effectivenessof WFP Livelihoods RecoveryInterventions noted that in severalcountries external funding constraintsforced WFP to abruptly cease, ratherthan to phase out, planned activities.This evaluation suggested the need formore planned and gradual transitionswhen faced with funding or pipelineconstraints.

The 2008 evaluation of WFP’s CapacityDevelopment Policy and Operationsfound that emergency response,especially in relation to recurrent shocks,is more effective when carried out in thecontext of longer-term strategies forcapacity building and resilience. Thisevaluation noted that operations oftenbegin with a ‘supply side focus’ andgradually evolve as their effectiveness isshown to be more ‘demand driven’.Examples from Ethiopia’s MERET andEFSRA programmes are cited wheredemand and supply for capacitydevelopment assistance meet andbecome driven by government processesand expectations.

This evaluation suggests that the term“handover” is in fact a poor fit for the

good practices of capacity developmentas it suggests an abrupt shift, ratherthan an ongoing process of shiftingresponsibilities for programmemanagement and implementation ascapacities are built. The risk ofsubstituting (by implementing withoutkey partners or embedding staff inpartner agencies to provide technicaland analytical support rather thandeveloping their ability to sustainablyconduct such work) rather than buildingcapacity of partners is also raised in thisevaluation and is a particular problemwhere WFP is focused on its central roleof strengthening programmingcapacities. The evaluation suggests thatWFP may need to pro-actively managethe transition to a WFP advisory ormanagement oversight role where thereis an ongoing need for an externalpartner to stay engaged to ensurecontinued quality, transparency andenhance credibility. In the early stagesafter a transition, WFP’s continuedinvolvement at such levels helps toreassure stakeholders that expectedlevels and quality of service willcontinue.

The 2009 strategic evaluation of the

5

Effectiveness of WFP LivelihoodsRecovery Interventions suggested thatexit strategies should include advocacywith donors and national governments todevelop policies and programmes thatcan address needs previously covered byWFP as it transitions out of a servicedelivery role. Sustainability of assets and livelihoodsdepends on adequacy of transfers,duration of assistance and often requirescomplementary programmes that morecomprehensively address vulnerability.The 2009 strategic evaluation of theEffectiveness of WFP LivelihoodsRecovery Interventions found that inEthiopia the debate about sustainabilityis partly around whether or not theobjective to ‘graduate’ people fromchronic food insecurity, through a

combination of PSNP transfers and otherfood interventions, is in fact achievable.It notes that while food assistance on itsown is seen as being inadequate,generating enough complementaryprogramming to allow ‘graduation’continues to be challenging.

The 2009 evaluation of the BangladeshCountry Programme found that whileaverage values of productive assets hadincreased, incomes of beneficiaries werestill low and just over half of participantshad been able to ‘graduate’ tomicrofinance programmes. In the 2010evaluation of the Nepal Country Portfoliothe timeframe for livelihood recoveryactivities was found to be too short tosustainably restore essential assets.

Lesson 3: WFP’s analytical capacities and experience delivering at scale are its greatest comparativeadvantage for contributing to sustainable safety nets.

Summary: WFP’s vulnerability assessment capacities have allowed it to help partnersinstitutionalize evidence based monitoring and targeting and make contributions to safetynet partnerships and consortia in terms of designing, monitoring and adjusting safety netprogrammes. Efforts to share and transfer vulnerability assessment skills and analysis alsoenable WFP to influence national policies. WFP’s ability to deliver at scale provides acomparative advantage for helping fill gaps in national safety nets.

Experiences from the field: The 2008evaluation of WFP’s CapacityDevelopment Policy and Operationsnotes that policy influence andinformation sharing through analysis,advocacy and regional networks areeffective in building political commitmentand agreements as a basis for adoptingnational policies and strategies. WFPvulnerability analysis and needsassessments in all regions and countrieswere an area of comparative advantageas reported in Country Portfolio and

operations evaluations. For example, inBangladesh, WFP vulnerabilityassessment and mapping (VAM) andother analytical efforts effectivelystrengthened the Government ofBangladesh’s capacity in resourceallocation and targeting for its safety netprogrammes.

In Rwanda, the analytical work funded byWFP and partners was strongly alignedwith the demand from governmentpartners for planning, and information on

6

hunger and food security, which led tothe development of the Food SecurityMonitoring System and helped to placefood security and nutrition on thenational agenda.

In Kenya WFP’s participation andleadership in key food security meetingsand steering groups allowed WFP to useits analytical capacities, particularlythrough VAM, helped ensure effectivecoordination of response by thegovernment at national and districtlevels along with that of other UNagencies and non-governmentalorganisations (NGOs).

In Egypt WFP provided capacity buildingassistance to the government by helpingestablish an advisory unit within theMinistry of Social Solidarity. Thisanalytical capacity building assistancewas intended to help the governmentidentify ways to reform Egyptian foodsubsidy systems.

The ability of WFP to deliver at scale isalso cited as a key comparativeadvantage related to safety nets. WFP’sunique and significant food logisticscapacities fill gaps in national food safety

nets and help build government capacity.The 2010 evaluation of the OccupiedPalestinian Territory PRRO noted thatWFP’s support to institutional feedingwas a critical means of augmenting thePalestinian National Authority’s decliningcapacity to meet the needs of those ininstitutional care, as an interim measureuntil the government was able to fullyresume its responsibilities.

However some livelihoods recoveryprogrammes tend not to take advantageof WFP’s capacity for delivering at scale.The 2009 strategic evaluation of theEffectiveness of WFP LivelihoodsRecovery Interventions foundconsistently that fewer recoveryactivities are implemented than plannedand FFA activities in particular seem tobe consistently small-scale with lowcoverage.

Many aspects of sustainable large-scalenational social safety nets fall outside ofWFP’s niche – including pension andinsurance schemes, categorical welfare(e.g. widows, disabled) and long terminfrastructure projects.

Lessons Regarding the Main External Factors Affecting Performance

Lesson 4: Predictable and adequate funding is critical for safety nets by enabling sufficient durationand value of the transfer and limiting pipeline breaks and other interruptions in assistance.

Summary: Challenges in fully funding WFP’s programmes have often forced countryoffices to rebalance priorities or limit programme implementation in ways that reduce theoverall value or regularity of transfers. When planned support does not arrive or is lessthan planned, the safety net effect of programmes is reduced.

Experiences from the field:Evaluations in Cambodia, The Gambia,

Egypt, Malawi, Ethiopia, Guatemala andTimor-Lest show that funding constraints

7

can lead to delays or interruptions intransfers or reductions in the amount ofassistance provided. Such tradeoffs canundermine the timeliness, predictabilityand adequacy of transfers and reducethe effectiveness of the intervention as asafety net.

The 2009 strategic evaluation of theEffectiveness of WFP LivelihoodsRecovery Interventions stressed that forlivelihoods recovery assistance to beeffective it also needs to be timely.

Delays in delivery of transfers werefound to restrict household investmentsin protecting livelihoods and riskmitigation measures in the 2010evaluation of the Ethiopia PRRO. Thisevaluation notes that timeliness andpredictability of transfers are criticalfactors for reducing negative copingstrategies and households which receiveunpredictable and lower than intendedlevel of transfers are much more likely toengage in distress sale of assets.

The 2010 evaluation of the Timor-LestPRRO found that while FFA activitieswere successful in some cases, a sevenmonth gap in service provision led toonly one third of the project’s targetsbeing achieved.

Evaluations in Egypt, Malawi, CentralAmerica and the Occupied PalestinianTerritory (OPT) found that fundingshortfalls require a country office torebalance priorities. Shortfalls appear tohave greatest impact on FFW/FFAlivelihoods oriented interventions ascountry offices prioritize life-saving GFDas well as more institutionalized schoolfeeding.

The 2010 evaluation of the OPT PRROfound that the recovery componentsfailed to meet targets, particularly inFFW, due to financial shortfalls that ledto the prioritization of reliefinterventions.

The reduction of WFP’s impact due tofunding shortfalls is also shown asrecovery interventions are cut backleading to widespread sharing of rationsand households only being able to workfor a short time on FFA projects in the2009 strategic evaluation of theEffectiveness of WFP LivelihoodsRecovery Interventions.

In Kenya, WFP plans to support a disasterpreparedness facility were meant toprovide a safety net for chronically foodinsecure communities to build assetsagainst recurrent droughts. However the2008 evaluation of the Kenya EmergencyOperation and CP found that theprogramme was unfortunatelysuspended shortly after becomingoperational due to demands to divertresources to GFD under the emergencyoperation.

Inadequacy of funding limits thetimeframes and volume of assistanceprovided in many livelihoods recoveryprogrammes. The 2009 strategicevaluation of the Effectiveness of WFPLivelihoods Recovery Interventions foundthat WFP food assistance is less effectiveat supporting recovery processes byrestoring key productive assets forbuilding stronger livelihoods becausetimeframes are often too short. Thisevaluation also suggested that suchactivities need to be implemented earlierand simultaneously with relief for foodassistance to support sustained

8

improvements in livelihoods.

In Central America, the 2006 evaluationof the Central America PRRO found that

FFW outcomes for asset creationachieved were limited overall becauserations were few and irregular.

Lesson 5: The country’s economic and policy contexts play a significant role in determining thespace for WFP to contribute to safety nets and the best combination of programme activities required.

Summary: Contextual factors found to affect WFP’s ability to achieve safety net outcomesincluded government policies, implementing partner capacities, stability, marketfunctioning and fluctuations in prices, and supportiveness of communities within a countryand individual communities.

Experiences from the field: Policies ofnational governments and the policyenvironment in a country can affect thedemand for particular types of WFPassistance, constrain or enable its abilityto implement programs based on goodpractice, and determine the impact ofcapacity building efforts.

The impact of national and local policieswhich can constrain implementationbased on good practice can be seen inschool feeding examples from Coted’Ivoire and Gambia. The 2010evaluation of Gambia School Feeding and2011 evaluation of Cote d’Ivoire Schoolfeeding show the negative effects ofnational policies that require payment forschool meals. Both evaluations foundthat the payments, where still required,reduced the value transfer of meals andthus the ability of school feeding to serveas a safety net. In Cote d’Ivoire theevaluation found that as a result of therequired payments the rations were notalways being given to the same children– since children in the same householdmay have taken turns to receive moneyfor meals. The evaluation notes that this

undermines learning, safety net andnutrition outcomes.

In the case of Malawi, government policyfocus was shifting in ways that createdan enabling environment for safety nets.The 2009 CP evaluation found that asfood security conditions improved in thecountry the government’s agenda wasshifting towards focus on socialprotection and developmentprogrammes. This paralleled a focus onfood security by new governmentleaders. The result of these changes inthe national policy environment were anincrease in ‘upstream support’ – orcapacity building, technical advice andpolicy inputs – related to socialprotection and development. WFP wasfound to be making efforts to meet theincreasing requests for food, technicaland policy inputs to government socialprotection programmes in the areas offood, nutrition, school feeding, HIV/AIDS,and through vulnerability assessmentand disaster preparedness work.

The 2010 evaluation of the EgyptCountry Programme documented WFP’s

9

efforts to contribute capacity buildingassistance and technical support toenable the government to review andrevise its national food subsidy systems.Despite these efforts, the evaluationconcluded that a possible lack ofgovernment commitment and politicalwill to reform food subsidies limitedWFP’s potential to achieve its capacitybuilding and policy goals in Egypt.

The capacity of implementing partnershas also been frequently cited as a keyfactor affecting WFP’s ability toimplement safety net programmes thatare timely, predictable and adequate.The 2007 evaluation of WFP SchoolFeeding in Emergency Situations foundthat low capacities of local partners andgovernments make responding to large-scale needs in a timely mannerchallenging for school feeding inemergencies. The 2006 evaluation ofthe Central American PRRO and 2009evaluations of the Malawi CountryPortfolio and Burkina Faso PRRO foundthat weak capacity among implementingpartners negatively affectedperformance and implementation(predictability and adequacy), andtimeliness. Similar concerns were raisedin the 2010 evaluations of theGuatemala and Timor-Lest PRROs as wellas the 2006 thematic evaluation ofTargeting in WFP Relief Operations.

Environments characterized by conflictand instability also present challengesand opportunities. The 2009 evaluationof School Feeding in Cote d’Ivoire foundthat prior to the conflict WFP support to anationally owned program was stronglyjustified, but once the conflict eruptedand the government lost control of 60%of the country, WFP linkages to the

government did not fully serve theeducation and safety net objectives ofWFP’s school feeding programme.

The interruption of government safetynets in conflict environments can alsocreate opportunities for WFP to fill gaps.The 2009 evaluation of Cote d’IvoireSchool Feeding also noted that the gapscreated by the conflict in areas no longercontrolled by the government led to WFPestablishment of emergency operationsin these areas aimed at protection ofhuman and productive assets. The 2010evaluation of the Nepal Country Portfoliofound that when the governmentreduced coverage of districts by one-third during the conflict WFP filled gapsin food distribution and advocatedagainst the government’s position toeliminate support for these areas.

Security restrictions and limitedaccessibility to areas affected byconflicts were also identified as limitingWFP’s ability to respond to large scaleneeds in a timely way particularly forschool feeding in the 2007 thematicevaluation of WFP School Feeding inEmergency Situations.

Community and institutional context iscited in a number of evaluations asaffecting the potential for WFP to achievesafety net objectives. In the 2010evaluation of Kenya School Feeding thelinkages between school feedingobjectives and factors which drivebroader learning, health and livelihoodsoutcomes are highlighted. Theevaluation stresses that school feeding inthe absence of appropriate learningenvironments and family/communitysupport is a weak intervention limitedmostly to food security. The 2011

10

evaluation of Cambodia School Feedingnotes that the types of threats causingvulnerability, as well as their frequencyand recurrence, determines the degreeto which school feeding can be aneffective safety net.

Market forces related to food and basicgoods has also been shown to affect theimpact of safety net interventions andalso affects the design of WFP safety netinterventions and WFPs ability to beflexible. The 2010 evaluation of the OPTPRRO notes that in places whereeconomic infrastructure exists and isfunctioning what matters most is theeconomic value of resources transferred,rather than the nature of the resourcestransferred. In these cases, where

market mechanisms are able to deliversupport by voucher or cash, suchmechanism are cited as having aconsiderable advantage and preferenceby the population.

The 2010 evaluation of the BangladeshCP noted that safety net outcomes aresusceptible to economic pressures suchas food prices – which means thatprogrammes must be regularly adjustedto keep pace with such trends. The 2010mid-term evaluation of the EthiopiaPRRO similarly noted that increased foodprices had reduced the value of cashtransfers in the PSNP, which requiredWFP and other partners to increase thevolume of food transfers to compensate

Lesson 6: Effective safety nets require comprehensive partnerships between the government, localorganizations and potentially international agencies. Such partnerships provide opportunities for greaterprogramme impact as well as policy influence.

Summary: Safety nets are often oriented to meeting sometimes overwhelming needs inthe countries where WFP works. WFP’s comparative advantages give it a significant role toplay in developing and implementing safety nets but effective partnerships are required tomaximize impact. Partnerships with national governments and other national actors arecritical to ensuring long-term sustainability and predictability. When WFP’s analytical skillsare applied to joint efforts to raise awareness of gaps and needs and advocate forestablishment or improvement of safety nets its voice is multiplied and more likely toimpact national policies and programmes.

Experiences from the field: In Kenyapartnerships played a key role inenabling school feeding to be effectivelyimplemented and achieve its safety neteffects. The 2010 evaluation of schoolfeeding in Kenya found that acomprehensive approach involving cross-sector, interagency collaboration isrequired to maximize the benefits ofschool feeding. This involves a denseand active network of partnerships with

government, UN agencies and NGOs.

In Ethiopia the complex partnership inthe PSNP led to improved coordinationand greater safety net effects. The 2010mid-term evaluation of the EthiopiaPRRO notes that the PSNP partnership,which WFP played a role in developing,established new ways of working, withstronger coordination mechanisms tosupport implementation.

11

Partnerships help enable WFP to havegreater policy influence on safety netissues. The 2008 evaluation of WFP’sCapacity Development Policy andOperations notes that studies led by WFPin Latin America demonstrate howpartnerships with academic,government, humanitarian anddevelopment organizations can raiseawareness of food security andnutritional issues and provide afoundation for influencing policies thatcreate or improve safety nets.

Active WFP participation in existingcoordination structures and socialprotection platforms can also helpposition WFP to influence policies relatedto safety nets. In the 2010 evaluation ofthe Rwanda Country Portfolio WFP’s roleas the co-chair of the UNCT thematicworking group on sustainable growth and

social protection is cited as positioningWFP to promote an agenda focused onnutrition, vulnerability and food security.Similarly, WFP’s interaction with keyRwanda government ministries inmonthly meetings enabled it to influencepolicy and strategy for national safetynets.

The 2011 strategic evaluation of WFP’sRole in Social Protection and Safety Netsfound that donors are increasinglydevoting their resources for socialprotection to programmes implementedunder well-coordinated national socialprotection platforms. The evaluationfound that WFP safety nets work is moreeffective and more likely to receivedonor support when WFP is activelyengaged in such platforms and works inpartnership with other agencies andinstitutions with complementary skillsand experience.

Lesson 7: Staff skills related to livelihoods, public administration/policy, and nutrition analysisshould be built in WFP country offices while reinforcing vulnerability and needs analysis skills and basicliteracy regarding safety nets and social protection to improve WFPs ability to engage in policydiscussions regarding the appropriate role of food assistance in safety net programs.

Summary: Safety nets, as a subset of social protection, are ideally owned by nationalgovernments. As such they are enabled and guided by national policies. In order tocontribute to the establishment and maintenance of effective food based safety nets, WFPstaff should engage in policy discussions and undertake other efforts to influence andenable policy decision-makers. In order to play this role effectively, WFP staff skills need tobe built in areas related to safety nets and social protection, as well as associated topicssuch as livelihoods.

Experiences from the field: Theability of WFP staff to engage withorganisations and institutions that playcritical roles in social protection policydiscussions is critical for enabling WFP to

have a lasting impact in safety netprograms. The 2009 strategic evaluationof the Effectiveness of WFP LivelihoodsRecovery Interventions found that WFPstaff need to be prepared with

12

appropriate skills and capacities toengage with a different set of institutionsthan they have traditionally, includingdifferent government agencies, theWorld Bank, NGOs and donors. WFPneeds to be able to articulate a clear rolefor itself within longer-term socialprotection programming and strategiesand needs greater capacity to engage inpolicy debates around food security andlivelihoods, safety nets and theappropriate role of food assistance.

The 2006 thematic review of Targeting inWFP Relief Operations stresses thatpolitical analysis is critical for targetingbecause political influences will alwaysbe evident in targeting processes. Theevaluation suggests that WFP needs totake account of the institutions andgovernance structures which play a rolein determining targeting (orenabling/undermining targeting) tomaximize the equity of distribution.

Capacity development initiatives focusedon policy-making stakeholders at inter-ministerial and ministerial levels arecited as an important means for WFP toinfluence programmes and policies in the2008 evaluation of WFP’s CapacityDevelopment Policy and Operations. Theevaluation notes that such efforts tobuild capacity of policy-makers isequally, if not sometimes more,important than ensuring adequatecapacities for implementation.

In the 2010 evaluation of the EgyptCountry Programme limited nutrition andvulnerability/poverty analysis capacityare cited as key impediments to ensuringhigh quality work and a sustained focuson key policy issues and governmentcapacity building.

Lesson 8: Close engagement with national governments in the design and implementation of safetynets can create dilemmas for humanitarian organizations in certain contexts, which should be carefullyconsidered by WFP.

Summary: Good practice in safety nets requires efforts to identify ways of buildinginstitutional national capacity for their long-term sustainability. However, as ahumanitarian agency, WFP must balance the interests of sustainable long-term safety netswith its need to maintain independence and related humanitarian principles.

Experiences from the field: Tensionsbetween humanitarian principles andprinciples of good practice for safety netscan arise specifically in relation to thedesire to build government capacity andprovide support to nationally ownedsafety net systems.

The 2009 strategic evaluation of theEffectiveness of WFP Livelihoods

Recovery Interventions noted that thesetensions have been highlighted inEthiopia where WFP support to PSNP hasboth been an important part ofgenerating government ownershipthrough a shift from relief to longer-termsupport for social protection while alsocreating issues for WFP in relation to itsindependence and ability to influenceand be critical of the government when

13

needed on humanitarian grounds.

In Nepal the same evaluation noted thatWFP deliberately limited its policy andprogram relationships with thegovernment for a period of time to avoid

politicization and maintain itsindependence during the conflict.However this is cited as limiting WFP’sability to develop government capacityand influence institutional policy-making.

Lesson 9: Targeting for safety nets should include community and household data, be conducted inmulti-agency partnerships where feasible, and seek to minimize exclusion and inclusion errors to theextent possible.

Summary: Targeting for safety nets requires complementing geographic targeting withadditional community and household data, which can help prevent exclusion errors. Multi-agency targeting for safety net programmes better inform policy-making and resistpoliticization. Where national policies and safety net systems require national coverageprogramme design should pay specific attention to inclusion errors.

Experiences from the field: Multiple evaluations suggest thatgeographic targeting needs to becomplemented with community andhousehold data to allow WFP to designeffective safety nets.

The Synthesis of WFP EvaluationsBearing on the Subject of Targeting2000-2006 notes that household foodeconomy or other in-depth studies ofsocio-economic status and copingmechanisms are particularly useful indeveloping livelihood building strategiesand such information is needed tosupport VAM analysis and to harmonizegeographic and household targeting.The same synthesis stresses thatvulnerability analysis at household levelis almost invariably required to identifythe most at risk communities andhouseholds.

The 2006 thematic review of Targeting inWFP Relief Operations states thatgeographical targeting does not appearto be explicitly linked to objectives for

protecting livelihoods or avoidingdistress sales of assets.

The lack of appropriate mechanisms toconduct local needs assessments is citedin the 2007 thematic evaluation of WFPSchool Feeding in Emergency Situationsas limiting the appropriateness andaccuracy of targeting strategies. The2010 evaluation of OPT PRRO notes thattargeting should be based onbeneficiary’s socio-economic status anddifferentiated needs.

Multi-agency targeting is noted as a goodpractice and an increasingly utilizedapproach in a number of evaluations.The 2006 thematic review of Targeting inWFP Relief Operations notes that wheremulti-agency involvement in targetingdecisions is lacking perceptions of biascan emerge and political pressures canunduly influence outcomes. Thisevaluation suggests that multi-agencystructures that involve and informpolitical decision-making throughtechnical advice are a key element of

14

successful targeting practice andrepresent a step forward in how WFPconducts targeting in recent years.

Exclusion errors can limit theeffectiveness of safety net programsespecially where the ‘supply’ ofassistance is not able to keep pace withneeds, where policies negatively impactprogramme fidelity and in cases wheregeographical targeting is used as thefirst targeting filter.

The 2009 strategic evaluation of theEffectiveness of WFP LivelihoodsRecovery Interventions found that thefundamental targeting issue in Ethiopia’sPSNP programme is that resources areinsufficient to cover all chronically foodinsecure households leading tosignificant exclusion of people who areas needy as those included in theprogramme. The 2010 mid-termevaluation of the Ethiopia PRRO notedthat the application of targetingmeasures meant to discern betweenacute and chronic food insecurity did notaccurately reflect the complex nature ofvulnerability in Ethiopia and led toexclusion errors.

In Kenya, the 2010 evaluation of KenyaSchool Feeding programmes found thatwhile WFP explicitly targets the mostvulnerable segments of the population,studies have shown that a million ormore malnourished children are found inthe more privileged districts of thecountry where no school feeding occurs.The 2010 evaluation of the Nepal

Country Portfolio similarly found thatwhile geographical targeting works wellfor reaching the majority of householdsfacing food insecurity it is not effective atreaching small groups of disadvantagedor those vulnerable communities withinbetter-off areas.

Inclusion errors can also arise due totargeting decisions in safety netprogrammes, particularly due to ‘self-targeting’ designs and programmes thatare mandated to cover an entire territory.

The 2010 evaluation of the NepalCountry Portfolio notes that self-targeting, where individuals andhouseholds are given the choice ofwhether to become beneficiaries , wasfound to be ineffective in FFW programsresulting in inclusion errors.

The 2009 evaluation of the MozambiqueCountry Programme found that despiteVAM data which shows clearconcentrations of vulnerability in certainparts of the country, a national mandateto extend school feeding to all provincesmeant that targeting was not based onvulnerability and led to likely inclusionerrors. The 2010 evaluation of theTimor-Lest PRRO similarly found thatMCHN and school feeding programmeswere deliberately and explicitly alignedwith government systems and prioritieswhich are country-wide and not targetedaccording to need which meant that froma vulnerability standpoint inclusion errorswere unavoidable.

Lesson 10: Monitoring and evaluation of safety nets should be linked closely to targeting, capturequalitative and quantitative data, and shared with partners to improve learning and programme design.

15

Summary: Monitoring and evaluation linked to targeting enable WFP to adjust programmeimplementation based on changes in needs and context to maximize impact. Monitoringshould capture qualitative and quantitative data to enable periodic review of safety netprogram designs and objectives during implementation to adjust for exclusion and inclusionerrors, implementation fidelity issues and changes in context. Lessons gathered throughmonitoring and evaluating safety nets should be shared and sought among partners tostrengthen collective efforts and WFP’s base of knowledge.

Experiences from the field:Monitoring of safety net programmesshould be closely inter-linked withtargeting and needs assessment andshould feed a continuous process ofprogramme adjustments andimprovements.

The Synthesis of WFP EvaluationsBearing on the Subject of Targeting2000-2006 describes targeting as acontinuous exercise, generally describedas monitoring. The 2009 strategicevaluation of the Effectiveness of WFPLivelihoods Recovery Interventions citesthe recurrent need for initialassessments and programme design tobe updated throughout implementationof programmes to reflect changinglivelihoods needs.

The idea of ‘rolling needs assessments’is discussed in the 2006 thematic reviewof Targeting in WFP Relief Operations.This review notes that such ‘rollingassessments’ enabled geographicaltargeting to be periodically revisedduring EMOPs implementation to allowfor adjustments based on changingconditions.

The 2010 evaluation of the GuatemalaPRRO notes that frequent changes in theoperating context require regularadjustments to exit strategies duringprogram implementation.

In addition to engaging in a periodic andfrequent and inter-linked monitoring andtargeting process, numerous evaluationsnote the need to change the way inwhich monitoring and evaluation areviewed and conducted within WFP toensure such analysis has a positiveeffect on program improvement.

The 2006 thematic review of Mother andChild Nutrition Interventions found that afairly mechanical reporting of outputswas driving monitoring and evaluation(M&E) at a country level. This minimizesthe potential utility of monitoring andevaluation and creates an environmentwhere staff view such efforts as a simpleaccountability reporting requirement.The evaluation found that this limitedapproach was creating a disconnectionbetween M&E and programmingadjustments.

The 2006 thematic review of Targeting inWFP Relief Operations recommendedthat WFP staff be encouraged to viewM&E as an essential tool for increasingprogramme effectiveness, rather than asimple accountability mechanism.Increased communication betweenprogramme staff and programmesupport staff responsible for M&E wasrecommended in the 2010 evaluation ofthe OPT PRRO as a way to help ensurethat monthly analyses are designed and

16

presented in a way that encouragesprogramme learning and adjustments.

Simply recording and reporting on basicquantitative indicators limits WFP’sability to learn about the more contextrelated factors that affect theperformance of safety net programmes.

The 2009 and 2010 evaluations of theMalawi and Mali CPs cited an absence ofgood monitoring data, especially datathat measures outcomes and impacts ofcommunity assets. The 2011 evaluationof the Nepal Country Portfolio found thatmore quantitative and qualitative data isneeded about changes to assets, well-being indicators and income as a resultof WFP interventions.

Sharing lessons and jointly analyzingqualitative and quantitative data withpartners including the national

government, UN agencies, the WorldBank and NGOs increases the potentialfor learning and adjusting programmesto maximize impact. The 2011 strategicevaluation of WFP’s Role in SocialProtection and Safety Nets found thatWFP was effectively collecting andsharing data with a wide range ofpartners in Senegal for its cash/voucherpilot projects. At the same time, theWorld Bank and other partners were alsosharing data from similar projects.Collectively this sharing of data wasbeing used to inform discussions of thenational social protection platform andfuture programme design. The SenegalWFP Country Office was also found to becollaborating with the Regional Bureau todevelop a regional evaluation ofcash/voucher programmes to comparelearning from experiences in Senegaland Burkina Faso.

17

Evaluation reports reviewed

Thematic Full report of the Strategic Evaluation of WFP’s Role in Social Protection and Safety Nets,OE/2011/006Management response to the Strategic Evaluation of WFP’s Role in Social Protection andSafety Nets, WFP/EB.A/2011/7-B/Add.1Synthesis report of the Joint Evaluation of Effectiveness and Impact of the EnablingDevelopment Policy of the World Food Programme (WFP), Volumes 1 & 2 and CountryStudies from Bangladesh, Ethiopia and HondurasFull report of the Strategic Evaluation of the Effectiveness of WFP Livelihoods RecoveryInterventions, OEDE/003/2009Summary report of the Thematic Review of WFP-Supported Mother-and-Child NutritionInterventions, WFP/EB.1/2006/7-C*Synthesis of WFP evaluations bearing on the subject of targeting 2000 – 2006Full report of the Thematic Review of Targeting in WFP Relief Operations, OEDE/2006/1Full report of the Evaluation of WFP’s Capacity Development Policy and Operations,OEDE/2008/3

School Feeding EvaluationsFull report of the Evaluation of WFP Cambodia School Feeding (2000-2010), OE/2011/008Full report of the Evaluation of WFP Cote d’Ivoire School Feeding (1999-2009), OE/2011/004Full report of the Evaluation of WFP Gambia School Feeding (2001-2010), OE/2010/017Full report of the Evaluation of WFP Kenya School Feeding (1999-2008), OE/2010/005Full report of the Thematic Evaluation of WFP School Feeding in Emergency Situations,OE/2007/06

Regional EvaluationsFull report on the Nutritional Dimensions of Social Safety Nets in Central America and theDominican Republic, June 2010Summary of Findings: OMC Region-wide Study on National Food Subsidy & Safety NetPrograms: Opportunities for WFP Capacity Support, EB/2/2009Summary of Findings: OMC Region-wide Study on National Food Subsidy & Safety NetPrograms: Opportunities for WFP Capacity Support, Second Edition, EB/2/2009

Country Portfolio Evaluations Country evaluation reports were selected by cross-referencing references to countries madein the recent WFP Safety Net Policy Guidance Note and CP evaluations in the Office ofEvaluation Database. Full report of the Evaluation of Bangladesh Country Programme (2007–2010),OEDE/2009/007Full report of the Evaluation of Egypt Country Programme (2007-2011), OE/2010/006Full report of the Evaluation of Malawi Country Portfolio (2000-2008), OE/2009/009Summary report of the Evaluation of Mali Country Portfolio (2003-2009), WFP/EB.1/2011/6-AFull report of the Evaluation of Nepal Country Portfolio (2002-2009 ), OE/2010/010Full report of the Evaluation of Rwanda Country Portfolio (2006-2010), OE/2010/016Full report of the Evaluation of Kenya Emergency Operation and Country Programme (2004-2008), OEDE/2008/002Full report of the Evaluation of Mozambique Country Programme (2007-2009),OEDE/2009/005

Protracted relief and recovery operations Full report of the Evaluation of Burkina Faso PRRO 10541.0, OE/2009/006Summary report of the Evaluation of Central America PRRO 10212.0, WFP/EB.2/2006/6-AFull report of the Mid-term Evaluation of the Ethiopia PRRO 10665.0 (2008-2010),OE/2010/009Summary report of the Guatemala PRRO 104570, WFP/EB.1/2010/7-DFull report of the Evaluation of Occupied Palestinian Territory PRRO 10387.1, OE/2010/014Full report of the Evaluation of Timor-Leste PRRO 10388.1, OE/2010/007

External Evaluation Reports (serve as secondary sources to further informlessons deriving from WFP evaluations) IEG (Independent Evaluation Group). 2011. Evidence and Lessons Learned from ImpactEvaluations on Social Safety Nets. Washington, DC: World Bank.IEG (Independent Evaluation Group). 2011. Social Safety Nets: An Evaluation of World BankSupport, 2000–2010. Washington, DC: Independent Evaluation Group, the World BankGroup.IFPRI. 2009. An Impact Evaluation of Ethiopia’s Productive Safety Net Program. ODI. 2009. Targeting of Social Transfers: A Review for DFID. IFPRI. 2005. Impact Evaluation of a Conditional Cash Transfer Program: The NicaraguanRed de Proteccion Social.IDS Centre for Social Protection. 2006. Study of Trends in PSNP Transfers Within TargetedHouseholds.Danish Institute for International Studies. 2011. The Evaluation of the Paris Declaration,Phase 2 Final Report.IDS Centre for Social Protection. 2009. DFID Social Transfers, Evaluation Summary Report.Devereux, S. and Coll–Black, S. 2007. Review of Evidence and Evidence Gaps on theEffectiveness and Impacts of DFID-supported Pilot Social Transfer Schemes.Hickey, S. and R. Sabates-Wheeler, G. Guenther and I. Macauslan. 2008. Promoting SocialProtection & Social Transfers: DFID the Politics of Influencing.Davies, M. 2008. Identifying Existing and Planned Baseline Information and Evaluation Plansin Connection with DFID’s Social Protection White Paper 3 Commitments and SocialTransfers Action Plan. World Bank. 2008. For Protection and Promotion: the Design and Implementation ofEffective Safety Nets. Grosh, del Ninno, Tesliuc, Ouerghi.World Bank. 2004. Targeting of Transfers in Developing Countries: Review of Lessons andExperience. Coady, Grosh, Hoddinott.World Bank. 2011. Social Protection Discussion Paper. Natural Disasters: What is the rolefor Social Safety Nets? Pelham, Clay, Braunholz.IFPRI. 2004. FCND Discussion Paper No. 172. Designing and Evaluating Social Safety Nets:Theory, Evidence and Policy Conclusions. Coady.IFPRI. 2008. Discussion Paper 00839. The Impact of Ethiopia’s Productive Safety NetProgramme and its Linkages. Gilligan, Hoddinott, Taffessee.

Other sources (to ensure consistency with prior WFP analysis and existing policy)WFP Internal Document. 2009. Notes from WFP Thematic Workshop on Safety Nets, 26 to28 October 2009, Addis Ababa, EthiopiaWFP. 2010. Revolution: From Food Aid to Food Assistance. Omamo, Gentilini, Sandström,et. al.WFP (2004) WFP and Food-based Safety Nets: Concepts, Experiences and Future

Programming Opportunities. EB.3/2004/4-A. AND UPDATED POLICY PAPER AS AVAILABLE

Acronyms

Cash for assets CFACountry Programme CPFood for assets FFAFood for training FFT Food for work FFWGeneral food distribution GFDMaternal and Child Nutrition and Health MCNHNon-governmental Organization NGOOccupied Palestinian Territory OPTOffice of Evaluation OEProductive Safety Net Programme PSNP Protracted Relief and Recovery Operation PRRO Take Home Rations THR Targeted School Feeding TSFVulnerability Assessment and Mapping VAMVulnerable Group Development and Feeding VGD/VGF