evaluation report - europa
TRANSCRIPT
Evaluation of the partnership between ECHO and UNHCR and of UNHCR activities funded by ECHO.
March 14 – May 23, 2005
Evaluation Report
Claude de Ville de Goyet, Anna Gorter, Riccardo Polastro
For SHER Ingénieurs-Conseils s.a. (www.sher.be)
This Evaluation Report has been financed by and produced at the request of the European Commission (ECHO/ADM/BUD/2004/01212).
The comments contained herein reflect the opinions of the consultants only.
The cost of this evaluation was 153.130 Euros.
Final Version – October 2005
FINAL REPORT Evaluation of the Partnership between ECHO and UNHCR between 2002 and 2004
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. i
Acknowledgements The evaluators express their appreciation for the excellent collaboration received from the staff of ECHO and UNHCR at headquarters and field level. In particular, the insight and experience of ECHO Technical Assistants in the countries have been most valuable. The team is also very thankful for the logistical support received from each office of UNHCR in the visited countries. Most important and enlightening were the long interviews and meetings with actors far away from the capitals. They generously shared their limited time with our team, as they did so many times in the preceding months with bilateral, international or other missions. The need to maintain a balanced and independent view has led us to meet with most actors without the presence of either partner: ECHO or UNHCR. We appreciate the understanding of the senior management of both agencies. Finally, a special mention should be made for the Senior Liaison Officer assigned by UNHCR/Geneva to accompany and assist our team for the duration of the mission. A challenging assignment, given the nature of our mission and the need to maintain its independence!
FINAL REPORT Evaluation of the Partnership between ECHO and UNHCR between 2002 and 2004
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. ii
Table of content
List of acronyms............................................................................................................................. iv Executive Summary ........................................................................................................................ 1 Main Report..................................................................................................................................... 9 Section I. Introduction ................................................................................................................. 9 Section II. UNHCR Capacity.................................................................................................. 10
Section 2.01 UNHCR’s mandate ....................................................................................... 10 Section 2.02 Persons of Concern to UNHCR .................................................................... 11 Section 2.03 Funding.......................................................................................................... 12 Section 2.04 Staffing and Recruitment .............................................................................. 12 Section 2.05 Administrative Management ......................................................................... 14 Section 2.06 UNHCR’s Partners........................................................................................ 14 Section 2.07 Needs-based planning ................................................................................... 15
Section III. Coordination by UNHCR...................................................................................... 15 Section 3.01 What is coordination ?................................................................................... 15 Section 3.02 Coordination within the UN family .............................................................. 15 Section 3.03 Coordination with ECHO and other donors.................................................. 17 Section 3.04 Coordination with Implementing Partners .................................................... 18 Section 3.05 Coordination with national counterparts ....................................................... 20
Section IV. Legal and Physical Protection ............................................................................... 22 Section 4.01 Mandate and Definition................................................................................. 22 Section 4.02 Protection : relevance, coverage, effectiveness, efficiency and impact ........ 23
Section V. Assistance.............................................................................................................. 26 Section 5.01 Definitions and mandate................................................................................ 26 Section 5.02 Assistance : relevance, coverage, effectiveness, efficiency and impact ....... 26 Section 5.03 Standards for assistance ................................................................................ 29
Section VI. LRRD .................................................................................................................... 30 Section 6.01 Definition and mandate ................................................................................. 30 Section 6.02 LRRD efforts of the European Union ........................................................... 30
Section VII. Monitoring, Reporting, and Evaluation................................................................. 31 Section 7.01 Monitoring..................................................................................................... 31 Section 7.02 Reporting....................................................................................................... 31 Section 7.03 Evaluation...................................................................................................... 33
Section VIII. Thematically funded activities .......................................................................... 34 Section 8.01 Background ................................................................................................... 34 Section 8.02 Protection staff deployment and training ...................................................... 34 Section 8.03 Surge.............................................................................................................. 36 Section 8.04 Project Profile................................................................................................ 37 Section 8.05 Security.......................................................................................................... 38 Section 8.06 Key conclusions and recommendations ........................................................ 39
Section IX. The partnership: A look to the future .................................................................... 41 Section 9.01 Importance of this partnership....................................................................... 41 Section 9.02 Supporting NGOs directly or through UNHCR ? ......................................... 41 Section 9.03 ECHO funding mechanisms.......................................................................... 43 Section 9.04 A matter of trust and transparency ................................................................ 44
FINAL REPORT Evaluation of the Partnership between ECHO and UNHCR between 2002 and 2004
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. iii
Technical Annexes (separated volume)
1 Main findings of the 2001 Evaluation
2 Thematic Funding
3 Cross Cutting Issues
4 Afghanistan/Pakistan
5 Chad
6 Guinea
7 Methodology and lessons for future evaluations General Annexes (separated volume)
8 Terms of reference
9 List of contacts
10 Maps
FINAL REPORT Evaluation of the Partnership between ECHO and UNHCR between 2002 and 2004
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. iv
List of acronyms ACD: Association for Community Development ACT: Action by Churches Together ACTED: Agence d’aide à la Coopération Technique et au Développement ACF: Action Contre la Faim AGS Japan: Action for Greening Sahel, Japan AIDCO: European Aid – Co-ordination Office ARC: American Relief Committee AUP: Aid to Uprooted People (special budget line EC) AWSDC: Afghan Women Skills Development Center BEST: Basic Education and Employable Skill Training BNCR: Bureau National pour la Coordination des Réfugiés (Guinée) BPRM: Bureau for Population, Refugees and Migration C&M: Care and Maintenance CARE: CARE International CG: Consultative Group CNAR: Commission National pour l’Accueil et la Réinsertion des Réfugiés (Tchad) COOPI: Coperazione Internazionale COP: Country Operational Plan CORD: Christian Outreach Relief Development DACAAR: Danish Committee for AID to Afghan Refugees DEV: Directorate General for Development EC DIP: Directorate for International Protection, UNHCR DIPECHO: Disaster Preparedness ECHO DOS: Department for Operational Support, UNHCR EC: European Commission ECHO: European Commission Humanitarian Office –Directorate General for Humanitarian Aid EDF: European Development Fund EEMET: Entente des Eglises et Missions Evangéliques au Tchad ERT: Emergency Response Team ESS: Emergency and Security Service EU European Union EVI: Extremely Vulnerable Individuals EWS: Early Warning System EXCOM: Executive Committee FAFA: Financial and Administrative Framework Agreement FAO: Food and Agriculture Organisation FLM/ACT: Fédération Luthérienne Mondiale/ Action by Churches Together FPHC: Frontier Primary Health Care FPA Framework Partnership Agreement 4 Rs: Return, Reintegration, Rehabilitation and Reconstruction continuum GAM: Gender and Age Mainstreaming GTZ: German Technical Cooperation HAPA: Humanitarian Action for People of Afghanistan HQ: Headquarters HIV/AIDS: Human Immunodeficiency Virus / Acquired Immunodeficiency Syndrome ICRC: International Committee of the Red Cross IDPs: Internally Displaced Persons IFRC: International Federation of the Red Cross and Red Crescent Societies
FINAL REPORT Evaluation of the Partnership between ECHO and UNHCR between 2002 and 2004
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. v
ILO: International Labour Organization IOM: International Organization for Migration IP: Implementing Partners IMC: International Medical Services IRC: International Rescue Committee JAI EC Directorate General for Justice and Home Affairs JICA: Japanese International Cooperation Agency JRS: Jesuit Refugee Service LRRD: Linking Relief, Reconstruction and Development MD: Medical Doctor MoRR: Ministry of Refugees and Repatriation (Afghanistan) MoU: Memorandum of Understanding MOSS: Minimum Operational Security Standards MRRD: Ministry of Rural Rehabilitation and Development (Afghanistan) MSF: Médecins Sans Frontières MUHD: Ministry of Urban Development and Housing (Afghanistan) NCA: Norwegian Church Aid NFI: Non Food Items NGO: Non Governmental Organization NRC: Norwegian Refugee Council OCHA: Office of the Coordinator for Humanitarian Affairs ODIC: Organisation pour le Développement Intégré Communautaire OMS: UNHCR Operational Management System OP Operational Partner OXFAM: Oxford Committee for Famine Relief PU: Première Urgence PGDS: Population and Geographic Data Section, UNHCR RELEX : DG External Relations SECADEV: Secours Catholique pour le Développement SGBV: Sexual Gender Based Violence SOP: Standard Operating Procedures SPD: Strategic Programming Dialogue TA: ` Technical Assistance TF: Thematic Funding ToR: Terms of Reference TWIN: Today’s Women International Network UN-DSS : United Nations Department of Security and Safety UNHCR: United Nations High Commissioner for Refugees UNICEF: United Nations Children Education Fund. UNSECOORD: United Nations Security Coordinator UNV: United Nations Volunteers VARA: Volunteer Association for Rehabilitation of Afghanistan VCT: Voluntary Counselling and Testing VRF: Voluntary Repatriation Form WFP: World Food Programme WHO: World Health Organization ZOA Refugee Care: Zuid Oost Azie Refugee Care
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 1
Executive Summary 1. A three-person team evaluated the partnership between ECHO and the United Nations High Commissioner for Refugees (UNHCR), as a follow-up to a 2001 evaluation. The evaluators, between themselves, visited four countries – Afghanistan, Burundi, Chad and Guinea1, and the border area of Pakistan2 – to review UNHCR's work in the period 2002-2004, with a special focus on activities supported by ECHO or relevant to the partnership between the two organizations. 2. This partnership has not always been easy. If the European commission has reiterated at policy level its intention to strengthen its relation with the UN System at large, ECHO’s support to UNHCR is showing a downward trend. Moreover, UNHCR, although it appreciates the usefulness of ECHO contributions, has occasionally questioned whether they warranted the effort. Indeed, the ECHO contribution is only one stream of European Community funding to UNHCR, averaging around 6% of its annual budget. 3. The main report is organized around: UNHCR Capacity; Coordination by UNHCR; Refugee Protection; Assistance to Refugees; Monitoring, Reporting and Evaluation; and Thematic Funding. UNHCR’s Capacity3 4. The evaluators based their conclusion on the review of the UNHCR mandate and its operational capacity in terms of: funding, recruitment and deployment of protection staff, financial management, procurement, and network of partners. 5. Dealing with refugees or Internally Displaced Persons (IDP) is distinct from providing relief after a tsunami or drought. Effective actors should have a strong legal mandate to intervene. UNHCR is the only agency able to provide this indispensable framework. In 1950, UNHCR received the mandate of “providing international protection to refugees and of seeking permanent solutions for their problems”. Its mandate covers refugees returning to their place of
1 The visit was short and coinciding with Easter holiday and ECHO’s technical assistant at the end of a contract break. It included only a three-day visit to N’Zerekore. As such the evaluator’s observations concern only N’Zerekore and not Kissidougou.
2 Although not part of the evaluation programme a very short visit of one and half day was organised to the border area of Pakistan for triangulation purposes with the Afghanistan UNHCR operations. 3 Section II, Paragraphes 5-17.
In legal or operational terms, no other single ECHO partner can match the capacity of UNHCR to provide protection and assistance to refugees.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 2
origin and, on an ad hoc basis, some Internally Displaced Persons. These “Persons of Concern to UNHCR” are also of primary concern to ECHO. 6. On the operational side, more than 6,000 UNHCR staff are deployed on a world scale, albeit sometimes with difficulties in terms of timeliness, average tenure and quality. While UNHCR relies mostly on its human resources to provide protection, it is, however, fully dependent on its extensive network of international and national partners to deliver assistance to the beneficiaries. Coordination by UNHCR4
7. The evaluators regarded UNHCR’s coordination of the international assistance to refugees as a most valuable factor for ECHO and other donors. A strong coordinating agency is required to reach a consistent level of services within and among camps. As observed by the evaluators in 2001 and 2005, ECHO contracting practices did not always support this authority. The undermining of UNHCR authority by partners directly funded by ECHO remains a concern. 8. UNHCR’s relationship with its own partners left much room for improvement. The partners complained, especially in Guinea, of a lack of communication or consultation by UNHCR. The evaluators concluded that some UNHCR staff members treat their NGO partners as low-cost contractors rather than collaborators with their own competence, appreciation of the needs of the refugees and an understanding of constraints.
4 Section III, Paragraphes 18-33.
• ECHO should implement the measures recommended in the 2001 evaluation: o To ensure that UNHCR is provided with complete and transparent information
regarding all ECHO funding o To require ECHO direct partners to sign a Memorandum of Understanding with
UNHCR regarding coordination • UNHCR should treat its “implementing” partners as … partners not as contractors.
• UNHCR should greatly improve its exchange of information with partners and donors. • The UN Coordinator or OCHA should ensure that the refugee assistance is congruent with
the needs of the local population and/or the long-term development priorities. • UNHCR should revise its Memorandum of Understanding with UNICEF and WHO to
ensure their predictability and commitment in education and health assistance respectively.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 3
As a corollary, partners and donors alike complained that UNHCR is not spontaneously sharing information with them. The evaluators believe that is a counterproductive feature of UNHCR corporate culture. Changes of attitude are urgently required. The level of satisfaction of UNHCR partners with the information received is a sensitive indicator of its success in coordination. Informing partners on proposed plans, policies or initiatives should not imply that decision-making becomes a shared responsibility. 9. UNHCR’s mandate is limited to the “persons of concern” and therefore does not include the local, host population. The evaluators concluded that there is a need for one of the existing UN agencies with a broader coordinating and/or developmental role to assume a higher level of coordination. This is particularly important when local populations are also affected by emergencies, when the presence of refugees adversely affects the local communities or when the focus is on durable solutions. Also, coordination of sectorial activities (health and education, for instance) should ideally be delegated to agencies specialized in this sector, in order to avoid duplication and role confusion. Legal and Physical Protection of the Refugees5
10. Protection is the reason d’être of UNHCR. The proportion of resources assigned to this role is difficult to estimate, but appears to be inadequate both in terms of budget and human resources. 11. UNHCR protection is provided first by maintaining a solid and visible presence at the government level and in the field. Although far from sufficient, this presence has considerably improved since the 2001 evaluation. A second measure is an efficient registration of refugees, an area where UNHCR has considerable expertise. The effectiveness of protection provided to in-camp refugees was good in all countries (except Burundi). The protection provided by the registration process could and should have been afforded to refugees outside the camps (i.e. those settled in border villages or urban areas) in several of the countries visited. 12. It is also important to remember that countries of asylum have the primary obligation to protect the refugees. Where the right to asylum is compromised, UNHCR may require political backing from donors and especially the EU member states. ECHO technical staff are not often equipped, politically, to mobilize this support.
5 Section IV, Paragraphes 34-47.
• UNHCR should increase the resources assigned to protection, especially of those refugees outside the camps.
• The EU should bolster its capacity to mobilize political support for UNHCR
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 4
Assistance to Refugees6 13. The evaluators agree that assistance is an integral part of UNHCR protection function. 14. It is important to ask: how much assistance is right? Assistance provided to the refugees in the camps was usually of a level far superior to that received by the local population or expected by the refugees when they return to their country of origin. The unbalanced assistance is partly the result of a rigid, often donor-imposed, compliance with ideal global standards and indicators. The provision of drinking water to camps in Chad is a glaring example of the adverse impact of such compliance on the Sahelian environment. ECHO direct funding to NGOs providing services to host communities somewhat corrected this unbalance – an example of where direct funding by ECHO to its own partners complemented UNHCR work. 15. The evaluators thought that, under some circumstances, more creative approaches, such as use of cash, might be more efficient to assist refugees and local communities. These alternatives should be explored in the future. Monitoring, Reporting and Evaluation7
16. Monitoring of results rather than outputs remains a difficult challenge for UNHCR. Since 2001, UNHCR has designed an extensive set of quantitative indicators designed to address this problem. It is a commendable initiative that has shown few results to date. The evaluators identified the need for simplifying those indicators and parameters and supporting UNHCR in
6 Section V, Paragraphes 48-59. 7 Section VII, Paragraphes 64-71.
• UNHCR and donors should adopt a more contextual and flexible use of existing international standards and indicators, which should not be seen as norms to be rigidly applied.
• UNHCR should reduce the broad gap between the assistance provided to the refugees in the camps, the standard of living of the local population and the situation that the refugees may expect on their return to their country of origin.
• ECHO should continue to support directly NGOs providing assistance to host communities.
• ECHO, in compliance with its commitment to abide by the “Good Donorship” principles and work plan (Stockholm, 2003), should support UNHCR in streamlining the use of indicators as well as its reporting system.
• ECHO and UNHCR should undertake joint evaluations on issues of mutual concern.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 5
their field implementation internally, in its Country Operational Plans, and with its partners delivering assistance to end-beneficiaries. 17. Reporting to ECHO has been a contentious issue. Although the timeliness of the reports has improved, the content needs further attention. Some reports lack critical analysis of difficulties encountered and do not contribute to a lessons-learned objective. A single consolidated report to all donors would be helpful to rationalize this cumbersome process. This goal may be incompatible with the use of a single ECHO form for all its partners, ranging from small NGOs to large specially mandated international organizations. Reporting to ECHO is only part of the reporting burden on field staff, which chains them to their desk, and prevents them from working in the camps, where they are most needed. 18. The internal evaluations carried out by UNHCR deserve high praise for their objectivity and quality. Joint ECHO-UNHCR evaluations would be beneficial and cost-effective in the future. Thematically Funded Activities8
19. UNHCR has been the first and largest beneficiary of ECHO thematic funding. Four projects were targeted: financial support to existing protection posts, the Project Profile, Surge protection posts and staff security. All four TF projects helped to significantly strengthen the operational capability of UNHCR. 20. In 2003/2004, ECHO funded 130 protection, or protection-related, posts in 13 African countries. The initial UNHCR proposal did not take into account the ‘normal’ vacancy rate (at the time over 10%), which resulted in a lack of strict compliance with the terms of the contract and misunderstandings with ECHO. Nevertheless, this influx of funding at a time of declining resources has increased the overall capacity of UNHCR. A durable impact on protection has 8 Section VIII, Paragraphes 72-88.
• Protection posts and security measures should be included, as appropriate, in projects submitted to ECHO Geographical Units. This aspect should not be covered by the TF mechanism.
• ECHO should continue funding Surge and the Project Profile under the TF mechanism.
UNHCR and ECHO should agree upon a set of indicators to prioritize future Surge deployments.
• UNHCR should continue to improve its human resources management to be able to field
its own protection staff for at least one year while increasing their actual presence in the camps.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 6
been achieved, with over 100 new protection positions established under the regular annual budget for 2005. 21. UNHCR field managers appreciated the Project Surge capacity to respond to unanticipated protection staff needs. They praised the speed of the deployment, and the quality of the Surge deployees selected and recruited by the International Rescue Committee, the Surge partner. This initiative fills a critical gap in UNHCR recruitment and field posting of its staff by providing quality protection personnel at a fraction of the cost of a UN staff member. The success of Surge, a temporary fix, should not prevent UNHCR from addressing urgently and permanently the perceived shortcoming in the management of its own human resources. 22. The implementation of Project Profile, a globally standardized registration system of Persons of Concern to UNHCR, is slowly taking off. The development of reliable software and a database was overdue but has now been completed. However, this was the first and easiest step towards a successful Project Profile. Entering all Persons of Concern into the database and ensuring its routine maintenance and quality control will require a significant amount of qualified personnel – who have not been planned or budgeted for. UNHCR will need to present a plan indicating who will be registered in this database, what human resources will be assigned and from where the funding will come for its routine maintenance once the system has been firmly introduced into all UNHCR operations. Sustained support from the international community and cooperation from all partners will be critical. 23. Security equipment and training to increase compliance with the UN Minimum Operating Security Standards, MOSS, have improved as a result of the TF support. How effective this support has been in terms of making UNHCR staff more able to operate in unsafe environments without taking undue risk could not be determined. 24. In addition to the current seven criteria used for selecting eligible TF projects, the evaluators recommend adding the principle of internal “subsidiarity”. Under this criterion, only projects that cannot be funded by geographical units, due to their non-emergency nature or broad geographical scope, would qualify. Looking to the Future9 25. This section deals with crosscutting conclusions and recommendations.
9 Section IX, Paragraphs 89-101
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 7
26. The essence of the partnership is not merely financial (respectively 5% of the UNHCR budget and 8% of the ECHO budget). Rather, this essence lies in the constructive influence gained by ECHO on large mandated partners. ECHO’s influence with UNHCR is not proportional to the level of funding by the EC and the Member States. Purchasing services does not make one a stockholder. The current focus on earmarked funding of well-circumscribed projects should be balanced with general support to improve the overall capacity of the partner. The strengthening of UNHCR with thematic funding only partially corrected the situation. 27. The question of whether ECHO should fund its own partners directly or multilaterally through UNHCR has no definite black or white answer. A combination of both approaches is advised. For ECHO, the right mix will depend on its reply to the question raised in the 1999 evaluation of the Commission Humanitarian aid: “Should ECHO be a mere banker …or a privileged catalytic partner that facilitates quality aid”. This partnership between ECHO and UNHCR may provide a critical test of how ECHO envisages its future role in the humanitarian world. 28. On the UNHCR side, planning is still done on the basis of the anticipated budget. The presentation of a needs-based appeal – whereby UNHCR would provide a comprehensive view of the requirements to meet all the needs of all the refugees, and not only those UNHCR plans to meet itself – is lacking. Some modest progress has been made in the formulation of the Country Operational Plans and this progress should be rapidly strengthened. By making improvements in a needs-based and, therefore, a results-based approach, UNHCR could benefit from the full flexibility envisaged under the FAFA. 29. The evaluators could assess the relevance and need for both the current geographic and thematic funding mechanisms, with one managed by the Geographical units and the other by ECHO 4 due the global and institutional nature of the projects. The thematic funding has proven to be the most effective tool in ECHO’s possession, in terms of building the management capacity of its mandated partners. 30. ECHO funding to UNHCR’s current refugee operations should steadily move towards a balance between funding of specific projects of special interest to ECHO and lightly or non-earmarked multi-donor contributions to an operational appeal. For this purpose, the evaluators consider it relevant that the programmatic funding mechanism envisaged by ECHO is implemented as soon as possible. Funding a percentage of UNHCR’s activities may lead to a loss
• ECHO should increase its strategic support to UNHCR through its thematic funding and proposed programmatic funding mechanisms.
• UNHCR should accelerate its adoption of a genuinely needs-based planning and results-
based monitoring and reporting. • The issue of miscommunications between the HQ of both partners should be addressed
through an enhanced Strategic Partnership Dialogue mechanism.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 8
of day-to-day scrutiny and accounting capacity. This loss will be more than offset by the gain in strategic influence and in long-term policy advances. 31. Finally, the report addresses what is perceived as the real issue in this partnership, which is a gap in communication between the headquarters of the two organizations10. Each organization is justifiably proud of its mandate and achievements. But each is unable to fully understand and accept the inherent constraints of the other. To quote an UNHCR senior official, “There is a need for an interpreter between the two cultures.” The result is a tendency to dwell on faults and mistrust, although things have been improving lately, probably due to the establishment of an UNHCR office in Brussels. The evaluators recommend that the Strategic Partnership Dialogue (SPD) be enhanced, to include a separate forum for high-level policy exchange which would address or avoid any future misunderstandings.
10 The same problem had been noted regarding the communications between UNHCR and its own partners.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 9
Main Report
Section I. Introduction 1) Providing assistance to victims of natural or man-made disasters is a long tradition of the European Union and its Member States. Indeed, all signatories of the First Geneva Convention in 1864 were European countries. Humanitarian assistance has always been an integral part of the external cooperation of the European Commission. This commitment culminated in the creation of the European Commission Humanitarian Office (ECHO) in 1992, which became a Directorate General in 2004. 2) Within the broad scope of the emergency humanitarian aid, the assistance to refugees and displaced persons holds a distinct place due to the complexity of the international implications, the number of victims and the protracted duration of their suffering. In 1951, the flow of refugees across Europe in the aftermath of the Second World War prompted the General Assembly of the United Nations to establish a High Commissioner Office for Refugees (UNHCR). In July of the same year; a special U.N. conference approved the Convention Relating to the Status of Refugees. This 1951 convention for the first time defined who is a refugee11 and stated his/her rights. 3) At the European Union level, the Commission has adopted three communications in 2001, 2003 and 2005 on the strengthening of the relations between the EU and the UN system. The legal and regulatory framework has also been clarified with the new Financial Regulation (Council Regulation N◦1605/2002 of 25 June 2002) and the Financial and Administrative Framework Agreement (FAFA) signed in April 2003 between the European Commission and the United nations. This policy was clearly reiterated in the proposed text of the new EU
11 A refugee is “a person who is outside his/her country of nationality or habitual residence; has a well-founded fear of persecution because of his/her race, religion, nationality, membership in particular social group or political opinion; and is unable or unwilling to avail himself/herself or the protection of that country, or to return there for fear of persecution”
The objectives of this evaluation were: “To review the follow-up given by both organisations to the 2001 evaluation” “To review the progress in their Partnership” “To evaluate the actual operations financed in a number of countries” “To evaluate the lessons learned from the ECHO’s funding of thematic activities at UNHCR”
This document is not a technical audit of ECHO-funded projects but a global management review of an inter-agency partnership based on observations in a few representative countries: Afghanistan, Burundi, Chad and Guinea. Although not foreseen in the evaluation programme, a short visit to the border area of Pakistan was included for triangulation purposes with UNHCR operations in Afghanistan.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 10
Constitution (2004): “The Union shall ensure that its humanitarian aid operations are coordinated and consistent with those of international organisations and bodies, in particular those forming part of the United Nations system12” … “The Union shall promote multilateral solutions to common problems, in particular in the framework of the United Nations13”. In its Communication to the Council and the European parliament on the 15 June 2005, the Commission stated that “The EU should, through both its policy work and its action in the field, contribute to supporting the UN in upholding and protecting the humanitarian space and principles, to ensuring safe access to vulnerable populations, to pursuing more coherent financing, to improving local capacity in disaster-prone areas, as well as to the strengthening of the UN’s own internal co-ordination role and the leadership role of the humanitarian coordinator”. 4) The protection of refugees is an issue of concern of several services of the European Commission: In February 2005, a Strategic Partnership Agreement signed between UNHCR and the Directorate General responsible for External Affairs (RELEX) as well as an exchange of letters between the High Commissioner and the Commission Responsible for Justice, Freedom and Security (JLS), further expanded UNHCR and EU collaboration. UNHCR is the most significant UN partner relative to ECHO’s financial contribution (€ 962 million since ECHO’s inception). In 2001, this partnership was evaluated 14 and found in need of improvements. These findings guided the evaluators in this mission. Section II. UNHCR Capacity
Section 2.01 UNHCR’s mandate 5) The mandate of UNHCR is established by its Statute, adopted by the General Assembly in 1950 and complemented by international legal instruments (the 1951 Convention and its 1967 Protocol). Under the Statute, the UNHCR has the function of providing international protection to refugees and of seeking permanent solutions for their problems. "Persons of concern" are not only those defined as refugees under the Statute, but also returnees, former refugees, who have recently returned from exile, stateless persons15, as well as internally displaced persons (IDPs) whom UNHCR has been called upon, on occasion, by the UN Secretary General to assist. 6) A broader interpretation of the mandate is provided by the Article 10 of the Statute: “The High Commissioner shall administer any funds, public or private, which he receives for assistance to refugees, and shall distribute them among the private and, as appropriate, public
12 Article III-321, Para 3 13 Article III-292, Para 1 14 Evaluation of UNHCR activities funded by ECHO in Serbia, Kosovo, Zambia and Guinea:
http://europa.eu.int/comm/echo/pdf_files/evaluation/2001/unhcr_synthesis.pdf 15 Stateless persons were included in the mandate of the High Commissioner by the UN General Assembly, through Resolution 50/152 of 21 December 1995, requesting UNHCR to promote state accession to the 1954 and 1961 Conventions relating to statelessness; to provide technical and advisory services to interested states pertaining to nationality law and practice; to provide appropriate training to its staff and to government officials; to disseminate relevant information and to enhance cooperation with other interested organizations.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 11
Numbers of persons of concern
0
5,000,000
10,000,000
15,000,000
20,000,000
25,000,000
30,000,000
1980
1983
1986
1989
1992
1995
1998
2001
2004
Refugees
Other persons ofconcernTotal persons ofconcern
agencies which he deems best qualified to administer such assistance.” This article provides he recognition of the links between protection and assistance and a better understanding of what is realistically required for protecting persons of concern, resulting in the provision of not just legal aid, but also the provision of shelter, food, water, education, health. 7) The Executive Committee (ExCom), created in 1958, is constituted of UN Member States. ExCom is the governing body approving the policies and programmes proposed by UNHCR and, therefore, the only empowered to officially endorse the operational applicability of its mandate. ECHO refers to UNHCR’s “Core Mandate” as its legal protection responsibility, applying a strict interpretation which excludes the provision of material assistance. ECHO may choose to support only some of its partner activities but it should refrain from using a terminology suggesting that there are different types of UNHCR mandates. Legally speaking, it is not the case.
Section 2.02 Persons of Concern to UNHCR 8) The total number of persons of concern to UNHCR has been decreasing steadily over the last 10 years. This was partly due to a decrease in the number of refugees to less then 10 million, while the number of other persons of concern, mostly those IDPs entrusted to the care of UNHCR, has been more stable, hovering around 9 million (see figure 1). The total number of IDPs is presently more than double that of refugees with a current estimate of roughly 25 million. The UNHCR policy regarding this pool of potential beneficiaries is not clearly delineated, although guidelines have been recently redefined by the High Commissioner. A
recent review of the Organisation’s response to IDPs describes it as mostly ad hoc with “a high degree of polarisation amongst UNHCR staff who are proactive and forceful about engaging in IDP situations and those who view the issue as beyond the mandate”16. This review recommends, in accordance with a recent evaluation by OCHA17, that “UNHCR clearly articulate its position on IDPs” and “advocate for
protection and assistance, including where UNHCR is not involved operationally. Clear parameters of involvement can best be achieved in the framework of the collaborative response by systematically explaining the reasons for entering or not each IDP situation”. “Should it fall formally under UNHCR responsibility?” remains an open question that the international community, and ECHO as one of the agents for change, must eventually address on a principled, rather than ad hoc, basis.
16 Mattar V. and White P. Consistent and predictable responses to IDPs, A review of UNHCR’s decision-making processes, UNHCR, EPAU/2005/2 March 2005. 17 Bagshaw S and Paul D, Protect or neglect? Toward a More Effective United Nations Approach to the Protection of Internally Displaced Persons, an Evaluation, Internal Displacement Division, the UN OCHA and the Brookings Institution-Johns Hopkins SAIS Project on Internal Displacement, November 2004.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 12
Meanwhile UNHCR’s work with IDPs, because of its human rights and practical experience, can be justified, its involvement in the aftermath of natural disasters, such as the Tsunami in December 2004, cannot.
Funding
9) Since 1995, UNHCR's yearly budget ranged between US$ 400 to 1,100 million. The EC’s contribution declined from over US$ 200 million (20% of UNHCR’s budget) to less then 80 million (8%), while ECHO funding declined even more sharply, from US$ 150 million (16%) to less then 50 million (5%). The decline over the last three years is much steeper when measured in Euros. Other EC funding sources for UNHCR include DG/RELEX, and DG/JLS.
10) Since 2000 on average 8% of ECHO’s funding goes to UNHCR, down from 20% in earlier years. In spite of the decline UNHCR still appreciated ECHO’s funding for its relatively “speedy approval and easy monitoring”, compared to other EC sources.
Section 2.04 Staffing and Recruitment 11) UNHCR has over 250 offices in more then 110 countries and up to 6,000 employees of whom about 1,400 are international staff. The functions of particular concern to ECHO are those of the Community Services officers, the Field Officers and the Protection Officers. Although the formal post descriptions focus on specific aspects of field work, all posts contribute to provide protection and assistance to the persons of concern to UNHCR In order to increase the timeliness of its response in emergency situations, UNHCR incorporated several mechanisms to avoid the formalities and shortcomings of the UN recruitment system when applied to agencies working mostly under emergency conditions (see table 1):
• The compulsory, in principle, periodic rotation of staff between HQ and the less desirable duty stations;
• The creation of an Emergency Response Roster, constituted from experienced and trained UNHCR staff members released from their posts to serve in an emergency response for up to three months;
• Bilateral agreements with Save the Children Fund and the governments of Norway and Sweden for the rapid deployment of short-term experts;
• The Surge Project (see Thematic Funding); • The fast-track vacancy announcements for expeditious internal recruitment for one year
assignments. • An International Professional Roster of potential candidates for employment.
0
200,000,000
400,000,000
600,000,000
800,000,000
1,000,000,000
1,200,000,000
1985
1988
1991
1994
1997
2000
2003
Total contributionsEC contributionsECHO contributions
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 13
Percentage of professional protection posts vacant in the field on January First
0
5
10
15
2000 2001 2002 2003 2004
Table 1 Staff provided outside of the UN recruitment system according to contractual arrangements (UNHCR, June 2005) Statistic on UNHCR Staffing by Various contractual arrangements 01-Jun-05
HQ Total Field Total Grand Grade M F HQ M F Field Total
JPOs 10 15 25 19 42 61 86
Surge 0 1 1 27 21 48 49
UNHCR/ICMC Resettlement Deployments 0 0 0 15 22 37 37
UNVs 0 0 0 256 229 485 485
Consultants (incl. RSD) 27 33 60 40 28 68 128
Expert Deployments (incl. emergencies) 1 0 1 18 9 27 28
Total 38 49 87 375 351 726 813 Most encouraging is the current consideration by the UN Secretariat of a new staffing system for humanitarian or protection agencies18. Considering the importance of this initiative, ECHO should carefully monitor it. 12) The vacancy rate for posts in UNHCR has been reduced from 10 to 5% in the period 2002-2004. In Chad and Guinea, evaluators could observe that the new mechanisms had indeed resulted in placing dedicated staff for longer periods ending the rapid turnover of previous years. 13) The problem is the perceived inability of UNHCR to move rapidly a sufficient number of its own skilled staff19 to the most difficult sites and ensure their presence, without gaps, in the camps rather than in management desk positions. In Guinea, the rotation of staff in 2003-2004 was for a few months (in general lesser than 6 months). The situation has partly improved.
18It is hoped that the new system will consider also the possibility of flexibility in the application of the no-family rule. Over 50% of UNHCR field posts are in no-family duty stations 19 As opposed to surge deployees or others in loan from partners.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 14
Inter-Governmental
5.4%
International NGOs
44.3%
Governmental28.4%
National NGOs21.8%
Section 2.05 Administrative Management 14) Financial management and reporting were a source of problems noted by the evaluators in 2001. UNHCR’s financial management system still does not easily permit one to identify, and report, which funds were used for which particular activity within a given timeframe. As a policy for donor relations, UNHCR considers all voluntary contributions as part of an annual pool. It has some rationale: From the end-of-line beneficiary perspective, earmarking of funds doesn’t bring an added value. When compelled to do so by a donor’s requirements, UNHCR “assigned” the most essential (and therefore best selling) activities to that donor’s contribution. It is an artificial and somewhat misleading device, as these activities would have been carried out anyway. Undoubtedly, the funds freed by this earmarked donation did permit UNHCR to offer essential services in a forgotten crisis somewhere else in the world. UNHCR expects that the Management Systems Renewal Project (MSRP), the new web-based financial management system, will enable better budgeting and workforce planning, as well as facilitate timely reporting, to meet donor requirements. MSRP is now being rolled-out and tested for finance, payroll, procurement, and piloted more broadly in some field offices in Europe. The application will be extended to other field offices through 2005-6, as details and bugs are worked out. 15) UNHCR has a well managed procurement system with strict internal guidelines for its own use and in a simplified version for its implementing partners. If the system is providing significant savings for commonly procured items, it is criticised by some partners as too slow and centralized. In Chad the evaluators were informed that in 2004 UNHCR’s implementing partners had been asked to procure materials through their own system to avoid bottlenecks and gain time. Apart from the recent automation, various initiatives are underway to improve UNHCR’s supply chain further, such as the review of Standard Operating Procedures and the strengthening of contingency planning and emergency preparedness measures, such as pre-positioned stockpiling.
Section 2.06 UNHCR’s Partners
16) UNHCR works with operational partners in virtually every location. In 2004, there were about 800 partners. Out of the 609 NGOs receiving funding from UNHCR (i.e. implementing partners), 75% are national and 25% international20. The international partners are receiving 50% of the UNHCR budget. UNHCR’s partners do enhance its global operational capacity significantly. In the camps visited by the evaluators, all basic field work was performed by the partners that can operate under difficult security conditions. Indeed, UNHCR, a UN agency, falls under the UN Department of Security and Safety, UNDSS, regulations,
20 A large part of national NGOs are affiliates of international organizations providing them extensive management and accounting support.
UNHCR funding of partners 1994-2004
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 15
which result in more stringent security measures, restricted movement and higher costs than is generally the case for the NGOs.
Section 2.07 Needs-based planning 17) Traditionally, the budgeting and planning of UN agencies are prepared in anticipation of an approved or anticipated funding. UNHCR programming and monitoring are based on resources and activities, not on expected results as called for by the logical framework approach. UNHCR confirmed to the evaluators that it plans to shift from resource-based to needs-based planning and that new guidelines were provided for the development of the Country Operational Plans (COP) for 2006. However, from the evaluator’s limited observations, needs- based planning has barely started.
Section III. Coordination by UNHCR Coordination, or rather the lack of it, is so frequently mentioned in humanitarian studies that it has become something of a cliché! This report will be no exception.
Section 3.01 What is coordination ? 18) What type of coordination is expected from UNHCR ?
• Providing a mere forum for exchange of information among actors ? • Promoting a consensual sharing of responsibilities among partners ? • Authoritatively assigning tasks to the actors following prior consultation ?
Protracted refugee operations in complex political and security environments may benefit more from the last definition given to coordination. Mandated authority is a prerequisite for effective coordination. Coordination will benefit from respect of and consultation with others.
Section 3.02 Coordination within the UN family
19) The relationship of UNHCR with the rest of the UN System is influenced, in part, by the fact, UNHCR is directly responsible and accountable to provide services (protection and assistance) to eligible individuals while other agencies have a normative or supportive role. Overlapping mandates and competition for funding and visibility are also playing a role. The 2001 evaluators focused particularly on the weak participation of the Office of Coordinator of Humanitarian Affairs (OCHA). In 2002-2004, the contribution of OCHA in Guinea, although limited, has been praised by NGOs and UN agencies. OCHA’s main asset in addition to its skill is, paradoxically, its lack of operational involvement. OCHA, not being a direct provider of humanitarian assistance, has no inherent conflict of interest, something UNHCR cannot claim.
No other ECHO partner can match UNHCR’s legal mandate or operational capacity to provide an extensive of services to refugees and displaced persons across the world. UNHCR remains a key humanitarian partner for the European Union.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 16
UNHCR cannot be an uncompromising champion of the short-term rights of its refugees and, at the same time, an honest broker when other competing interests or priorities are in play. 20) Assistance to refugees or returnees poses a distinct challenge: In refugee situations the focus is on the well being of the refugee community whereas when refugees return to their homes the focus is on their reintegration into the community. Nevertheless, the assistance to both groups must be integrated within the broader environment, and consider the interface with other populations and the long-term rehabilitative or developmental imperatives of either the host or the returning country. Failure to do so for the refugees can generate counterproductive discrepancies between living standards in the camps and the host communities and importantly, their expectations when they repatriate. In the countries visited, UNHCR ability to see beyond the immediate needs of the in-camp refugees was inevitably variable: from an exclusively refugee-oriented assistance in Guinea; the assignment of funds for non-refugees or non-returnees communities in Chad and Burundi; to deep involvement in long-term solutions and development in returnees’ communities in Afghanistan.
21) There are other key UN interfaces for UNHCR: Coordination with WFP was generally very good. More controversial was the interaction of UNHCR with sectorial agencies such as UNICEF (education and nutrition) and WHO (health). UNHCR sees itself as the agency ultimately responsible across sectors but also within the sectors. In the health sector, WHO and UNICEF also claimed coordination responsibility of the response for health or water services.. As a result, in Chad, three health coordinators and four nutritionists supervised, advised and monitored the health activities in the refugee camps, confounding and overloading NGOs unnecessarily. One agency should assume the sectorial lead.
UNHCR should negotiate a clear and binding Memorandum of Understanding with UNICEF and WHO whereby those agencies would assume technical supervision in education and health respectively. ECHO should financially and politically encourage those two agencies to either fully assume this operational role or accept the operational authority of UNHCR.
The UN Representative or OCHA should ensure, with donor support, that the refugee assistance be congruent with the needs of the local population and/or long term development priorities.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 17
Section 3.03 Coordination with ECHO and other donors 22) This evaluation mission endorses the findings of the 2001 evaluators in that “coordination can only be derived from mutual interest and added values, not from formal contractual, administrative and financial relationship”. It is equally applicable to the partnership between ECHO and UNHCR, as well to the relationship between UNHCR and its “implementing” partners. 23) At country level: The 2005 evaluation team found the relationship between ECHO and UNHCR to be very good in Afghanistan, Burundi and Chad. It could not be directly assessed in Guinea where the team could not meet ECHO staff.21 Undoubtedly, the personalities and the experience of the ECHO/TAs were critical in achieving this smooth collaboration locally. The US Bureau of Population Refugees and Migration (BPRM)22, another major contributor to UNHCR, was seen as having a stronger and more positive influence on UNHCR’s policies and strategies than the EU, despite the latter’s larger combined financial input. This is thought to be consequence of a perceived ECHO “unilateralism” (i.e. directly funding its own implementing partners23), and its inability to direct and guide the contributions of its Member States. Thematic funding of UNHCR has fortunately contributed to restore, in part, ECHO’s strategic influence on UNHCR.
24) Coordination requires broad vision and providing donors with a convincing plan that generates the necessary financial support. The current process to assess needs within UNHCR’s global appeal focus on requirements for what UNHCR perceives to be its own role given , not what is needed to attend all the basic needs of all the refugees. The global appeal should be expanded on the model of the inter-agency CAP managed by OCHA. The Global Needs Assessment and Unmet Needs – 2006 prepared by UNHCR is a first step in that direction24.
21 The regional ECHO Technical Assistant covering Guinea is stationed in Abidjan. He was unavailable due the recent termination of his short-term contract. But secondary sources indicate that there is still room for improvements in furthering a constructive dialogue between both partners. 22 The team leader interviewed the senior management of BPRM in Washington. 23 BPRM reported that only 15% of its budget goes directly to US NGOs, the rest being channelled through lead international agencies: UNHCR or Red Cross. 24 Global Needs Assessment and Unmet Needs – 2006, PCOS / Working draft, 12/05/05
UNHCR Global Assessment of Needs (Appeal) should present to the international community all critical needs of all the refugees, not just those to be managed by UNHCR.
ECHO should pragmatically seek the balance between unilateral and multilateral funding most likely to benefit the refugees while strengthening the international coordination. It should be a planned policy, not driven by the flow of proposals or the availability of funding.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 18
Section 3.04 Coordination with Implementing Partners 25) Coordination is absolutely essential considering the large number of parties involved: in Afghanistan over 100 local and international NGO’s; in Chad over 25; and in Guinea, with a largely reduced caseload, still there are over 15 NGO partners. While recognizing that “no other organization has the capacity, expertise or mandate to take on such a coordination role,” and that …” this is accepted by all NGO partners”, the 2001 authors found serious problems of a lack of confidence in UNHCR. They wrote: “Lack of consultation and dialogue, competence and attitude of certain key UNHCR staff, delays in programming and contractual decisions and lack of internal coordination within UNHCR (Branch Offices and Sub-Offices); and an undermining of UNHCR coordination by ECHO partners …that could …become detrimental to field security”. The 2001 evaluators were particularly critical of the Guinea programme 26) In Guinea, the mistrust between UNHCR and many international NGOs is still evident in early 2005.It prevails despite a marked and steady improvements in the quality and stability of the UNHCR staff. Trust, once seriously marred by personality conflicts and bad publicity, takes a long time to re-establish 27) In Chad, all partners praised the current coordination at field and national levels. This was not the case in early 2004. The present willingness of the UNHCR coordinators in Chad to consult and listen to NGOs before making a decision was a wise approach as in any case; NGOs, if ignored, will not shy away from presenting forcefully those views afterwards. 28) Coordination supposes a policy of information-sharing with donors and partners. On one side, many felt that UNHCR’s corporate culture is adverse to fluid and timely exchange of information. This is especially true with problems encountered or on policies and strategies in the making. Partners and donors had to “seek bits of information to try and compile a picture of what the UNHCR was up to”. On the other side, UNHCR perceives an ECHO’s tendency for fault-finding in its UN partners, which does not encourage UNHCR genuine transparency.
UNHCR should treat its “implementing” partners as … partners not as contractors. Both ECHO and UNHCR should discontinue the use of the qualifier “implementing” in reference to the internationally-mandated UN partners of ECHO or those working with UNHCR. Real partners are mutually implementing their own programs in a synergic and mutually supportive manner. Where not done yet, Branch Offices of UNHCR should designate an existing staff member to act also as NGO Liaison Officer. S/he should have a particular capacity to listen, dialogue and influence operational decision making in his/her own organisation.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 19
29) UNHCR’s special relationship with local NGOs is a valid argument for channelling part of the ECHO assistance funding through UNHCR. Some local partners, although part of a strong international network, reportedly did not perform well in Guinea and Chad. In Guinea, only one local NGO was still in partnership (see photo). In Afghanistan, there were many capable NGOs, which had continued to operate for decades under different regimes: a few of them collapsed under the sudden flood of resources, and demands, thrown upon them by donors post-2001. Some, dependent on only one or two donors, have now difficulty surviving between contracts.
UNHCR contracts do not adequately cover all the operating costs, a serious burden given the widely known and prolonged delays in negotiating contracts with UNHCR. UNHCR’s capacity building support to local NGOs25 is appreciated, but also self-serving: workshops on accounting or supply management are intended more to maintain acceptable accounting standards, than prepare for a future without international sponsors. In line with its commitment to capacity building, UNHCR should do both.
30) Finally, an important coordination aspect is the management of the camps. UNHCR designated different NGO partners to directly manage day-to-day activities in the camps on its behalf. The role of camp manager was undefined. The authority was not respected by other partners, or by UNHCR staff. The evaluation team heard grievance of interferences in daily life decisions by UNHCR field officers or assistants “who do not know the refugees and the situation” as well as by the managing NGO26. Although only delegating authority (but not responsibility), UNHCR staff27, who divide their time between several camps and office work, should better appreciate and utilize the closer exposure of the camp managers to the realities. Progress has been noted in early 2005: A job description of camp manager has been developed and a pilot co-sharing of responsibility is being negotiated with IFRC in Chad, with ECHO encouragement. The direct benefits for the refugees from this innovation need to be assessed at a later date.
25 International NGOs are also using local partners but those specialized on humanitarian response tend to place less priority on the training and long-term capacity building of their partners. The latter often prefer to work with UNHCR. 26 Occasionally, it sounded as the overprotective reaction of a housekeeper before a parental decision concerning children under his/her daily care. 27 Too often, the UNHCR “staff members” were inexperienced UN Volunteers or not actually staff members in the case of Surge deployees.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 20
31) The assertion that ECHO undermines UNHCR’s authority, and capacity to coordinate, remains a concern. Even if this evaluation team does not agree with the 2001 evaluators that full financial control is a requisite for effective coordination by UNHCR, they both urge ECHO to act in support of strengthening UNHCR’s coordination authority. The evaluators found that some partners who were directly funded by ECHO were not sharing information critical to the effective management of the camps. Some dissent can be productive, but uncoordinated and loose-ended activities are not.
The above two measures are currently adopted by most large donors.
Section 3.05 Coordination with national counterparts
32) Under international law, the authorities of the country of asylum have the primary obligation for the protection of refugees. UNHCR is the internationally mandated agency with whom those authorities can work. A strong UNHCR-host government relationship is an asset for the international community, the refugees, and ECHO and its partners; but in building a relationship with a host government UNHCR can be perceived as being too close, especially where government policy is less than ideal. Ultimately if UNHCR has a good relationship it facilitates the work
of humanitarian organisations; conversely, if it in conflict with the government, basic concessions could disappear. Yet, there are cases where UNHCR’s relationship with a government has been considered too tolerant or subservient, such as in Guinea (2001), Burundi and Nepal (2005). In Guinea, the new management team has changed the perception that it was “too close” with the authorities. In Afghanistan, the transfer of the operational responsibility of a camp to the national partner was bringing its own set of problems: difficulties of the NGO formerly managing the camp or of the UNHCR experts in adjusting their attitude may lead to bruised national pride and building of resentment.
ECHO should support the coordination authority of UNHCR and implement the following measures recommended in the 2001 evaluation: - To ensure that UNHCR is provided with complete and transparent information regarding all ECHO funding to every partner working on the concerned “caseload” - Require ECHO direct partners to sign a Memorandum of Understanding with UNHCR pledging to accept coordination.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 21
33) In the countries visited, local counterpart ministries or departments were often technically weak and under-resourced. Although UNHCR has invested resources to strengthen their government counterparts with vehicles, equipment, staff training, advisors, etc., these institutions, nominally in command, remain weak and under-resourced compared to the international actors. Perhaps, Western countries increasingly tightening up their own asylum policies, should increase their aid to UNHCR’s strengthening of the national capacity of the host nations.
The primary responsibility for refugees is with the host government, often the weakest link in the humanitarian chain. A multi-year commitment is required to strengthen the local capacity of the national counterparts. This activity may be a good candidate for TF.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 22
Section IV. Legal and Physical Protection
Section 4.01 Mandate and Definition 34) UNHCR is invested with a clear mandate to “provide protection to refugees28”. Protection begins with securing their admission to a safe country of asylum, the granting of refugee status and ensuring respect for their fundamental human rights, including the right not to be forcibly returned, or refouled, to a country where their safety or survival is threatened. UNHCR’s responsibility ends only with the attainment of a durable solution. Durable solutions for refugees include: repatriation, resettlement in a third country, or local integration in the asylum country. Repatriation, the internationally preferred alternative, is underway in Guinea29 and, on an unprecedented scale, in Afghanistan (over 3.5 million returnees in three years).
35) ECHO regards the legal30 and physical31 protection as the primary area of expertise and responsibility of UNHCR. UNHCR, however, included a strong component of assistance (food, shelter, health, education, water and even recreation) as an integral part of protection, hence the increasing preponderance of so-called “protection-related” activities. The inevitable link between protection and assistance where host countries cannot bear the burden is strongly supported by the international community and recognized by the evaluators. However, ECHO’s: heavily funding of UNHCR’s assistance activities is perceived as ambiguous in regard to ECHO’s insistence that UNHCR focus primarily on its “core” protection mandate. 36) Legal protection is based on the ability of UNHCR to negotiate and collaborate with the host governments, a responsibility and comparative advantage no other agency or partner does have.
Physical protection of refugees, a concern highlighted by recent abuses reported internationally,
28 The 1951 United Nations Convention relating to the Status of Refugees and its 1967 Protocol. 29 Settlement to USA, Canada, Australia or Europe is certainly the solution preferred by some refugees. The mere possibility of it has served a strong disincentive to consider repatriation from Guinea to Liberia. 30 Legal protection issues include working with governments on ensuring that proposed new or amended refugee legislation and refugee status determination structures and procedures adhere to international norms and standards. It includes training and building capacity of local institutions. 31 Physical protection problems include harassment, extortion, physical assault, sexual and gender-based violence and exploitation (SGBVE), infiltration of refugee camps by armed elements, forced military recruitment, diversion of assistance, armed attacks on refugee camps.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 23
requires that their safety and security from risks inherent to their status be maintained at all times. Whether the mandate requires that refugees be specially protected from all risks common to host populations remain a matter of disagreement between the evaluators and some protection officers of UNHCR. 37) IDPs are not considered under the refugee convention or by any UN statutes. However, on certain occasions, UNHCR has been mandated by the UN Secretary General to intervene. Increasingly, there is an impression that UNHCR is eager to become more involved. In Afghanistan and Burundi, UNHCR’s programme has been extended to include IDPs. In Burundi, 10% of UNHCR operational budget has been specifically allocated for IDPs for their reintegration. UNHCR, within the UN system, remains the best equipped agency to answer to the needs of IDPs.
Section 4.02 Protection : relevance, coverage, effectiveness, efficiency and impact
38) The 2001 ECHO evaluation noted that “the share of protection resources was not always commensurate to the objectives” and that “a sensitive point for the protection mandate of UNHCR was the technical capacity of the agency to carry out registration exercises”. It concluded that “the present support of ECHO to core protection activities must be maintained and developed through systematic strategic dialogue”. 39) The relevance of UNHCR protection is well recognized by all partners and donors. 40) The coverage of the protection given to refugees by UNHCR left much room for improvement. A large number of refugees, persons of primary concern to UNHCR, are
spontaneously settled with the local population along border areas (in Burundi, Chad and Guinea) and in some urban locations (Pakistan). They are not receiving formal legal or physical protection as they are simply not registered or monitored. They may not need assistance, but should be able to receive protection. UNHCR brought the attention of the evaluators to the implications of Article 2 Paragraph 6 of the OAU refugee convention32 which states, "For reasons of security, countries of asylum shall, as far as possible, settle refugees at a reasonable distance from
the frontier of their country of origin." While UNHCR does “encourage the integration of refugees into local communities to the extent possible”, the agency considers that “Article 2.6 does not permit encouraging this integration close to the frontier of their country of origin”. At the same time UNHCR claims to “provide international protection, to the extent possible, and to facilitate basic assistance to those who chose to remain close to the frontier of their country of origin”. UNHCR considers “important that the issues of protection and assistance to refugees living outside refugee camps be separated from the issue of protection and assistance to refugees living close to the frontier of their country of origin”. 32 Convention Governing the Specific Aspects of Refugee Problems in Africa, Addis Ababa, 6-10 September 1969
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 24
Although the distinction is indeed pertinent, the evaluators consider that the lack of registration, the most basic form of protection, and assistance to both categories remain a serious gap that UNHCR has not sufficiently addressed. UNHCR is too narrowly focussed on those refugees willing to move into camps. Another coverage issue is UNHCR’s limited access to some refugee groups due to security restrictions (e.g. Burundi/RDC border). Under those circumstances, UNHCR must rely on a number of partners that are actually present and working in those areas. Finally, UNHCR was given the opportunity to undertake the determination of Rwandan refugees by the Burundian government, it was unable to assert its protection mandate when refugees seeking asylum were refouled in June 2005! As stated by UNHCR, “they needed more political support from the international community”. 41) Effectiveness of protection depends on several factors: First, it is best achieved by the sheer presence of UNHCR and other international actors in refugee camps, border areas and settlements. Such presence must be strengthened through regular and extended field visits, strong links and good communication with refugee communities, and increased knowledge of the situation (e.g. obtained through “situation analyses” as piloted in 10 countries in 2004 as an ECHO/UNHCR effort.) 42) Registration, followed by periodic verifications, is the most effective tool to guarantee the refugee status to those entitled to it. In the refugee camps visited, elaborate verification measures
had been generally adopted to address the initial problems of refugee registration. When effectively implemented, the UNHCR methodology reduced the number of multiple or irregular registrations. The benefits of the registration process extended to the organisation of voluntary repatriation: well tuned in the camps in Guinea and Pakistan/Afghanistan. Registration still needs considerable progress in the Great Lakes region. Registration is generally not available to refugees outside the camps. 43) The location of the camps can be critical: border areas are considered of greater risk as they are more easily exposed to the volatile environment from where they flee. Immediate relocation of refugees, according to UNHCR standards (minimum 50km), has not always been possible due to the environment constraints (Chad) or the risk of potential ethnic clashes (Burundi) or by the stance adopted by national and local administrations. While minor incidents occurred in the border camps in Chad, 169 Congolese refugees where killed in 2004 in Gatamba, Burundi. As a priority, UNHCR successfully relocated them to other camps. Within the camps, physical
security is guaranteed through local police forces trained and supervised, with variable success, by UNHCR and in Burundi by the UN Peace Keeping Forces. 44) In two major ongoing repatriation exercises, Afghanistan and Burundi, UNHCR protection is also extended to returnees who are often confronted with risks specific to their status: land tenure issues, complicated by illegal occupations or loss of documentation, ethnic discrimination or simply the lack of recognition of a school diploma earned abroad. These issues
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 25
have been addressed by UNHCR, but with partial success, as their resolution requires time and labour intensive interventions and commitment from the government. 45) The efficiency of the protection is more difficult to assess. Under funding of protection was a serious issue: During the period under review the budget for UNHCR’s protection activities was far below that for assistance. As an average in sub-Saharan countries visited, it represented a 10% of the overall budget, while in Afghanistan it is only 4%. Donors, including ECHO, tend to concentrate the largest part of their funding on more visible assistance activities. Although, the second objective of the ECHO geographical units’ country global plans is to support international agencies in the execution of their mandate, ECHO’s relative investment for protection and coordination is often also limited to a 4 to 5 %. In addition to protection posts, including protection-related field and community services officers, ECHO funded legal assistance and physical protection activities including for the prevention of Sexual Gender Based Violence. Upgrading UNHCR protection is a matter of improving its human resources, a relatively small expense compared to the assistance programmes. The thematic funding has contributed to alleviate UNHCR protection staffing needs in all locations visited, mostly through the Surge project. It is, however, only a gap-filling measure. 46) Increasing efficiency is not merely a matter of more funding, but also requires joint efforts among all partners. In the countries visited, protection was a shared responsibility with national and international partners. These played a highly valuable role by first increasing the sheer presence in the field including under the direct supervision of UNHCR and by contributing also to a broad spectrum of essential protection activities (i.e., legal services, peace building and Sexual Gender Based Violence education). Above all, those partners constitute a necessary outspoken advocacy network for refugee rights. Their independent monitoring of UNHCR’s relationship with the government is a contribution not always appreciated by some UNHCR officials. 47) The impact of the protection granted by UNHCR to refugees and returnees has been generally positive. Host governments were collaborating fully in Chad, Guinea and Afghanistan. A recent exception has been observed in Burundi. Under such challenging and recurrent circumstances, UNHCR must receive strong political bolstering from the international community if it is to fulfil its mandate. ECHO technical assistants may not have the status, or feel comfortable, to mobilize the support from the EU delegation in this respect.
UNHCR must increase its protection presence outside the camps and in particular in the border areas. UNHCR should increase the relative funding of its protection activities and ECHO should increase its relative investment in protection and coordination activities with UNHCR.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 26
Section V. Assistance
Section 5.01 Definitions and mandate 48) As described earlier, protection not only has a legal, but also a physical dimension. In the increasingly competitive world of humanitarianism, this has led to a debate in which UNHCR has been criticised for having become too focussed on assistance, at the expense of its protection obligations. To argue where ‘physical protection’ ends and ‘non-essential’ activities begin is fraught with problems; there are many points of view, most of which reflect the different perspectives of the commentators. Simply put, where resources are limited, logically the focus should be on legal protection and basic life-sustaining assistance. In other words, UNHCR should be careful to concentrate on the specific work for which it was created, and avoid spreading its resources too thinly. Too many assistance activities are non-essential and finally adversely affecting relationship with host populations and complicating repatriation.
Section 5.02 Assistance : relevance, coverage, effectiveness, efficiency and impact
49) The 2001 evaluators commented: “… care and maintenance (C&M) activities by UNHCR (are) arguably the most controversial among the four key functions...” and “…its effectiveness is made very vulnerable by negative factors such as lack of (its) own qualified technical field staff, sudden budget cuts or weakness of implementing partners”. The evaluators had the opportunity to discuss with many partners of UNHCR and all concur that, given the constraints faced by the many actors, in practice UNHCR is an effective implementing mechanism to oversee the assistance to refugees. 50) The needs, at least for in-camp refugees, had been correctly
assessed, mostly in consultation with the partners. Beneficiaries’ complaints ventured by their representatives were on matters of details: e.g. quality/quantity of food rations or limited telecommunication facilities. Mostly the living standards were far above what refugees had at home, and this was mentioned as an important factor for their reluctance to repatriate to countries just recovering from conflict and war. This issue of level of sustainability and appropriateness of standards of assistance must be addressed by UNHCR in consultation with its partners. The approach that more assistance is better is often counterproductive and not in the best interest of the refugees and the host population.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 27
51) In general, distribution of assistance among beneficiaries in the camps was satisfactory and equitable. In Afghanistan although 90% of returnees received a small cash payment, only 20% could be served by the shelter project. Ironically, it was the poorest, those without land, who were not eligible. Notable exceptions are the absence of registration and coverage of unknown numbers of refugees in Guinea (where registration of unaccompanied minors, some present 10 years, has recently been initiated), in host villages at the Sudanese-Chad border and in urban areas in Burundi and Pakistan33. Too often for the refugee, it is a dilemma between full material assistance, which could lead to dependency in the camps or no assistance at all. 52) As a 2004 UNHCR evaluation reported, the Organisation’s response to the new emergency in Chad was much too slow to start; but once the relocation of refugees was decided, it became more effective34. The assistance provided in the camps was good and beneficiaries received essential care: shelter, food, non-food items, water/sanitation, education and health services.
53) Indicators showed good quality care, close to meeting UNHCR’s standards, with a few exceptions, such as the quantity of water in some of the camps in Chad, where availability is the problem, and the quality of health care in one camp in Chad. 54) Considering the effectiveness of the UNHCR-funded shelters built in Afghanistan, evaluators believe that a more thorough technical consultation on earthquake resistance would have been appropriate, especially as ‘UNHCR’s’ design is now also used as a
standard by other organisations. 55) Regarding efficiency, evaluators do agree with the 2001 evaluators that: “A culture of cost-effectiveness needs to be developed in humanitarian C(are) &M(aintenance), to which ECHO could valuably contribute”. The use of Surge staff by UNHCR and the contracting of local NGOs are somewhat offsetting the high cost of international staff and of a MOSS compliant infrastructure. UNHCR’s decisions in Afghanistan to focus on a few key sectors, to provide returnees with cash grants instead of in-kind packages of food and non-food items, and to subcontract the actual cash disbursements to a local broker agent for a modest
33 A full registration of urban refugees in Burundi has been planned to take place in 2005. 34 Real-time evaluation of UNHCR’s response to the emergency in Chad, Evaluation and Policy Analysis Unit, UNHCR and the Regional Bureau for Africa, August 2004, EPAU/2004/07.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 28
commission are examples of efficiency. The evaluators repeatedly noted opportunities in the camps visited where partial cash payments might have saved considerable logistical costs for UNHCR or WFP, and strengthened the dignity of the refugees by allowing them some choice and avoiding the exchange of donated goods in local markets. Cash as a partial substitute to food and non-food items should be considered in situations where a local market may be able to meet the needs of the refugees. 56) Finally, long lines (see photo) were noted in one camp in Chad for some food distribution resulting from , according to the NGO managing the camp, delays in the registration and the availability of lists from the new ProGres database.
57) Assistance was provided to the local population mostly by NGO partners using their own resources in Guinea, as UNHCR gave no direct support to neighbouring villages35 but nevertheless allowed NGOs to use its UNHCR-funded vehicle and staff for this purpose. In Afghanistan, UNHCR through the construction of permanent housing, the so-called shelters and water points served returnees and the rest of the community. In Chad, in view of the precarious living conditions, 5% of UNHCR’s programme budget had been designated for development projects with the local population. In Burundi, 10% of the reintegration budget is directed to non-returnees. 58) Apart from the overall positive impact of the assistance, an unplanned adverse impact was seen in Chad, where the local population saw the firewood and water essential for its own existence being rapidly depleted by the camps to meet the unsustainably high standards often promoted by the international community.
35 According to the 2001 evaluation, in the earlier years in Guinea, UNHCR was financing “a structural, longer-term assistance provided to a basically unsustainable local health system”.
In line with its objectives of preserving the dignity of the refugees, UNHCR and its partners should carefully explore creative alternatives, such as cash or vouchers, to the traditional distribution of donations and services. ECHO’s direct support to its partners assisting both refugees and host populations is an illustration of efficient use of resources to address perceived gaps or shortcomings of UNHCR.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 29
Section 5.03 Standards for assistance 59) In January 2004, the first edition of a Practical Guide to Systematic Use of Standards & Indicators in UNHCR operations was produced. The standards were closely aligned with other international standards, such as the Sphere Minimum Standards and the Millennium Development Goals, although some of UNHCR’s indicator values are higher than those of Sphere, such as the minimum daily quantity of water. Sphere and the UNHCR guide do recommend a flexible application of the standards in coherence with the local context: “it can be the case that the
Minimum Standards exceed normal everyday living conditions. Since this can give rise to resentment, local conditions must be taken into account, and programmes should always be designed with equality of the affected and surrounding populations in mind”36. Indicators, once defined, do seem to have their own life and the evaluators could not get away from the impression that standards were seen by the field staff as rigid norms to be complied with in order to satisfy distant HQ and donors.
36 The Sphere Project, Humanitarian Charter and Minimum Standards in Disaster Response, Geneva, Switzerland, 2004.
Evaluators do recommend shifting away from a dogmatic use of UNHCR or SPHERE standards towards a more contextual and flexible utilization.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 30
Section VI. LRRD
Section 6.01 Definition and mandate The High Commissioner is mandated to “seek permanent solutions for the problem of refugees”. 60) The 4Rs strategy of UNHCR refers to the linkages between the four processes of repatriation, reintegration, rehabilitation and reconstruction. “UNHCR is a major player in one and a half of the 4Rs: repatriation is a prime UNHCR responsibility but UNHCR engages only in” initial” reintegration37”. 4Rs is the equivalent of LRRD used by the EC to underline the continuum from relief to development. Convention Plus (C+) is an international effort initiated by UNHCR to improve refugee protection worldwide and to facilitate the resolution of refugee problems through multilateral special agreements. The concepts and goals of C+ and 4Rs are not innovative but their marketing is.
Section 6.02 LRRD efforts of the European Union
61) The evaluators in 2001 did not direct any specific recommendation to UNHCR. The 2005 observations and recommendations can be found in the annexes on Afghanistan, Chad and Guinea. The present section will focus on ECHO LRRD efforts. 62) In all countries, the EC delegation was actively seeking funding mechanisms to assist the host communities. The catalytic role of the ECHO/TAs was essential. Their personal relationship with the senior officials of the delegation, as well as their
seniority and experience, were vital to the success of these efforts. Mechanisms used by each delegation were distinct: EDF in Guinea, for the province of Guinée Forestière38, and in Burundi, the use of the Aid for Uprooted People (AUP) budget line in Afghanistan (see country annex). 63) Two countries, Norway and Denmark, created a fund to support transitional work bridging relief to development. In our opinion, a similar transition financial mechanism should be adopted by EU Member States and the EC even if it adds another dimension in the coordination process towards a seamless EU response.
37 Repatriation and Reintegration: An appraisal of progress in Afghanistan, a report commissioned by the UNHCR Branch Office in Kabul, March 2005. 38In final negotiation step at the time of the visit (European development funds were suspended in Guinea)
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 31
Section VII. Monitoring, Reporting, and Evaluation Monitoring and reporting were seen in the 2001 report, as “a conflicting issue”. Although, it seemed less acute in 2005, it still needs to be addressed in a comprehensive manner covering both internal and external reporting.
Section 7.01 Monitoring 64) Both partners agree for the need of SMART indicators in proposals submitted to ECHO, internal Country Operational Plans (COP) or Logical Frameworks to monitor progress and measure results rather than outputs. For this purpose, UNHCR has issued a guide with 52 standards and core indicators. Reportedly, those indicators were tested in all camps with more
than 5000 refugees. In 2003, 50% of the questionnaires received in HQ provided data for 75% of the indicators. Compliance declined in 2004. In Chad, a pilot country, the evaluators observed serious practical difficulties in collecting detailed sectorial and protection data for the indicators. If ECHO finds that UNHCR’s monitoring of its implementing partners is deficient, NGOs found the demand for data excessive and the interpretation and feedback disappointing. They also complained about the nature of the indicators, as being too quantitative, not qualitative. In our opinion, the entire set of indicators
should be drastically reduced. Protection indicators (26 conceived in HQ, 12 selected for field implementation) remain impractical and need simplification. Similarly, the indicators presented in most of the Country Operation Plans may appear impressive, but they are inapplicable in reality.
Section 7.02 Reporting 65) Reporting from UNHCR to ECHO is a contentious issue. Delayed reporting has been a significant complaint of ECHO over the past years. Factors are the need for UNHCR to receive and compile the reports from its implementing partners and UNHCR’s centralized approval process. The timeliness of reports has improved lately. Similar progress should also be made by
UNHCR should start from a modest set of indicators and parameters, ensuring full compliance by its partners and acceptance by all donors. ECHO should consider providing thematic funding to fine tune the standards and indicators and serve as agent of change for their adoption.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 32
ECHO in acting promptly upon the reception of those final reports39. Quality of UNHCR reporting nevertheless remained poor. The sample of reports reviewed by the evaluators contained mostly official statistics; statements of the mandate and generic description of activities with little effort to identify lessons learned and operational difficulties. In this regard, reports were not much different from the original project proposals. Reports on activities funded under the thematic funding mechanism are generally more informative, although they tend to be focused on specific outputs in response to queries and concerns from ECHO. 66) The format of these reports is the core of the problem. On one side, ECHO aims to standardize the reporting from all its partners, from small NGOs to large internationally mandated organizations, into one single format. UNHCR, faced with a large diversity of donors pursues the same goal to standardize its reporting through the promotion of its own global report as a single reporting tool for all donors. Both positions, though advantageous from an internal administrative point of view, are mutually incompatible. 67) In its mission statement40, ECHO considers that it has “a special responsibility in the implementation of the Good Humanitarian Donorship Principles and Implementation Plan” adopted in June 2003.41. Of particular interest here, is the principle 23 which “encourages the development of standardized formats for donor reporting”. Adopting one single format for all donors is meant to facilitate the reporting by humanitarian actors and should be actively supported by ECHO. However, the present UNHCR Global Report does not adequately reflect the results, successes and shortcomings, achieved with donor funds. In a true partnership, both parties must find the best compromise meeting the needs for transparency and accountability in terms of results rather than processes. 68) Excessive reporting is not without consequence for the refugees, as UNHCR field staff consistently complained of an increasing overload of work for reporting. “Time spent in the office is time not spent in the field where staff are needed to provide protection and services” 42. 69) Some UNHCR interlocutors suggested the convenience of increasing the number of field posts for the preparation of reports to donors and ECHO in particular. There is a purposely tight separation maintained by UNHCR between the function of donor relation and reporting, and the protection and assistance functions. Unless those report writers have also an operational protection or assistance responsibility, they tend to serve the institutional interest of donor relation rather than contribute to lessons learning, dialogue and genuine transparency. 70) Identifying and consolidating formats that would serve both internal and external reporting needs would go a long way to increase UNHCR staff’s field presence. This streamlining of UNHCR reporting requirements (internal and external) can only be done with the 39 On July 2005, UNHCR reported outstanding payments for the amount of almost € six millions, of wich reportedly there had been no communication from ECHO for over 90 days on payments for € 2.4 millions.. This backlog of closures is currently addressed by ECHO-6. . 40 ECHO AID STRATEGY 2005 (22/12/2004) 41 Principle 10 reads: Support and promote the central and unique role of the United Nations in providing leadership and co-ordination of international humanitarian action 42 Reportedly, the most time consuming demand for reporting would be generated by UNHCR HQ vertical programs
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 33
pressure and support of the donor community, and ECHO in particular – another candidate for Thematic Funding?
Section 7.03 Evaluation 71) The team was favourably impressed by the quality and independence of two key evaluations carried out by UNHCR (Chad 2004 and Afghanistan 2005). Additional evaluations on specific issues were also made available. Those documents served as institutional memory in an organisation with rapid turnover of staff and are quoted in this report. “Repeated attempts by UNHCR at securing the participation of a team member selected by a donor [in the Appraisal of Progress in Afghanistan] proved unsuccessful”43. Reportedly, ECHO was also approached by UNHCR. Perhaps the evaluation units in UNHCR and in ECHO should centralise all requests initially directed to the geographical units or the TA in the country. An active support from ECHO to similar initiatives should be encouraged.
43 Para 8, Appraisal Progress in Afghanistan (march 2005)
Given the relevance of evaluations and the positive record of UNHCR to evaluate objectively its own activities, the responsible ECHO and UNHCR desks should increase their communication, inform each other of the different evaluation activities and eventually consider joint evaluations in the future.
ECHO and UNHCR should agree on a common results-based format for reporting. Although recommended by the 2001 evaluation, the present Global Report from UNHCR is not an alternative, nor is a single ECHO form suitable for NGOs projects as well as for the multi-donors programmes of large mandated international organizations. ECHO should offer thematic funding to UNHCR in order to streamline its internal andexternal reporting to alleviate the burden on UNHCR field staff and increase the organization’s transparency.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 34
Section VIII. Thematically funded activities
Section 8.01 Background 72) The 2001 evaluation recommended: “ECHO to become a more predictable donor and concentrate funding on UNHCR activities with optimum added value”. ECHO, in addition to its traditional geographical funding, introduced in 2002 a new horizontal co-financing instrument called thematic funding (TF) “that supports international organisations to improve their performance with start-up, innovative or other suitable measures that provided added value to their core humanitarian response capacities and that have a multiplying effect on the wider humanitarian community”44. 73) UNHCR was the first to have received Thematic Funding and is still the only organization benefiting from three ECHO funding decisions: M€ 11 in 2002 (100% of the TF envelope), M€ 11 in 2003 (66%) and M€ 5 in 2005 (the percentage will fall dramatically as the number of partners benefiting from this type of funding increases)45 .The three TF decisions focused on the following four projects:
• Protection Posts: a budgetary support for protection and protection-related posts Support was provided from November 2003 to December 2004.
• Surge capacity mechanism: a deployment scheme to meet sudden unanticipated need for protection staff. Support has been provided since the end of 2003.
• Project Profile: a globally standardized registration system of persons of concern to UNHCR. Support has been provided since 2002.
• Staff Security: compliance with the UN Minimum Operating Security Standards (MOSS). Support was provided in 2002 and 2003.
Section 8.02 Protection staff deployment and training
74) ECHO supported 130 international and national protection and protection-related46 posts over a 13-month period in thirteen African countries47. These posts, for the most part, already existed. This allowed UNHCR increased flexibility in the use of un-earmarked funds from other donors. During the ECHO 2004 monitoring in Guinea and at several UNHCR-ECHO SPD, the impact of such support was discussed. This resulted in the observation of the distribution imbalance between Headquarters, capitals and field; the number of unfilled posts that have led to protection gaps; and the costly and frequent use of staff on short-term missions.
44 ECHO’s Note for the file. Thematic/Programmatic Funding: State of the Play and Outlook 2005/2006. The actions envisaged under TF are covered by article 4 of the Council Regulation (EC) N° 1257/96 on humanitarian aid. 45 Between 2002 and 2004 UNHCR received 66% of the overall TF envelope: it was the only international organisation to benefit from it in 2002 while in 2005 a total of 10 ECHO partners are entitled to thematic funding. 46 Protection-related posts cover all assistance activities including sectorial and cross cutting issues 47 47The following countries were identified as areas of the greatest unmet needs: Central African Republic, Chad, Côte d'Ivoire, Ethiopia, Guinea, Kenya, Liberia, Rwanda, Sierra Leone, Sudan, Somalia, Republic of Congo, and Uganda.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 35
75) The key issue is whether this budgetary support has allowed UNHCR to improve its recruitment and deployment capacity and ultimately to increase its overall field presence. It is only possible to answer this through careful analysis of the global data rather than from anecdotal observations in the field. Table 1, compiled from a large number of UNHCR documents, suggests the following:
• From January 2000 to the end of 2003, the caseload and number of protection staff tended to decline, while the amount of voluntary contributions received was rapidly increasing.
• The rate of protection posts vacant at the end of the year was being reduced by UNHCR before the TF contribution was in effect.
• The number of posts sharply increased at the end of 2004. Table 2 Trend in the number of protection or protection-related posts
1999 2000 2001 2002 2003 2004 Contributions (US$ M.)1 703 780 820 945 961 Persons of Concern (M.) 21.8 19.7 20.5 18.0 17.1 17.5 Protection Posts 2
Total Field
356 324
348 317
319 289
321 286
330 296
436 400
Posts filled Total Field
311 283
306 274
285 257
289 261
311 283
384 345
Vacancy rate in the field 12.6 13.6 11.1 8.7 4.4 NA3
Number of posts (total) Per million Persons of Concern
16.3 17.7 15.6 17.8 19.3 24.9
1 Voluntary Budgetary contributions and Persons of Concern estimated on 31 December. 2 Posts are reported on the first of January of the following year. JPO or non-professional staff are not included 3 A large number of new posts has just been established, effective 1st January 2005, making an estimate of the vacancy rate meaningless. This data suggests that the objective of substantively increasing the capacity of UNHCR’s protection field presence in the field was achieved at the end of the TF project. Credit should, however, be shared with other donors, such as BPRM, who funded the recruitment of 27 new protection posts at field level – of which 12 were to be mainstreamed in the UNHCR budget within a year.
ECHO should continue funding protection posts in new interventions or in ongoing programmes in forgotten crises. Protection posts should be covered by geographic rather than thematic funding.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 36
Section 8.03 Surge 76) “In the case of suddenly emerging needs, qualified persons with experience in protection and related fields will be deployed through the Surge Project for a short period of time to assist in selected interventions. Surge is used only in situations where the demands for protection staff exceed UNHCR capacity to respond in the short term”48. IRC, the implementing agency, maintains a roster of over 160 pre-screened professionals. Candidates from the roster are proposed to UNHCR, and upon approval from UNHCR, IRC recruits and deploys the Surge protection officers. Those deployees, although not UN staff, for all practical purposes function as such. Since this short-term field deployment scheme was introduced in July 2001, through April 2005, the total number of deployments has reached 127, with good gender balance and nationality distribution. Their skills and functions span the entire scope of UNHCR activities, i.e. protection as well as assistance. 77) Surge provides an efficient tool for UNHCR, as the average time from request to deployment averages 36 days. Since the end of 2003, ECHO TF has become the largest source of funding for the Surge project. It is also a cost-effective tool: it provides highly motivated staff who “do not hesitate to go deep in the field” at less costly salary scales compared to UNHCR regular staff, too often bound to his/her desk by administrative tasks. 78) As with any good administrative tool, Surge has also provided an element of convenience to managers. UNHCR managers have on some occasions used Surge to respond to field request (e.g. Panama and Venezuela) without making clear to donors their criteria for prioritization. In Chad and Burundi, the creation of new UNHCR posts has been delayed, because the deployment of Surge provided a temporary fix. Tenure, normally 6 months, was almost routinely extended to the limit of 11 months and, occasionally, a new Surge candidate was requested to fill the gap left by a departing Surge deployee. As a first step towards the development of an explicit set of criteria for future deployments, UNHCR should share with ECHO the total number (broken down by geographical areas) of requests received for Surge posts. 79) For the deployed personnel, it is an opportunity to work with UNHCR. For UNHCR it is a practical source for external recruitment – it injects fresh blood into protection activities and field test them for possible recruitment.
48 From Decision number: ECHO/THM/210/2003/2000
UNHCR should recognize that Surge and other mechanisms are only palliative solutions. UNHCR should revise its global human resource management to ensure the presence of its own staff in the camps. ECHO should continue funding Surge under the TF mechanism.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 37
Section 8.04 Project Profile 80) Since 2002, ECHO has substantially supported a new standardised and uniform registration system with an investment of 5.8M€. The ECHO contribution represents 64% of the total donor contributions between 2002 and 2005 and may exceed 80% in 2005. This innovative project will still need considerable investment before it is mainstreamed in UNHCR country budgets in 2007. The software, ProGres, was developed in 2003 and tested in Ghana and Turkey by mid-2004 and then rolled out49 in 17 other countries. In 2005, 10 follow-up missions will be carried out, after which UNHCR plans to roll out the software in a further 30 countries. During the field visits, the level of project implementation varied, human resources were often inadequate and the needs not sufficiently anticipated or planned for.
81) Significant management and protection gains can be anticipated from the application of a standardized system for registration, identification and documentation of individuals rather than households. Caseload administration and services offered to the refugees, from food distribution to repatriation will improve also. It should be particularly helpful managing UNHCR’s complicated urban refugee caseload across the world. However, the added cost of maintaining the database will be significant and is something not yet foreseen by UNHCR. Every department in UNHCR and every donor expects that Profile will provide them with personalized detailed data which will meet their particular sectorial or issue-based interest. This high level of expectation may not be realistic. Profile must concentrate on the core data (name, date of birth, gender, nationality and country of origin), while not raising too much the hopes and expectations of all the other parties involved, for any type of additional information. 82) . Project profile is designed to incorporate the future use of biometrics. One useful electronic intervention witnessed is the iris scanning done in Pakistan before repatriation. Acting 49 Rolling out means electronically transferring the data from the existing local databases (in variable format) into the ProGres format and training the staff in the use of the software.
Earmarked contributions to Project Profile 2002-2004
0%10%20%30%40%50%60%70%80%90%
100%
2002 2003 2004
AustraliaNetherlandsUSAECHO
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 38
as a deterrent against fraud, this technology has reduced recycling and double registration to nil. No data could be obtained to analyse the cost/returnee using this technology, but it would be useful to investigate it. The evaluators have no factual evidence to comment on the use of fingerprint technology regarded by UNHCR as more cost-effective, easier and more reliable than iris recognition. 83) Project Profile’s main benefits are in providing individual and personalised documentation for each refugee (instead of household as in the old system) and in facilitating and improving the management of protection on a global basis with a web-based system. It provides a database support to the existing registration mechanism of UNHCR. It should not be seen primarily as a tool for reducing double registration and assisting in the distribution of assistance, though it will help. Much of that work still needs to be done by people on the ground with the traditional canvassing, which is well done by UNHCR. As indicated by UNHCR, where ProGres has been introduced, ”not all numbers have gone down – but knowledge about caseloads has improved significantly”.
Section 8.05 Security
84) ECHO funded 60% of UNHCR’s global programme for staff safety and security funding, which was essentially invested in direct expatriate and local staff costs and equipment to comply with the UN Minimum Operating Security Standards (MOSS). These personnel costs amounted to €1,867,257. The ECHO contribution allowed UNHCR to flexibly meet new and unplanned equipment and human resources requirements. In the countries visited, UNHCR occasionally provided physical security advice to partners’ staff, and several of them received equipment. Before staff can go to the field they must successfully complete an interactive course available on CD-ROM.. Completion of the test is compulsory for all UN staff and permits UNHCR security advisors to concentrate their training on action-learning techniques.. 85) At field level, it is impossible to determine which security expenses have been charged to TF. It is difficult to assess how much ECHO’s investment has improved the safety of the staff. The global number of security incidents reported to the UN reflects the level of violence and insecurity, not necessarily the effectiveness of the UN’s security. UNHCR HQ and field-based Field Safety Advisers agreed that “MOSS has contributed, among other factors, to a bureaucratization of security, there is a natural level of non-MOSS compliance, since funding is not always guaranteed”. Security rules and regulations, when applied too rigidly, can have adverse implications on UNHCR operations. They tend to hamper the delivery of protection and
UNHCR should provide ECHO with a clear estimate regarding the amount and source of the human resources required to maintain this system’s core data. Meanwhile, ECHO should continue supporting Project Profile under TF. UNHCR must diversify its sources of funding in order to avoid donor dependency.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 39
material assistance in high-risk areas such as Afghanistan or Burundi, as compared with their implementing partners50.
Section 8.06 Key conclusions and recommendations 86) For ECHO to be an influential actor in the overall humanitarian community it should have the flexibility to channel significant and sustained funding to correct what it perceives to be shortcomings in its major partners. This will require a high-level dialogue and mutual respect among two equal partners with similar goals. It also demands a coordinated approach with other major donors. This investment should be judged over the long term, reflected in trends, and not on the minutiae of detailed technical or financial reports. 87) ECHO has established seven criteria for the selection of activities to be funded under TF. They include the likely impact on the victims of humanitarian crisis, the relevance for the partner’s mandate, the improvement of the partner’s core capacities, the project feasibility, the implementation capacity of the partner, the partner’s willingness to share the results and finally the budgetary impact on the capacity to respond quickly or to develop flexibility in its allocation of resources. The evaluators suggest adding the principle of “subsidiarity”, because TF should support projects that cannot be funded by geographical units primarily, given their broad geographical scope or their capacity-building nature. Table 2 shows the compliance of the four projects with the “seven plus one” criteria. Table 3 Criteria for thematic funded projects
Criteria Protection posts Surge Profile Security Impact on victims YES YES YES indirect Relevance to mandate YES YES YES YES UNHCR core capacities YES YES YES YES Feasibility & risk YES YES Uncertain YES Implementation capacity YES YES YES YES Sharing results - - YES YES Budgetary impact YES YES YES YES Principle of “subsidiarity” NO YES YES NO Projects such as funding protection posts or security improvements do not meet the principle of subsidiarity as they can (and therefore should) be funded by ECHO Geographical Units. ECHO should exercise flexibility in the duration of its TF. Although the concept is to support innovative ideas for their initiation, withdrawal on schedule from some projects could be 50 For a global analysis of the delivery of aid vital operations, see “Maintaining a UN humanitarian presence in periods of high insecurity: learning from others”. OCHA, May 2004.
Security interventions should be included in proposals submitted to ECHO geographical units and not be funded under the thematic funding instrument.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 40
counterproductive and detrimental to their long term success.. Key objectives for strengthening the partner’s capacity cannot always be achieved within three years. 88) In addition to the topic of emergency preparedness funded from 2005, areas for future thematic funding for UNHCR include:
• The strengthening of capacity to offer protection to refugees and returnees outside the camps: UNHCR in many countries was not able to provide minimum protection (registration) to refugees at the border or in urban areas.
• A greater and sustained investment in developing the capacity of the national counterparts; a strong and competent governmental counterpart can only facilitate ECHO’s and its partners’ programmes.
• The adoption and testing of a limited number of standards and indicators, which would lead to a result-based management approach. At present, the sets of indicators included in the Country Operation Plans or introduced in the camps are excessively ambitious.
• The review and simplification of the internal and external reporting system in UNHCR. Reporting requirements, mainly internal, are keeping the field staff at their desks rather than in the camps. In line with its commitment to support the Good Donorship principles and work plan, ECHO should join with other donors and offer a complete review of the UNHCR reporting system, aiming for simple and informative reports for both internal and donor use.
Thematic funding is currently the only tool available to ECHO for helping its partners to build their capacity and address areas in need of managerial improvement. This tool should be maintained and strengthened. Criteria should be used with flexibility.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 41
Section IX. The partnership: A look to the future
Section 9.01 Importance of this partnership 89) Each partner sees this partnership in the context of its own mandate and priorities. ECHO, as a “global player and a reference donor”51 and UNHCR, as an agency with a unique international mandate, have much at stake in this partnership. UNHCR values the relationship with ECHO more for what it represents than for its financial importance52. Both ECHO and UNHCR are major policy actors and have no alternative but to collaborate as equal partners, each one implementing its own programme in complementary and synergic manner. 90) The overall influence of ECHO on the policies and programmes of UNHCR could be greater. As indicated above, excessive focus on direct or visible impact on end beneficiaries, unilateralism through ECHO’s own implementing partners and perhaps the technical status of ECHO field representatives may have contributed to this perceived lack of influence. On UNHCR’s side, a strong corporate culture, sometimes perceived by operational as well as implementing partners as bordering on arrogance, makes persuasion at high level the most productive approach from donors or governments wishing to address constructively its major shortcomings. 91) Still, there will be limits to this cooperation, as ECHO’s mandate and priorities may not always neatly coincide with those of UNHCR. Areas of converging interest should be consistently agreed at strategic level rather than decided at operational level. For instance, if UNHCR’s assisting IDPs should be welcome by ECHO, in the absence of other competent party, its venture in development projects or in post tsunami relief could be challenged.53.
Section 9.02 Supporting NGOs directly or through UNHCR ? 92) ECHO funding NGOs partners, directly or through UNHCR, has advantages of its own for the refugees and ECHO. The right mix boils down to how ECHO sees itself:: “Should ECHO be a mere banker, or a ‘gap-filler’ (in political and/or funding terms) …or more importantly, should it aim to be a privileged, catalytic partner (to the UN, NGOs, sister
51 Mission Statement 2005 52 5% of UNHCR annual budget 53 Yet, definitely assisting host communities may be critical, as in Chad, but UNHCR should pass along, with the funding, the full responsibility for negotiating, approving and supervising the many small projects to other large UN development agencies or NGOs.
ECHO’s influence on large UN agencies is limited, when compared to the European level of funding and humanitarian tradition. ECHO should increase its strategic support to UNHCR in order to address the shortcomings of this key partner
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 42
Commission services and Member States) that facilitates high quality aid for the most vulnerable, where, when and how required?”54 93) To avoid undermining of UNHCR authority in the field, ECHO should either channel most of its funding through this coordinating authority or contractually request its direct partners to abide by UNHCR authority. Funding NGOs directly will remain the best alternative when addressing, for instance, immediate life-saving needs or providing assistance to groups overlooked or unattended by UNHCR. Programmes directed at both refugees and host communities are prime examples. Funding of the same NGO programme by both ECHO and UNHCR should not be seen as a risk of duplication, provided transparency is maintained by both partners. 94) The Table 2 below lists the perceived comparative advantages of channelling ECHO assistance funds through UNHCR or directly to NGOs: UNHCR remains the only alternative for legal and physical protection. Table 2 Relative advantages of funding UNHCR or NGOs
54 Evaluation of the commission humanitarian aid under article 20 of regulation 1257/96, (1999)
UNHCR advantages NGOs advantages
A mandated UN agency with responsibility to coordinate all actors and with regular access to senior government officials
Operational ECHO-partners are independent
Exclusive competency responsible for the protection of the refugees.
Specialized skills and capacity to deliver rapid and effective assistance
Strong skills in camps setting and management on a global scale
Often but not always lower operating costs
Supervision capacity and field presence ensuring consistent standards within and between camps
Able to meet the needs of host populations and other groups not reached by UNHCR.
Sub-contracting and capacity building of national partners
More presence in the field especially when security is a concern
Centralized (low cost) procurement Project submittal and negotiation are faster Flexibility of line items changes once the project has been approved by UNHCR
Greater control and visibility for ECHO
Simple process for ECHO: one large project with the burden of field monitoring placed on a trusted partner.
Detailed and direct monitoring / auditing by ECHO
The benefits of a partnership
The control afforded by a contractual agreement
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 43
Section 9.03 ECHO funding mechanisms In addition to the robust and tightly managed geographical funding (via the geographical units), two other mechanisms are potentially available to ECHO: thematic funding (see section 8) and the proposed programmatic funding 95) Geographic funding tends to benefit well-circumscribed activities with concrete, i.e. measurable or visible, immediate benefits to a well defined group. From an auditor point of view, it has definite administrative advantages. From a strategic point of view, ECHO loses some of its strategic ability to influence UNHCR’s overall policy and coordination. ECHO will be perceived more as the co-owner of some projects artificially identified within a broader programme than a trusted stakeholder. When possible, funding of specific projects should be substituted with blanket programmatic funding of specific appeals. 96) Programmatic funding is designed to finance a percentage of the humanitarian appeal (CAP or similar) for a given country or region. As such, it should remain under the geographical units. Under the FAFA procedures, this support needs not be earmarked for narrowly defined activities. Perhaps, ECHO would benefit to adopt what has been called negative earmarking: i.e. excluding specified expenses incompatible with the regulations or policies of ECHO. In the case of UNHCR, ECHO’s loss of direct control and monitoring will be more than offset by the benefits of policy influence over the principle actor in matters of refugee and IDPs. As said by ECHO, its core function does not limit ECHO to assistance to end of line beneficiaries. Sometimes reforming the health insurance system will serve the sick more than providing direct care and medicines.
To achieve its objectives of closer collaboration and have an impact on the decision making in UNHCR over the long term, the EC may need to act more and more as contributing States by providing a regular, predictable contribution or quota to the core budget. The creation of an EC budget line for an EU annual contribution to UNHCR will, one day, be ineluctable. 97) The thematic funding should remain a mechanism to address specific systemic or global needs and therefore continue to be managed collegially between ECHO units. Its management and evaluation should be judged by its impact, that is by whether positive changes have been achieved and, if not, why. The thematic funding has been conceived as a vehicle to strengthen the capacity of mandated partners in areas of concern to ECHO. It should not be reduced to a mere technical instrument by focussing on detailed outputs or temporal results (seeing the tree and not the forest).
ECHO geographical units should increasingly opt for “lightly” earmarked programmatic funding to the global appeal of UNHCR. If need be, ECHO should consider adjusting the financial implementing procedures of the FAFA. For this purpose, UNHCR should accelerate its adoption of a genuinely need based planning and results based monitoring.
Final Report Evaluation of the Partnership between ECHO and UNHCR (2002-2004)
ECHO/ADM/BUD/2004/01212
SHER Ingénieurs-Conseils s.a. 44
Section 9.04 A matter of trust and transparency 98) The key component to further strengthen the collaboration between ECHO and UNHCR is to improve communication and mutual trust among partners. The evaluators are concerned by the miscommunication between the management staff of the HQs of the two agencies. They do not always share nor understand the terminology, and nor the basic culture of the other party. As one said, “You need an interpreter between the EC and UN”. There is a similar issue of communication at the operational level between UNHCR and some of its partners (donors or NGOs). For multi-donor operations, the establishment of a technical steering or monitoring committee with all key partners (implementing partners included of course) would offer UNHCR a convenient mechanism for collective consultation prior to reaching decisions on strategic or operational matters. 99) Garbled communication leads to mistrust and an inclination to dwell on the perceived shortcomings of the other. At UNHCR this manifests in an overly defensive attitude. Partnership precludes a financial donor-grantee relation in which one is simply the contractor of the other. It applies to the relation between ECHO and UNHCR as well as between UNHCR and its own partners. This parallel error in the respective relationships was striking in the field. In an environment where UNHCR is regularly under-funded, it cannot afford to turn its back on ECHO. On the other hand, ECHO should remember that this is a partnership with an internationally mandated organisation. Gauging the importance of a partnership between ECHO and UNHCR merely in financial terms would be serious misjudgement. 100) The Strategic Partnership Dialogue (SPD) established in 2001 has helped improve dialogue and communication between the partners. However, it seems that its format has become too formalized and focused on specific interventions. The audience is perceived as too broad reducing the exercise to an operational review of the refugee situation at field level. The present SPD meetings are necessary but they are not responding to the need for a forum at the senior policy level between head and principal managers in an atmosphere suitable to identify and solve the real obstacles for a closer partnership! In this forum, decision-makers should feel free to admit their own institutional constraints, while seeking common solutions wherever possible. Meetings should be held alternatively in Brussels and Geneva. 101) Both ECHO and UNHCR have great achievements to their credit, resulting in legitimate pride. They also have inherited rigid and, at times, counterproductive procedures that cannot be changed overnight. Protection and assistance to refugees can only benefit from a stronger long-term partnership.
All three proposed or existing ECHO funding mechanisms should be applied in the partnership between UNHCR and ECHO. Thematic funding should continue to be managed by the existing committee under ECHO-4 while the proposed programmatic funding would be best placed under the respective geographical units.
Final Report
Evaluation of the Partnership between EC
HO
and UN
HC
R (2002-2004)
ECH
O/A
DM
/BU
D/2004/01212
SHE
R Ingénieurs-C
onseils s.a.
45
Recom
mendations* from
the Evaluation of the Partnership between EC
HO
and UN
HC
R
(2002 and 2004)
ECH
O
U
NH
CR
U
NH
CR
’s Capacity
ECH
O should realize that in legal or operational term
s, no other single EC
HO
partner can match the capacity of U
NH
CR
to provide protection and assistance to refugees. E
CH
O m
ay choose to supp
ort only som
e of its partner activities but it shou
ld refrain from
using a term
inology sug
gesting that there are differe
nt types of U
NH
CR
man
dates “S
hould
IDP
s fall form
ally under U
NH
CR
responsib
ility?” rem
ains an o
pen
question that the internatio
nal comm
unity, an
d EC
HO
as one of the agents for
change, m
ust eventually a
ddress on a principled, ra
ther than ad h
oc, basis E
CH
O sho
uld carefully m
onitor a new
staffing system
for hum
anitarian or protection a
gencies consid
ered by the UN
Secretaria
t
Coordination
ECH
O should im
plement the m
easures recomm
ended in the 2001 evaluation: •
To
en
su
re th
at U
NH
CR
is p
rovid
ed w
ith co
mp
lete an
d
transp
arent in
form
ation
regard
ing
all EC
HO
fun
din
g
• T
o req
uire E
CH
O d
irect partn
ers to sig
n a M
emo
rand
um
o
f Un
de
rsta
nd
ing
with
UN
HC
R re
ga
rdin
g co
ord
ina
tion
EC
HO
should financially an
d politically enco
urage U
NIC
EF
and WH
O to either
fully assume the opera
tional role or accep
t the op
erational authority of U
NH
CR
UN
HC
R
should treat
its “im
plementing”
partners as
…
partners not as contractors. U
NH
CR
should greatly improve its exchange of inform
ation w
ith partners and donors U
NH
CR
should revise its Mem
orandum of U
nderstanding w
ith UN
ICEF and W
HO
to ensure their predictability and com
mitm
ent in education and health assistance respectively
Final Report
Evaluation of the Partnership between EC
HO
and UN
HC
R (2002-2004)
ECH
O/A
DM
/BU
D/2004/01212
SHE
R Ingénieurs-C
onseils s.a.
46
EC
HO
shou
ld prag
matically
seek the
bala
nce betw
een
unilatera
l an
d
multilatera
l funding m
ost likely to benefit the refu
gees w
hile strength
ening the
intern
ational coord
inatio
n. It should be a plann
ed policy, not driven by the flow
of prop
osals or the availability of fund
ing
A m
ulti-ye
ar comm
itme
nt is required to strengthen the local cap
acity of the
nationa
l counterp
arts. This activity m
ay be a good ca
ndid
ate for TF
Where not do
ne yet, Bra
nch Offices o
f UN
HC
R shou
ld desig
nate an existing
staff me
mb
er to act also as NG
O Lia
ison Officer. S
/he should have
a particu
lar capacity to
listen, dialogu
e and influence o
perational d
ecision m
aking in his/her ow
n organisation
UN
HC
R’s ca
pacity build
ing support to local N
GO
s is appreciated, but also
self-serving: w
orkshops on accounting or supp
ly ma
nagem
ent are inten
ded
more to m
aintain acceptab
le accounting stan
dards, than pre
pare for a future w
ithout internation
al sponsors. In line
with its com
mitm
ent to capacity buildin
g, U
NH
CR
should d
o both
UN
HC
R staff shou
ld better appreciate and utilize th
e closer exposure of the
camp m
anagers to the re
alities.
Th
e UN
Co
ord
inato
r or O
CH
A sh
ou
ld en
sure th
at the refu
gee assistan
ce is con
gru
en
t with
the n
eeds o
f the lo
cal po
pu
lation
an
d/o
r the lo
ng
-term
develo
pm
ent p
riorities
Legal and Physical Protection of the Refugees
EC
HO
should increase its relative investm
ent in pro
tection and coord
ination
activities with U
NH
CR
U
NH
CR
should
increase the
resources assigned
to protection, especially of those refugees outside the cam
ps. U
NH
CR
, with
in the UN
system, re
mains the best e
quipped agency to answ
er to th
e needs of ID
Ps
T
he
EU
sh
ou
ld b
ols
ter its ca
pa
city
to m
ob
ilize p
olitic
al s
up
po
rt for U
NH
CR
Assistance to R
efugees U
NH
CR
and donors should adopt a more contextual and
flexible use
of existing
international standards
and indicators, w
hich should not be seen as norms to be rigidly
applied EC
HO
should continue to support directly NG
Os providing
assistance to host comm
unities.
UN
HC
R and donors should adopt a m
ore contextual and flexible
use of
existing international
standards and
indicators, which should not be seen as norm
s to be rigidly applied U
NH
CR
should reduce the broad gap between the assistance
provided to the refugees in the camps, the standard of living
of the local population and the situation that the refugees
Final Report
Evaluation of the Partnership between EC
HO
and UN
HC
R (2002-2004)
ECH
O/A
DM
/BU
D/2004/01212
SHE
R Ingénieurs-C
onseils s.a.
47
Regard
ing efficiency, evaluators do agre
e with th
e 2001 evalu
ators that: “A
culture o
f cost-effective
ness
needs to
be deve
lope
d in
hum
anitarian C
are
&M
aintena
nce, to wh
ich EC
HO
could valuably contrib
ute”
may expect on their return to their country of origin
Regard
ing efficie
ncy, evaluators do agree
with th
e 200
1 evaluators that: “A
culture of cost-effectiveness needs to be
develop
ed in huma
nitarian C
are &
Ma
intenance, to w
hich E
CH
O could valuab
ly contribute
” U
NH
CR
should b
e careful to concentrate on the specific w
ork for wh
ich it was
created, and avoid spreading its resources too thinly.
Un
der so
me circu
mstan
ces, mo
re creative app
roach
es, su
ch as u
se of cash
, may b
e mo
re e
fficie
nt to
as
sis
t refu
ge
es
an
d
loc
al c
om
mu
nitie
s. Th
es
e a
ltern
ativ
es s
ho
uld
be ex
plo
red in
the fu
ture.
LRR
D: A
transition fin
ancial m
echan
ism in M
ember S
tates and in th
e EC
would facilitate LR
RD
even if it adds an
other dim
ension in the coordin
ation
process tow
ards a seam
less EU
response.
Monitoring, R
eporting and Evaluation EC
HO
and
UN
HC
R
should increasingly
carry out
joint evaluations. EC
HO
, in compliance w
ith its comm
itment to abide by the
“Good
Donorship”
principles and w
ork plan
Stockholm,
2003, should support UN
HC
R in stream
lining the use of indicators as w
ell as its reporting system.
EC
HO
an
d U
NH
CR
sh
ould
agree on
a com
mon
results-based
forma
t for
reporting. Althou
gh recomm
ended by the 2
001 evaluatio
n, the presen
t Glo
bal R
eport from
UN
HC
R is n
ot an alterna
tive, nor is a single E
CH
O form
suitable for
NG
Os
projects as
well
as for
the
multi-do
nors program
mes
of
large
mand
ated intern
ational org
anizations.
ECH
O
and U
NH
CR
should
increasingly carry
out joint
evaluations. U
NH
CR
reporting
to E
CH
O
has been
a conten
tious issue.
Althou
gh the
tim
eliness has im
proved, the conte
nt needs further atte
ntion
UN
HC
R sho
uld start from a
modest set of ind
icators and para
meters, ensuring
full com
pliance by its p
artners and acce
ptance by a
ll donors.
EC
HO
an
d U
NH
CR
sh
ould
agree on
a com
mon
results-based
forma
t for
reporting. Althou
gh recomm
ended by the 2
001 evaluatio
n, the presen
t Glo
bal R
eport from
UN
HC
R is n
ot an alterna
tive, nor is a single E
CH
O form
suitable for
NG
Os
projects as
well
as for
the
multi-do
nors program
mes
of
large
mand
ated intern
ational org
anizations
Thematically Funded A
ctivities U
NH
CR
and ECH
O should agree upon a set of indicators to
prioritise future Surge deployments.
ECH
O should continue funding Surge and the Project Profile
under the TF mechanism
.
UN
HC
R and EC
HO
should agree upon a set of indicators to prioritise future Surge deploym
ents. Protection posts and security m
easures should be included, as appropriate, in projects subm
itted to ECH
O G
eographical U
nits. This
aspect should
not be
covered by
the TF
Final Report
Evaluation of the Partnership between EC
HO
and UN
HC
R (2002-2004)
ECH
O/A
DM
/BU
D/2004/01212
SHE
R Ingénieurs-C
onseils s.a.
48
In addition
to the current seven criteria used for selecting elig
ible T
F pro
jects, the evalu
ators recom
me
nd adding
the princip
le of internal “subsidiarity”. U
nder
this criterion, on
ly projects that cannot be fun
ded by geograp
hical units, d
ue to
their non-em
ergency nature or bro
ad geograph
ical scope, wou
ld qualify
EC
HO
support to UN
HC
R’s current refugee op
erations should pro
gressively m
ove tow
ards a
bala
nce
betwe
en fund
ing of
specific
projects of
special interest to
EC
HO
and lig
htly or non-earm
arked multi-d
onor contrib
utions to an
appeal.
For
this p
urpose, the
evaluators
consider
it relevant
that the
programm
atic funding m
echanism
envisage
d by EC
HO
be imple
me
nted as
soon as possible.
Them
atic fu
nding
is currently the
only to
ol available to E
CH
O for h
elping its
partners to
build
their
capacity
and
address areas
in ne
ed of
man
agerial im
provem
ent.
Th
is too
l shou
ld b
e m
aintained
and
strengthene
d.
Criteria
should be
used with flexib
ility. E
CH
O sho
uld continu
e funding pro
tection posts in new
interventio
ns or in
ongoin
g program
mes in forgotte
n crises. E
CH
O shou
ld exercise flexibility in the duratio
n of its TF
. Altho
ugh it is a start-
up support, schedu
led w
ithdrawa
l from so
me projects of key im
portance to
E
CH
O
could be
prem
ature
and counterprod
uctive. T
he key
objective o
f strengthe
ning the partner’s capacity can
not always b
e achieved with
in three
years In add
ition to the topic of E
merg
ency preparedness fund
ed since 2005 and
w
hat is already mentio
ned, areas of future ne
eds for TF
include also:
• T
he strengthenin
g of capacity to offer protection to refu
gees and return
ees outsid
e the cam
ps •
A
greater and
sustained
investment
in deve
lopin
g the
capacity of
the
national cou
nterparts
mechanism
. U
NH
CR
should continue to improve its hum
an resources m
anagement to be able to field prim
arily its own protection
staff for at least one year while increasing their actual
presence in the camps.
The
success of
Surge,
a
temporary
fix, should
not
prevent U
NH
CR
from
addressin
g urg
ently and
perm
anently
the
perceived
shortcoming
in the
m
anage
ment of its ow
n hum
an resources
UN
HC
R w
ill need to present a p
lan indicating w
ho will be re
gistered in the
P
roject Pro
file database, what hu
man
resources will b
e assigned a
nd from
where the fund
ing will com
e for its routine m
aintenan
ce once the system has
been firm
ly introduced into
all UN
HC
R op
erations U
NH
CR
must d
iversify its sources of fundin
g for Project P
rofile in order to
avoid don
or depen
dency. S
ecurity interventions sho
uld be
included in
proposals sub
mitted
to EC
HO
geograph
ical units an
d not be fu
nded u
nder the them
atic fun
ding instrument.
Looking to the Future The issue of m
iscomm
unications between the H
Q of both
partners should
be addressed
through an
enhanced Strategic Partnership D
ialogue mechanism
.
The issue of miscom
munications betw
een the HQ
of both partners
should be
addressed through
an enhanced
Strategic Partnership Dialogue m
echanism.
Final Report
Evaluation of the Partnership between EC
HO
and UN
HC
R (2002-2004)
ECH
O/A
DM
/BU
D/2004/01212
SHE
R Ingénieurs-C
onseils s.a.
49
ECH
O
should increase
its strategic
support to
UN
HC
R
through its thematic funding and proposed program
matic
funding mechanism
s T
he question
of w
hether
EC
HO
should
fund its
own
partners
directly or
multilatera
lly through U
NH
CR
has no black or wh
ite answer. A
com
binatio
n of both approaches w
ill remain advisa
ble E
CH
O
geograph
ical un
its shou
ld increasing
ly opt
for “ligh
tly” earm
arked
programm
atic funding to the g
lobal ap
peal of U
NH
CR
. If need be, E
CH
O
should consider adjustin
g the financia
l imp
lementing
procedures of the F
AF
A.
UN
HC
R should accelerate its adoption of a genuinely needs-
based planning and results-based monitoring and reporting.
UN
HC
R
Glo
bal A
ssessment
of N
eeds
App
eal
should
present
to the
intern
ational com
mu
nity all critical n
eeds of all th
e refugees, not just those to
be m
anaged
by UN
HC
R
To
facilitate the
process of
increasingly
“lightly”
earm
arked pro
gramm
atic fund
ing by EC
HO
geograp
hical units, U
NH
CR
should
accelerate its adop
tion
of a gen
uinely n
eed base
d planning and results b
ased mon
itoring
* Key recom
mendations in B
old S
econdary recomm
endations in
Italic