international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · web...

65
EVALUATION REPORT An evaluation of the project “Towards Inclusion of Persons with Disabilities in Nepal” funded by the Danish Ministry of Foreign Affairs under the Civil Society Fund managed by CISU. By Dorthe Skovgaaard Mortensen & Rita Shrestha

Upload: doankien

Post on 27-May-2018

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Evaluation REPORT

An evaluation of the project “Towards Inclusion of Persons with Disabilities in Nepal” funded by the Danish Ministry of Foreign Affairs under the Civil Society Fund managed by CISU.

By Dorthe Skovgaaard Mortensen & Rita Shrestha

Page 2: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Content

1. Summary..................................................................................................................................................2

2. Introduction..............................................................................................................................................5

2.1 Methodology..........................................................................................................................................6

3. Relevance of the project...........................................................................................................................7

Conclusion to the relevance of the project...................................................................................................8

4. Effectiveness.............................................................................................................................................9

4.1. Achievements under objective 1:..........................................................................................................9

Sub conclusion and recommendations to objective 1............................................................................14

4.2 Achievements under objective 2:.........................................................................................................17

Sub conclusion and recommendations to objective 2............................................................................18

4.4. Achievements under objective 3:........................................................................................................19

Sub conclusion and recommendations to objective 3............................................................................20

5. Efficiency................................................................................................................................................20

Conclusion to the efficiency of the project.............................................................................................25

6. Sustainability..........................................................................................................................................27

Conclusion to the sustainability of the project.......................................................................................28

Annex 1: ToR...................................................................................................................................................... i

Annex 2: Review programme.........................................................................................................................viii

Annex 3: District map....................................................................................................................................... ix

Annex 4: Monitoring visits to district chapters.................................................................................................x

Annex 5: key answers from visits to district chapters......................................................................................xi

1

Page 3: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

1. Summary The National Federation of the Disabled (NFDN) has with the support of Disabled Peoples Organizations Denmark (DPOD) implemented two consecutive phases of the project “Towards Inclusion of Persons with Disabilities in Nepal”. The first phase was reviewed in November 2011. The present evaluation is an assessment of the second phase of the project (2013 – 2016), with a particular focus on decentralization under objective 1.

The project aims to contribute to a situation where persons with disabilities (PWDs) are fully integrated in the Nepalese society in accordance with the United Nations Convention on the Rights of Persons with Disabilities (UNCRPD). The project has been funded by the Danish Ministry of Foreign Affairs under the Civil Society Fund managed by CISU.

The evaluation is based on a desk study, consultations with NFDN and field visits to six project districts. On an overall note the project is assessed to be highly relevant and consistent with the needs of PWD and Nepal’s legal obligation to protect persons with disabilities.

Due to the detailed focus on the relatively challenging decentralization process, the report may appear predominantly critical. It is therefore important initially to emphasize that the project components relating to internal good governance and advocacy at the national level have been implemented competently, responsibly and with a very satisfactory outcome.

Thus, NFDN demonstrates capacity and legitimacy to conduct effective and efficient policy advocacy at the national level. The multi-stringed advocacy approach developed and applied by the national board has evidently led to significant legislative improvements for PWD in revised policies and action as well as in the new constitution.

The apparent ability to balance between a confrontational (hardliner) and a constructive (diplomatic) advocacy engagement with the authorities, without compromising either of the strategies, places NFDN in a very strong and unique position for advocacy at national level.

The evaluation also finds NFDN to be well in the process of becoming an effective and transparent organization which follows good governance principles, but the changes are most noticeable at the national level, whereas the effort to develop organizational capacity at district level; enabling the district chapters to follow NFDN good governance principles, policies, systems, workflows, etc. is less evident.

NFDN has established 10 district chapters as a direct output of the project, but the implementation of the structure has been inefficient and consequently less effective as an umbrella structure for DPOs at district level. The decentralized structure is frail and the role of the chapters (being the umbrella for local DPOs) is somewhat unclear due to the lack of regular trainings, meetings and sharing forums and internal coaching, mentoring and feedback systems targeting the local disability rights campaigners (employed by the project) and the district committee members.

The NFDN chapters are recognized by the local authorities who value their efforts to mobilize persons with disabilities, and facilitate access to public funds and ID-cards, but the potential role and effect of the chapters would be greater if they had emerged out of a desire for an umbrella structure among the local

2

Page 4: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

DPOs, and if the chapters had been supported and capacitated to lead common awareness and advocacy campaigns and provide capacity building to local DPOs in accordance with their individual capacity gaps.

The evaluation notes that the mid-term review of phase I pointed to the need for continuous support to the districts. The inadequate support in phase II is consequently not caused by ignorance of what is required of a decentralization process, but rather the lack of strategic considerations, priority and adequate resource allocation by NFDN; the board, project staff and the project steering committee.

The evaluation team regards the decision not to link the district chapters to the regional structure a failure in the design process; a failure which must be corrected before more district chapters potentially emerge. In general, it appears as if the decentralized NFDN structure has emerged without a thorough discussion of key questions relevant to establishing an effective and efficient decentralized structure such as:

What is the actual purpose of a decentralized structure? Which districts have a need and desire for a decentralized structure? What are the roles and responsibilities of the NFDN district representatives, and how are they

equipped to fulfill these? How will NFDN at central level support the decentralized structure and continuously ensure that it

maintains adequate capacity to perform the assigned roles and responsibilities?

The project also included an integrated disaster risk management (iDRM) component which had low priority when implemented, but became more relevant following the earthquake in 2015.

The project has managed to deliver a number of immediate outputs, which potentially and prospectively can enable NFDN to become a proponent of evidence-based advocacy on iDRM. Thus, the issue of iDRM remains relevant to the wide spectrum of disability groups under NFND, and measures could suitably be taken by NFDN to create awareness of iDRM, build the disaster risk reduction capacity of DPOs, and coordinate DPOs at district level to increase the collective visibility of disability issues in relevant disaster response structures.

The evaluation is convinced that the national level achievements in regard to good governance and advocacy will be sustained after the project, due to the quality of the outputs and especially due to NFDNs dedication towards strengthening the national level governance and impacting the lives of PWDs through strategic advocacy.

The evaluation is on the other hand concerned that the sustainability of the district structure is too weak and that the dependency on facilities and dedicated human resources to conduct activities and facilitate the work of the committees will lead to a situation where the majority of district chapters close (or ceases to be active) when the project ends.

3

Page 5: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Recommendations:

#1: The NFDN district chapters can potentially serve as a coordinating and capacity building body for district level DPOs, but the formation of district chapters should occur as a result of a need and desire for coordination and capacity building among the DPOs, rather than a centrally defined idea to create a decentralized structure. NFDN is consequently recommended to choose a bottom-up approach to the formation of district chapters. Such approach entails that NFDN defines the purpose and benefits of a decentralized structure and communicates this to the district level DPOs, who upon request can be facilitated in forming a district chapter.

After the bottom-up formation of a chapter, NFDN should assist with systematic and continuous organizational capacity building and leadership training which enables the chapter to fulfill its roles and responsibilities as an umbrella structure; acting on behalf of others, defending the interests of others, building capacity of others, etc.

#2: To avoid becoming a service organization for individual PWDs, the district chapters should not be tasked to establish self-help groups. Instead, NFDN should systematically collect and document lessons learned and good practice models for establishing self-help groups (based on experiences gained under the project) for the purpose of being able to provide advice and support to DPOs who wish to form self-help groups.

It is recommended that NFDN is being assisted in the process of collecting and documenting lessons learned and good practice models for establishing self-help groups, and that the documentation focuses not only on the formalities of establishing a self-help group, but also on how to identify the actual needs and interests of PWDs and provide support accordingly.

# 3: NFDN is recommended to use the remaining months of the project to document its experiences with national level advocacy and to systematically share relevant tools and experience with the district chapters, so that they will be empowered to make a significant contribution to monitoring the implementation and effectiveness of the NPPAD. Experiences from the project districts may subsequently be used in new districts.

# 4: iDRM might not be a top priority for NFDN, but the issue is relevant and deserves more attention. NFDN is consequently recommended to review the existing iDRM documentation and revise it in light of the experience gained in responding to the earthquake in 2015. NFDN national board and regional office representatives must subsequently be trained to become effective proponents of evidence-based advocacy on iDRM. This includes capacity to train district chapters and DPOs on the issue of iDRM.

# 5: It is relevant for NFDN to have a decentralized structure, but it is strongly recommended to reconsider the organization of the structure; i.e. consider how to create an efficient link between region and district level, and how to establish effective reporting/information systems, support systems, etc. in order to strategically develop an efficient and cost-effective structure.

4

Page 6: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

# 6: To ensure the most appropriate use of NFDNs human and financial resources are recommended prospectively, only to establish district structures in those districts where a structure is relevant; i.e. where there is more than three DPOs and a genuine need for coordination.

# 7: To facilitate the distribution of district chapters NFDN is recommended to systematically document experiences (lessons-learned) from the ten project districts and develop a model for establishing and maintaining district chapters; including a training plan and curriculum for the strategic capacity building of the chapters (which can be applied by the regional offices).

# 8: A decentralized structure requires continuous attention. Key recommendations from the mid-term review (phase I) are therefore repeated. NFDN is recommended to:

orient the district chapters and continuously guide and motivate them for better performance, reducing communication gap and building good relation

provide some district level activities to empower the district chapter work for empowering its member organizations (DPOs) so that they will be able to self-sustain

and contribute to common issues

The latter bullet includes strengthening the leadership and advocacy capacity of DPOs as well as their capacity to conduct local resource mobilization.

2. IntroductionNFDN is a national umbrella Organization representing more than 300 Disabled People’s Organizations (DPOs), and serves as such as the coordinating body of DPOs, government agencies and (I)NGOs working with disability issues in Nepal.

The evaluation of phase II of the CISU1 funded project “Towards Inclusion of Persons with Disabilities in Nepal” is commissioned by the DPOD to allow for an independent assessment of the project’s relevance, effectiveness, efficiency, sustainability and impact.

The second phase of the project concludes 8 years of project cooperation between the DPOD and NFDN. The first project phase (2009 – 2011) was evaluated in November 2011 after which a new project phase was developed, with an extended outreach to 10 districts in a decentralized structure.

The second project phase has been implemented by NFDN with an objective to achieve the following:

1. NFDN follows Good Governance principles as an effective and transparent organization, in order to work for ensuring the rights of persons with disabilities in Nepal.

1 CISU - Civil Society in Development is an independent association of more than 280 small and medium-sized Danish Civil Society Organisations (CSOs).

5

Page 7: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

2. The Disabled Protection and Welfare Act and NPPAD are in line with the UNCRPD, with the NPPAD being implemented at national level and in the 5 regions and 10 districts where NFDN has a secretariat.

3. NFDN has created a foundation for becoming a proponent of evidence-based advocacy for inclusive disaster risk management (iDRM) in Nepal.

The first and the second objective focusing on good governance, capacity building of NFDN district chapters and policy advocacy at national level represents a natural extension of efforts in Phase I, while objective 3 constitutes a new focus area for NFDN.

The work under objective 3 was down-scaled and phased out mid-2014 as a consequence of not being a strategic priority for NFDN2.

The evaluation will in accordance with the ToR provide an overall assessment of the project, but mainly focus on objective 1 and 2, and particularly the decentralized structure and the achievements at district level.

2.1 MethodologyThe evaluation, carried out in November – December 2016, consisted of a desk study of relevant project documents (including the production of an inception note and question guide), consultations with NFDN and a field visit to six selected districts in Nepal.

The evaluation has applied a participatory and exploratory approach involving NFDN national board and regional/district committees, implementing staff at national and district level and beneficiaries in self-help groups. The findings have been triangulated with perspectives of external stakeholders.

The evaluation program included the following activities:

Output/outcome assessment with key NFDN staff and DPOD staff based on the content of the Logframe and other relevant project documents.

Focus-group interview with representatives of the NFDN board to understand the relevance and outcome of the project mainly regarding good governance and the decentralized organizational structure (objective 1), and national level advocacy (objective 2).

Meeting with external stakeholders (My Right and Ministry of Women Children and Social Affairs) to understand the wider political context and NFDNs role as advocate for PWDs.

Field visit to the following selected project districts:

Kailali (Far Western Region) Surkhet(Midwestern) Kapilbastu (Western Region) Sindhupalchowk (Central Region) IlLam and Jhapa (Eastern Region)

2 According to ToR both partners acknowledge that the incuision of iDRM was pushed from the side of DPOD.

6

Page 8: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

The project has had a keen focus on the organizational development of 10 district chapters under objective 1. The aim of the field visit was consequently to understand the decentralized (district) structure and assess the relevance, effectiveness and sustainability of the results achieved at district level.

During the field visit following meetings/focus group interviews have been conducted:

Regional meetings with the NFDN regional coordinators and representatives from the regional committee to obtain their assessment of the relevance of the project, the achievements and the sustainability of achievements.

Individual interviews with NFDN’s disability rights campaigners to cover issues regarding the organizational project set-up, coordination, capacity and project implementation, including the appropriateness of the applied approaches, methods and tools.

Focus-group interviews with representatives of the district committees focusing on the relevance and outcome of the project mainly regarding good governance and sustainability (objective 1), and local level advocacy (output 2.5).

Focus-group interviews with representatives from the self-help groups focusing on the relevance and achievements of the SHG, and on the support given to the SHG and the sustainability of the group.

Meetings with relevant district authority to understand how NFDN work in the district and the effect of its advocacy work.

Meeting with representatives of a neighboring non-project district committee to understand the broader NFDN structure, including the difference in set-up between the project districts and the non-project districts, and how these structures respectively are conducive to achieving the rights of PWDs.

The field visit was concluded with a debriefing with NFDN and representatives from DPOD for the purpose of presenting key findings and testing their validity prior to writing the evaluation report.

Terms of Reference for the evaluation are attached in Annex 1. For a detailed program of the field visit please refer to Annex 2.

The evaluation has been conducted jointly by a Danish and Nepali consultant and the following chapters provide their assessment of the relevance and the outcomes and outputs of the project, as well as the efficiency of the project set-up, including the partnership between DPOD and NFDN. Recommendations for possible new approaches and improvements to NFDNs work are given consecutively in the report.

3. Relevance of the project- The extent to which the project intervention is consistent with local needs, national legal obligations

as well as partners’ and donors’ priorities.

The project’s overall development objective is to reach a situation where PWDs are fully integrated in the Nepalese society in accordance with the United Nations Convention on the Rights of Persons with Disabilities (UNCRPD).

7

Page 9: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

The UNCRPD was ratified by Nepal in 2010. The project was consequently designed in a political context where the rights of persons with disabilities were acknowledged, but where disability policies and procedures needed to be aligned with the UNCRPD, and where several entitlements of PWDs remained to be enforced by the national and local authorities. Thus, inconsistencies between UNCRPD and the Disabled Protection and Welfare Act (1981) and the existing National Policy and Plan of Action (2006) needed to be addressed. Furthermore, with a new constitution in the making NFDN saw a window of opportunity to influence policies and legislation positively on issues of PWDs.

When the analysis is moved from the political level to a more practical level, there are several government provisions providing opportunities for PWDs at local (district/regional) level. These opportunities include the distribution of Disability Identity Cards, free education for children with disabilities in government schools, budget allocation for disability issues3 and representation of PWDs in the District Disability Coordination Committees. However, to make the most of these opportunities there is a great need for awareness raising and local coordination among DPOs; a need which is acknowledged not only by NFDN, but also by district and national authorities which confirm the need for information sharing and coordination among local level DPOs. Being the national umbrella organization for PWDs, NFDN has a legitimate position as the coordinator of DPOs at national as well as local level.

The project (especially objective 1 and 2) is well in line with key priorities of NFDN and DPOD; namely organizational capacity building of the disability movement and advocacy for the rights of PWDs.

Conclusion to the relevance of the project Despite Nepal’s ratification of the UNCRPD and its legal obligations towards PWDs, the project is implemented in a political and social context where there is no comprehensive legal framework protecting the human rights of PWDs, and where a persistent culture of discrimination, social neglect, stigmatization and inadequate public services continue to pose serious challenges to PWDs. In a context of that nature, the project is assessed to be highly relevant and consistent with the needs of PWD and Nepal’s legal obligation to protect PWDs.

The development objective aiming at a situation where PWDs are fully integrated in the Nepalese society in accordance with the UNCRPD, is found to be in excellent agreement with the priorities and focus areas of NFDN. Furthermore, NFDN demonstrates good knowledge of the relevant disability legislation and practical insight into policy processes.

Lastly, the project is also found relevant in an international context, including the Danish priorities and policy commitments to promote human rights as set out in the Danish development strategy “The right to a better life” and the Civil Society Policy of the Danish Ministry of Foreign Affairs.

4. Effectiveness- The extent to which the project objectives have been or are likely to be achieved.

3 PWDs are part of a 15% allocation for PWD, Dalit, Single women ,Madhesi and marginalized groups of the total tax revenue)

8

Page 10: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

The aim to fully integrate PWDs in the Nepalese society, is pursued through three project components4:

1. Organizational development focusing on good governance procedures and creating a decentralized organizational structure (objective 1).

2. Policy advocacy focusing on aligning national policies with the UNCRPD and policy implementation (objective 2).

3. Capacity building for advocacy focusing on building a foundation for evidence-based advocacy on inclusive risk management (objective 3).

In order to examine the extent to which the project objectives have been or are likely to be achieved (section 4.1. – 4.3) it is important to understand the implementation context.

The political context in Nepal has been seriously affected by the aftermath of the conflict which brought an end to the King’s direct rule and led to the declaration of Nepal as a Federal Democratic Republic in 2008. Disagreements between parties in the appointed Constituent Assembly, charged with writing Nepal's permanent constitution, led to years of political turmoil. The country has consequently been governed by an interim constitution until September 2015.

Although there has been consensus among the members of the Association of International NGOs in Nepal that the environment for civil society remained conducive, the somewhat unpredictable political situation has impacted the methods and the timing for advocacy. Furthermore, the devastating earthquake that hit Nepal on 25. April 2015 put the project implementation temporarily on hold.

The project Logframe has, as a consequence of the above, been revised and the project period extended with two years. The revision of the Logframe consists primarily of changes in the number of outputs for outcome 2 and 35.

The findings in regard to effectiveness are based on interviews with the NFDN board, implementing staff, project participants at district level, and are triangulated with NDFN representatives at regional level and district authorities.

4.1. Achievements under objective 1: [Objective: NFDN follows Good Governance principles as an effective and transparent organization, in order to work for ensuring the rights of persons with disabilities in Nepal].

The objective combines activities and results at central (national) and decentralized (district) level. According to the project document, capacity building and organizational development constitutes the foundation for the project strategy. The aim is to ensure that the required individual competencies and structures are in place at national and district level, and that all NFDN representatives share the same

4 The project components are closely linked to the three project objectives.5 The down-scaling and phase-out of objective 3 by 2014 was decided with the consent of DPOD and CISU, since the component did not represent a priority area for NFDN.

9

Page 11: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

knowledge. The outcome is expected to be a common understanding of roles and responsibilities which will help all involved individuals gain ownership of NFDN and develop it as a strong umbrella organization6.

The achievements relating to objective 1 have largely been within the sphere of NFDNs control and thus only affected minimally by the context in which it was implemented.

A key output under objective 1 has been the expansion of the district structure with additional 5 district chapters7. The evaluation team has verified that a decentralized structure, consisting of district committees, disability rights campaigners (staff) and self-help groups, is established in 10 districts across Nepal’s five regions (for a map indicating the selected districts please refer to annex 3). NFDN has provided a list of a total of 101 self-help groups, but it has not been possible to determine the actual number of functioning self-help groups by the end of phase II, as no systematic monitoring of the groups has been conducted by NFDN.

A key intention of the decentralized structure is to facilitate access to local government funds allocated for disability issues, and to contribute to the distribution of disability ID-cards which ensures certain privileges to PWD (depending on which card category they hold).

Interviews with the national board, the disability rights campaigners, district committee members and district authorities (Women, Children and Social Welfare Department, Municipality and District Development Committee) confirm this intention and it appears that the decentralized NFDN structure has contributed significantly to an increased request for government funds and distribution of ID-cards to PWDs, with the self-help groups as active participants.

“There is a fixed government budget for PWDs. We are actively taking advantage of that budget, if not, the budget is reallocated to other issues”.

The district authorities recognize NFDNs coordinating role, and members of the self-help groups confirm that the district chapters have raised their awareness about funds and ID-cards for PWD, and assisted the groups accessing the earmarked funds as well as the ID-cards. Other actors, such as the Women, Children and Social Welfare Department and individual DPOs also play an active role and achievements are consequently a result of joint efforts.

“For the ID-card distribution to PWDs the district chapter has worked very closely with the Women, Children and Social Welfare Department. Two days a month the Women, Children and Social Welfare Department

allocates time for ID card distribution. On those days, there will also be a representative from district chapter”.

“The district structure is very important as most of the line agencies and district authorities look for the NFDN representation … they seek advice from NFDN not from individual DPOs”.

The self-help groups have been established to increase NFDNs outreach to PWDs, but apart from facilitating access to public funds and ID-cards, the district chapters have had only limited activities to offer

6 Quoted freely from the project document, p. 177 Other 5 chapters were formed in phase I and continued under phase II

10

Page 12: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

the groups, including some short-term training on good-governance, rights and income generating activities.

“I am a person with low vision. The incense8 making training was fruitful for me. I got some materials and I used the materials to make incense at home. I earned Rs 1500 selling it, but there were no materials to

continue”.

The trainings are reported not to be adequately followed-up, and the self-help groups are generally requesting more regular support from the district chapters. Without any regular follow-up and support, the effect of the trainings is limited.

A district authority points out that activities are limited to trainings only; trainings which are often inconsistent with the actual needs and interests of PWD. It was consequently recommended that the district chapters should aim to identify the actual needs of PWDs and work on those rather than what the donors require them to work with.

Another evident output under objective 1 is the development of a Good Governance Guideline Manual. The manual outlines the principles and practices of good governance within four clusters: equity and participation, transparency, accountability and rule of law. Furthermore, it stipulates that the capacity of NFDNs elected representatives and staff shall be developed to enable them to carry out their specific roles and responsibilities. Capacity development shall per the manual be achieved through regular orientation about organizational policies, guidelines and systems, regular workshops, meetings and sharing forums and internal coaching, mentoring and feedback systems9.

It is apparent that the internal process of developing the guidelines and clarifying administrative procedures has impacted the governance and effectiveness of NFDN at the national level.

National board members and staff confirm unanimously that the election of a new board and the employment of a programme manager (in project phase II) have prompted not only commitments but actual actions to develop new and better systems and procedures. These include new staff and financial guidelines, and job descriptions containing clear roles, responsibilities and lines of authority.

“Before, the roles and responsibility between staff and board was confused. The project has brought a big change and given rise to new practices and procedures in NFDN!”

The lines of authority between staff and board have been separated, and operational systems formalized and implemented in practice, which allow staff to plan and implement the day-to-day project activities, while the executive board (the president, general secretary and treasurer) are acting on their role as overall responsible; i.e. approving final project documents, monitoring project results, etc.10.

8 A substance that is burnt to produce a sweet smell.9 Cf. the Good Governance Policy p. 10-11. 10 The NFDN board has previously had a practice of controlling every action of the organization. The practice constituted a concern among partners, including DPOD, and a thorough discussion on governance issues led to the separation of authority between board and staff.

11

Page 13: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Furthermore, a five-year strategic plan was developed, and transparency and accountability has been increased through the implementation of official e-mails and an internal calendar, and through more systematic database documentation and web publication of documents in accessible formats.

NFDNs national board members’ express ownership of the process, but recognize that it takes time for good governance practices to materialize throughout the organization.

“Changing behavior is difficult! It is easy to make documents and systems, but harder to make use of these documents and systems”.

National board members and staff, disability rights campaigners and selected representatives from the district committees have all received training/orientation on NFDNs Good Governance Guidelines Manual and the new systems and procedures. However, there is a significant difference in how the guidelines, systems and procedures have been internalized at national and district level respectively.

The orientation to the district representatives was provided relatively early in phase II, and there has subsequently not been any additional training on the issue of good governance for the previous or new district committee members (elected August 2016). Thus, of six district committees interviewed only one committee (Surkhet) indicates that it is familiar with the Good Governance Guidelines, and has applied some of the learnings on good governance. There is consequently very limited evidence that the orientation on good governance has trickled down and resulted in new and better governance practices in the districts.

“Every three years there will be change in the committee members. There is no system to orient or train the new committee”.

Whereas there appears to be a large difference in the organizational capacity of the district chapters, including their ability to build capacity among the DPOs, most district chapters find that the project has contributed to assemble the main local DPOs.

“After launching the project, it helped us to organize ourselves and reduce conflicts among DPOs. Before the chapter, we had a feeling of individuality, now we have a group feeling!”

“This project has brought all DPOs together and increased the common understanding among the DPOs. Due to the project, each DPO has its own working area, this has helped reduce duplication of work”.

There is an indication of increased coordination among DPOs in the districts, but only few chapters have served as an actual umbrella body of DPOs; building capacity of DPOs, making joint advocacy and coordinating the relations to local government agencies and NGOs working in the disability sector.

The chapters serving as an umbrella for the DPOs (mainly Jhapa11, and to some extent Surkhet and Sindhupalchowk) are chapters with several DPOs and where the disability rights campaigner has been given responsibility to support the DPOs.

11 In Jhapa all DPOs in the district are included in the chapter, also the DPOs not associated with NFDN.

12

Page 14: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

According to NFDN, all disability rights campaigners have been informed of their roles and responsibility in writing and roles and responsibilities has also been discussed and refreshed during monitoring visits and annual learning and sharing meetings. However, a number of campaigners (particularly from the districts with very few DPOs) expressed uncertainty about their roles and responsibilities and the chapters have not been able to act effectively as an umbrella structure for DPOs. Furthermore, it must be mentioned that the evaluation team came across several reports of conflicts at district level12; between disability rights campaigners and district committees, among the committee members themselves, and between district chapters and DPOs (mainly non-NFDN members).

“Formation and monitoring of self-help groups should not be the work of the district chapters. This kind of activities should be done by DPOs who are working at village level. The role of the district chapters should be to improve the governance of DPOs and build their capacity. DPOs should be strong to work at local level”.

“Before we were more active in our own DPOs, now we are more focused at sustaining the SHGs”.

To be able to assess the effect of the project districts the review team visited three non-project districts (Rupandehi, Morang and Banke) where NFDN has district committees similar to the committees in the project districts.

The main purpose of the committees in the non-project districts is the mobilization of local resources. In Rupandehi NFDN was recently approached by a large majority of the DPOs who wanted to form a committee that could coordinate the DPOs and meet with the district authorities for the purpose of fundraising and increasing the visibility of PWD/DPOs. In Banke and Morang the committees have existed for many years and represent all district DPOs13.

The non-project committees have limited activities and are generally run without any funding from NFDN (or external donors14). However, in Rupandehi and Banke the district committees join the regional activities (supported by the Norwegian Federation of Organizations of Disabled People) and are using the facilities of the regional office since the district and the regional NFDN headquarter are coinciding.

"We have not received any support from national level but from the regional level. Being in the same district we are closely linked with the regional office".

According to the non-project district committees the district representation is increasing NFDNs outreach and effectiveness in regards to local fundraising to the DPOs. Furthermore, the committees coordinate the DPOs in the district and carry out some limited joint advocacy activities. In order to further increase the effectiveness of the district structure the committee in Banke suggests that NFDN recognizes the district coordination committee, not as a loose structure, but as a "legalized" entity.

12 Conflicts of different types were reported in Kailali, Kapilbastu and IlLam.13 In Banke also other organizations working in the field of disability are included in the committee.14 The district committee in Morang has previously received funding from Plan Nepal.

13

Page 15: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Sub conclusion and recommendations to objective 1NFDN is in the process of becoming an effective and transparent organization. The evaluation finds that the objective to follow good governance principles and become more effective and transparent has largely been achieved at national level, whereas the change is much less visible at district level.

The evaluation team has seen a significant commitment to the issue of good governance among national board members and staff. Substantial achievements have been made in the form of good governance principles and new systems and procedures, some of which are well under implementation in the board and at the NFDN secretariat. It is noticed however, that efforts to develop a similar capacity at district level; enabling the district chapters to follow NFDN good governance principles, policies, systems, workflows, etc. and develop a strong decentralized umbrella structure, has been less significant.

The project district committees and disability rights campaigners demonstrate a fairly good understanding of their task to mobilize PWDs to claim government funds and ID-cards, and the chapters are recognized for their contribution to creating awareness and facilitating assess to government services. The establishment and facilitation of self-help groups has however consumed a lot of time and resources and has turned district chapters into DPO-like structures that serve (and facilitate access to services to) individual PWDs/groups of PWDs rather than an umbrella structure that coordinates and builds capacity of local DPOs15.

NFDN may have fulfilled its ambition to establish a district structure in ten districts, but the structure is frail and its content remains somewhat unclear. Consequently, a significant number of district committees and disability rights campaigners’ express uncertainty about their roles and responsibilities in regard to being an umbrella for district DPOs, and many chapters possess insufficient capacity (and limited resources) to effectively mobilize and build capacity of the DPOs at district level.

The main difference between the project and non-project district committees is that the three non-project committees which were consulted all have been formed by a bottom-up approach (i.e. on demand from the district DPOs) for the purpose of coordinated lobbying efforts towards district authorities.

The limited financial and human resources allocated to the non-project districts structure is apparently not affecting the effectiveness of the local resource mobilization and the simple/light organizational set-up of the not-project districts is found to be more sustainable compared to the project districts. However, without a formalized structure and a minimum of facilitation and technical support from NFDN (national or regional level) the effort to build a strong disability movement of capacitated DPOs at district level remains limited in the project as well as the non-project districts.

The evaluation team is convinced that NFDN only will be able to ensure the rights of persons with disability, if it works systematically and strategically through the DPOs. At district level, this entails that the NFDN chapter acts as an umbrella structure; coordinating the entire disability sector, leading common awareness and advocacy campaigns, and providing capacity building to DPOs in accordance with their capacity gaps. In that regard, Jhapa district chapter is found to be a best practice case – which can be used as a model for other districts16.

15 As appears from section 5, NFDN is fully aware of the organizational challenges in forming self-help groups and therefore plans to discontinue this practice.

14

Page 16: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

The preconditions for assuming the role as umbrella for local DPOs are – in addition to the obvious need for a critical number of DPOs to coordinate and facilitate – a common understanding of roles and responsibilities, sufficient organizational and professional capacity, and last but not least a desire for an umbrella structure among the DPOs.

The evaluation team does not consider the weakness at district level a matter of having engaged the wrong persons, but a matter of deficient support. The district representatives in general appear sincere and interested in creating better conditions for PWD, but the chapters have not been facilitated by regular trainings, meetings and sharing forums and internal coaching, mentoring and feedback systems as stipulated in the Good Governance Guideline, and there is only little indication that the very limited orientation being provided to district representatives has been trickled down and transferred to others.

The different conflicts identified at district level; between disability rights campaigners and district committees, among the committee members, and between district chapters and DPOs, is not necessarily evidence of anything other than a commonly high level of organizational and political conflict in the Nepalese society. Nevertheless, the evaluation team notes that there is a far greater risk of conflict if the people who make up the district structure have only a vague understanding of their roles and responsibilities, and if the DPOs do not acknowledge the benefit of an umbrella structure and take active ownership of the structure. In such a situation, there is fertile ground for frustration and conflict, which is likely to lead to an inappropriate use of available human and financial resources.

The evaluation team confirms that NFDN has a potential role to play at district level (i.e. in the districts with a critical mass of DPOs), but it has not yet managed to create an effective district structure which is suitable for replication. In general, it appears as if the decentralized structure has emerged without a thorough discussion of key questions such as:

What is the actual purpose of a decentralized structure? Which districts have a need and desire for a decentralized structure? What are the roles and responsibilities of the NFDN district representatives, and how are they

equipped to fulfill these? How will NFDN at central level support the decentralized structure and continuously ensure that it

maintains adequate capacity to perform the assigned roles and responsibilities?

If NFDN fails to provide clear answers to the above questions and to act upon these, it will at best continue to have a generally weak and ineffective decentralized structure in a few randomly selected districts.

The evaluation team could provide some here-and-now recommendations, such as providing a thorough training of the new district committee members, but these recommendations are short-term and will not contribute significantly to strengthening the decentralized structure.

The recommendations are instead of a more principled nature and will hopefully help to ensure the basic preconditions for a strong decentralized structure.

16 Based on findings in Jhapa (and to some extent Surkhet) the purpose of the district chapter appears more evident if the chapter plays an active role providing technical support and building capacity of the DPOs, so that they can defend their members' best interests and thereby ensure the rights of persons with disability in Nepal.

15

Page 17: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Recommendations:

#1: The NFDN district chapters can potentially serve as a coordinating and capacity building body for district level DPOs, but the formation of district chapters should occur as a result of a need and desire for coordination and capacity building among the DPOs, rather than a centrally defined idea to create a decentralized structure. NFDN is consequently recommended to choose a bottom-up approach to the formation of district chapters. Such approach entails that NFDN defines the purpose and benefits of a decentralized structure and communicates this to the district level DPOs, who upon request can be facilitated in forming a district chapter.

After the bottom-up formation of a chapter, NFDN should assist with systematic and continuous organizational capacity building and leadership training which enables the chapter to fulfill its roles and responsibilities as an umbrella structure; acting on behalf of others, defending the interests of others, building capacity of others, etc.

#2: To avoid becoming a service organization for individual PWDs, the district chapters should not be tasked to establish self-help groups. Instead, NFDN should systematically collect and document lessons learned and good practice models for establishing self-help groups (based on experiences gained under the project) for the purpose of being able to provide advice and support to DPOs who wish to form self-help groups.

It is recommended that NFDN is being assisted in the process of collecting and documenting lessons learned and good practice models for establishing self-help groups, and that the documentation focuses not only on the formalities of establishing a self-help group, but also on how to identify the actual needs and interests of PWDs and provide support accordingly.

4.2 Achievements under objective 2:[The Disabled Protection and Welfare Act (DPWA) and National Policy and Plan of Action on Disability (NPPAD) are in line with the UNCRPD, with the NPPAD being implemented at national level and in the 5 regions and 10 districts where NFDN has a secretariat].

Due to an unpredictable policy environment, which has been outside the direct control of NFDN, the focus of the second objective has only been on national level policy advocacy by the NFDN board.

Despite the changeable policy environment, the context for advocacy has been favorable, and NFDN has managed to achieve significant legislative changes in the favor of PWDs. The board17 has engaged actively in the drafting committee for the DPWA, and has as such contributed to the revision of the Disability Act, which is currently awaiting the final approval in parliament. The board has also continuously been advocating for a revision of the NPPAD and it has worked closely with the Ministry of Women, Children and Social Welfare, which has outsourced to NFDN to prepare a draft NPPAD in line with the UNCRPD.

17 Represented by the NFDN president.

16

Page 18: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

A final draft NPPAD has been developed based on multi stakeholder consultations with DPOs, (I)NGOs, etc, and it is now in the process of finalization in the Ministry of Women, Children and Social Welfare, where a formal process will ensure that the policy and action plan is robust and will remain relevant for many years. The process was at the time of the evaluation expected to be finalized by 1st of March 2017.

The Ministry confirms that it has a well-established and close working relationship with NFDN. NFDN is regarded to have comprehensive knowledge of disability issues and a solid understanding of policy matters and processes, and as an umbrella for most DPOs it is a valued partner for the ministry.

The board has also been active in the process that led to the new constitution which was enforced in September 2015. Through intensive advocacy it managed to have improved entitlements for PWDs incorporated in the constitution. These entitlements concern fundamental rights and social justice, participation at government level, acknowledgement in line with other minorities, and social welfare.

NFDN applied a multi-stringed advocacy approach to influence the new constitution. The strategy included national mass protest, 11 days of hunger strike and informal consultations with parliamentarians.

Particularly the latter strategy is a new approach which NFDN was inspired to test after meeting DPOD partners from Uganda and Ghana in 2014. NFDN board members began to meet parliamentarians informally where after a trusting relationship has been built gradually. Currently the board is having regular meetings with the Youth Caucus in parliament18, which also invites NFDN board members to observe parliamentarian debates on disability issues.

According to the NFDN board, and confirmed by the Ministry of Women, Children and Social Welfare, the project has contributed to major changes at policy level; the parliamentarians have become more attentive to disability issues and the disability legislation submitted to parliament is consistent with the UNCRPD. What remains, is the implementation of the legislation.

With the political process being utterly delayed NFDN has not yet been able to take the NPPAD to the ten project districts for implementation. Most district committees express that they have some experience and capacity to work with advocacy, but they have generally not received any strategic capacity building on advocacy from NFDN19.

Sub conclusion and recommendations to objective 2NFDN demonstrates capacity and legitimacy to conduct effective policy advocacy at the national level. The multi-stringed advocacy approach developed and applied by the national board has led to significant legislative improvements for PWD in the revised policies and action plans (DPWA and NPPAD) as well as in the new constitution.

The applied advocacy approach includes three different strategies:

18 The Youth Caucus is the entity speaking on behalf of PWDs in the parliament.19 Two districts have not received any form of support or advice on advocacy, one district answers that it has not requested any support, another district has received support to interact with likeminded organizations and thereby increased visibility of the chapter, and one district says it has received unspecified support for advocacy.

17

Page 19: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

1) A constructive engagement with the Ministry of Women, Children and Social Affairs; where NFDN has played a substantial role in revising existing legislation in accordance with the UNCRPD. The engagement with the ministry is based on mutual trust, well-defined roles, and a recognition of NFDN being the umbrella (representative) of the majority of DPOs in Nepal.

2) Informal alliance building with selected parliamentarians in order to exchange knowledge and views on current disability issues, and getting access to and influencing parliamentarian debates.

3) A more confrontational engagement mobilizing crowds of PWDs in order to demonstrate the strength of the disability movement by numbers and its willpower.

The combination of the three strategies could have proved counterproductive, but in the case of NFDN it has proved to be mutually reinforcing. The apparent ability to balance between a confrontational (hardliner) and a constructive (diplomatic) advocacy engagement with the authorities, without compromising either of the strategies, places NFDN in a very strong position for advocacy at national level.

The district chapters have for natural reasons not yet been engaged in advocacy for the implementation of the NPPAD. Some districts may, by the virtue of their own efforts, be ready to conduct policy monitoring, but the evaluation is concerned that several districts have inadequate capacity to contribute strategically to monitoring the implementation and effectiveness of the NPPAD when the policy and action plan is finally approved by government.

Recommendation:

# 3: NFDN is recommended to use the remaining months of the project to document its experiences with national level advocacy and to systematically share relevant tools and experience with the district chapters, so that they will be empowered to make a significant contribution to monitoring the implementation and effectiveness of the NPPAD. Experiences from the project districts may subsequently be used in new districts.

4.4. Achievements under objective 3: [NFDN has created a foundation for becoming a proponent of evidence-based advocacy for inclusive disaster risk management (iDRM) in Nepal]

The evaluation has had only limited focus on the results under objective 3 and the analysis is consequently limited.

The results achieved in relation to objective 3 have all been realized in 2013-14. In accordance with the revised results framework NFDN has reviewed legislation relevant to disaster risk management and developed an advocacy tool on inclusive disaster risk management (iDMR).

18

Page 20: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Furthermore, a review has been conducted of the Flagship 4 Community Based Disaster Risk Reduction plan, based on data collected by the disability rights campaigners in a total of 20 VCD in 9 out of the 10 project districts20.

NFDN has subsequently included iDRM in its 5-year strategic plan, and in line with its strategy conducted an iDRM training-of-trainers in cooperation with Unicef and Nepal Red Cross Society.

The advocacy tool on iDMR is distributed to NFDN regional offices, project districts and other DPOs, and representatives from the districts have received orientation about the tool. Unfortunately, none of the district committee members are familiar with the tool, and among the disability rights campaigners it is only the ones from Surkhet, Jhapa and Jumla who declare to be aware of it; two were active collecting district data for the review, and one remembers the orientation provided on iDRM, an orientation which according to the campaigner was very important, but insufficient.

After the fieldwork in Nepal the evaluation team has come across an elaborate study “Disaster, Disability & Difference” of the challenges faced by persons with disabilities in the post-earthquake Nepal. The study is published by Social Science Baha with the support from UNDP and NFDN in May 2016.

The study indicates that DPOs are highly under-represented within the institutional structure of disaster response in post-earthquake Nepal. “The findings showed that: a) local DPOs working for the rights of PwDs in each of the four districts were not meaningfully included in the Protection Cluster; b) disability-oriented organizations who did participate in the Protection Cluster were mostly INGOs that arrived post-disaster; and c) coordination between international and local organizations was low. In Nuwakot, for example, a National Federation of Disabled-Nepal (NFDN) representative participated in the Protection Cluster as the representative of all the DPOs in the district: “We presented our work. Other NGOs also presented their work. But it was just sharing meeting. Nothing happened afterwards21.”

Sub conclusion and recommendations to objective 3The iDRM component had low priority in NFDN when implemented, but became more relevant following the earthquake in 2015.

The project has delivered a number of immediate outputs, which potentially could enable NFDN to become a proponent of evidence-based advocacy on iDRM. There is however, no evidence that the review or the advocacy tool developed by the project have been used actively to advocate for more inclusion of PWDs in DRM-strategies at national level, and at district level there is only very limited recognition of the advocacy tool. Thus, the evaluation finds no evidence of any change at outcome level.

The low level of awareness on iDRM may be caused by a replacement of some of the disability rights campaigners and the election of new district committees. Yet, it is an example of poor transfer of knowledge within the districts and underpins the need for continuous orientation and support from the national level.

20 The review report was presented to the review team, but the advocacy tool, which is in Nepali, has not been reviewed by the team.

21 Cited from the study page 32.

19

Page 21: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

The issue of iDRM remains however relevant to the wide spectrum of disability groups under NFND, and the study by Social Science Baha indicates that NFDN has a role and an interest in a continued focus on iDRM.

Further measures could suitably be taken by NFDN to create awareness of iDRM, build the disaster risk reduction capacity of DPOs, and coordinate DPOs at district level to increase the collective visibility of disability issues in relevant disaster response structures (e.g. District Disaster Recovery Committees and Protection Clusters), thereby increasing emergency preparedness and reducing the risk of PWDs in disasters.

Recommendation:

# 4: iDRM might not be a top priority for NFDN, but the issue is relevant and deserves more attention. NFDN is consequently recommended to review the existing iDRM documentation and revise it in light of the experience gained in responding to the earthquake in 2015. NFDN national board and regional office representatives must subsequently be trained to become effective proponents of evidence-based advocacy on iDRM. This includes capacity to train district chapters and DPOs on the issue of iDRM.

5. Efficiency- The extent to which the project set-up (capacity, cooperation and coordination) has delivered value

for money

The efficiency of a project is - to a large extent - based on the availability of sufficient capacity and adequate structures and systems in the implementing organization NFDN, but also on the cooperation between NFDN and DPOD.

The project was designed in a context where NFDNs organizational structure, the administrative competencies, the transparency and the general understanding of work processes, roles and responsibilities needed to be strengthened22. The awareness of the organizational challenges has been fueled in the relationship between NFDN and its partners, including DPOD. Initially, the challenges gave rise to some mistrust and dissent in the partnership relation, but according to the NFDN board, the recent cooperation with DPOD have been very good and beneficial for the organization.

“DPOD has a very appreciative approach; they do not request us to implement their approach, the project is based on our needs. They alert us to the international context and guide us. We are given the opportunity to

decide ourselves per the context”.

The good governance component (objective 1) has contributed to build/improve leadership capacity, communication structures, reporting systems etc. at national level, and even though it takes time to

22 Cf. The following statement in the project document: “NFDN has during the last couple of years underestimated the value and importance of good governance and sustainable organisational development and DPOD can contribute with guidance and support in these areas” (page 5).

20

Page 22: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

institutionalize and implement new practices, the NFDN board and secretariat demonstrate dedication and determination to revitalize the organization and exercise good governance.

The efficiency of NFDNs advocacy approach (objective 2) has largely been demonstrated by the achievements described in section 4.4, which testifies that NFDN possesses sufficient power, legitimacy and capacity to influence the national policy-making process and achieve significant policy change. NFDN is yet to prove its efficiency in policy monitoring and public accountability when the NPPAD and DPWA are finally approved by government and ready for implementation.

The evaluation reveals less efficiency when it comes to NFDNs decentralization efforts; i.e. the establishment of and the support to the ten district chapters.

The decentralization component has per design been integrated in the larger good governance component (objective 1). The lack of an explicit focus on decentralization i.e. an independent objective regarding decentralization and associated outputs and indicators, may be a contributing factor to the relatively low attention granted to the component. Furthermore, the scattered distribution of the 10 project districts combined with inadequate human and financial resources for visiting the districts have also contributed to the limited direct interaction between NFDN at central level and the district chapters23 (for an overview of the distribution of project districts please refer to annex 3).

Considering the analysis and the recommendations provided by the Midterm evaluation (November 2011), it is evident that NFDN, prior to phase II, was made aware that the decentralization effort requires thorough and continuous attention from the national office24. Unfortunately, the recommendations from phase I has not been followed and the ten project districts have not been allocated the necessary direct support by NFDN, which has impacted the efficiency of the structure.

During the inception of phase II NFDN developed a thorough Monitoring and Evaluation System, and an elaborate Project Implementation Guideline. These guiding documents are signs of good organizational practices, but have not been applied to a satisfactory degree.

The district chapters have mainly been monitored through a quarterly reporting system different than the one suggested in the Monitoring and Evaluation System. Each disability rights campaigner develops a quarterly activity plan against which the budget is released. After completing the plan the campaigner submits an activity report describing the quantitative achievements of the quarter. The quarterly report is in most cases shared with the district committee and in rare cases with the regional office. The report contains no measures regarding outcome and impact of the project.

The follow-up on the reports consists of a brief feedback by the project officer including an example of the best report submitted that quarter.

23 The review team was informed about a previous dispute between NFDN staff and board over the responsibility for monitoring. This dispute is considered solved and the responsibility is primarily assigned to project staff.24 Cf. The following recommendations from the Midterm Evaluation: 1) NFDN secretariat should orient the district chapters and continuously guide and motivate them for better performance, reducing communication gap and building good relation, 2) NFDN should provide some district level activities to empower the district chapter as well as SHGs, 3) NFDN should work for empowering its member organizations so that they will be able to self-sustain and contribute in common issues, and 4) Capacity of the project staffs (DISABILITY RIGHTS COORDINATORs) should be built up for effective implementation of the project activities at the field level.

21

Page 23: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

“The quarterly reports do not cover the challenges we experience. It is quantitatively explaining how many ID-cards distributed, how many meetings held, how many visits to self-help groups, etc.

We get only limited feed-back on our reports, but may receive an example of a report from another district. It is a bit discouraging!”

The supervision and confirmation of district activities rarely happens by way of monitoring visits to the districts, but by regular e-mails and phone calls. Of the ten districts, only one district (Sindhupalchowk) has received a monitoring visit every year from 2014-2016. Among the remaining nine districts one district (Jumla) has received no visits, another district (Kapilbastu) one visit (for an overview of the monitoring visits please refer to Annex 425).

The NFDN regional structure, with staff and offices in the five regions26, has by design not played any active role in monitoring or supervising the ten project districts. Generally, the link between the regional and district structure is weak, and there is only limited coordination and information sharing between the project districts and the regional NFDN staff/office27.

The far too few visits to the districts (from the central, but also the regional office) has resulted in inadequate hands-on support to the capacity building and supervision of the disability rights campaigners and the district committees, including insufficient introduction to new committee members and training opportunities for all board members as per their needs. The limited attention and direct interaction between national/regional level and the district chapters has generally led to a weak decentralized structure, where several district representatives are unclear about their roles and responsibilities being an umbrella organization for DPOs (cf. the analysis in section 4.2). Furthermore, the relevance of being an umbrella is hampered in at least three districts (Kapilbastu, Baitadi and Jumla) by the fact that there are only two or less DPOs and as such no need for coordination.

Training of district representatives has only been provided in limited doses. Many of the planned 2-days training sessions were converted to shorter orientations, exclusively for the disability rights campaigners and selected representatives of the district chapters. The training approach applied by the central office has been a hands-off, learning-by-doing approach, which has been appropriate in the few district chapters with motivated disability rights campaigners and strong district committees, but for most districts chapters the trainings have only had very little effect. Furthermore, all trainings/orientations have been carried out without a prior needs assessment and the trickle down of learning has not been adequately institutionalized. Some persons trained indicate that they did some subsequent training of the district committee members (or self-help groups), but there is no evidence that knowledge and skills have been transferred to the new district committees28.

Some project districts have, for different reasons, experienced several replacements among the disability rights campaigners, but the position has generally been challenging. The disability rights campaigner is formally employed by the district committee, but is instructed by and reporting to the NFDN project officer

25 The overview was provided to the review team during the review. 26 The regional structure is supported by the Norwegian Federation of Organisations of Disabled People.27 The review has evidence of only three districts (Kaski, Kailali and Batadai) forwarding a copy of the quarterly report to the regional office, and in all cases, it is a recently established practice. Furthermore, there is very limited evidence of visits and support to the districts from the regions. 28 The new district committees consist of former and new members.

22

Page 24: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

in Kathmandu, with whom they communicate but seldom meet. Many of the disability rights campaigners have consequently experienced a conflict of interest.

“There might be less challenges if the disability rights campaigner was appointed by the central office rather than the district chapter”.

“As disability rights campaigner I am under the board, … The board treat the campaigner as a staff, not as an expert that can help them, there is a question of acceptance by the board”.

Some district committees tend to consider the disability rights campaigner to be their secretary and demand that he/she is available for the committee. On the other hand, the project officer requires that the disability rights campaigner reaches a certain target with respect to visits to self-help groups, provision of ID cards, etc.

The motivation to visit and supervise the self-help groups appears relatively low among many disability rights campaigners. They indicate that they spend a significant number of days in the field visiting the groups, but the evaluation team has only been able to confirm a routine of regular visits to the self-help groups that are associated with other organizations, also providing support to the groups29. Furthermore, the scattered and in some cases impassable geographical location of the self-help groups makes the supervision visits burdensome. In Sindupalchwok for example, there is no regular transportation facilities and it takes 2-3 days to reach some SHGs by walking.

Half of self-help groups visited by the evaluation team30 express that they receive very few or no visits by

the disability rights campaigner. This is verified by meeting minutes in those self-help groups that make records of activities.

“If there is a meeting or program the president will usually be invited to the district office rather than they visit us.”

From the second quarter of 2015 the disability rights campaigners began receiving instructions from the national office as to which four self-help groups to visit the coming quarter. According to one of the disability rights campaigners the instructions led to more visits, but the campaigners have allegedly never received any clear direction in regard to the purpose and content of the visits to the self-help groups.

Thus, the general lack of regular supervision and monitoring tends to be repeated at district level, with the result that a share (most probably a larger share) of the self-groups established by NFDN31 and reported to be functional, in fact are dysfunctional; i.e. not meeting regularly, not carrying out activities, etc. The

29 This applies mainly to Surkhet, Jhapa and Ilam, where all or some of the SHGs are associated with other organizations (e.g. the Disable Empowerment Center (Surkhet), Netherland Leprosy Relief (Jhapa/Ilam) and Karuna Foundation (Ilam)) who provide funding and technical assistance.30 The team has visited at total of 10 self-help groups spread across 6 project districts. 31 Some of the groups were evidently established and functioning before the district chapter (and receiving significant support from others), but counted as a self-help group under NFDN. Furthermore, neither the disability rights campaigners nor the project staff have met/monitored the self-help groups on a regular basis, and the review team has doubts about the actual number and organizational status of the self-help groups.

23

Page 25: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

evaluation team has identified a large discrepancy between the reports provided by the disability rights campaigners and the actual situation in a number of groups visited during the evaluation; for example, in the joint District Status Report (ultimo 2015) it is stated that the groups visited in Kapilbastu and Sindhapulchok meet monthly and that the disability rights campaigners are called for these meetings (but not necessarily attending). Neither of this information can be verified by the evaluation team, which was informed that the groups have no regular meetings or no meetings at all. Moreover, in Kailali the self-help group was a well-functioning larger community based organization (established before the project) with income generating activities and support from different sources other than NFDN. In 2016 the group was visited by the district chairperson once and once by the disability rights campaigner when he, per the group, only came to check their papers.

The best functioning and most active self-help groups are found in Surkhet and Jhapa, and to some extent Ilam (please refer to the activity assessment chart below). In Surkhet the disability rights campaigner is also the president of the local Disable Empowerment Center, which purpose is to provide support to self-help groups (including the groups established by NFDN). In Jhapa the self-help groups are easy accessible and some receive support from Netherland Leprosy Relief, and in Ilam a number of groups are also linked to support from Netherland Leprosy Relief and the Karuna Foundation. It is consequently difficult to attribute the support to these groups to the district chapter.

Assessment of the self-help group activity levelActivity level D 1 D 2 D 3 D4 D5 D6High XY XZ XYZMedium/highMedium/low X Y Y XLow X*X: answer by DRCs, Y: answer by District Committee, Z: answer by Regional office

According to NFDNs implementing staff (at national as well as local level), the project has been successful in mobilizing PWDs, making them visible in the local communities and linking them to sources of local government and non-government resources, but it has been challenging to facilitate and monitor the decentralized self-help-group structure. Based on this experience, NFDN clearly indicates that it will not continue accommodating the informal structure of self-help groups within the umbrella structure. It should prospectively be the role of the DPOs to establish and facilitate such groups.

The activity budget is not arranged according to objectives and it is consequently not possible to establish the actual size of the budget for the individual objectives/project components or to make an assessment of the value for money. However, it is assumed that the decentralization component constitutes a substantial part of the total activity budget; and if it proves to be true, that component has not been able to deliver significant value for money.

Conclusion to the efficiency of the project NFDN has demonstrated a high level of efficiency in building internal capacity, structures and systems as well as conducting policy advocacy at national level.

24

Page 26: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

The intention to establish a decentralized NFDN structure is found to reasonable considering the general decentralization process in Nepal, the authorities' request for coordination and cooperation between disability organizations and the need for capacity building of DPOs at district level; but the implementation of the structure has been inefficient and consequently less effective as an umbrella structure for DPOs at district level.

The scattered distribution of the ten NFDN districts, the lack of strategic selection of districts based on relevance of a district chapter, the missing link to the regional structure and particularly the limited human and financial resources allocated to support and monitor the district chapters, have been contributing factors to the inefficiency of the project component.

It is noted that the mid-term review of phase I pointed to the need for continuous support to the districts. The inadequate support to the districts in phase II is consequently not caused by ignorance of what is required of a decentralization process, but rather the lack of strategic considerations, priority and adequate resource allocation by NFDN; the board, project staff and the project steering committee.

In the current decentralized structure the district chapters are linked directly to the national level, bypassing the regional level (located closer to the districts). The organogram below clearly illustrates an unmanageable structure, with no logical connection and cooperation between the different organizational levels and an inefficient use of resources for monitoring and support of the district chapters.

The evaluation team is aware that the project has attempted to pilot a district structure, and had the districts been less and/or had there been more proximity between the district chapters it would have been easier for NFDN national office to support and monitor the districts and to accumulate learning and experiences which could be used to expand the structure.

The somehow disorganized decentralization process has led to a disjointed and somehow ineffective and unmanageable decentralized structure, where the district chapters have been tasked with forming self-help groups and generally been left with too little direct support (and very limited customized or needs-based support), resulting in ambiguities about their roles and mandate. Such a situation provides fertile ground for frustration and conflict, and is likely to result in an inappropriate/inefficient use of available human and financial resources.

The evaluation team regards the decision not to link the district chapters to the regional structure a failure in the design process; a failure which must be corrected before more district chapters potentially emerge. The current situation calls for a fundamental reconsideration of the organization of the NFDN decentralized structure.

25

Page 27: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Furthermore, the training approach applied at district level is found to be extensive rather than intensive. There has been no systematic capacity building of the district chapters, and the expectation that the chapters would grow by a learning-by-doing approach and that any new knowledge would trickle down automatically has not been met. The failure to trickle down knowledge could be a matter of some people wanting to keep the knowledge to themselves in order to stay in power, or a matter of people receiving insufficient knowledge to be able to transfer it to others. The evaluation team is of the impression that the latter constitutes the primary explanation for the limited knowledgebase and the lack of a common understanding of roles and responsibilities within the district chapters.

The project has consequently not yet resulted in an efficient model for establishing and maintaining district chapters.

Recommendations:

# 5: It is relevant for NFDN to have a decentralized structure, but it is strongly recommended to reconsider the organization of the structure; i.e. consider how to create an efficient link between region and district level, and how to establish effective reporting/information systems, support systems, etc. in order to strategically develop an efficient and cost-effective structure.

# 6: To ensure the most appropriate use of NFDNs human and financial resources are recommended prospectively, only to establish district structures in those districts where a structure is relevant; i.e. where there is more than three DPOs and a genuine need for coordination.

# 7: To facilitate the distribution of district chapters NFDN is recommended to systematically document experiences (lessons-learned) from the ten project districts and develop a model for establishing and maintaining district chapters; including a training plan and curriculum for the strategic capacity building of the chapters (which can be applied by the regional offices).

# 8: A decentralized structure requires continuous attention. Key recommendations from the mid-term review (phase I) are therefore repeated. NFDN is recommended to:

orient the district chapters and continuously guide and motivate them for better performance, reducing communication gap and building good relation

provide some district level activities to empower the district chapter work for empowering its member organizations (DPOs) so that they will be able to self-sustain

and contribute to common issues

The latter bullet includes strengthening the leadership and advocacy capacity of DPOs as well as their capacity to conduct local resource mobilization.

26

Page 28: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

6. SustainabilityThe extent to which the benefits of the project are likely to continue after donor funding has been withdrawn.

The benefits achieved at national level as a result of good governance and advocacy activities (objective 1 and 2) are consistent with NFDNs targets for strategic organizational development and effective policy advocacy at the national level. NFDN demonstrates genuine commitment with respect to sustaining and further developing these achievements.

The sustainability of the district structure is less evident.

The establishment of the district chapters has largely been an end in itself; not a means to develop a strong umbrella structure. The efforts to develop an effective and sustainable structure have thus been insufficient.

Most district committees are committed to continue, and the district authorities are eager to see that happening. However, the district structure (the chapters) is closely linked to the district offices and the disability rights campaigners whom will not continue to be funded when the project is completed.

A self-assessment of the sustainability of the chapters, conducted among the six chapters under evaluation, shows a relatively pessimistic assessment in half of the districts.

*X: answer by DRC, Y: answer by District Committee, Z: answer by Regional office

The self-assessment contains no differentiated information of whether the answers on sustainability concerns the chapter as such or only the district committee, but it is apparent that the issue of staff (i.e. disability rights campaigners) carries some weight and that the rights campaigners, who will lose their jobs when the project ends, are significantly more pessimistic about the sustainability of the chapters.

“If there is no staff, there will be no office, and no support to DPOs, and again DPOs will work independently”.

“The structure is important. We wish to continue and expect that something will come from the center”.

Some districts are actively trying to raise funds for office facilities, but only one district have succeeded to ensure funding32.

32 Jhapa district chapter is linked with a project funded by Nederland Leprosy Relief, and will consequently have funds operating expenses associated with an office

27

Self-assessment of the sustainability of the district chaptersSustainability D 1 D 2 D 3 D4 D5 D6

High YMedium/high X XY Z XYZMedium/low Z Y XZLow X X

Page 29: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Conclusion to the sustainability of the project The evaluation is convinced that the national level achievements in regard to good governance and advocacy will be sustained after the project, due to the quality of the outputs and especially due to NFDNs dedication towards strengthening the national level governance and impacting the lives of PWDs through strategic advocacy.

The evaluation is on the other hand concerned that the sustainability of the district structure is too weak and that the dependency on facilities and dedicated human resources to conduct activities and facilitate the work of the committees will lead to a situation where the majority of district chapters close (or ceases to be active) when the project ends.

Some districts may remain with a district committee in a moderated form as a platform for DPOs, equally to the District Coordination Committees which existed in some districts prior to the project.

28

Page 30: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Annex 1: ToR

Evaluation of the project:

Towards Inclusion of Persons with Disabilities in Nepal, phase II,

January 2011 – December 2016, NFDN

1. Background

In Nepal, Disabled Peoples Organisations – Denmark (DPOD) has supported the umbrella organisation National Federation of the Disabled – Nepal (NFDN) since 2009, aiming at building capacity of the organisation to advocating for the rights of PWDs nationally and at local levels.

NFDN has as the federation 300+ member organisations across Nepal, both consisting of national and local organisations as well as single and cross disability organisations. With its head office and project office situated in Kathmandu, NFDN further has five regional offices and five regional committees under NFDN, one in each region of Nepal. The regional offices are currently upheld with support from the Norwegian Federation of Organizations of Disabled People. The committees consist of elected persons representing member organisations residing within the relevant region. Further, out of the 75 districts in Nepal, the project under evaluation has supported district offices in ten districts (in phase I it was five districts, with an added five in phase II) – two in each region, with affiliated district working committees in the same districts. Additional to the project, there are in 22 other districts across the country district coordination committees informally organised under NFDN.

Nepal has in September 2015 ratified a new Constitution, including a range of provisions for political representation of persons with disabilities. The Constitution outlines a federal system, with seven federal states, whose demarcations are contested, and the implementation of the new structure of the country may be postponed until the settling of those disputes – thus, the timing is impossible to predict. It is foreseen, though, that with the new structures, regions will seize to exist and will be replaced by the federal states, being designated with more power and authority from national level. It is also foreseen that the districts will remain, but power may be designated to lower, more local, levels.

Currently DPOD and NFDN are implementing the second phase of the project Towards Inclusion of Persons with Disabilities in Nepal. The first and second objectives have been key to the project throughout the project, focusing on building the capacity of district committees of NFDN in 10 selected districts, as well as policy advocacy at national level. The third objective has been down-scaled and phased out early in the project period. The previous project phase (phase I) had a similar focus as objective one and two, including five of the 10 phase II districts. Phase I was reviewed mid-term in November 2011, and recommendations and conclusions can be found in Annex 1. The objectives of the current project phase are:

Page 31: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

1. NFDN follows Good Governance principles as an effective and transparent organization, in order to work for ensuring the rights of persons with disabilities in Nepal.

2. The Disabled Protection and Welfare Act and National Policy and Plan of Action on Disability (NPPAD) are in line with the UNCRPD, with the NPPAD being implemented at national level and in the 5 regions and 10 districts where NFDN has a secretariat.

3. NFDN has created a foundation for becoming a proponent of evidence-based advocacy for inclusive disaster risk management (iDRM) in Nepal.

The first objective contains the development of organisational good governance guidelines and the implementation both at national level, having been key initiators and driving force of the development of the guidelines, as well as subsequently in the ten project districts. The same guidelines are being implemented in the regional offices and committees under the FFO project. Additionally, the district staff and committees are supported to conduct activities in their districts with regards to fulfilling the rights of persons with disabilities. This includes awareness raising activities in the district head quarters, coordination of member organisations’ activities, advocacy with district authorities for issues like ensuring disability identity cards for persons with disabilities, scholarships and assistive devises, and they raise funds from government to conduct activities like health camps and income-generating activities training.

Under the first objective is also the organisation of and support to ten self-help groups in each district, which are primarily supported and monitored by the district staff member, called district rights campaigner. The self-help groups do similar types of activities as the district working committee, but with their advocacy targeting the ward authorities.

The second objective covers advocacy at national level. Activities here are mainly carried out by NFDN board members, and consist of meetings with Ministries and parliamentarians on policy changes, as well as involving relevant strategic partners. With NFDN’s involvement, the Disability Act has been revised and is in the final process of being approved, and on basis of this, NFDN has worked with the Ministry of Women, Children and Social Welfare to prepare the National Policy and Plan of Action on Disability (NPPAD) – being designed the task of submitting a draft to the Ministry. However, in between Nepal has been drafting a new Constitution, in which NFDN was heavily involved in ensuring rights of persons with disabilities in the Constitution. Thus the NPPAD is being finalised only at this point in time. Although preliminary results include that some Ministries are currently planning and budgeting on basis of the NPPAD, the actual implementation still awaits.

The third objective was downscaled throughout the project period, and work under this objective was finalised by mid-2014. It is acknowledge from both DPOD and NFDN that the inclusion of iDRM during project development was to a large extent pushed from the side of DPOD, while not being an organisational priority of NFDN (which has now changed after the earthquakes in 2015). The results accomplished by 2014 was the development and distribution at district level of an advocacy tool on IDRM, inclusion of iDRM in NFDN’s 5 year strategy plan, a review of DRM related programmes in selected VDC’s (village development committees – a lower level structure of Nepal) in nine of the ten project districts. Without support of the project, NFDN has since conducted a ToT training on iDRM with UNICEF and Nepal Red Cross Society.

It is expected that key learnings of the current project will feed into the ongoing planning of a next project, with a larger focus on linking more closely the regional (to be federal, once the federal system stipulated in

ii

Page 32: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Nepal’s new constitution will be implemented) and the member organisations at district level with regards to building capacity to continuously advancing the rights of persons with disabilities at local levels. The upcoming project is envisioned to be co-funded with the Norwegian Federation of Organizations of Disabled People, who are currently supporting NFDN’s five regional committees and offices.

2. Objectives of the EvaluationThere are three objectives of the evaluation, which are:

1. An overall assessment of the project:to assess project achievements regarding relevance, efficiency, effectiveness, impact and sustainability in relation to the set project objectives, plans and activities.

2. With specific focus on the progress and results at district level :to assess and document good practices at district level regarding sustainable achievements.

3. A smaller part of the evaluation will look at the partnership:to assess the NFDN – DPOD partnership in terms of the partners’ experiences of its strengths, weaknesses and added value.

3. Scope of Work The evaluation shall comprise but not necessarily be limited to the following elements and evaluation questions:

Ad. 1: Assessing project achievements

The evaluation will examine to which extent the project has produced the planned results and achieved its objectives as well as the overall effectiveness and appropriateness of approaches, methods and tools applied in the implementation of the project.

Concerning the project as a whole (across all three objectives), evaluation questions include: Are achievements in keeping with needs and priorities of the beneficiaries? Are achievements relevant to the policies and strategies of NFDN, DPOD policies and strategies, and relevant national policies? Were activities and strategies sufficient to realise agreed outputs? Is it likely that the project will have the planned impact? Were the division of roles and responsibilities between board and staff conducive to reaching the objectives? To which degree have financial resources and inputs (funds, expertise, time etc.) been converted to results/outputs (i.e. have these been delivered as agreed)? Could it have been done better, more cheaply and more quickly? Has project and financial management practices, procedures and mechanisms, budgets, plans of action, M&E systems, etc. been followed and has been efficient? To which degree has NFDN at all relevant organisational levels obtained ownership of the project and its achievements? Have attempts been made to promote harmonisation with others; thereby reducing administrative costs of the project implementation (e.g. through establishment of strategic partnerships or

iii

Page 33: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

alliances with other partners)? Are achievements sustainable, including the cooperation with relevant external strategic partners?

Concerning the project component under project objective one, additional evaluation questions include: Regarding national level good governance what were the challenges and to which extent has it been addressed? Which role has the guidelines played, how has the process been managed and how has the guidelines been implemented?

(work related to district level and self-help groups are covered under evaluation objective 2.)

Concerning the project component under project objective two, additional evaluation questions include: What are good achievements and approaches to advocacy both at national and district level? Which good advocacy practise has been applied, including use of strategic partners, and to which extent are the practices borne by individuals rather than organisational practices? To which extent has there been sufficient advocacy strategy planning? To which extent is advocacy at local levels informed or inspired by content of and/or approaches to advocacy at central level?

Concerning the project component under project objective three, additional evaluation questions include: To which extent is achievements on iDRM sustainable, and to which extent have they been taken forward outside of the project scope (eg. by NFDN during the aftermath of the April and May earthquakes in 2015)? With a renewed focus within NFDN on iDRM, how can the achievements under the project support the strategic focus?

Ad. 2: Assessing district level good practises

The evaluation will assess progress and results at district level and in relation to the regional and national level as well as document good, sustainable practises.

Concerning the district level achievements and practises, evaluation questions include: Has the approaches applied at district level been effective and appropriate? To which extent has the roll out of one approach been conducive to the achievements of sustainable results? To which extent has elements of individualised support to each district based on the particular needs identified been conducted and has it been effective? What are good achievements and approaches to mobilising and building capacity of the disability movement at district level, and why was the project successful in reaching these achievements? What were the challenges sought to be addressed by the project interventions at district level, and to which extent have they been addressed? Which role has the good governance guidelines play in the achievements at district level, how has the guidelines been implemented and how has the process been managed in the districts?

Concerning interlinks with and support to district committees, evaluation questions include: What is the character and role in achievements of interlinks and support between 1) self-help groups and the district committee, 2) the district committee and the regional committee, 3) the district committee and the national level, and 4) the relation to and involvement of non-member DPO’s at district level?

Concerning district level functioning in comparison to district coordination committees in non-project districts, evaluation questions include: Comparing district coordination committees (non-project districts)

iv

Page 34: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

and district working committees (project districts), how are the different district set-up functioning and how is it conducive to achieve rights of persons with disabilities at district level?

Concerning the self-help groups, evaluation questions include: What are the key achievements of the self-help groups? Are the groups considered relevant by their members and to which extent is the functioning of the self-help groups sustainable? Has the support given to the self-help groups been appropriate and effective?

Ad. 3: Assessing the NFDN-DPOD partnership

The evaluation will provide suggestions on how to further the current partnership and provide input to the DPOD partnership strategy process to be undertaken in 2016-2017.

Assess the NFDN – DPOD partnership in terms of the partners’ experiences of its strengths, weaknesses and added value with a view to provide suggestions on how to further the current partnership and provide input to the DPOD partnership strategy process to be undertaken in 2016-2017.

4. Method of workPrior to the field work, a desk review of the below list of documents (section 8) will be conducted.

The methodology to be applied during field work will be participatory through consultations with staff, board and committees of NFDN, as well as stakeholders external to NFDN, such as self-help groups, member organisations, the District authorities, Ministry of Women Children and Social Affairs, Disability Unit and relevant strategic partners.

With regards to the objective one, the evaluation will fall in two parts. The first part will take place together with the remaining part of the evaluation and will include a pilot visit to one of the five regions of Nepal, visiting the two relevant districts as well as an additional district with a coordination committee. Based on tools developed by the evaluation team during preparation, data will be gathered from the said districts and region, and simultaneously the tools will be tested and reviewed.

The evaluation will conclude on the evaluation objectives two and three, and preliminarily on objective one, on basis of which the evaluation report part I will be produced.

Subsequent to the first field visit and without the participation of the team leader, the team member supported by the NFDN project secretariat (a separate specific ToR will be prepared for this part of the assignment, depending on the scope decided during end of field work part I), will make visits to the remaining four regions of Nepal and to the majority of the eight remaining project districts to collect data based on the revised tool. Once accomplished, the data will be submitted to the team leader, who will prepare the evaluation report part II, concluding on the evaluation objective one.

The regional visits within each five region will include visits to the regional office, the district working committee and the staff member in two project districts (or where not possible, the staff member and a representative from the committees will travel to the regional head quarter), persons from a few self-help groups, together with visiting a district coordination committee in an adjacent district.

v

Page 35: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

The collective field visit by the full team is expected to end with a debriefing workshop for key stakeholders at NFDN. The draft report part I and part II will each be distributed to NFDN project secretariat and board and to DPOD for comments and further discussion. After receiving the comments, the Consultant(s) will finalise the reports and submit to DPOD and NFDN.

5. Output An inception note A debriefing note or PowerPoint presentation of part one that summarises the findings and

recommendations of the first part of the evaluation. This is presented at meeting/seminar/workshop at the end of the review mission in which NFDN participates.

Two evaluation reports (part I and part II) of not more than 12 pages each (part II is expected to be significantly shorter than part I), excluding annexes and executive summary. The reports shall comprise an executive summary of not more than three pages. Conclusions and recommendations shall be stated in a separate section.

6. Time frame The first part of the field work will take place from app. 9th to 18th November (10 days for the team

leader including travel time – for the team member it will amount to app. 8 days). Beforehand, there will be 3 days for preparation (team leader – 1 day for the team member), and afterwards there will be 4 days of compilation and drafting of the report (team leader – 1 day for team member for input). After second part of the evaluation (the scope of the assignment with the team member will be drafted by the end of the field work of the first part of the evaluation), there will be 2 days for compilation and drafting of the report.

The part I draft report will be delivered to DPOD and NFDN no later than 10 th December. Feedback from NFDN and DPOD will be submitted within 10 days of the draft report, after which

the final report will be submitted to NFDN and DPOD within one week.

7. Evaluation Team Composition and responsibilitiesTeam leader: Dorthe Skovgaard Mortensen.

Team member: a Nepali consultant (to be determined).

The team leader is responsible for preparing the outputs. The team member is responsible for supporting the input during data collection and for providing input on content of the outputs.

NFDN administrative manager, project coordinator and finance officer as well as DPOD programme coordinator will act as resource persons and have an active part in the Evaluation. For the first field work, a Nepali-English translator will be engaged to support the team leader.

8. Key DocumentsKey documents made available for the Evaluation during the preparation period:

Project documents of phase I and II, and supporting documents Updated budget, including expenditures to date Implementation plans Baseline report, M&E plan, latest monitoring reports

vi

Page 36: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Quarterly and annual reports Phase I mid-term evaluation report Other key program documents, material, tools, etc. DPOD/Partner policies and strategies.

vii

Page 37: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Annex 2: Review programmeDate ProgrammeTuesday 8th

Inception meeting evaluation team

Wednesday 9th

Meeting with external stakeholder (MyRights) to get an external view on the disability sector and the broader Nepalese socio-political context.

Meetings at NFDN office:

Introduction and evaluation briefing with NFDN Initial output/outcome assessment with implementing staff Focus group interview with 3-5 board members

Thursday 10th

Field visit, travel to :o Kapilbastu (Western Region)o Kailali (Far Western Region)Friday

11thSaturday 12thSunday 13thMonday 14thTuesday 15th

Final meetings with NFDN board staff to follow up on issues arising from visits to the districts and to conduct a rapid partnership assessmentMeeting with MoWCS

Wednesday 16th

Half day for debriefing noteDebriefing meeting

Following the initial visit to two districts, the remaining districts have been reviewed by Rita Shrestha according to the plan below:Date District Meetings22-23 Nov Sindhuplachowk 1. SHG-2

2. DISABILTY RIGHTS CAMPAIGNER3. Local Government (Municipality, DDC)4. District Chapter members

28 Nov- 1 Dec

Surkhet and Nepaljung

1. SHG-22. DISABILTY RIGHTS CAMPAIGNER3. Local Government (Municipality, DDC)4. District Chapter members5. Regional Office6. Non project DPO

11-16 December

Jhapa, Ilam and Morang

1. SHG- 42. DISABILTY RIGHTS CAMPAIGNER -23. Local Government (Municipality, DDC)- Ilam and

Jhapa4. District Chapter members-

Ilam and Jhapa5. Regional Office- Ilam and Jhapa6. Non project DPO - Ilam and Jhapa

viii

Page 38: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Annex 3: District map

ix

Page 39: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Annex 4: Monitoring visits to district chapters

District 2016 2015 2014 Total Surkhet II 2 Sindhupalchok I I (I) I 3 (4) Kaski I II 3 Jhapa I (I) II 3 (4) Ilam I II 3 Kailali I I 2 Kapilbastu I 1 Nuwakot (I) (I) II 2 (4) Baitadi I I 2 Jumla 0 Total 6 (7) 4 (7) 11 21 (25)

x

Page 40: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Annex 5: key answers from visits to district chapters

1. Focus group interview with SHGsAim: to assess the relevance and achievements of the SHG, and the sustainability of the group.

District 1 District 2 District 3 District 4 District 5 District 6How often do you meet?

SHG 1: Once a monthSHG 2: Once a month

SHG 1: No regular meetingsSHG 2: No meetings

SHG 1: Twice a monthSHG 2: No regular meetings

Regular meetings

No regular meetings

SHG 1: Once a monthSHG 2: Once a month

Out of 10 self-help groups 6 groups meet regularly.How often do the DRC visit you and what is the nature of the visits?

SHG 1: Once a month (book keeping, progress analysis, needs assessment)SHG 2: Once every two months

SHG 1: Is invited to the NFDN office rather than receiving visitsSHG 2: Never

SHG 1: Regularly (bookkeeping and application of local funds)SHG: Once or twice a year

Once a year during application of local fund at DDC.

Occasionally (The group does not know the name of the DRC)

SHG 1: Twice a yearSHG 2: Once every two months

There is a large difference in the number of visits from DRC, some SHG are visited regularly, mainly groups in the district where the DRC is responsible also for an organization supporting SHGs. Some receive no visits, but the common feature is one to two visits a year.

What kind of support have you received from NFDN district chapter?

SHG 1: savings and credit activities + links to local level cooperatives and health servicesSHG 2: health treatment, assistive devices, revolving funds, training

SHG 1: One trainingSHG 2: Help to access local funds (by the president of the chapter)

SHG 1: Building linkages with the DDC, leadership trainingSHG 2: No support (except help by a member to apply to the municipality).

Have not received anything after being affiliated with NFDN

Orientation about different disability groups, and leadership training (the latter by committee members).

SHG 1: Help to write proposals to the VDC, orientation on disability rightsSHG 2: Help to form the group and collect data on PWDs.

Common support is assistance to apply for local government funds in the VDCs, and shorter trainings/orientations. Some indicate that they have received only limited support, which is consistent with the relationship to the DRC. while others have received

What are the key achievements of the SHG?- Did NFDN

contribute to

SHG 1: able to everything ourselves, recognized

SHG 1: -SHG 2: Awareness of rights, confidence

SHG 1: empowered and able to speak in front of

A saving group, trainings, tailoring, and

Preparing a plan for the development forum;

SHG 1: distribution of assistive devices

xi

Page 41: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

these achievements? – if yes how?

and proudSHG 2: ID-cards, influence schools to build ramps

to speak people. Increased funding from VDCSHG 2: Not able to do much

collecting data on PWDs.

demanding assistive devises, capacity building and if possible a disability friendly hall.

through VDC budget.SHG 2: To work as a team, manage office running costs, collect money

Do you expect to continue receiving that support next year?- What will

happen if you don’t receive it?

SHG 1: There is continued support from DEC/INF.SHG 2: Will continue with funds from the local budget

SHG 1: Not dependent on NFDN, was able to access local funds before the chapter was formed.SHG 2: No difference when there is no support

SHG 1: We will continueSHG 2: Support is very important, it will help us to build linkages to other groups.

Was sustainable before becoming a SHG under NFDN. Receiving help through NLR.

Cannot continue without the support of the district chapter.

SHG 1: Will continue without NFDN support. Receives help through NLR.SHG 2: Can continue without the support of the chapter.

Not all groups were formed by NFDN, and several groups (district 1, 3 and 6) are tied to other organizations providing support (same districts indicate a high activity level). Some are actively accessing local government funds. There is a great will to continue in most groups, however at least four groups (two from district two, one from district 3 and one from district 5) are not considered sustainable.

Self-assessment by DRC (X), District committee (O) and regional office (V)

SHG

activ

ity

leve

l

High XO XV XOVMedium/highMedium/low X O O XLow X

2. Focus group interview with district committee members Aim: to assess the relevance and outcome of the project mainly in regard to NFDN decentralized structure, good governance and sustainability and local level advocacy.

District 1 District 2 District 3 District 4 District 5 District 6How many members are there currently in the district committee?

8 (total of 10 organizations in the district but 2 are not DPOs)

8 (out of a total of 11 DPOs

6 DPOs 7 DPOs 2 DPOs 13 DPOs (all district DPOs – including non-NFDN members)

xii

Page 42: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

How often do you meet?

Quarterly or per need

No meetings, new board (since August)

Quarterly meetings

Quarterly meetings

Quarterly meeting

Monthly meetings

What would you say are the most significant changes achieved by the project?

Organization and mobilization of DPOs

A platform to discuss issues, joint DPO meetings, more funds for joint activities, joint advocacy.

Not easy to bring all DPOs under the umbrella

More active DPOs

Facilitation of SHGs, collection of data on CWDsin schools and PWDs employed, more joint lobby activities

Recognition by local GO. Easy access to GO officials.

Establishment of self-help groups, receiving funds from local GO.

Coordination of DPOs, reduced duplication of work, confidence and trust building, coordination with NLR

How often do persons from the national level (staff/board) visit the project?

Once a year (by staff or board members)

Two visits in 2016

Once a year

Once a year by the coordinator (board members more often)

Once to twice a year

Self-assessment by DRC (X), District committee (O) and regional office (V)

Boar

d ac

tivity

le

vel

High XMedium/high

XO X X XOV XV 0V

Medium/low

0V

Low

NFDNs decentralized structure:How important is the NFDN representation at district level for the national structure?

NFDN is an umbrella representing all PWDs. It is very important to have a district level structure to support DPOs. It is not possible to reach the DPOs from central level.

It is important to empower PWD and bring their voice to the central level, but no delegation of power/authority

Due to the district chapter, it is possible to obtain cooperation with other organizations (e.g. Karuna Foundation)

Monitoring activities of DPOs and providing inspiration and motivation to DPOs

- The district structure is important since line ministries look for NFDN for any activity, and they seek advice from NFDN, not individual DPOs

xiii

Page 43: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Can you give any examples of how the district chapter has mobilized and built capacity of the disability movement in the district?

If there is an issue the district chapter advocate for the issue, e.g. ID cards, transportation discount

ID-card distribution (in cooperation with WDO)

- - Active use of the budget for PWDs, if not the budget is reallocated to other issues.

ID card distribution (in cooperation with WDO), and a database of PWDs in the district.

What do you think will happen with the district chapter activities when the project ends?

Will continue, but without staff

If there is no office there won’t be any coordination among the DPOs

Will continue, but without staff. May have support from local companies.

Will try to mobilize local resources to continue.

Wish to continue and expect that something will come from the center.

Will continue, but will have limited facilities. Has support from NLR

What will happen with the SHGs when the project ends?

All will continue with support from VDCs

2-3 SHG will continue as they have access to local funds.

After association with Karuna Foundation the SHG will continue to function.

- Estimates that 8 out of the existing 10 SHGs can continue

About 50% will continue.

Do you feel that you have received sufficient support from NFDN at the national level?

No, the support has not been enough, but we have received technical support

No, not enough.

Yes, when we need or request for it.

No, would like more feed-back and faster response on issues.

- Yes, we get regular advice, but there is no financial support.

Self-assessment by DRC (X), District committee (O) and regional office (V)

Sust

aina

bilit

y High OMedium/high

X XO V XOV

Medium/low

V O XV

Low X XQuestions on Good Governance:Are you familiar with NFDNs Good Governance Guidelines?

Yes No No No No -

How have you applied the Good

Yes, some:Clear documenta

No No No No No (new committee

xiv

Page 44: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

Governance Guidelines at district level?

tion, transparent budgeting and minutes from meetings

members)

Has the committee been trained on NFDN systems, workflows and policies, and on the Good Governance Guidelines?

Two members were trained, after training the other members were trained. Learnings has not been implemented.

The past president was trained. Some documents are there, but the new committee has not gone through it.

- The previous vice chair participated in a training on good governance

None of the current members have been trained orreceived any documents.

-

Is NFDN concerned about issues of good governance in you district? Do feel you get sufficient support to exercise good governance practices?

No Don’t know.

No - - No

Questions on advocacy:Do you feel that you have sufficient capacity to conduct advocacy?

Have worked with advocacy for many years

We feel that we are capable

Yes - - Yes, have concrete examples of advocacy activities.

How has NFDN at national level supported your advocacy work?

No advice or coordination.

Supported interaction with likeminded organizations and thereby increased visibility of the chapter

Have not requested for support

- No, received no directions or participated in any national advocacy campaigns

Yes

Self-assessment by DRC (X), District committee (O) and regional office (V)

Boar

d

High 0 0V XMedium/high

X X X XV 0VMedium/ X 0V

xv

Page 45: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

advocacy level

lowLow

Questions on iDRMAre you familiar with the advocacy tool on iDRM?

No, but the DRC has been trained

No No - No No, maybe the previous committee knew about iDMR

In your opinion, what could have been done differently in the project?

Project period too short. Most activities were carried out by the DRC. When the DRC was not there, the office had to be closed.

Forming SHGs and visiting them once or twice a year does not make sense.

- Capacity building of the district board. We don’t know how about our roles and responsibilities.

DPOs should form SHG, rather than NFDN

3. Interview with district staff (DISABILTY rights campaigner)Aim: to obtain their assessment of the organizational project set-up, coordination, capacity and project implementation, including the appropriateness of the applied approaches, methods and tools.

District 1 District 2 District 3 District 4 District 5 District 6When was the district involved in the project? And when were you employed?

2013 (current DRC employed in 2013)

2013 (current DRC employed in 2013)

2013 (current DRC employed in 2013)

2013 (current DRC employed in 2013)

2009 (current DRC employed in 2013)

2009 (current DRC employed in 2013)

How many (active) SHGs do you have in the district?

26 SHGs all associated with DEC and active

10 SHGs, only few are active

10 SHGs formed by NFDN (25 formed by VDC with support from Karuna Foundation). Not all

10 SHGs, the nearest group is 30 km. from the office, the furthest is 100 km.

10 SHGs, about 6 are active.

11 SHGs, all are active

xvi

Page 46: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

SHGs are active.

Please explain briefly about your key roles and responsibilities as DISABILTY rights campaigner.

50% support to SHGs40% advocacy, 5% reporting, 5% DPO work

20% representation in different events15% monitoring and communication, 10% coordination, 5% documentation, 5% SHG visit.

40% linkage and coordination (SHG), 20% monitoring, 20% advocacy, 20% reporting

Coordination of SHG, sending progress reports, share information about PWD/SHG entitlements. Providing service/referrals to PWDs visiting the office.

Motivating SHGs, providing service to PWDs who come to the office to seek advice e.g. in regard to identity cards.Use approx. 10 days a month to visit SHGs.

25% advocacy40% reporting /monitoring from SHGs35% technical support to DPOs

How often do you visit the SHGs and what is the nature of the visits?

Quarterly, as per guidelines provided by central office + upon request

As per guidelines provided by central office

Regular visits, some SHGs every month. Due to budget and geography not all SHGs have been supported (mostly visits to the SHGs linked to Karuna Foundation)

6 SHGs have been visited twice in 2016. Due to travel difficulties, it has not been possible to visit the other SHGs.

- Quarterly, as per guidelines provided by central office + after need. There is easy access to the SHGs in the district.

To whom do you report?

Central office, informs the committee.Feed-back mainly on information gaps.

Central office, informs the president of district committee.Feed-back mainly on language.

Central office, sometimes to the regional office.Not much feed-back.

Central office. If we don’t send the report there is a follow up from the national level.

Central office. The reports do not cover challenges, but are quantitative; i.e. how many cards, how many meetings, how many visits, etc. No feed-back as

Central office, committee signs, do not sent to regional office. Feed-back mainly on content and format

xvii

Page 47: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

such.How often do you communicate with programme staff at the national level, and how often do they visit the project?

Weekly, and a yearly visit

Regular phone conversation. Two visits in 2016.

Regular communication. Visit once a year.

Regular communication per mail. One yearly visit.

Two visits since 2013.

Once a week, and one visit a year (2015 two visits)

What trainings have you received?

GG, resource mobilization, iDMR, strategic planning. Afterward trained SHGs in GG.

GG, proposal writing. Not done any subsequent training but supported DPOs to write proposals

GG, proposal writing, bookkeeping, time managementAfterwards training of some SHGs.

GG, finance, strategic planning. Some training (e.g. finance) was provided by NLR.

GG. GG, reporting and accounting. Afterward trained the district committee on GG

Do you feel that you have sufficient capacity / support to conduct advocacy?

Yes, experienced conducting advocacy

Not confident to do it alone.

Yes - - Basic knowledge

Do you see an increase in the inclusion of PWDs in the local networks and developments forums?

Yes, inclusion in ward citizen forum, cooperatives, women groups.

Due to government policies, most groups and networks include PWDs

Yes, inclusion in mother’s groups, ward citizen group, etc.

Yes, inclusion in and youth groups, disaster reduction groups

- Yes, inclusion in user’s groups, ward level committees, youth committees etc.

Are you familiar with the advocacy tool on iDRM (distributed at district level)?

Yes, been training, but not used it.

Yes, read through it, but not used it. Saw that it was relevant, after the earthquake

Yes, and used it for disaster mapping in two non-VDCs.

- - Yes, have made a iDMR survey, and gone through the tool.

Do you feel that you have received sufficient support from the national staff?

Do not request support from national level

Yes Yes When I have challenges, I contact the coordinator who

No Yes, always.

xviii

Page 48: international.handicap.dkinternational.handicap.dk/media/215080/evaluation-report... · Web viewEvaluation REPORT. An evaluation of the project “Towards Inclusion of Persons with

refer to Mourice, who inform the board and they respond.

What do you think will happen with the district chapter activities when the project ends?

No staff, but there will be a platform where the DPOs can meet. Maybe the name will change to District Coordination Committee (like before the project)

If there is no staff, there will be no office, and no support to DPOs, and again DPOs will work independently.

If we get funding from local business companies (ongoing negotiations) the chapter will remain. If not, the office will close.

I wish that the project will be extended. We need more years to make the work sustainable.

- The chapter will continue (project and support from NLR)

In your opinion, what could have been done differently in the project?

Formation of SHG is not the role of the chapter, but DPOs. Instead the chapter should improve GG of DPOs. Should have worked with all the VDCs.

Regular impact monitoring from central level. Saving and credit for SHGs.

Maybe less challenges for the DRCs if they were appointed by the central office, rather than the district chapter. + More support from the SHGs.

The VCDs (SHGs) are far from the head quarter; when PWDs come to seek advice, there is no one at the office and they get annoyed.

- If the SHGs had seed money they may have been more active.

xix