6ghmeforum.org/.../uploads/2019/07/ghana-cso-toc-report-phase-… · web view27/02/2018 · csos...

Civil Society Organisations in evaluation systems

STRENGTHENING THE ROLE OF CIVIL SOCIETY ORGANISATIONS IN EVALUATION SYSTEMS: WORKSHOP REPORT

ACCRA, GHANA26th - 27th February 2018

i | P a g e


ACRONYMSCLEAR-AA Centre of Learning on Evaluation and Result Anglophone Africa CSOs Civil society OrganisationsECD Evaluation Capacity DevelopmentGWM&E Government Wide Monitoring and EvaluationGMEF Ghana Monitoring and Evaluation Forum INGOs International Non-Governmental Organisations JADF Joint Action Development ForumJSR Joint Sector Reviews.LNGOs Local Non-Governments Organisations M&E Monitoring and Evaluation MoF Ministry of FinanceNES National evaluation systemsNGOs Non-Governmental OrganisationsNSD National Savings Direction RBOs Religions Based Organisations SSP Sector Strategic PlansUN United NationsUNICEF United Nations International Children's Emergency FundVOPE Voluntary Organisation of Professional Evaluator

ii | P a g e


1. Background In 2016, the Centre for Learning on Evaluation and Results Anglophone Africa (CLEAR-AA) collaborated with Twende Mbele, to research the current roles Civil Society Organisations (CSOs) play in Government Monitoring and Evaluation Systems. The research was conducted through a series of workshops in three targeted countries: Uganda, South Africa and Benin. The body of research produced from the workshops, allowed for an in-depth analysis on the levels of engagement between civil society and the state. The work shaped a series of recommendations on how these countries are able to strengthen their government M&E systems. The recommendations derived from the project, were primarily produced from a state-centric perspective, with little focus placed on the needs of civil society. To complement this work, an additional four workshops were conducted in Rwanda, Kenya, Ghana, and Uganda over the course of 2018 and the beginning of 2019. The aim of these workshops was for civil society participants to grapple with the extent to which CSOs contribute to their Government M&E Systems, through using the CLEAR-AA framework - 10 functions of a National Evaluation System. Furthermore, the workshops acted as a Theory of Change (ToC) training, which saw participants confronting challenges of CSOs’ participation in their countries’ National Evaluation System. This report outlines the results from the CSO workshop held in Ghana. The Ghana CSO workshop was carried out in collaboration with the Ghana Institute of Management and Public Administration (GIMPA). The workshop was held in Accra, Ghana for two consecutive days between the 26th-27th of February 2019.

2. IntroductionResearch shows that National Evaluation Systems (NES) have been emerging in various African countries, differing in maturity, capacity and effectiveness (Porter and Goldman 2013). African governments are implementing evaluation systems to measure outcomes and impacts of their investments in social services, infrastructure and other public goods. As M&E continues to grow as a discipline across the continent, it is essential to map out the various stakeholders roles and responsibilities in their contribution towards the NES.

Civil Society Organisations contribute significantly to the social and economic development of the country as they often play an intermediary role within society. Within an M&E evaluation ecosystem, CSOs are an important source of evidence generation and building the capacity of a country’s monitoring and evaluation system. The purpose of the CSO TOC training workshop in Ghana was to facilitate a structured discussion amongst leaders of Civil Society Organizations, to establish the current role as well as future aspirations of civil society in the building of the nascent Ghanaian National Evaluation System. The Ghana Institute of Management and Public Administration (GIMPA) hosted civil society participants, from various sectors, to understand the different perspectives within the Civil Society Organisations.

1 | P a g e


Using the CLEAR-AA 10 Functions of a National Evaluation System as the methodological framework, the workshop discussions indicated both strengths and weaknesses of CSOs in their contribution towards the NES in Ghana (refer to Annexure 1). This report outlines the challenges CSOs face in influencing the NES through a problem analysis exercise. A Theory of Change model, was formulated thereafter, as participants explored the possible solutions to combat these challenges.

3. Problem Statement CSOs play a significant role in a country’s evaluation system(s). In Ghana, numerous national and community-based NGOs undertake development projects with funding support from donors. To prove project effectiveness and account to their donors, these NGOs periodically commission evaluations of their projects. Major NGOs in Ghana who occasionally commission evaluations of their projects include CARE Ghana, PLAN Ghana, JICA, IBIS, World Vision, and Action Aid. There is a strong incentive for CSOs to improve their performance through the application of recommendations, as this will translate to secure funds for future projects. However, even though CSOs are currently generating monitoring and evaluation data through their projects and contributing to the M&E ecosystem in Ghana, there is limited interaction between CSOs and the Government M&E systems.The reasons why CSOs should contribute to Government M&E systems are numerous. First, government institutions are held accountable for the delivery of goods and services. The evidence generated through civil society communicates the voice of citizen experiences at grassroots levels where government institutions often struggle to reach. In addition, CSOs act as strong advocates for the most marginalised populations in the country – fighting for the rights of women, children, refugees and citizens with disabilities. Furthermore, CSOs engage in service delivery projects themselves and often have elaborate M&E reporting structures that generate evidence around service delivery performance. This evidence is of great value to various line ministries that are running national programmes under which these civil societies led services fit but where there is insufficient coordination to link the work of civil society into the evidence systems being generated by the state.

a. Purpose of the workshop

The purpose of the workshop was twofold: i) to train participants on how to develop a Theory of Change (ToC) for a programme or project, and ii) to understand from a CSOs perspective the problem/s hindering CSOs to contribute to their government led evaluation systems, which could inform future country interventions. The objectives of the workshop were to:

a. Develop a ToC based on how CSOs can engage in government led evaluation systems.

b. Identify areas where CSOs either collaborate or contest government evaluation systems

c. Foster critical and analytical discussions among civil society leaders and actors on the role of civil society in the NES

2 | P a g e


d. Identify ways to strengthen participation of CSOs in Government led evaluation systems going forward

b. Workshop participants

The key informants were chosen based on their position within the selected CSOs, as well as their knowledge and insight on the current status quo between government and CSOs engagement. Participants attending the workshop included the Planned Parenthood Association of Ghana (PPAG), Integrated Social Development Centre (ISODEC), Ghana Centre for Democratic Development (CCD), Ghana Anti-corruption Coalition (GACC), Centre for Development Initiatives (CDI), Water aid Ghana, Wash Alliance International, Christian Aid as well as several media outlet agencies.

c. Limitations

A small number of CSOs participated in the workshop; therefore, the views of the CSOs who participated in the workshop are not representative of the entire CSO sector in Ghana. Second, the selected CSOs were included in the workshop, on the grounds of convenience or based on established rapport with GIMPA. Third, workshop participants were not familiar with CLEAR-AA concepts used in the workshop, such as the “Ten functions of evaluation systems”. Lastly, CSOs generally engage with sectoral ministries and not specifically with heads of government. Some of the participant responses may be referring to their experience with sector ministries and not the centre of government coordinators of NES/GWM&E system like the ministry of finance, etc.

4. Ghana Country ContextGhana has experienced consistent economic growth over the past thirty years, through the successful implementation of various poverty reduction programmes and economic reforms. The country has achieved remarkable improvements in the standard of living, with poverty reduction rates exceeding other African countries. Both absolute and extreme poverty rates have radically decreased in the past 25 years. The country has attained remarkable gains in living standards, public health, and education. Ghana is one of the few countries in Africa that has been able to produce reliable and comparable data of human welfare indicators, spanning across two decades (Molini, V & Paci, P, 2015, p. 3). The Ghana Living Standards Survey (GLSS) is a multi-dimensional survey, gathering statistics on individual and household welfare. Seven rounds of GLSS data has been collected since 1987, becoming a critical tool in informing government decision-making. Historically, the Ministry of Finance (MoF) and the National Development Planning Commission (NDPC) combined to form the central management agencies responsible for the M&E framework in Ghana. This evolved in 2017, when the Ministry of Monitoring and Evaluation was established to track the presidential priorities outlined in the governing party’s manifesto prior to coming to power in 2016. As such, the MoF is a main government institution responsible for the mobilisation, allocation and management of financial resources while the

3 | P a g e


NDPC has driven the coordination of planning and performance reporting from District Assemblies, Municipalities and Metropolitan areas. The NDPC also advises Parliament on the performance of public policy/programmes and recommends improvements. The technical responsibility for coordinating the national M&E system rests with the NDPC in collaboration with Ghana Statistical Service (GSS) (CLEAR- AA, 2013, p.9) while the political responsibility for performance against Presidential priorities is with the MOME.Since the mid-1990s, Ghana has seen a major shift in the relationship among the State, development partners and CSOs with respect to development policy dialogue in particular and public policy making in general (A.G, Abdulai & R. Quantson, 2009, p.114). CSO participation in policymaking has been warranted for various reasons, ranging from the inadequacy of state institutions in meeting the demands of its citizens to making development interventions more sustainable, and more recently, to the protection of the rights of the vulnerable in society (A.G, Abdulai & R. Quantson, 2009, p. 119). In 2005, the Ministry of Finance and Economic Planning introduced an open-door policy aimed at encouraging civil society to make inputs into the preparation of the national budget. The Ministry of Finance and Economic Planning (MOFEP) invites civil society actors through open advertisements in the print and electronic media to submit memoranda on issues that could inform the prioritization of resource allocation in the national budget (A.G, Abdulai & R. Quantson, 2009, p. 139).However, the limited impact of Ghanaian CSOs on policy outcomes in other sectors can be explained through numerous challenges. Primarily, establishing enabling legal and/or political environment for civil society actors is only but one of several factors necessary for enhancing CSO effectiveness in policy making. A variety of other obstacles, such as limited information on government policies, limited CSO capacity, absence of effective structures for engagement as well as inadequate commitment on the part of the political elite to genuinely engage civil society (A.G, Abdulai & R. Quantson, 2009, p. 128).

5. Workshop Findings SummaryThe capacity restrictions encountered by CSOs, according to respondents, particularly financial and human resources, largely affect the ability for CSOs to be involved in government M&E systems. With the exception of a few, most CSOs lack the capacity for policy analysis and the research abilities needed to engage effectively with the state. A fragmented Civil Society in Ghana further weakens the collaboration and agreement on key policy issues. CSOs lack a coordinated front to engage with Government effectively at the level of building a centralized evaluation system. CSOs are often criticised to be pre-occupied with their individual goals than with collective national goals. Despite the fact that all CSOs are striving towards the same ultimate objectives of effecting purposeful social change to enhance the wellbeing of people, harmonisation between these organisations and bodies, remains relatively weak.

4 | P a g e


The Ten Functions of Evaluation Systems was useful to understand the roles and responsibilities various civil society representatives played in evidence production and where they fit into the existing Ghanaian national evaluation system (See Annexure 1 & 2).

6.1. CSOs contributions to the National Evaluation System

See Annexure 1 for a visual representation of the Ten Functions of an Evaluation System. To understand the status of CSOs in the National Evaluation System or Evaluation System/ Space, we asked workshop participants to identify functions, which CSOs have been contributing to within the CLEAR-AA Ten Functions framework.

a. Planning and Setting Direction Participants consider CSOs in Ghana to have the ability to influence policy at the local and national level due to the variety of work across all sectors. CSOs are often consulted by local ministries, because they are able to provide local knowledge.

b. Evidence Production and ResearchParticipants highlighted their contribution to evidence production. CSOs generate various evaluations, research, surveys and reports that recommend areas for improvements at policy level. Participants indicated that CSOs in Ghana are increasingly producing research and M&E evidence mainly as a result of donor requirements. The use of that evidence however does not necessarily infiltrate the government system. CSOs conduct surveys at a grassroots level, and use community scorecards to assess service delivery.

c. Performance MonitoringParticipants noted the CDD “I Am Aware” project as the prime example of CSO performance monitoring. The District League Table (DLT) implemented by CDD since 2011 aims to increase social accountability for development in order to improve responsiveness in service delivery. Secondly, the “I am Aware” initiative has inspired citizens, the District Assemblies and the MMDAs to use the District League Table evidence to monitor the state of service delivery in the country.

5 | P a g e


Both projects have been successful in addressing data/ information gaps in government evaluation systems and allowing information to be easily accessible and understood by citizens.

d. Use of Evidence FindingsCSOs in Ghana currently use evidence in the preparation of funding proposals. CSOs depend on evaluations to design and scale up interventions. CSOs produce evidence through conducting research and evaluations. The evidence produced influence their programming, planning and budgeting.

6.1.2. Challenges CSOs in Ghana experience in their contribution towards the NES

a.Institutional Arrangements

CSOs are not involved in the creation of new ministries/ institutions or new policies. CSOs do not have the mandate to execute M&E at a national level, as government is the strongest player in the system and has the directive to inform direction.

b.Shaping Axiology’s and NormsParticipants mentioned the need for CSOs to shift their norms and positions to please government/donor agenda. CSOs are weak in their ability to hold government accountable. Although civil society desire to influence norms and values at the larger scale, there is a lack of influence on political party manifestos, which sets the pace for the national development agenda. CSOs only shape values indirectly.

c. Planning and Setting DirectionAlthough the participants agreed that planning and setting direction is one of their strongest contribution in the evaluation ecosystem, they felt that there are various limitations to their ability to influence government planning and policy direction. Government provides oversight of the M&E system therefore produce frameworks, guidelines, standards and policy for evaluations. CSOs are not aware of the government planning cycle, which limits their ability to become included. The primary outcome of the workshops indicated the desire for CSOs to become involved in their country’s policy planning and implementation cycle. CSOs experienced limitations within influencing Government planning at a National, Sectoral level and Local Government as government actors do not generally consider CSOs contributions of evidence generation.

d.Evidence use

6 | P a g e


Similarly, participants thought that CSOs have minimal influence in their use of evidence by government. Although the evidence generated by CSOs is used for internal purposes and to inform their own work, the evidence used by government in still somewhat missing. This can be attributed to the lack of a sufficient system, which would allow for knowledge sharing and evidence access.

7. Problem AnalysisIn the previous session, participants began to identify several challenges CSOs experience in their participation in the NES and the wider evaluation ecosystem in Ghana. The fishbone diagram identifies many possible causes for a problem. It immediately sorts ideas into useful categories.

This session allowed participants to explore their understanding of the problem, through examining the causes and effects. We introduced participants to several tools used to analyse a problem which included; the Problem tree, Five why’s and the Fishbone analysis. For this workshop, a Fishbone problem analysis was used to understand why CSOs are not fully participating in the national evaluation system or evaluation ecosystem in Ghana. The resultant problem analysis is presented in Figure 1. According to the workshop participants, the overall problem is CSOs contribution to the Government wide M&E system does not adequately influence change in policy and programmes. The following is a discussion of underlying causes to that problem.

Figure 1: Problem Analysis

7 | P a g e


8. Addressing the ProblemAfter identifying the problem(s), we asked participants to think of ways the different issues they identified as contributing to the problem can be dealt with. We used a Theory of Change process to develop solutions. The participants developed a simplified Theory of Change depicting how to address the problem “CSOs contribution to the Government wide M&E system not adequately influencing change in policy and programmes”. Figure 2 below illustrates a theory of change developed by participants.

The theory of change model points out four essential areas of intervention. First, participants recommend government and CSO stakeholder meetings and the creation of policy planning guidelines, which will involve CSOs. Through the timely release and dissemination of these guidelines, CSOs will be able to contribute to policy direction at the sectoral and national level. Second, the

8 | P a g e


participants secondly proposed the formation of a CSO coalition and the development of a communication strategy. This will allow for greater awareness of government timelines and time frames and keep CSOs consistently in loop. Third, through the training of CSOs in M&E, with the aid of fundraising, CSOs believe that their ability to generate evidence will be enhanced. This will ultimately improve their capacity to contribute towards government decision-making and strategies. Lastly, CSOs desire to build a platform, which allows for knowledge sharing and engagement between government and CSOs.

9 | P a g e


Figure 2: Theory of Change addressing the problem of CSOs contribution to the Government wide M&E system not adequately influencing change in policy and programmes

10 | P a g e

Activity 1: Government and CSO stakeholder meeting(s).

Activity 2: Create policy-planning guidelines, which involve CSOs.

Activity 3: CSO Coalition building.Ecosystem

Activity 4: Create and disseminate a comm. strategy (i.e. timelines

Activity 5: Training CSOs in M&E.

Output 1: Release and dissemination of policy planning guidelines.

Output 3: CSOs trained in M&E, which will increase their ability to generate and use reliable evidence.

Output 4: Bottom- up approach to policy planning which includes the participation of CSOs in decision-making.

Output 2: An increased awareness of political/government processes and timelines.

Outcome 1: Enhanced Planning Cycle.

Outcome 3: Improved CSO capacity to produce evaluations and use evidence.

Outcome 2: Increased stakeholder engagement in Planning and Setting Direction.

Increased CSOs influence in Planning and Setting Direction

Activity 6: Fundraising

Activity 7: Create a system/platform for CSOs to

Outcome 4: Increased CSO influence in the political manifesto.


9. Concluding RemarksIn developing a CSO-government engagement model for the evaluation system in Ghana, there is a need for political support, buy-in from both CSOs and government, and the allocation of resources (human capacity, time and funds). Partnerships are key to the inclusion of CSOs in the evaluation system, between different CSOs and between CSOs and government. A communication strategy will be essential to building and sustaining these partnerships. In addition, information from the participating CSOs strongly highlighted the need for further capacity development in the form of training, which will enable them to produce reliable evidence to be used by the state. Ghana has developed an enabling political environment where CSO involvement is beginning to be encouraged, however several challenges, which inhibit this partnership, need to be addressed.

11 | P a g e


REFERENCESAhmad, R. (2008). Governance, Social Accountability and the Civil Society. JOAAG, Vol.3. No.1.Chaplowe, S, G, and Engo-Tjéga, R, B. (2007). Civil Society Organizations and Evaluation. Evaluation 13(2) 257 –274. Ghaus Pasha, A. (2004). The role of civil society organisations in governance. 6th Global Forum on Reinventing Government Towards Participatory and Transparent Governance 24 – 27 May 2005, Seoul, Republic of KoreaKhan, K, Waheed, A and Iqbal, S. (2003). Monitoring and Evaluation: Civil Society Organisations’ Competitive Edge in Effective Poverty Alleviation. The Pakistan Development Review 42: 4 Part II (Winter 2003) pp. 909–924.

Lazaro, B. (2015). Comparative Study on the Institutionalisation of Evaluation in Europe and Latin America. Programme for Social Cohesion in Latin America.Louise, A and Langley, P (2004)”Ambiguities of global civil society” Review of International Studies 30: 89-110.

UNDP (2014). Strengthening Civil Society Organisations for Responsive and Accountable Governance in Rwanda. Joint Programme Report for 31013-2018. Joseph Antwi-Boasiako (2019): Drivers of civil society organisations’ participation in Ghana’s governance processes, Development in Practice, DOI: 10.1080/09614524.2019.1585416Abdulai, A.G., and R. Quantson. 2009. “The Changing Role of CSOs in Public Policy Making in Ghana.” Ghana Social Science Journal 6 (2): 114–151.

12 | P a g e


Annexure 1: CLEAR-AA Ten Functions of Evaluation Systems

13 | P a g e


Annexure 2: CLEAR-AA Ten Functions of Evaluation Systems Terminology

CLEAR-AA Conceptual framework for NES CLEAR-AA’s 10 Functions of a National Evaluation System is a common holistic framework, which can be used as a guideline to define the parts, and functions of a NES, diagnose, and identify areas of strengths or readiness and entry points for a NES. Understanding the different functions within a NES, and how they relate to each other, is crucial for both defining a range of sub-systems and providing a more nuanced analysis of the capacity of an evaluation system and the ways different stakeholders interact within it. For this workshop, the 10 Functions were used to understand areas where CSOs are strong and are contributing to the system as well as their weakest areas or functions. This provided an insight into areas where collaboration or engagement can be strengthened or established. CLEAR-AA defines these functions as:

1. Defining Results and Planning: Evaluation systems, if empowered to play a role in governance, are key to defining programmatic results, bringing evidence to planning processes, and setting programmatic direction.

2. Performance Monitoring: While monitoring performance is primarily within the remit of monitoring and evaluation systems, evaluation systems alone play some role in performance assessment.

3. Institutional Arrangements: Taking a systems approach to national evaluation systems requires a focus on the interlinkages between the component parts of the system. Institutional arrangements vary across national evaluation systems, but what is common is that there are core custodians of these arrangements, and it is a characteristic function that requires dedicated attention. This can include both normative and practiced roles of stakeholders, as well as policies, legislation, and other rules of the game.

4. Evidence Production and Research: the process of undertaking an evaluation simultaneously generates and uses evidence or various kinds. While evaluation and research processes have important differences, evidence production is a key function of evaluation systems. Similarly, evaluation systems face similar boundaries, constraints, and contextual considerations as research in the region, and the fluidity between evaluation and evidence generation functions, organizations, and individuals merits specific consideration.

5. Evaluation Practice: An evaluation system has an important role to play in developing evaluation practice. As an emergent profession with a lively debate around the competences necessary for professional evaluators, the systematization of evaluation in the region is pulled into an important cycle with the evolution of evaluation practice. There is currently a process of co-definition happening between those institutionalising evaluation systems, and those practicing evaluation in the region. This is

14 | P a g e


evident in the emergence of collectively developed competency frameworks, quality assurance frameworks, etc.

6. Disseminating Evaluation Results: Evaluation systems play an important role in disseminating evaluation results. The mechanisms and extent through which they do this varies, but dissemination, to users and a wide range of additional stakeholders, is an important function that makes an evaluation system effective in its support to evaluation use.

7. Using Evaluation Findings: An evaluation system must use evaluation results to be effective. This use is on many levels, and takes place through various mechanisms, and includes planning systems and programme re-design, but also advocacy, accountability, shifting culture and norms, etc. This function bolsters other evaluation functions but given the range of standalone activities and stakeholders that have a virtually exclusive interest in evaluation use, it merits specific consideration.

8. Building Capacity: Given both the variation in capacity around evaluation practice, and the emergent systematization or evaluation, capacity building is an inherent component of evaluation systems. This includes building capacity for evaluation practice, but also within each component of evaluation system, to ensure the whole functions, and each component works.

9. Shifting Norms: Through evaluation systems, organizational norms and practice should change. This change should be both through, and as a result of the evaluation systems. Through systematizing evaluation practice, norms will change around evidence use. Additionally, based on the results of evidence and through systematizing its use, norms will shift around the way decision makers engage with evaluation processes.

10.Shaping Axiology’s: Evaluation systems are designed around values systems, in the way they define and measure results. Through a systemic practice of evaluation, particularly through the local ownership of evaluation systems in Africa, the process through which values are reinforced is shifting.

15 | P a g e

6ghmeforum.org/.../uploads/2019/07/ghana-cso-toc-report-phase-… · web view27/02/2018 · csos...

Documents