Prepared by Iñigo Retolaza and Elena Díez Pinto Democratic Dialogue Project

EVALUATING MULTI-STAKEHOLDER DIALOGUES: A COMPARATIVE ANALYSIS

A background paper prepared for the

Generative Reflection Workshop: Assessing the Impact of Democratic Dialogues

Carter Center, Atlanta, January 24-25 2007

May God keep us from single vision

and Newton’s sleep

William Blake 1.- PAPER BACKGROUND The Carter Center and the United Nations Development Program convened a workshop to reflect and learn about evaluation and assessment of dialogue processes, by exploring the subject from diverse approaches and practical experiences in Latin America, the Caribbean and other regions. The workshop intends to generate relevant knowledge on this field in order to provide practitioners, donors, and social and political leaders with elements to analyze and improve the depth, quality and impact of dialogue interventions. In particular, to discuss a set of useful indicators to measure results and impact from dialogue processes and lay the foundation for developing an effective impact assessment methodology. This paper constitutes an input for this workshop. 2.- AIM AND STRUCTURE OF THE DOCUMENT This paper aims to serve as an input to stimulate further reflection on the field of evaluation by making a comparative analysis of three dialogue evaluations: Jamaican Civic Dialogue, Destino Colombia and the Argentinian Dialogue. The approach and methods used by each of the evaluation teams will be analysed drawing attention to: i) the theoretical approach applied, ii) the actors responsible for the evaluation, iii) the production of knowledge and learning, iv) the tools used, v) the outcomes achieved, vi) the societal learning and change process, vii) role of donors and lenders. The final section synthesises some of the main findings and invites for further reflection regarding: i) monitoring and evaluation mechanisms for effective impact and assessment, ii) democratic dialogue as a societal learning and change process, iii) managing complexity in multi-stakeholder dialogue processes, iv) democratic governance and multi-stakeholder public dialogue, and v) long-term donor support. 3.- BRIEF INTRODUCTION TO THE COUNTRY CASES The three cases have a basic common ground: all of them were supported in one way or another by an international organization. Nevertheless, the context in which these experiences occurred was clearly different. The Colombian dialogue took place in the midst of a longstanding armed conflict1 while in Jamaica the country was facing “low economic output, high levels of crime, violence and social alienation and staggeringly high debt to GDP ratios” i. In contrast, the Argentinean dialogue was launched as a consequence of a deep institutional crisis in which democratic institutions which were unable to tackle ever increasing and deep structural socio-economic degradation.2 Table 1 summarizes basic aspects of these dialogue processes.

1 Diez Pinto E., De León A., 2000, Destino Colombia: 1997-2000. A treasure to be revealed, UNDP: New York, 2000. 2 UNDP, 2004, Evaluación del Diálogo Argentino, UNDP: not published

Table I: Basic Aspects of Dialogue Processes Destino Colombia Jamaican Civic Dialogue Argentinian Dialogue

Purpose3

Shared vision; dealing with conflict

Shared vision; capacity building; personal development/leadership

Shared vision; strategy/action planning; decision making; consensus building

Time span4 and dialogue phases

March 1996-July 2000 3 phases: 1st phase: assembling the promoter and dialogue groups, dialogue design and funding, sharing international experiences 2nd phase: scenario design 3rd phase: scenario national dissemination

July 2002-January 2004 4 phases: 1st phase: pre-project investigation and design 2nd phase: scenario design 3rd phase: initial engagement phase 4rd phase: deep engagement

January 2002-October 2004 3 phases: 1st phase: launching of dialogue and setting the agenda among key actors 2nd phase: enhanced dialogue table, support to the Presidential electoral process 3rd phase: sectoral tables

Conveners Five recognized national leaders

UNDP Government with Catholic church and UNDP support

Dialogue stakeholders

Businessmen, guerrilla, paramilitary, academics, social activists, NGO leaders, journalists, union leaders, senators, military,

Military, police, government, youth, church, social leaders, businessmen, judiciary

Business associations, NGOs, government, political parties, grassroots organizations, bank associations, university, religious organizations

Methodology Civic scenario Civic scenario Sectoral Roundtables

The three dialogues basically pursued building a shared vision of a common problem and all were multi-stakeholder, that is, assembled a microcosm of the country society. The Jamaica and Argentina dialogues lasted two years while Colombia lasted four. The dialogues of Colombia and Jamaica used civic scenario methodology. Participants of these dialogues were influential leaders and individuals but did not come to the dialogue as formal representatives of their organizations. In Argentina, a sectoral roundtable methodology was used. Sectoral action plans were defined through a series of round tables in order to establish and implement new public policies. 4. - COMPARATIVE ANALYSIS OF DIALOGUE EVALUATIONS In this section the approach and methods used by each of the evaluation teams will be analysed and compared, drawing attention to: i) the theoretical approach applied, ii) the actors responsible for the evaluation, iii) the production of knowledge and learning, iv) the tools used, v) the outcomes achieved, vi) the societal learning and change process, vii) role of donors and lenders. In reading this analysis two observations must be kept in mind. First, evaluations occurred at different time frames. The Jamaican and Colombian exercises were undertaken long after the respective dialogues were over (1-2 years later), while the Argentinean assessment started at the concluding stages of the third and final dialogue phases (Sectoral Round Tables). This difference in time frames, in turn, accounts to different outputs and outcomes of the dialogue processes. Second, none of the three dialogues had any sort of pre-established monitoring system at place. No indicators were defined before the dialogue started and no formal entity was in charge of implementing a learning-oriented monitoring system.

3 Characterization of purpose after Pioneers of Change, 2006, Mapping dialogue. A research project profiling dialogue tools and processes for social change, GTZ: South Africa 4 The time spam includes pre-dialogue, dialogue and dissemination phases.

4.1. Evaluation theoretical approach The three evaluating exercises had some basic common ground as well as some peculiarities regarding the theoretical approached used. All three cases were i) critical and reflective-learning oriented, ii) context-based, iii) relational and inclusive, in terms of how knowledge and learning were produced, iv) dialectic, v) qualitative, and vi) propositional in their final recommendations. The theoretical approach used in the Jamaican case was based on Sustained Dialogue’s theory of change and on the World Bank’s Social Capital Implementation Framework. The Argentinian case was grounded on evaluation as an integral and continuous process, a constructivist approach, and on participatory and dynamic approaches. The Colombian case was based on the transformative potential of dialogue and orders change, and the use of the learning history methodology. These approaches are summarized in Table II.

Table II: Theoretical approaches used in evaluating dialogues

COUNTRY CASE THEORETICAL APPROACH

Jamaica Sustained Dialogue’s Theory of Change World Bank’s Social Capital Implementation Framework

Argentina Evaluation as an integral and continuous process Constructivist approach Participatory and dynamic

Colombia Transformative potential of dialogue Third order change5 Learning history

4.2. Who evaluated the process and why? In every evaluation, a team was assembled and lead by an international expert accompanied, by national colleagues, and hired by donors (UNDP and others to a lesser degree). This was done to assure a high quality of the evaluation process and final product, and objectivity and balance. In particular, national consultants were involved for having a deeper and broader understanding of the local historical, social, economic and political context where the dialogue took place, for knowing the relations between actors and for having access to local leaders. 4.3. How was learning and knowledge produced? Although the three cases followed a relational and participatory approach to evaluation, participants interviewed during the evaluation were approached mainly as information givers and as process and content analysts. Interviewees’ opinions were somehow framed by indicators and criteria determined beforehand by the evaluation team. The Argentinean case used a more interactive approach as it engaged participants at an early stage of the evaluation. A group of participants were brought together to help the evaluation team define areas and indicators to be assessed. An epistemological contradiction may be found here as the three evaluation teams acknowledged learning and knowledge as a negotiated process among significant actors, and yet, the teams initially set the ground rules for further generation of learning and knowledge. By doing so, the evaluators

5 Pruitt and Waddell (2005) propose three orders of change which depend, among others, of the desired outcome: first order change , second order change (reform), third order change (transformation). See section 6 in this document.

became the labellers of what needs to be known6. To a certain point the external consultants determined, at least initially, what was important to be known. The Argentinean case was the one most clearly affirming the need to co-design and implement a more flexible evaluation process, thus minimizing this contradiction. To an extent, every group of participants interviewed became an interpretive community of their own dialogic process. 4.4. What tools were used? Every country case followed mainly a qualitative approach to evaluation using tools corresponding to this approach. Table III summarizes the tools used for assessing the impact of dialogue.

Table III: Tools for dialogue evaluation

COUNTRY CASE EVALUATION TOOLS

Jamaica

Group and individual interviews based on levels of engagement Assessment of basic project data (e.g. minutes, summaries, etc.) Individual case studies Cost-effectiveness analysis Gender-based participation matrix Participants’ engagement analysis matrix Narrative analysis

Argentina

Individual interviews Literature review Web search e-survey for participant self-evaluation Expert study on the impact of dialogue on the media Groups sessions for initial framing and final feedback Narrative analysis

Colombia

Individual interviews Group interviews Group sessions for final feedback Narrative analysis

In every exercise a common set of tools were applied (narrative analysis, individual interviews and group sessions). The Colombian learning history strongly constructed its dialogue assessment on the narratives of the interviewees and the Jamaican case used individual case studies comprising participants’ quotations and opinions. The Argentinean case used narratives along with time lines to illustrate how participants viewed and experienced the different stages the process went through. These narratives were used in all cases to: i) pull out new critical and experiential learning around dialogue processes as mechanisms for deepening democracy and dealing with complex social systems conflicts, ii) compare these opinions with some of the indicators and theoretical frames determined as the basis for dialogue assessment, and finally iii) to assess the personal impact dialogues had on participants’ worldviews, human/institutional relationships and lives. Clearly, the evaluations were embedded by a dialogic approach themselves. But there were also some differences. The Argentinean case added in a very interesting quanti-qualitative analysis regarding media coverage of the three dialogue process phases. It was about understanding the impact the dialogue had in the media and how the latter covered the process. Also, the Argentinean evaluation team used an e-survey to reach many of the dialogue participants (125 replied to the questionnaire) who could not be interviewed. This tool allowed many more voices and

6 See IDS (2006) for a more complete account on labelling in development practice. In this policy brief Eyben and Moncrieffe remind us the importance of labellers: “the issue in not whether to label, but which labels are created and whose labels prevail to define a whole situation or policy area, under what conditions, with what effects”

perceptions to be included. The Jamaican case critically looked at the cost-effectiveness aspect of the dialogue. A budget expenditure analysis was utilized as the base tool for such analysis. In conclusion, dialogue impact assessment allows the opportunity for innovative uses of qualitative research tools traditionally circumscribed to social sciences. It also opens up the door for the application of several social marketing tools which can throw new insights into the impact dialogue processes have on significant actors such as mass-media, political parties, social movements, private sector, etc. Also, it suggests the need for deeper understanding on how qualitative tools can contribute to measure the impact of public multi-stakeholder dialogue processes on societal learning and change. 4.5. Overall outcomes achieved7 The outcomes initially for each evaluation were different due to the specific and contextual nature of the initiatives. Nevertheless, the overall intended outcomes envisaged for the three cases aimed at addressing the following aspects: i) deep historic socio-economic and structural institutional inequalities, ii) the need for installing a dialogic culture and enduring trust relationships among national leaders and other stakeholders as a means for implementing deeper participatory governance systems.

COUNTRY CASE OVERALL INTENDED OUTCOMES

Jamaica “a vibrant culture of democratic governance that guarantees political participation, poverty reduction and social equity.”

Argentina “basic consensus on central issues to be addressed as a mean to cope with the crisis”

Colombia “formulate a vision but also and alternatives for solution of probles facing the country”

Of the three evaluation cases, the Jamaican Civic Dialogue evaluation report was the only one that clearly and explicitly rescued the intended outputs outlined in the Civic Dialogue Project.8 This clarity of outputs somehow allowed the Jamaican evaluators to contrast what was expected initially and what was achieved afterwards and analyse factors affecting the accomplishments of the proposed outcomes. Further, by outlining a clear and inductive theoretical evaluation framework evaluators were able to develop a detailed account of intended and unintended outcomes. The other two cases used a more deductive and constructivist approach to analyse the outcomes and to draw lessons from the dialogue processes. Although the three cases explain, in general, intended and unintended outcomes, there is not a clear distinction of each type. This may have happened as a result of not having defined the intended outcomes initially, in particular in the Argentinean and Colombian cases. Making this distinction is of paramount importance not only to understand complex social learning processes--such as multi-stakeholder dialogues—but its effects on the individual (personal), organisational (institutional) and societal dimensions. We draw attention onto these dimensions in the next section.

7 See annex I for a detailed outcome analysis matrix 8 1. Increased and improved civic dialogue process, 2. Learning History of the Project produced, 3. Targeted

research papers produced in support of the process, 4. Leadership capacity for all dialogue participants strengthened, 5. Nationwide communication of scenario based civic dialogue process, 6. Policy/ Strategy Formulation and other upstream activities.

4.6. The societal learning and change process9 Not all evaluation reports kept a “balance” when analysing the individual, organisational and social impacts of the dialogues undertaken. The Argentinean report drew heavily on institutional and social aspects but failed to address the impact of dialogue on the personal (individual) dimension, that is, on participants themselves. In contrast, the Jamaican and Colombian evaluation considered the three dimensions, highlighting the personal and organisational aspects by using case study boxes and direct quotations from participants. The following matrix presents a more detailed analysis of how each evaluation report dealt with the three dimensions.

SOCIETAL LEARNING

AND CHANGE

DIMENSIONS

A COMPARATIVE ANALYSIS

Individual dimension

Personal self-awareness

The Argentinean report does not address this dimension. The Jamaican and Colombian evaluations reported successful impacts as participants questioned and enriched their worldviews by interacting with diverse individuals through dialogue. Also improved capacity for listening thanks to the dialogic conversational approach utilized. By far, the Colombian report explores better than any other the personal impact of dialogue on participants, including changed mindsets, improved listening skills, inclusive and systemic vision developed, and greater tolerance towards difference and diversity.

Personal empowerment and social behaviour

The Argentinean evaluation comments on the strong leadership and support exercised by the UNDP Representative, even before dialogue started. However, there are no reports on how or if the process empowered powerless individuals. The Jamaica evaluation used a case study, the changed life of a community youth leader, to illustrate the impact of participating in the Leadership Forum (dialogue). The report also comments on personal empowerment of participants and enhanced social networking resulting from the civic dialogue which lasted after the dialogue process was over. The Colombian case informs about dramatic shifts on how former enemies and adversaries related to each other both in private and public spaces. The Colombian learning history also comments on how every participant disseminated the results of the civic scenario exercise among their own networks, institutions and peers, across sectors and geographies.

Organisational dimension

Adaptive organisational arrangements

The Colombian evaluation reports the lack of a managerial team hired on a full-time basis to coordinate the overall dialogue process, a leadership style too oriented to technical aspects which missed important political and strategic dimensions, and a shortage of funds which hampered efforts for a widespread dissemination of results. As regards to Jamaica, the evaluation reports a project team that took upon itself the facilitation of the process in its different stages, including supporting community-based projects without hiring new staff or incorporating additional dialogue methods, tools and skills. This situation resulted in diminished project effectiveness and staff loosing focus on its purpose. Although the dialogue project was responsive to emerging demands resulting from dialogue participants it did not come up with new organisational arrangements to cope with them. The Argentinean evaluation reports good management in organizing the complexity and uncertainty throughout the public dialogue. Also, each dialogue stage was marked by a change in both official leadership and facilitation. For instance, a first stage was convened by the government and subsequent stages were lead by religious and civil society organisations. The Catholic Church kept a very active role throughout the process as well as the UN system which continued serving as promoter of the dialogue and provided technical support and funds. This flexible articulation between i) shifts in convening roles, ii) a sustained political, technical and financial support, along with iii) a responsive State, became crucial for the successes accomplished along the process.

9 The societal learning process requires an interactive and iterative learning process which happens between the individual, institutional and social realms. For a detailed account of the social learning processes happening in multi-stakeholder processes see Woodhill J., 2004, Facilitating complex multi-stakeholder processes. A social learning perspective, Working Document, Wageningen International: not published; more information on multi-stakeholder processes in http://portals.wi.wur.nl/msp/

Establishment of post-dialogue monitoring and support structures

Jamaica and Colombia reports noticed a decrease in post-dialogue follow-up activities due to lack of funding and institutionalization of monitoring and support systems. The Argentinean evaluation comments on how the government developed new decrees, laws, public policies and government agencies to implement and monitor key agreements resulting from the process (i.e. a new Sub-secretariat for Institutional Reform and Democratic Strengthening was set in place, Decree 565 Family Right to Social Inclusion was endorsed, the Unemployed Men and Women Heads of Households Program was launched).

Social dimension

Spaces for multi-stakeholder public dialogue and participation

In Colombia spaces for further dialogue and participation were opened by organizing strategic reflection workshops about the scenarios and their implications with additional society groups and organizations. This process of dissemination was modest due to limited funds. In Jamaica and Argentina spaces were developed for broader involvement of social, religious and political actors but in both cases the possibility for participating in these public deliberations was mediated by corporate representation (business sector, religious representatives, social and union leaders, members of parliament, and so on) limiting direct citizen participation. The civic dialogue in Jamaica was able to support, relate to and get feedback from some community-based participatory decision making spaces.

Sustainable partnerships for collaborative action

The Colombian exercise resulted in a limited engagement of the broader society and of political parties. Diálogo Argentino succeeded in implementing sustainable partnerships among different actors and collaborative action among diverse and often confronted social, political and religious actors. The Jamaican evaluation reported partnerships resulting from the civic scenario exercise and how these ended up in collaborative action among the project and several communities as well as set up of new citizen forums (i.e. Kingston-Saint Andrews Action Forum10). It did failed, though, in achieving strong support and sustainable alliances with political actors (congressmen and political party leaders) which hindered deeper State responsiveness.

Policy formulation and implementation through dialogic practices

In terms of real and tangible policy making and implementation, the most concrete and successful case is reported from Argentina, and was possible due to i) State strong responsiveness, ii) cross-sectoral political will and combined leadership, iii) nationwide social pressure, iv) multilateral lender and donor support, and v) cross-sectoral alliances. In the case of Colombia, the dialogue group intentionally did not seek strong advocacy and communication with congressmen (although a few were included in the dialogue group). In the Jamaican case, although the Civic Dialogue Project explicitly sought promoting a vibrant culture of democratic governance, it did not achieve a strong and fruitful linkage between civic scenarios and public deliberation with policy making.

4.7 Role of international donors and supporters The three evaluations reported lack of involvement on the part of donors and supporters after the dialogue phase concluded. For instance, for establishing formal structures to monitor impact or to support implementation of agreements resulting from dialogue (public policies, citizen active engagement and oversight, cross-sectoral alliances). This disengagement in the post-dialogue phase may have limited sustainability of agreements and nurturing active social networks capable of managing conflicts through non-violent means. 5. - CONCLUSIONS AND RECOMMENDATIONS FOR FURTHER REFLECTION The evaluation reports along with the comparative analysis developed in this paper suggest a rich array of lessons learned. Also, they trigger deeper reflection and action for improving implementation and assessment of future dialogue processes. 5.1 Monitoring and evaluation mechanisms for effective impact and assessment The lack of common assessment approaches and tools makes difficult to structure a comparative and comprehensive monitoring and evaluation system. This system should be flexible, open and inclusive in order to complement pre-defined indicators and criteria with additional ones stemming from specific

10 A public platform set in place under leadership and guidance from some of the members belonging to the Leadership Forum

contexts and participants. It should also consider both qualitative and quantitative approaches to evaluation. A flexible and comprehensive monitoring and evaluation system would enable effective quality control procedures and contribute to accomplishing broader and constructive short, mid and long term impacts. Permanent social auditing for the implementation of agreements becomes crucial in countries where democracies are young or dysfunctional, State institutions may be weak and corrupted, and political cultures reproduce discriminatory attitudes and behaviours. 5.2 Democratic Dialogue as a societal learning and change process Democratic dialogue processes and their evaluations must integrate the three dimensions present in every societal learning and change process: personal, organisational and social. When these dimensions are not actively sought and achieved, the impact multi-stakeholder public dialogues on deepening democracy are low, insufficient and often counterproductive. Not including these dimensions in the dialogue process design may generate future resistance or distrust for democratic dialogue as i) a valid and effective mechanism for peaceful conflict resolution, ii) an accepted and shared approach for promoting more inclusive and democratic governance systems, and iii) an enabling mechanism for personal and group learning and change. 5.3. Managing complexity in multi-stakeholder dialogue processes Complex multi-stakeholder dialogues such as the ones presented in this paper demand a paradigm shift in terms of how these processes are designed, facilitated and managed. Linear and Cartesian management styles do not apply to complex processes which happen to be driven by uncertainty, unintended events, systemic relationships, asymmetric power dynamics, historic conflicts, etc. If Democratic Dialogues are meant to be mechanisms capable of dealing with complex social systems and longstanding conflicts, then they must be conceived and managed differently. A new paradigm should comprehend approaches and tools deriving from Complexity Theory, Systems and Quantum Thinking, Constructivist and other non-positivist paradigms. These new set of conceptual frames and worldviews seem to be more updated and appropriate for the challenges these progressive and deliberative processes pose on us as dialogue practitioners (i.e. facilitators, managers, donors, participants, promoters or conveners). 5.4. Democratic governance and multi-stakeholder public dialogue Democratic Dialogues challenge the existing traditional governance systems based entirely on representative democracy. They advocate for open-ended, inclusive and participatory governance systems where active citizen engagement is sought as a means to deepen democracy and solve complex social and political conflicts. This shift towards participatory governance systems implies, among others: i) cross-sectoral, multi-stakeholder collaborative learning and action, ii) new institutional arrangements and procedures, iii) institutionalized spaces for public deliberation, iv) deep and sustained changes in personal attitudes, behaviours and mindsets. 5.4. Long term donor support Support offered by donors must go beyond the preparatory and dialogue phases and include the dissemination and implementation stages too. It should consider mid and long-term support to make sure democratic dialogues become part of a more comprehensive strategy aimed at deepening and strengthening the democratic culture and practices in societies, institutions and citizens. If political will is lacking or institutions are weak, agreements will only be partially implemented or not implemented at all. Second and third order changes, urgently needed in transitional democracies and developing countries, will not happen. Disenchantment with and distrust of dialogue—both a tool for democratic governance and a democratic practice—is something that may be prevented by having adequate and long-term support from donors.

6. BIBLIOGRAPHY Benhabib S. (ed.), 1996, Democracy and Difference. Contesting the boundaries of the political, Princeton: Princeton University Press Bohm D., 1996, On Dialogue, London: Routledge Brookfield S., 2005, The power of critical theory. Liberating adult learning and teaching, San Francisco: Jossey Bass Diez Pinto E., De León A., 2000, Destino Colombia: 1997-2000. A treasure to be revealed, UNDP: New York Eyben R., 2006, (ed.), Relationships for aid, London: Earthscan Flick D. L., From debate to dialogue. Using the understanding process to transform our conversations, Boulder: Orchid publications Gerzon M., 2005, Strategic Outlook on Dialogue, UNDP: not published Hajer M., Wagenaar H. (ed), 2003, Deliberative policy analysis. Understanding governance in the network society, Cambridge: Cambridge University Press IDS, 2006, The power of labelling in development practice, Policy Briefing Nº 28, IDS: Sussex Isaacs W., 1999, Dialogue and the art of thinking together, New York: Doubleday Nemeroff et al, 2005, Evaluation of the Civic Dialogue for Democratic Governance Project, UNDP: not published Pioneers of Change, 2006, Mapping dialogue. A research project profiling dialogue tools and processes for social change, GTZ: South Africa Pruitt B., Waddell S., 2005, Dialogic approaches to global challenges: Moving from “dialogue fatigue” to dialogic change processes, Working paper prepared for the Generative Dialogue Project, available at www.generativedialogue.org/resources Sánchez Melean J., 2006, Analytical framework for measuring dialogue impact, Carter Center: not published Schein E., 1999, Process consultation revisited. Building the helping relationship, New York: Addison-Wesley Senge P. et al, 2005, Presence. An exploration of profound change in people, organizations and society, New York: Doubleday UNDP, (forthcoming), Handbook on Democratic Dialogue, New York: UNDP UNDP, 2004, Evaluación del Diálogo Argentino, UNDP: not published UNDP, 2003, III Learning Workshop Argentine Dialogue (draft copy), UNDP: not published VeneKlassen L., Miller V., 2002, A new wave of power, people & politics. Oklahoma: World Neighbours Woodhill J., 2005, Facilitating complex multi-stakeholder processes. A social learning perspective, Working Document, Wageningen International: not published Yanow D., 2000, Conducting interpretive policy analysis, London: Sage Publications Young I. M., 2000, Inclusion and Democracy, Oxford: Oxford University Press

Annex I: Outcome Analysis Matrix

COUNTRY CASE OUTCOMES ANALYSIS

Jamaica11 The evaluation outlined the outcome analysis based on the theoretical approach established beforehand12

i) Despite the effective use of the Civic Scenario Model among the Leadership Forum13, an overall weak conceptualisation and design of the project led to scope drift.

ii) Project implementation, while driven by a highly dedicated Secretariat, was weakened by gaps in participation, strategic planning, and appropriate skills

iii) The project produced some significant impacts among the Leadership Forum and at the community level, but had difficulty in influencing decision makers and other key national actors.

iv) A lack of effective monitoring and evaluation weakened the project’s capacity for quality control and made it difficult to prove the impacts it achieved.

v) The project had a number of learning experiences since launched that have yet to be fully examined for the use of policy makers, development workers, and other dialogue practitioners.

Argentina14 The report divides the outcome analysis in relation to the three stages the process went through.

1st stage: i) Consolidation of the dialogue as a tool for generating consensus. ii) Contribution towards social appeasement in the face of the crisis and the risks of social hostility

and dissolution. iii) Actions to address the emergency and reform of the social policy: Men and Women Heads of

Households Program and the Remedial Program. iv) Progress in the design of State policies, i.e. the prescription of generic drugs in health and the

reactivation of housing and employment. v) Identification of basic consensus for the transition. 2nd stage: The Enhanced Table15 organized two events of huge public impact: The Journey for Peace, a pronunciation against violence on the first anniversary of the 2001 institutional crisis, and a series of public debates with presidential candidates, oriented at involving them in the construction of consensus and the joint development of a Governance Agenda based on the agreements reached in the dialogue process. 3rd stage: The Enhanced Table was consolidated as a platform widely representing civil society, and thematic tables were established. The Dialogue also promoted new national security initiatives in association with the Ministry of Justice and Security; participated in dialogue initiatives advanced by the Ministry of Defense, and developed new Environment, Science, and Technology spaces. Moreover, through its Political Reform Table the Dialogue participated in convening a new space with civil society, coordinated by the Ministry of the Interior and the Head Office of the Cabinet of Ministries, for the promotion of political reform. The government created a new Sub-secretariat for Institutional Reform and Democratic Strengthening.

Colombia16 The report divides the findings in three realms: i) factors encouraging group learning and reflection, ii) the influence the process had on participants, and iii) on the country

Factors encouraging group learning and reflection i) Destino Colombia succeeded in bringing together diverse sectors of society, including

adversaries and enemies, in spite of the enormous limitations posed by the armed conflict ii) The internal rules agreed upon by the group made possible honest, tolerant, and respectful

dialogue that, in turn, allowed the group to gain in confidence and to be open to new ideas and thoughts.

iii) The scenario methodology was enlightening in that it led to reflections and learning about the country.

iv) Many of those interviewed referred to the camaraderie and good humor that prevailed during the encounters in Quirama17. These were essential factors in the building of bridges and confidence.

11 Extracted from Nemeroff et al, 2005, Evaluation of the Civic Dialogue for Democratic Governance Project, UNDP: not published 12 Sustained Dialogue’s Theory of Change, World Bank’s Social Capital Implementation Framework 13 A multi-stakeholder platform comprised by national leaders launched and promoted by the project as the container of the Civic Dialogue. 14 Extracted from UNDP, 2003, III Learning Workshop Argentine Dialogue (draft copy), UNDP: not published 15 As a result of the 1st stage, a multi-stakeholder platform was set in place as a container of the dialogue process for the second stage 16 Extracted from Diez Pinto E., De León A., 2000, Destino Colombia: 1997-2000. A treasure to be revealed, UNDP: New York, 2000. 17 The location where the dialogue events took place

Influence the process had on participants i) Changed mental maps in members of the group ii) Brought together a diverse group of leaders that was able to hold a dialogue and discuss the

most pressing problems facing Colombia. The dialogue created a safe space to exchange ideas, thoughts and proposals.

iii) Discovering the human and true dimensions of the other person. The Quirama meeting fostered that members of the group learn from the worlds of others. It also allowed them to discover their values and their lives, with which they influenced each other:

iv) Collective knowledge of the realities of the country. Destino Colombia offered the possibility of discovering the realities of the country and to sensitize the members of the group about the visions of others

Influence the process had on the country i) A massive national dissemination strategy18 allowed an estimated of 15 million Colombians to

learn about the initiative and the scenarios prepared by the Destino Colombia.dialogue group. But the influence of Destino Colombia on the country could also be appreciated on the basis of two situations: ii) The influence that the members of the group exercise on the organizations, sectors or groups

from which they came. iii) The generation of strategic thought and conversation in society stemming from the

presentations and workshops the project organized with various sectors and organizations in different regions of the country.

18 “The scenarios and results of the process of Destino Colombia were disseminated by means of videotape and a TV program, which was shown on national television and the preparation of a magazine that was widely distributed. Also, more than 750,000 press inserts were printed and distributed. To date, more than 400 presentations and dissemination workshops have been held. By one estimate a minimum of 15 million Colombians in one way or another had contact with the words Destino Colombia, as a personal experience on all the TV chains that transmitted a special program simultaneously”, extracted from Diez Pinto E., De León A., 2000:65,