partnerships for sustainable change...2015/08/01 · partnerships for sustainable change 5 lucia...
TRANSCRIPT
Partnerships for sustainable change
with transformative impact
Erasmus Universiteit Rotterdam
Erasmus School of Accounting & Assurance
Executive Program
Corporate Social Responsibility
Rotterdam School of Management
Partnership Resource Centre
July 2015
Lucia Helsloot
Begeleiding: Rob van Tulder
Marieke de Wal
Partnerships for sustainable change 2 Lucia Helsloot, July 2015
Preface
More than a year ago I started this project, writing a paper to complete the Executive Program
Corporate Social Responsibility of the Erasmus School of Accounting & Assurance. I found a place at
the Partnership Resource Centre of the Rotterdam School of Management to work on this project.
I decided to focus on the question how partnerships could contribute to sustainable change. In the
practice of international development I saw the predominant resource orientation of many partnerships
between NGOs and businesses, in literature identified as philanthropic partnerships. In the context of
decreasing government funding many NGOs increased their collaboration with businesses, but mainly
driven by the need to increase their financial resources. However, the collaboration with businesses is
also an opportunity to increase impact. That is what I wanted to explore more: how can partnerships
achieve impact towards sustainable change, or, how can partnerships have transformative impact?
This paper is the result of a long process. I’m happy that finally I’ve been able to complete the project
and want to thank all my colleagues at the Partnership Resource Centre for their support, in particular
Marieke de Wal and Rob van Tulder. I also want to thank Jos van Heijningen and Michiel van der
Borght of Plan Netherlands, and Sarah O’Neill of Akzo Nobel for being so open and responding to my
questions.
Lucia Helsloot [email protected]
Partnerships for sustainable change 3 Lucia Helsloot, July 2015
Table of contents page nr.
1. Introduction………………………………………………………………………………. 4
2. Methodology……………………………………………………………………………… 5
3. Partnerships……………………………………………………………………………… 5 3.1. The concept of partnerships……………………………………………………….. 5
3.2. Types of partnerships………………………………………………………………. 6
3.2.1. Partnerships by sector………………………………………………………… 6
3.2.2. Partnerships by level of commitment……………………………………….. 7
3.2.3. From philantropical towards transformative partnerships…………………. 9
3.3. Key factors for successful partnering……………………………………………… 11
3.4. Impact of partnerships……………………………………………………………….. 13
4. Case study: the partnership between Plan Netherlands and Akzo Nobel……… 17
4.1. Introduction…………………………………………………………………………… 17
4.2. The Plan – Akzo Nobel partnership……………………………………………….. 18
5. Conclusions and challenges…………………………………………………………… 20
Annex 1 : Data analysis……………………………………………………………………… 22
Annex 2: Measuring outputs, outcomes and impact…………………………………… 29
Sources………………………………………………………………………………………….. 31
Partnerships for sustainable change 4 Lucia Helsloot, July 2015
1. Introduction
NGOs and businesses are increasingly engaging in partnerships. It would therefore be interesting to
understand more about the conditions for these partnerships to have transformative impact. This is the
central research question of this paper: under what conditions can partnerships have transformative
impact?
To answer this central research question the following sub questions will be answered:
1. What is meant with partnerships?
2. What makes a partnership transformative?
3. What is known about the conditions for partnerships to have transformative impact?
4. What can we learn from practice (case study) about the abovementioned conditions?
5. What are lessons learnt, challenges and dilemma’s?
After a short explanation of the methodology used in this study (chapter 2), the concept of
partnerships is explored. In chapter 3 attention is given to different typologies of partnerships, and the
characteristics of transformative partnerships are further explored. In addition the most important key
factors for successful partnering are identified. Chapter 3 ends with exploring the current discourse on
impact of partnerships. In chapter 4 the partnership between Plan Netherlands and Akzo Nobel is
elaborated as an illustrative case study. The paper ends with some conclusions and challenges in
chapter 5.
Partnerships for sustainable change 5 Lucia Helsloot, July 2015
2. Methodology
This paper is an exploratory study, based on literature study, and illustrated by a case study.
Literature has been selected based on recommendations by the Partnership Resource Centre, the
Executive Program Corporate Social Responsibility, as well as on my own practice. I’ve used literature
which I used in my work as a senior policy officer and evaluation manager at Partos, the association of
Dutch NGOs working in international development. In addition I used literature found on the internet,
by using key words, including: partnerships, transformative, transformational, impact and
effectiveness. For the case-study I used the Plan and Akzo Nobel websites.
In addition to the literature review I interviewed some key persons, involved in the partnership between
Plan and Akzo Nobel.
I decided to carry out an illustrative case study to learn more about the reality of a partnership. I
selected the partnership between Plan and Akzo Nobel, because I wanted to learn from practice, from
an organization, profiling itself on transformative goals (girls’ empowerment)1, and intending to make
their partnerships more transformative2.
The case study is based on interviews and document/websites analysis. The data were analysed by putting the data from interviews, literature and websites in a table (see annex).
3. Partnerships
3.1. The concept of partnerships
Partnerships can be defined as “collaborative arrangements in which actors from two or more spheres
of society (state, market and civil society) are involved in a non-hierarchical process through which
these actors strive for a sustainability goal.” (Glasbergen 2007, in Partnership Resource Centre
(2011)).
Partnerships are often cross sector partnerships: “inter-organizational relationships in which actors
from two or more spheres of society (state, market and civil society) are involved in a collaborative
process through which these actors strive for a jointly defined goal (Drost, 2013).
This basic definition is compatible with other terminology used for cross-sector collaboration, including
‘multi stakeholder initiatives’: ‘interactive processes in which business, CSOs and possibly other
stakeholder groups interact to make business processes more socially and/or environmentally
sustainable’ (Van Huijstee, 2012).
However, while the ‘partnership’ concept also refers to collaborations between one company and one
CSO, Multi stakeholder initiatives always cover interactions involving more stakeholders at a sector or
issue level.
‘Global action networks’ is another terminology used to describe collaborative arrangements between
the above mentioned different actors, but these networks may have many more (thousands) of
participating organizations and tens of thousands of people participating, many more than may know
1 which are often not shared by businesses. E.g. the evaluation of the selection procedure of the Sustainable
Water Fund (FDW) and the Facility for Sustainable Entrepreneurship and Food Security (FDOV), shows that the
118 respondents (representing 81 unique partnership projects) considered the donor requirements in relation to
gender more difficult than anything else and not helpful at all. 2 Veldhuis (2013)
Partnerships for sustainable change 6 Lucia Helsloot, July 2015
each other. They come together because they are participants in a system that they want to move in a
certain direction, and they want to create greater coherence between their activities and that of others.
Partnerships mostly have a more limited and well-defined objective such as producing a report or
constructing a water system (Waddell, 2011).
Cross-sector partnerships (or multi stakeholder initiatives) have become increasingly prominent during
the past years. This has particularly been the case in relation to complex (sustainability) problems, for
which organizations on their own have not been able to find adequate answers or approaches.
While companies tend to externalize the negative social and environmental effects of their business
operations, national governments lack the power to regulate international markets. Civil society, on the
other hand, lacks the resources to achieve the broad range of changes needed for a more sustainable
world.
Multi-stakeholder initiatives combine the capacities and resources of several stakeholder groups to
respond to the complexity of e.g. sustainability problems. (Van Huijstee, 2012)
“The issues we face are so big and the targets are so challenging that we cannot do it alone.
When you look at any issue, such as food or water scarcity, it is very clear that no individual institution,
government or company can provide the solution” (Paul Polman, CEO Unilever, in Gray and Stites, 2013)
3.2. Types of Partnerships
3.2.1. Partnerships by sector
As the figure below shows, partnerships can be formed between businesses and NGOs, business and
government, NGOs and other civil society members/communities or government and communities.
Businesses may also engage directly with communities, but in practice, NGOs often broker or serve
as liaisons between businesses and communities.
Figure 1: Partnerships by sector (Gray and Stites, 2013)
Partnerships for sustainable change 7 Lucia Helsloot, July 2015
3.2.2. Partnerships by level of commitment
Many researchers have proposed typologies to capture differing levels of commitment by businesses
to partnerships designed to promote sustainability. An important typology is the Collaboration
Continuum, as developed by Austin and Seitadini in 2012, which show four types of cross-sector
collaboration:
- The philanthropic collaboration, in which the relationship between partners is that of donor and
recipient.
- The transactional collaboration, where the primary motive for business is improving profitability
or market share.
- The integrative collaboration, in which businesses start to consider how to balance bottom-line
considerations with social and ecological concerns.
- The transformational collaboration, involving co-creation and aiming to initiate change on
societal level and changing the rules of the game. The business’ partnership efforts empower
and give equal voice to stakeholders and communities with whom they partner.(Gray, NBS
2013)3.
Gray and Stites (2013) built on this model, in making a graphic representation of several forms of
business- NGO partnerships, arrayed along two axes (see figure below). The X-axis displays the
scope of the partnership, which increases with the number of players and sectors involved and the
size of the problem arena (e.g. local, regional, national, global). The Y-axis displays the degree of
shared ownership and responsibility exhibited by the partners.
Figure 2: Sustainability Continuum (Gray and Stites, 2013.)
3 E.g. the UN Global Compact (2012)) also recognises philanthropic and transformational partnerships, but
instead of transactional and integrative collaboration they identify:
- Opportunistic partnerships: program or action that brings short-term economic gain and may leverage
core competencies but in an ad hoc manner, and
- Strategic partnerships: core competencies used to develop markets, products, and services deliberately;
but it is often a two party agreement and set within an existing system.
Partnerships for sustainable change 8 Lucia Helsloot, July 2015
From the lower left-hand corner to the upper right hand corner the partnerships increase in scope,
complexity and shared responsibility.
Partnerships in the lower left-hand corner largely represent philanthropic partnerships. In this level of
response, businesses’ sustainability activities involve providing welfare to society by responding to
government regulations or by providing charitable giving. These are threat induced, compliance or
charity driven responses.
Partnerships in the upper-right area of the figure represent the partnerships with so-called
transformational or transformative engagement. It is a type of collaboration in which all key
stakeholders are involved in sustained interactions, which are equally responsive to all partners’
needs: collaborative governance4. The emergence of this kind of governance should be understood in
the context of the changing roles of governments (‘do more with less’) and the increasing complexity
of problems.
Businesses that engage in transformative partnerships not only embed sustainability in every aspect
of their operations and tie it into their strategic objectives, but are also interested in managing and
integrating stakeholder expectations. Their partnership efforts empower and give equal voice to
stakeholders and communities with whom they partner.
This type of collaboration can be found in Base-of-the-Pyramid strategies, partnerships in which
businesses work closely with income-poor communities around the globe to develop new sustainable
business opportunities that are locally embedded and generate value for all the partners. (Gray and
Stites, 2013)
Partnerships can develop along the collaboration continuum. The steps from philantropical towards
transformational partnerships appear to be in a chronological order. Partnerships may likely begin as a
philantropical or transactional type of partnership and then progress along the continuum, as they gain
experience and enjoy success in their partnership efforts. Although a partnership can start anywhere
on the continuum, and may skip levels in the process of increasing engagement, the upper right levels
of the partnership typology do appear later in the partnership’s development. Considerable previous
learning and investment is necessary to reach the final stage of the continuum (Gray and Stites,
2013). We will see this in the case study in chapter 3.
From the perspective of working towards a more sustainable and inclusive world, transformative
partnerships might be more interesting than philanthropic partnerships, because transformative
partnerships:
- are less dependent on the generosity (and thus unpredictability) of the donating partner.
- aim to initiate change on societal level and change the rules of the game (systemic change).
- are believed to be more effective (see paragraph 2.4 for more elaboration).
Shelagh Gastrow, executive director of Inyathelo, the South African Institute for Advancement, an organization
helping to build a stable civil society and democracy in South Africa:
“When providing support to a non-profit organization, the company should explore if it is contributing to systemic
change, as South Africa has to transform urgently. Whilst there is room for pure charitable giving to support the
poor and make poverty bearable, the real focus should be to shift the levels of poverty.”
(Traditional business approaches to aid and charity are being abandoned, Paul Klein, theguardian.com, 8 August
2014)
4 Collaborative governance has two key ideas: 1) non-governmental stakeholders are participating in the work of
government and 2) the intention of developing a more comprehensive understanding of the issues or problem
under consideration than government could achieve on its own. Governance should be understood as the
processes of managing the delivery of public good. (Gray and Stites, 2013).
Partnerships for sustainable change 9 Lucia Helsloot, July 2015
Generating additional income (often in philanthropic partnerships) has always been one of the most
important drivers for NGOs to engage in partnerships with the private sector and it still is, particularly
in the context of decreasing government funding. The question thus is how partnerships, based on
these predominantly financial drivers, can develop towards transformational partnerships, which are
desired from the perspective of impact on sustainable (and systemic) change. This question will be
elaborated in the next paragraph.
In addition to generating additional income, the Partnership Resource Centre (2011) mentions another
important drivers for NGOs engaging in partnerships: increased impact (in relation to the goal of the
NGO). This will be elaborated in par. 2.4.
3.2.3. From philanthropic towards transformative partnerships
Transformative partnerships are characterised by a relative high degree of complexity (more players,
more sectors, bigger problem area) and a high degree of shared responsibility.
There is not just one model or structure for a transformative partnership; rather, there is a spectrum of
structures, ranging from partnerships with a discrete number of partners and formalized governance
(multi-stakeholder construct) to network constructs which are more loosely organized and broadly
inclusive (UN Global Compact, 2012).
In answering the question how a philanthropic partnership could evolve in a transformative
partnership, the characterization of both types of partnership by Austin and Seitanidi (2012) is used
(see figure 3 below).5
Nature of relationship Philanthropic Transformative Level of engagement Low High Importance to mission Peripheral Central Magnitude of resources Small Big Types of resources Money Core competencies Scope of activities Narrow Broad Interaction level Infrequent Intensive Trust Modest Deep Internal change Minimal Great Managerial complexity Simple Complex Strategic value Minor Major Co-creation of value Sole Conjoined Synergistic value Occasional Predominant Innovation Seldom Frequent External system change Rare Common
Fig. 3: Characteristics of philanthropic and transformative partnerships (Austin and Seitanidi (2012).
Level of engagement
In striving for a transformational partnership, the partners need to increase their level of engagement.
Veldhuis (2013) stated that, in addition, the partners in a partnership should have the same
perspective on this nature of the relationship: an aligned degree of engagement.
An unaligned degree of engagement within partnerships does seem to be undesirable since this may
lead to a dependent relationship of one of the partners (mostly the NGO). This risk seems to be
5 Because of time constraints only some of the characteristics are elaborated.
Partnerships for sustainable change 10 Lucia Helsloot, July 2015
particularly present when the partner (NGO) is very small compared to the other party and is engaged
in no or few other partnerships.
The degree of engagement was based on several criteria including the definition of partnership (based
on the proposition that the more partnership definitions are alike, the higher the degree of
engagement), the forms of CSR activities employed (based on the proposition that the more forms of
CSR activities are employed within a partnership, the higher the degree of engagement) and the
extent of goal alignment (based on the proposition that goal alignment implies a high degree of
engagement).
Veldhuis (2013) also states, based on a literature review, that it is likely that partnerships that have a
higher degree of engagement achieve more impact.
Figure 3: Scale for the degree of engagement
Co-creation of value
Another important characteristic of transformational partnerships is co-creation of value. From the very
moment that partners start to explore a possible –transformational- partnership, they should focus on
co-creation of solutions around joint priorities and leverage the core competencies such as convening
power, resources, local presence, technical expertise, and network of all partners.(UN Global
Compact 2012).
External system change
Transformational partnerships aim at external system change. Many systemic issues span multiple
geographies and require long term efforts towards instituting new policies and rules and shifting
behavioral norms. As such, scale and sustained impact are important. Transformational partnerships
have in-built capacity to reach scale and leave a lasting impact.
Moreover, transformational partnerships have the potential to address highly complex problems.
Managerial complexity and impact
The management of the relationship is more complex than in other types of partnerships (more
organizational activities are covered), but the impact might be higher.
The effects of a transformational partnership do not only influence social, economic, or political
systems but also change each organization and its people in profound, structural, and irreversible
ways. (UN Global Compact (2012) and Austin and Seitanidi (2012)).
Partnerships for sustainable change 11 Lucia Helsloot, July 2015
3.3. Key factors for successful partnering
Market, state and civil society actors engaged in cross-sector partnerships all over the world point to
five, recognizable, critical success factors6 for partnering (Drost (2013) ):
1. Clarity of roles, responsibilities and ground rules7
Pooling the competences of each partner maximizes the complementary strengths of each
sector. A clear description of roles, responsibilities and ground rules enhances transparency
and helps preventing misunderstanding.
2. Clear understanding of mutual benefits
The value of cross-sector partnerships lies in the potential to create positive outcomes for
each of the partners. Ensuring mutual benefit is required for commitment of the partner to the
partnership, but also for creating ownership and sustaining the partnership outcomes.
3. Clear vision of objectives
In successful partnerships partners define and agree on partnership objectives and develop a
strategy on how to reach these objectives. This is important for achieving the desired
outcomes of the partnership.
4. Clear communication, shared planning and decision making
Good (participative) partnership governance, including good communication, shared planning
and decision making, ultimately leads to mutual understanding of each other’s needs and
requirements, and shared ownership.
5. Good leadership
Leadership based on the notion of equity between the key partners is based on credibility,
brokering, bridging divides and guiding rather than directing.
Although individuals from state, market, and civil society sectors generally agree on the five above
mentioned most critical factors for successful partnering, there are some differences too. Based on
their research, the Partnership Resource Centre states that actors from civil society organizations tend
to put greater emphasis on understanding the needs of (local) beneficiaries. And that state actors
more often point to the importance of taking time in the initial starting up phase of the partnership (see
figure 4 below).
This being said, it cannot be ignored that in recent years we see an increasing blurring of boundaries
between profit and non-profit. Non-profit organizations have put more emphasis on entrepreneurial
characteristics, and profit-organizations have got more attention for non-material issues. In addition
new kinds of hybrid enterprises are rapidly appearing.
Blurring the profit / non profit boundaries
As an example of the blurring profit / nonprofit boundaries Porter & Kramer (2011) described the case of
WaterHealth International, a fast-growing for-profit, which uses innovative water purification techniques to
distribute clean water at minimal cost to more than one million people in rural India, Ghana and the Philippines. Its
investors include not only the socially focused Acumen Fund and the International Finance Corporation of the
World Bank, but also Dow Chemical’s venture fund.
Other examples can be found in the rapidly expanding literature about social enterprises.
6 Critical success factors help answering the question “what must we do to be successful?”, which is not the same
as key performance indicators, which help answering the question “what have we achieved?” 7 Ground rules are base-line principles and working practices agreed between partners.
Partnerships for sustainable change 12 Lucia Helsloot, July 2015
Figure 4: Perspectives from state, market and civil society actors (From The Partnership Resource Centre, 2013).
Gray and Stites (2013) distinguish four key factors that greatly influence partnership outcomes: - external drivers or contextual factors,
- motivations for partnering,
- partner and partnership characteristics and
- process issues.
Whereas external drivers are largely outside the control of the partners, process dynamics are largely
within the partners’ collective sphere of influence. Consequently, paying attention to the process and
adjusting it as the partnership moves forward are two critical ways that partners can increase the
chance of successful and sustained outcomes.
They come with the following set of recommendations for optimal partnering, to ensure that
partnerships add value for all partners involved:
- Adopt a problem centric rather than a firm-centric model of stakeholders
- Frame the partnership as a learning process
- Construct fair processes and manage conflicts
- Don’t expect to come up with a quick solution
- Ensure voice for all participants. They add that If partnerships are to significantly enhance the
voice of less powerful communities, a participatory rather than a service focus need to be ensured.
- Set evaluation criteria
- Allow time for representatives’ constituencies to review and ratify agreements
- Develop leaders competent in partnership skills.
It is important to know these key factors for successful partnering, because they are important factors
for sustaining the partnership. However, it is only indirectly influencing the partnership’s effectiveness
or (social) impact. In the following paragraph the question of the partnership’s effectiveness or social –
Partnerships for sustainable change 13 Lucia Helsloot, July 2015
impact will be elaborated on. The recommendations of Gray to frame the partnership as a learning
process, and to set evaluation criteria are relevant for getting insight in the impact of partnerships and
will thus come back in the next paragraph.
3.4. Impact of partnerships
The United Nations, in collaboration with a number of large multinationals, believe in the huge
potential impact of partnerships, as a way to deliver transformative impact across sectors and
geographies (van Tulder et al. (2014), the Global Compact Lead Task Force on UN – Business
Partnerships (2011)).
However, in the literature about the impact of partnerships much is assumed or supposed.
Partnerships are supposed to create a ‘collaborative advantage’ by generating additional knowledge
and resources. By working together in partnerships the parties are supposed to be more effective (on
own organizational objectives and on partnership level) and achieve greater impact.
There is no evidence yet of the supposed increased impact, because little research has been done
into impact of partnerships (among others Karen Maas in P+ Magazine, and Partnership Resource
Centre 2011).
It is clear that more efforts need to be done to get more evidence about the –assumed- impact of
partnerships, the more because the assumed increased (social) impact is an important driver for NGO
engagement in partnerships.
The setting and committing to a clear framework for measuring outcomes is one of the most pressing
issues in UN partnerships with private companies. This should give insight in outputs and outcomes,
rather than just inputs. (UN Global Compact, 2012).
Impact refers to the effects of the program on society (socially, economically and environmentally),
often the ultimate goal of interventions
The impact level needs to be distinguished from other results levels, such as outputs and outcomes
and from concepts as effectiveness and efficiency.
Outputs refer to distribution of goods or services (e.g. repair of roads or distribution of books), whereas
outcomes encompass new practices and behaviours (e.g. new procedures for recycling waste).
Impact, which is the most difficult of the three to measure, capture improvements in sustainability (e.g.
reduced air pollution) or in material well-being (e.g. reduction in infant mortality rates).(Gray and Stites,
2013). For the interrelatedness of the different result levels (outputs, outcomes, impact), see e.g.
annex 2.
Impact assessment may provide an understanding of the ultimate benefits for local societies, and is
important, both for accountability and learning. However, impact assessments need thorough
investigation, and investment.
With her dissertation ‘Why doing good is not good enough’ Kelly Liket (2014) has put the importance
of measuring impact on the agenda again. And although the debate on impact measurement focuses
on the impact of NGOs the same holds for the corporate social responsibility programs within the
business sector. There has been much research into the contribution of CSR to the company's
success, but the company’s contribution to actually solving societal problems has got much less
attention (Liket, 2014).
Measuring impact of partnerships is even more complicated than measuring impact of individual
organisations, because the impact of partnerships include separate levels:
- to what extent has the common objective of the partnership been achieved?
- to what extent have the individual goals of the partners been achieved?
Partnerships for sustainable change 14 Lucia Helsloot, July 2015
Karen Maas (2014) mentions two more challenges, relevant to the impact of partnerships:
- the counterfactual, which should give evidence for the claim that without the partnership the
results could not have been achieved, and
- the attribution of the impact to the individual partners.
In addition, attention should also be paid to:
- unexpected outcomes, related to e.g. societal issues, beyond the context of the focal issues of the
collaboration;
- the process of collaboration itself;
- short-term as well as long-term results (these may differ).
Gray and Stites (2013) summarize possible partnership outcomes (accruing to each sector) as follows
(see table 1 below).
Table 1: Possible partnership outcomes8 (Gray and Stites, 2013)
Business NGOs Government Community Environment
Positive Improve CSR
reputation; ensure
license to operate
Greater focus on
efficiency and
accountability
Improved project
designs
Greater voice in
policy making
Environmental
advocacy
Supply chain
improvements
Enhanced
reputation
Greater
transparency and
acceptance of
plans
Improved quality
of life
Deal with
complexity
Innovative
products
Achieve needed
funding
More efficient
resource usage
Gain models that
can be used for
other projects
Institutionalized
attention to
problem
New markets Strengthen data
management
Integrated service
delivery
Environmental
conditions
improved
Attractiveness to
employees
Meet sustainability
targets
Retain control of
lives
Gain critical
competencies
Garner greater
public
accountability
Gain culturally
suitable products
and outputs
Integrate
sustainability in
core business
practices
Insight into
economic and
demographic
trends
Build networks for
self-reliance
De facto rules for
regulating
industries
Improve
interagency
coordination
Negative Perceptions of
greenwashing
Suffer tainted
reputation
Need to deal with
conflict
Inequitable
outcomes
Continued
degradation
cooptation9 Less thorough
study of research
Need to balance
subgroup vs.
greater public
interest
Replacement of
nature with
human-produced
products
Reduced funds
8 This table is about outcomes, encompassing new practices and behaviours, and being a necessary step on the
pathway towards the desired impact (improved sustainability and/or living conditions of people). 9 In general research finds the risks of partnering to be higher for NGOs than for businesses (Gray 2013).
Partnerships for sustainable change 15 Lucia Helsloot, July 2015
Transformational partnerships and impact
The impact of transformative partnerships might be higher than the impact of other types of
partnerships (already mentioned in par. 2.2). The effects of a transformational partnership do not only
influence social, economic, or political systems but also change each organization and its people in
profound, structural, and irreversible ways. (UN Global Compact (2012) and Austin and Seitanidi
(2012))
To conclude this paragraph some reflections are given about the current discourse about impact
measurement in the development sector as well as some recommendations (do’s and don’ts) for
maximum impact of partnerships.
Current discourse
In the development sector the current discourse on effectiveness and measuring impact is not (yet)
focused on partnerships. However the issues in the current debate, i.c. the questions as mentioned
below, are also relevant for partnerships:
- How could or should the impact of a development project be measured? Impact on whom or
what? How to involve local people? And when are the results reliable?
o Within the development sector several pilots are taking place, including the World
Citizens Panel of Oxfam Novib (www.worldcitizenspanel.com), combining qualitative
and quantitative methods, and using data collection tools such as ICT and apps, but
also the Oxfam GB effectiveness reviews and Cordaid’s flourishing communities
index.
o To which extent should counterfactuals be used, answering the question what would
have happened in the absence of the intervention. It is often suggested that impact
and effectiveness can only be measured by using a control group (that did not receive
the intervention), the so called Randomized Controlled Trials (RCT’s). In the practice
of often unique partnership constellation establishing control groups is often very
difficult to realise (website Partnership Resource Centre). In addition RCT’s , by their
nature, are restricted to evaluate clearly defined (often one-dimensional) activities,
which are implemented in a single environment. Development programs are often
complex programs, with multiple (sometimes hard to quantify) objectives. In addition
external validity of RCT results is a major problem: there is no guarantee that what
works in one environment will also be effective elsewhere (van der Gaag et al, 2015).
Furthermore there are also ethical considerations for deciding not to use control
groups.
o How to deal with attribution. To what extent can the identified change be attributed to
the interventions of the organization or partnership? Causal links are complex and can
often be interpreted in various ways. In the MFS II Joint Evaluation of International
Lobbying and Advocacy (Arensman et al, 2015) the evaluators worked with process
tracing and contribution analysis. They conclude that these methods are useful, but
very resource intensive and thus costly.
Do’s and Don’ts for maximum impact of partnerships:
- Start with impact thinking from the very beginning of a (partnership) program. The evaluation
design should be integrated into the project design. If you don’t, you cannot properly evaluate,
and you will never know whether it will work. (Liket, 2014). This touches on one of Gray’s
recommendations for successful partnerships, i.c. to set (agreed-upon) evaluation criteria
early in the process (see par. 2.3).
Partnerships for sustainable change 16 Lucia Helsloot, July 2015
- Evaluate the progress towards objectives (common as well as individual objectives), but
evaluate the process itself as well, to ensure that the partners’ interactions remain transparent,
credible and constructive for all involved.
- Try to make the partnership transformative. The different engagement levels (see figure 2)
help to distinguish partnership outcomes, with those focused on reputation and image in the
bottom left and those with wider societal impact in the top right.
- Aim for all partners having the same high degree of engagement, because then the
partnership is likely to have more impact than partnerships where this is not the case. This
assumes similar visions on the partnership and common objectives.
- Frame the partnership as a learning process. With the increasing levels of responsibility and
complexity that partners face as they move from partnerships in the lower left (philanthropic
partnerships) to the upper right of the figure (transformative partnerships) in par. 2.2., partners
necessarily have learned to capitalize on the creative tensions inherent in the partnership.
Partnerships for sustainable change 17 Lucia Helsloot, July 2015
4. Case study: the partnership between Plan Netherlands and Akzo Nobel
4.1. Introduction
This chapter elaborates on the partnership between Plan Netherlands and Akzo Nobel, as an example
of how a partnership may work in practice. The argumentation for chosing this case study has been
explained in chapter 2. Methodology. The case study is an example of organic growth of a
partnership, from a philantropical towards a more transformative partnership.
Plan Netherlands
Plan Netherlands (Plan NL) exists since 1975 and is a member of Plan International. Founded over 75
years ago, Plan International is one of the oldest and largest children's development organisations in
the world. Plan's vision is of a world in which all children realise their full potential in societies that
respect people's rights and dignity. Plan works in 50 developing countries across Africa, Asia and the
Americas to promote child rights and lift children out of poverty.
The programs of Plan NL focus on education, employment, food security, WASH (water, sanitation
and hygiene), protection against violence and emergency aid.
Participation of children, environmental sustainability and gender equality are integrated into all
programs.
Plan NL profiles itself on girls (“Girls First”, that however does not mean girls only). In their vision
investing in girls is the smartest investment for ending poverty. Educated girls have higher incomes,
marry at an older age, are better able to care for their children, know the importance of education for
their own children (including girls!), and invest their income in improvement of the family situation. In
the programs aiming to improve the situation of girls, Plan also involves men and boys.
Akzo Nobel
With its recently launched Human Cities initiative Akzo Nobel aims to highlight its commitment to
improving, energizing and regenerating urban communities across the world.
Akzo Nobel has defined six ways to improve, energize and regenerate urban communities around the
world:
1. Cities should be more colorful;
2. Urban heritage needs to be embraced;
3. People must connect to make cities come alive (transportation infrastructure);
4. Providing proper education for young people is fundamental;
5. Citizens need space to rest and play; 6. Urban design must consider climate change.
From 2012 to 2014 Akzo Nobel ranked first in the influential Dow Jones Sustainability Index.
The firm has a sustainability programme, with a main focus on environmental issues: “Planet
Possible”. Within that programme Akzo Nobel enters “long-term partnerships with suppliers,
customers and other key stakeholders with mutual benefits. For example, helping local suppliers to
develop safe and sustainable work environments”. Within the “Planet Possible” program the company
also aims to “actively participate in communities around the world, creating a positive impact”.
The company presents itself as having integrated sustainability in each and every aspect of the firm
(Akzo Nobel website).
Partnerships for sustainable change 18 Lucia Helsloot, July 2015
4.2. The Plan – Akzo Nobel partnership
Some twenty years ago Akzo Nobel and Plan started their partnership in the so-called Education
Fund. AkzoNobel and Plan NL both participate in the Board of this Education Fund.The money came
and comes through Akzo Nobel, from corporate funds, as well as employee donations and special
fund-raising events. Until now, more than 50 projects in 15 countries in South America, Africa and
South East Asia have been supported.
In 2013 and 2014, the Education Fund focused on Brazil (the Natal region in particular), where Plan,
Amsterdam ArenA Advisory and various other partners joined forces to contribute to the cultural and
social legacy of the Arena das Dunas stadium during and beyond the FIFA World Cup in 2014.
Common goal
The partners in this partnership have a clear shared goal10
or shared value proposition: strengthening
vocational training for deprived youngsters in Brasil (Natal region)11
. As a result of this training, the
chances of these youngsters to find employment will increase.
This common goal serves both partners: it increases impact on youth economic employment (Plan)
and contributes to the Human Cities strategy (education) of Akzo Nobel.
Co-creation, joint decision-making and learning
The partnership evolved on the basis of co-creation in which decisions are jointly taken in the Board of
the Education Fund, and implementation in the project is jointly organised. Both parties continuously
reflect on further development of the partnership and its programmes.
Synergistic value, types of resources and trust
The match between the partners is based on:
- international orientation and network of both partners
- trust, loyalty, long term relationship
- complementary expertise. Akzo Nobel provides financial assistance, but also expertise
(painting). Plan NL provides their knowledge of the local context, and of project management
in development projects.
Strategic value and level of engagement
The degree of engagement of both parties has become quite high and strategic12
. The partnership
program is increasingly connected to the core business of Akzo Nobel. The partnership has become
an integral part of the overall strategy (‘Human cities approach’).
The partnership is equally important for Plan NL as well, because it increases Plan’s impact (of their
youth economic empowerment programme). In addition Akzo Nobel has proven to be a stable partner
and financial resources for many years.
10
See par. 3.2.3.: goal alignment implies a high degree of engagement. 11
Based on market scan by Accenture, which identified hospitality/tourism and construction as interesting sectors.
with employment opportunities 12
Among others based on the proposition that a shared vision on the partnership and goal alignment imply a high
degree of engagement (see par. 3.2.3.).
Partnerships for sustainable change 19 Lucia Helsloot, July 2015
Internal change
The change in the relationship has been accompanied by internal changes in Plan as well. Corporate
partnerships are no longer part of Plan’s fundraising department, but integrated in the programme
department. Corporate partners are no longer seen as purely philanthropic.
Internal drivers
For Plan, increased impact is an important driver for the partnership, but resources mobilization and
increased brand awareness as well. In the interview Akzo Nobel mentions new market opportunities
and reputation as important drivers for the partnership, as well as employee engagement.
From a philantropical towards a more transformative partnership
At the start of the partnership, some twenty years ago, Akzo Nobel’s contribution was mainly providing
financial assistance for the educational projects and programmes, being implemented by Plan.
Over the years the relationship changed considerably. The partnership started as a philanthropic
partnership: focused on charitable giving. In this stage, Plan had clearly greater expectations of the
partnership than Akzo Nobel, and decided to invest in the relationship, to increase the degree of
engagement.
Anno 2014 both parties share the idea that the partnership should be linked to the core strategy of the
business. Akzo Nobel takes the lead in organising vocational training in painting and Plan takes the
lead in recruiting deprived youngsters (from slums13) to be trained as painters. The intensity of the
collaboration is increasing. The nature of their relation has clearly changed; started as a philantropical
partnership, in recent years the relationship has moved towards a more transformative stage.
Differences in the approach of each organization are not considered problematic. The partnership has
built-up mutual trust and is now seen as a good arena for (critical) dialogue and learning. For instance
Plan uses this arena to discuss the need for more focus on girls’ empowerment. AkzoNobel uses this
arena to stimulate Plan to prove their added value to the partnership (based on data), important
because charitable giving has been replaced by a more functional perspective on partnerships.
Impact
The outcome (impact) of the partnership is monitored through the monitoring & evaluation system of
Plan. Akzo Nobel has its own management information system, including the monitoring of sales.
There is no joint outcome or impact measurement at partnership level.
AkzoNobel - PLAN
PLAN – AkzoNobel
1 2
13 Plan can prove that their recruitment of youth leads to a higher success rate (less failure; lower costs of
recruitment) than recruitment by AkzoNobel themselves, which probably makes it interesting for the partners to
sustain the partnership cooperation.
Partnerships for sustainable change 20 Lucia Helsloot, July 2015
5. Conclusions and challenges
The conclusions and challenges as mentioned in this chapter are based on the chapters 1-4, but also
on the lessons learnt and obstacles as mentioned in the data analysis in the annex.
The partnership can become an arena for dialogue, where both partners, having developed
mutual trust, in a safe environment can learn from each other and work on transformative change.
In the case study we have seen that both partners had ambitions to influence each other in the
collaborative dialogue. Plan aimed to increase their partner’s understanding of the importance of
gender objectives. Akzo Nobel aimed to convince the NGO that doing good is not good enough to
get funds.
However, dialogue is not very easy. Partners have the feeling that they have to put much effort in
getting a dialogue. A possible explanation is that, in case the partnership is part of the HR and/or
communication departments, these are not the departments where the content of programs are or
should be discussed. The dialogue might be improved when the operational departments are
more involved.
For both partners it is important to really understand each other: a good understanding of mutual
benefits, and of the motives of your partner for partnering.
Plan has learned that understanding the company implies understanding the motives of the
company for partnering. Is it employee engagement, then how does that work? What is it that
makes that people get engaged, and proud of the company? Is it part of the core business, then
how does that work? How do the business’ financial strategies work?
In this case study the NGO found it equally important to build a strong evidence-based proposition
towards the company.
Plan has learned that building a strong proposition towards the business is important, and that
evidence strengthens this proposition. Evidence of successful recruitment of deprived youth by
Plan proved their added value to their partner. The importance of continuously providing this kind
of evidence, asks for tailor-made Monitoring and Evaluation.
In the case study the company’s internal support for the partnership has been mentioned as a
crucial factor for the partnership. Although this could be seen as part of ‘good understanding of
mutual benefits’, it is worth to be mentioned as a key success factor as well.
From this case study it appears that a NGO with local offices seem to be more attractive as
partner for businesses than NGOs cooperating with local partner organisations. This international
network is an interesting asset for international business in case of scaling up the collaboration.
To keep the partnership alive it is important to continuously invest in the relationship, to connect
the partnership program to the core businesses of both partners and to keep on learning and
jointly developing the partnership program.
Being related to the core business of the company, makes the partnerships more strategic and
therefore more stable than being part of specific company CSR programmes. In a situation where
the partnership is only a philantropical relation, the receiving partner (mostly the NGO) is very
dependent on the donating partner, who can suddenly withdraw its funding.
Partnerships for sustainable change 21 Lucia Helsloot, July 2015
In the case study both partners have the feeling that they have to put much effort in the
partnership to keep it going. This may be an area for concern. Attention for the key success
factors for partnerships as mentioned in this paper, may help to get the partnership going, but the
problem might also have other causes (including the possibility that the partnership is not related
to the operational departments).
Partnerships for sustainable change 22 Lucia Helsloot, July 2015
Annex: Data Analysis Analysis of (interview and literature) data
Partnerships for sustainable change 23 Lucia Helsloot, July 2015
Plan Akzo Nobel Observations / remarks / analysis
1. Partnership
definition
No definition, but key concepts
mentioned: mutuality,
complementarity, impact, business’
core business, transformative14
(together, co-creation).
Different kinds of partnerships
(more or less formal/intensive)
No definition, but criteria:
- fit within (one of the 6 themes of)
the Human Cities strategy
- positive contributing to reputation
- commercial interest (=
transactional)15
Both partners don’t use an explicit partnership
definition. Nevertheless they have an implicit idea of
what a partnership should be. Both share the idea
that the partnership should be linked to the core
strategy of the business, but comparing their
criteria/key concepts with the collaboration continuum
we find that Plan wants the partnerships to be
transformative, AN transactional.
2. Partnership
strategy /
policy
3 years ago partnerships have been
moved from fundraising department
to program department
Partnerships are ideally based on
equality and complementarity in
knowledge and experience. By
bundling forces, new initiatives may
emerge and impact may increase
(annual report 2012-2013).
In partnerships Plan also aims to
bring about change in the
company’s behavior, e.g. supporting
In the past the partnership was part of
the community program. Major driver
was employee engagement /
retention.
Currently partnerships are integral
part of the overall strategy (“Human
Cities”).
Long term commitments (after a pilot
phase).
Fit with one of the six themes in the
For both partners partnerships are of –increased-
strategic importance , related to the operational / core
business
14 From strategic perspective Plan prefers transformative partnerships, not only because of the impact and mutuality they aim for, but also because of the fact that in
philantropic partnerships the NGO is too dependent on the donor. 15
Although part of the “sponsorship and brand activation”department, partnerships are no longer seen as –philantropical- sponsorships. In the context of budget cuts and staff
losing their jobs, philantropical partnerships are no longer accepted within the company. Partnerships need to serve commercial interests (=transactional partnerships). The
partnership with Plan is an exception, because of its history.
Partnerships for sustainable change 24 Lucia Helsloot, July 2015
companies in taking their
responsibility e.g. in the value chain
(annual report 2012-2013) or
gender issues (interview)
Partnerships also aim at resources
mobilization16
(money and/or in
kind).
Focus on international firms.
Plan chooses her corporate
partners based on their contribution
to the Plan organizational and/or
strategic objectives, complementing
factors (e.g. complementary
expertise) and ethical screening.
In this case the partnership fits
within the Youth economic
empowerment program, focusing on
youth from slums, esp. girls17
.
Human Cities strategy (education)
3. Goal The partnership in the so-called Education Fund has a clear shared goal or For this specific partnership AN and Plan have
16
In 2012/2013 Plan NL got almost 2 million euro of corporate partners (in a total of 47,9 million euro). 17 Youth economic empowerment is a good base for collaboration with the private sector, because the theme is closely related to the purpose of any business. Plan motivates
their focus on girls with evidence that investing in girls is more beneficial to community development than investing in boys. However, AN seems to have no specific interest in,
but also no problems with gender empowerment objectives of Plan NL.
Partnerships for sustainable change 25 Lucia Helsloot, July 2015
shared value proposition. The partnership aims at vocational training for
deprived youngsters (in Brasil (Natal region)18
. As a result of this training, the
chances of these youngsters to find employment will increase.
Plan recruits youth from slums19
, AkzoNobel provides expertise for
vocational training of painters.
defined a common goal: strengthening vocational
training for deprived youngsters. This goal serves
both partners: it increases impact on youth economic
employment (Plan) and contributes to the Human
Cities strategy (education) of AkzoNobel.
Added value
per partner
- increased impact20
(mainly)
- resources mobilization
- increased brand awareness.
- new market opportunities
- PR, reputation
The partnership has added value for both partners.
Increased impact is important for Plan and Brasil is an
interesting market for AkzoNobel.
Added value
for the other
partner
(resources,
competen-
cies)
Plan thinks they are an interesting
partner for AkzoNobel, because of
their:
- local presence (local Plan
offices are better asset than
relationships with local partner
organisations);
- knowledge of local context;
- knowledge of implementing
international development
programs with social impact;
- International Plan network,
which makes it easy to scale up
to other countries.
AkzoNobel is interested to work with
AN thinks they are an interesting
partner for Plan because:
- AN is an international company
- AN is a proven loyal partner (>20
years)
- AN has expertise (on products
and painting) and money to start
up vocational training for painters
(without them Plan might not
have been able to train painters)
AN also contributes in money, and in
kind (paint), to the partnership
The match is based on:
- international orientation and network of both
partners
- trust, loyalty, long term relationship
- complementary expertise
18
Based on market scan by Accenture, which identified hospitality/tourism and construction as interesting sectors with employment opportunities. 19 Plan can prove that their recruitment of youth leads to a higher success rate (less failure; lower costs of recruitment) than recruitment by AkzoNobel themselves, which
probably makes it interesting for the partners to sustain the partnership cooperation. 20
One of the strategic objectives of Plan NL for 2015 is to increase the impact of the programs for empowerment of (adolescent) girls, through cooperation with corporate
partners and public-private partnerships (annual report 2012-2013).
Partnerships for sustainable change 26 Lucia Helsloot, July 2015
Plan, because (in their own words)
Plan helps AN:
- in making decisions about which
projects to support through the
(Education) Fund (website AN).
- in project management
(elaboration of objectives,
strategy and implementation).
Added value
for society
Increased number of deprived youngsters (male and female) trained and
(hopefully) employed, with a living wage.
Although this is valuable in itself, and certainly a
benefit for society, the project will also contribute to
systemic change (changing rules of the game) if the
project will succeed in improving gender relations.
Strategic
value /
importance
Quite important, because of (in their
own words) increased impact,
resources, and stable long term
(>20 years!) relationship21
.
The importance of the partnership
seems to be increasing because of
increasing intensity of collaboration
and increasing closer relation to
core programmes
On their website Plan NL refers to
their partnership with AkzoNobel
(and the Education Fund ’94), being
one of their 12 corporate
Quite important, because (in their
own words):
- the partnership is integral part of
the overall strategy (“Human
Cities”) , and Plan is the only
partner on one of the 6 themes
(education)
- history (the way it started and the
long term collaboration (>20
years))
- the Plan International network
can open new markets for AN.
(e.g. in Brasil)
- collaboration with Plan, being a
strong brand, strengthens the AN
For both partners the partnership is quite important
and strategic.
21
In contrast with government funding from calls, which are only for a couple years.
Partnerships for sustainable change 27 Lucia Helsloot, July 2015
partnerships. Here they quote the
chief HR Officer of AkzoNobel22
.
brand
On their website (under the headings
of “partners” and “sponsorships”)
Akzo Nobel refers to their partnership
with Plan, being one of their 12
partnerships. Reference is made to
the Plan website and information is
given about the partnership program.
Governance Both partners participate in the Board of the Education Fund, and collaborate
in implementation. Account managers from both partners collaborate and
bring people together.
The program is monitored through the Plan PME system. Akzo Nobel’s
management information system focuses on sales and reputation, and not
on other achievements of partnership programs. However, in the case of a
film documentary, they did measure the number of people reached.
Joint and equal responsibility for decision making, co-
creation.
The monitoring in this partnership is different from the
Plan-Accenture partnership. Accenture has its own
system for measuring the achievements in their
partnership portfolio (interesting to learn about).
Obstacles The feeling that they have to put
much effort in furthering the
partnership
- The feeling that they have to put
much effort in furthering the
partnership (compared to other,
more pro-active partners)
- more capacity needed, to realize
all ambitions
- different visions on partnerships
- differences between Akzo Nobel’s
ambitions and local needs
Both partners have the feeling that they have to put
much effort in the partnership to keep it going.
22
"Through the partnership with Plan, Akzo Nobel supports and invests in the training of young people all over the world, helping to develop their talent. We believe that there
can be no innovation without education. Providing good education to young people should be a fundamental concern of all of us who live, work, or are associated with cities".
(Marten Booisma, Chief HR Officer Akzo Nobel)
Partnerships for sustainable change 28 Lucia Helsloot, July 2015
- Akzo Nobel’s aim to be preferred
supplier in the partnership
Lessons
learnt
Building a strong evidence-based
proposition towards the company is
important.
For equal partnering and a good
proposition it is really important to
invest in relations and understand
the motives of the company for
partnering.
Continuous joint learning and co-
creation is important for keeping the
partnership alive.
Relating the partnership program to
the core business of the company
makes the partnership more
strategic and stable.
Internal support is important. For
Akzo Nobel this means connecting
the partnership to the international
core business
Partnerships for sustainable change 29 Lucia Helsloot, July 2015
Annex 2 : Measuring outputs, outcomes and impact
A. Example of a results chain:
(www. energypedia.info)
Partnerships for sustainable change 30 Lucia Helsloot, July 2015
B. Model for impact measurement (Karen Maas, in P+ Magazine, July + August 2014)
Partnerships for sustainable change 31 Lucia Helsloot, July 2015
Sources - Arensman, Bodille, et al. (2015), MFS II Joint Evaluation of International Lobbying and Advocacy,
Wageningen University.
- Austin et al., July 2012; Downloaded from nvs.sagepub.com by guest on April 22, 2013 Nonprofit and
Voluntary Sector Quarterly 41(5)).
- Drost, S. & Pfisterer, S. (2013). How to make cross-sector partnerships work? Critical success factors for partnering. Partnerships Resource Centre: Rotterdam. - Gaag, Jacques van der et al. (2015), MFS II Joint Evaluations Final Synthesis Report, Amsterdam
Institute for International Development.
- Gray, Barbara and Jenna P. Stites , Network for Business Sustainability (2013). Sustainability
through partnerships, Capitalizing on collaboration.
- Maas, Karen, 2014, Samen goed doen, in P+ Magazine jaargang 12, nr. 5 (mei/juni 2014)
- Maas, Karen, 2014, Nieuw Model om impact te meten, in P+ Magazine, jrg. 12, nr. (juli / augustus
2014)
- Ministry of the Foreign Affairs of the Netherlands (2010). Public-private partnerships. Ten ways to
achieve the Millennium Development Goals.
- Porter, Michael E. and Mark R. Kramer, Creating Shared Value, in Harvard Business Review jan-febr
2011.
- The Partnerships Resource Centre, 2011. The State of Partnerships Report, Dutch Development
NGOs facing the Partnering Challenge.
- UN Global Compact LEAD Task Force on UN-Business Partnerships (2011).Catalyzing
Transformational partnerships between the United Nations and Business. And A 10-step guide for
creating effective UN-Business partnerships.
- van Huijstee, Mariette (2012). Multi-stakeholder initiatives, A strategic guide for civil society
organizations. SOMO.
- Van Tulder, van Tilburg, Francken and da Rosa (2014). Managing the transition to a sustainable
enterprise. Lessons from frontrunner companies.
- Van Tilburg, Van Tulder, Francken en da Rosa (2012). Duurzaam ondernemen waarmaken. Het
bedrijfskundig perspectief.
- Van Tulder, Rob (2011). With great power comes great responsibility, Fourth Max Havelaar Lecture,
Rotterdam School of Management Erasmus University, 27 october 2010, published in 2011.
- Van Tulder et al. (2004). De strategische stakeholderdialoog.
- Veldhuis, Tom (2013). CSR and business-NGO partnerships. An assessment of partnerships and
impact at Accenture. Master thesis Business Administration, Global Business & Stakeholder
Management.
- Waddell, Steve, 2011. Global Action Networks, Creating Our Future Together.
Partnerships for sustainable change 32 Lucia Helsloot, July 2015
- Several articles in Vice versa (online, in Dutch), including:
- Promotieonderzoek Kellie Liket: ‘Waarom ‘goed’ doen niet goed genoeg is’. Door: Marjolein
Quist en Siri Lijfering op 7 april 201423
- Kellie Liket: ‘Beleid Ploumen niet ondersteund door wetenschappelijke kennis’. Door:
Marjolein Quist op 24 april 2014
- Discussie proefschrift Kellie Liket: is impact meten mogelijk? Door: Marjolein Quist op 17
april 2014
- Reflecteren op methoden van impactevaluaties. Door: Caroline van Slobbe op 10 februari
2015
23 For the dissertation of Kellie Liket (Why 'Doing Good' is not Good Enough; 2014)
see http://repub.eur.nl/pub/51130. See also www.impactmeten.nl.