conference on monitoring and evaluation for responsible ... · conference on monitoring and...
TRANSCRIPT
Conference on Monitoring and Evaluation for
Responsible Innovation, Wageningen 19-20 March 2015
M&E for responsible innovation AND scaling
By: Seerp Wigboldus , Centre for Development Innovation,
Wageningen UR, the Netherlands
In opening
Our innovation abilities increasingly exceed our ability to foresee longer-term effects of the outputs and outcomes of innovation processes
This is a core concern which inspires ideas regarding responsible (research and) innovation
Scaling (up) processes put a multiplier effect on this concern
However, research and innovation processes are ill-equipped to engage responsibly with scaling processes
Increasing demand for ‘impact at scale’
Research and innovation often considered as a “finding out what works” after which the idea is to “do more of the same” (replication, multiplication, spreading, adoption, dissemination, transfer, etc.)
“What works”, or what is considered “responsible”, will be defined within certain system boundaries, domain boundaries, geographic boundaries, etc.
Scaling processes (doing more of the same) usually escape such boundaries leading to impact across scale levels, domain boundaries, etc.
The ‘scaling (up)’ part of responsible
innovation is what connects strongest to
societal processes
Addressing societal concerns
“When pathways to scale become roads to hell”
Scaling processes are essentially about societal processes
– who decides what should go to scale and not? Who benefits from scaling (up) and who doesn’t? Who is responsible for negative impact at scale? Who decides what is responsible?
Responsible innovation and responsible
scaling are not necessarily the same
If innovations are considered to be ‘responsible’ (relating to
e.g. inclusiveness, sustainability), will they also remain
being ‘responsible’ when they are scaled up?
Scaling up means entering new scale levels, new domains,
new geographical areas, etc. – how will we know
innovations will remain ‘responsible’ when they cross those
boundaries?
The use of asbestos (at the time an innovative product)
went to scale in an enormously successful way. We now
pay the price for lack of foresight by having to scale down
the presence of asbestos in older buildings
A need to expand perspectives on responsible
innovation
Little or no explicit reference to scaling processes in the concept of responsible innovation, while these are the very processes which raise concerns
Environmental degradation is not the result from some innovation process that went wrong – it is the scaling up of the use of certain products and the application of certain practices that really got us in trouble
E.g the plastic in oceans is a result of the scaling up of the use of plastic (packaging) materials
Defining “responsible”
At what scale level do we define responsibility?
● Product level?
● Process level?
● Production system level?
● Locality level?
● Landscape level?
Cumming, Cumming and Redman (2006) hypothesize that many of the problems encountered by societies in managing natural resources arise because of a mismatch between the scale of management and the scale(s) of the (ecological) processes being managed
Testing logics
The often-assumed logic
One application of best-practice, best-bet, best-fit innovation is good news, more applications of the same is better news
Critical questions
Is this always true?
Is this true in most cases?
Is this rarely true?
From innovation to scaling – beware of
fallacies
Menter (2004) on fallacies along the lines of scaling up:
The ecological (inference) fallacy (or cross-level fallacy): what
works at one level will work at another.
The composition (inference) fallacy: what is good for one
person is good for everyone (if one village was able to increase
income through growing a new crop, all villages in the region
could do the same).
The exception fallacy , which is sort of the reverse of the
ecological fallacy, such as in stereotyping: If one researcher
does something, all researchers assumed to be like that.
The argument can be extended to what we consider to be
“responsible”
Scaling up in agricultural development and
innovation cannot escape complexity
Implications of scaling up the use of a new crop goes beyond cropping system boundaries
Implications of scaling up in agriculture goes beyond agricultural system boundaries
Understanding systems ≠ understanding success and failure of scaling processes
Scaling up may seriously distort equilibria, distort proportions, distort wider context
We may want innovation to be disruptive, but scaling up means putting a multiplier on disruption – can we anticipate implications?
Processes of scaling (up) will tend to move
anything into the domain of the complex and
‘chaotic’
Even introducing a (simple) device may lead to unanticipated (e.g. health) consequences
Scaling (up) moves things into new domains, new geographies, and new scale levels
Experimental scaling (up) is hard to do
Need to get an idea about cross-scale, cross-level, cross-domain, cross-mandate, cross-temporal implications
↑Level
ofnet
benefit/value
Level of scaling up →
?
?
?Deserving label of being
“responsible”
Level of scaling up →
?
This is serious stuff
Inertia is a concern for innovation, but also a concern once scaling up innovation (s) has led to undesirable situations
Once farmers have started using agrochemicals and have seen economic benefits, it is hard to get them to reduce its use after environmental damage is found out
But also, once we have lost biodiversity as a result of scaling up agricultural production, it is hard to restore it
It is scaling processes which may race the effects of ‘innovations’ toward planetary boundaries and beyond
Scaling processes race us toward tipping points (Scheffer, 2009, 2011)
A vision for responsible innovation and scaling
Responsible innovation is a relative concept, relative to scale levels, relative to context, relative to other products and processes
Scaling (up) is not a mere extending of that which emerged from (responsible) research and innovation
Water is good, but too much water is not good. Food is good, but too much food is not good. Use of soils for agriculture is good, but too much... Etc.
It is through processes of scaling (up) that things which may have even started off as something good, become critical and even harmful
Hence: From design, (responsible) research and innovation needs to have potential future scaling (up) in mind
Late lessons from early warnings – improving
theories of innovation and theories of scaling
European Environmental Agency, 2013
Toward responsible theories of scaling (ToC)
Wrongly assumed scaling mechanisms, an example
It used to be (and perhaps still is) widely believed that the accumulation of wealth by the rich would be good for the poor as some of the increased wealth of the rich would trickle down to the poor.
We may see this as an assumed scaling mechanism. In 2011, however, OECD, in the report “Divided we stand. Why inequalities keep rising”, challenged the trickle-down theory (which relates to scaling processes) as unfounded in many cases.
Articulating “theory of scaling”
Along the same lines as the “theory of change”, articulate how scaling is expected to happen and what its effects are expected to be
Articulate assumptions related to this, including related risks
Develop critical reflection questions to be used regularly to assess not just the “success” of scaling, but also unanticipated effects across domains, scales and time
M&E to inform strategic guidance
● In design: activating foresight analysis
● During implementation: activating critical reflection and adaptive management
An illustration: Rubber in Xishuangbanna, China
Monoculture rubber was introduced some 50 years ago as a
way out of poverty and as a “green” development option
(apart from other reasons)
This massively went to scale, now covering more than a
quarter of the entire prefecture (half the size of Bhutan)
With the increase in scale of rubber production and further
(scientific) research, gradually more and more concerns were
voiced
Now, rubber cultivation in Xishuangbanna is considered to be a
big problem, mainly because of its negative environmental
effect
But, the economic benefits have made people addicted to it –
it is very hard to change to more responsible rubber cultivation
If only this had been foreseen 40 years ago...
This example is rather typical for many agricultural
development processes around the globe
Some labels we may want to put on this:
Rollercoaster development
Tunnel-vision innovation
Monorail scaling
Suggested description of what scaling-inclusive
vision for responsible scaling may involve
Responsible in terms of process:
- Well-informed ex ante and ex durante about relevant cross-
scale, cross-domain, and cross-temporal influences and effects;
- Meaningfully involving stakeholders in design, decision-making
and M&E of the scaling initiative;
- ‘Potential negative side-effects can be mitigated and or adapted
to.
Responsible in terms of effect:
- Positive outcomes can be sustained across domains, scale
levels and time (sustainability and environmental integrity);
- Disadvantaged groups share in positive outcomes
(inclusiveness);
- Effects and side-effects do not undermine system/ stakeholder
resilience.
Taking scaling processes seriously
If we are to take responsible (research and) innovation seriously, it also implies a need to take scaling processes seriously
This means treating them more seriously than as mere processes of adoption, transfer (of technology), rolling out, replication, etc.
Are conventional research and innovation processes equipped for this?
No!
Are conventional M&E processes equipped for this?
No!
Upcoming papers
Taking scaling processes seriously in agricultural research, innovation and development (part 1) - Toward an integrative approach
Taking scaling processes seriously in agricultural research, innovation and development (part 2) - Application of an integrative approach in case studies on green rubber in China and agroecology in Nicaragua
Probably to be published later this year in Agricultural Systems
Suggested M&E to keep an eye on whether responsible innovation (as process) remains being responsible as products, services, processes are scaled up
Strategic design and implementation phase
Design of innovation (both in terms of process as well as in terms of
outputs) needs to have future scaling in mind
Scenarios for potential scaling up need to be developed early on to
assess potential risks and limitations which may be involved if “this”
were to go to scale
M&E needs to check to which extent such processes are in place (on
paper, functional, or actively used in decision-making/selection
processes)
M&E design and implementation phase
Development and use of indicators and critical reflection processes
related to the extent to which the label of “responsible” keeps
applying when things go to scale
Effective M&E processes to assess the above
1. How can M&E responsibly support the management and governance of innovation processes towards a sustainable and equitable future?
2. How can M&E contribute to deeper reflexivity and transparent decision making?
- Effective (responsible) M&E of scaling up (that which emerges from “responsible” innovation), starts with including related issues/questions already at the design of innovation processes
- This involves articulating strategic questions regarding what will make us consider innovation (including scaling) to be “responsible” - this is to be translated to information needs and needed M&E (reflection) processes.
3. What are the prerequisites for taking responsibility for systemic change in terms of: *M&E professional’s roles & responsibilities; values and principles; competencies, *M&E process design, focus and approach, *institutional changes needed to support M&E for responsible innovation.
Develop agreement on
- What is understood by “responsible” and translate this to concrete,
monitorable categories per case
- Who is considered to be responsible for what and what this means
for quality requirements they need to comply with
Strengthen foresight capacities (incl. scenario analysis) in relation to
an ability to foresee longer-term/cross-scale/cross-domain effects of
innovations going to scale
Monitor the abovementioned categories in relation to what is meant
by “responsible”
Regulatory frameworks will often need to back this up since there is
usually a short-term price tag attached to being “responsible”
Questions, comments, additions, further
ideas, related experiences?
Key question: To what extent does the argument make sense?
If relevant,
● what other lines of thinking/experiences might support the argument?
● what are implications for views on responsible innovation?
● what are implications for the role which M&E needs to play?
● what may need to change in conditions for M&E to make this possible?