Developmental Evaluation Cameron D. Norman PhD CE Principal CENSE Research + Design Adjunct Professor, Dalla Lana School of Public Health, University of Toronto

@cdnorman August 28, 2013

Outline

• Thinking about the problem(s)

• What is Developmental Evaluation?

• Methods

• Case Examples

• Discussion & dialogue

Same Evaluation Tools / "Different Thinking Tools

Complexity

Example: Organizing a children’s party

Evaluation & Measurement: Problems & Opportunities

What is an effective coffee date?

Is that the right question?

What is an effective tree?

What is the best practice for being a tree?

What is a fully optimized tree?

Do you know what is inside before you crack it open?

Confusing moving things with static things

The Importance of Theory

What is Developmental Evaluation?

Developmental Evaluation

•  First proposed by Michael Quinn Patton (1994)

•  Combines Utilization-Focused Evaluation with concepts from Complexity Science

•  Recognizes that social innovation and resiliency require a different form of feedback to enable programs to adapt and thrive in changing conditions

•  An approach to evaluation, not a specific method.

•  Strategic learning / real-time evaluation

Developmental Evaluation

•  Assists social innovators in learning strategically about how to intentionally develop their program while acknowledging to complexity

•  Supports adaptation to changing conditions

•  Can identify and nurture promising interventions

•  Documents innovators’ actions

•  Identifies emergent processes and outcomes; enhances resiliency

•  Determines if an innovation is ready for formative evaluation

Complicated Systems Complex Systems Single causality Multiple causality

Designed & intended Adaptive & emergent

High degree of certainty High degree of uncertainty

Assumed predictability Anticipation of events

Focus on ‘boxes’ Focus on ‘arrows’

Structures determine relationships Structures & relationships interact

Linear Non-linear

Noise, tension & flux undesirable Noise, tension & flux embraced

Adaptation to static environment Interaction with static environment

Reaction Engagement

Adapted from http://www.healthandeverything.org

Program Improvements Program Developments

1. Add a new topic to a training curriculum

1. Change the scope, sequence and delivery of curriculum

2. Provide staff training to enhance skills

2. Change job descriptions, reconceptualize fundamentals of the job

3. Expand recruitment effort to a target area

3. Question and re-evaluate recruitment strategy

4. Expand staff to serve increasing numbers using the same model

4. Expand staff, change the model to suit new scale and scope of practice

5. Fine-tune programming based on participant feedback

5. Change programming to fit ever-changing realities

6. Add more content to provide additional information

6. Change the way the audience interacts with content altogether

Design Thinking

Developmental Design

Methods & Methodology

Mindfulness

Ethnography

Network Analysis

Timelines

Visual Thinking

Gigamapping http://www.systemsorienteddesign.net/

Idea Generation

Knowledge Mobilization

Strategic Learning

Discussion

Cameron D. Norman PhD  

@cdnorman

censemaking.com