measuring impact in the voluntary sector', by new philanthropy capital
TRANSCRIPT
![Page 1: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/1.jpg)
vMEASURING IMPACTAnne Kazimirski, Head of Measurement & Evaluation, NPC
Connect@Evolve, Monday 15th June 2015
![Page 2: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/2.jpg)
NPC: TRANSFORMING THE CHARITY SECTOR
2
NPC works at the nexus between charities and
funders
Charity
SectorFunder
Increasing the impact of charities
eg, impact-focused theories of change
Strengthening the partnership
Eg, collaboration towards shared
goals
Increasing the impact of funders
eg, effective commissioning
Consultancy & Think tank
![Page 3: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/3.jpg)
THE FOUR PILLARS APPROACH TO
MEASURING IMPACT
3
Map your
theory of
change
Prioritise
what you
measure
Choose
your level
of evidence
Select your
sources and
tools
Effective measurement
framework developed
Strategic vision
Leadership
Case for impact measurement
![Page 4: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/4.jpg)
DEVELOPING AND USING
A THEORY OF CHANGE
![Page 5: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/5.jpg)
WHAT IS A THEORY OF CHANGE?
Links activities intermediate outcomes final outcomes
A description of how activities lead to outcomes
5
- Clarifies what the activities aim to achieve and how
- Provides the case for why achieving intermediate outcomes is important
- Provides a structure for identifying what can be measured
![Page 6: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/6.jpg)
HOW TO REPRESENT A THEORY OF CHANGE
6
Planning Triangle Logic Model Outcomes Chain
However you represent your theory of change, it should be supported by a written description.
![Page 7: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/7.jpg)
7
Children’s emotional
resourcefulness
improves
Counselling
Clients’ ability to support
their children's healthy
development improves
Clients’ emotional or
psychological
difficulties decrease
Mother / Child
interaction
improves
Clients’ capacity
for self care
increases
EXAMPLE (SIMPLIFIED) THEORY OF CHANGE
MOTHERS’ COUNSELLING SERVICE
Activities Intermediate Outcomes Final outcome
![Page 8: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/8.jpg)
PRIORITISING OUTCOMES
![Page 9: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/9.jpg)
PRIORITISE OUTCOMES THAT:
• Are directly influenced (rather than indirectly supported)
• are important / material to the mission
• are not too costly to measure
• will produce credible data
9
![Page 10: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/10.jpg)
CHOOSING THE RIGHT
LEVEL OF EVIDENCE
![Page 11: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/11.jpg)
COUNTERFACTUAL
• Comparing the world with your
organisation in it with what the
world would be like without it.
• Control group
• Attribution
11
![Page 12: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/12.jpg)
LEVELS OF EVIDENCE
12
Randomised
control trial
Anecdotes /
quotes
Before and
after survey
Self-reported
change
Case
studies
Control
groups
Credibility
Basic Advanced
![Page 13: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/13.jpg)
SELECTING DATA
SOURCES AND TOOLS
![Page 14: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/14.jpg)
• Quantitative data (numbers)
• Statistical estimates
• Prevalence of views, attitudes and experiences
• Admin data or questionnaires (paper, web, etc.)
• Qualitative data (words)
• Detailed understanding
• In-depth interviewing (telephone or face-to-face)
• Observation
• Stakeholders’ views
Proportion of beneficiaries
whose outcomes have
improved, and by how
much.
What did beneficiaries
think, did it make a
difference to them? How?
How could it have been
better?
DIFFERENT TYPES OF EVIDENCE
![Page 15: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/15.jpg)
EVALUATION TRAPS
15
Collect data that matters, and work together
Don’t force
squares into circles
& don’t collect
arbitrary data
![Page 16: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/16.jpg)
INSPIRING IMPACT: MEASURING UP! AND
THE IMPACT HUBMeasuring Up!
• online, step-by-step self-assessment tool
• looks at the way you plan, evidence, communicate and
learn from the difference your work makes
The Inspiring Impact Hub
• pulls together resources relevant to improving impact
practice
• enables users to search and filter results according to their
needs
inspiringimpact.org
16
![Page 17: Measuring Impact in the Voluntary Sector', by New Philanthropy Capital](https://reader030.vdocuments.mx/reader030/viewer/2022032700/55d34d2fbb61eb42628b4692/html5/thumbnails/17.jpg)
ACTIVITY – BENEFITS AND CHALLENGES
17
Two questions to discuss:
• What are the benefits of measuring impact?
• What are the challenges of measuring impact?