nef measuring what matters

Measuring what mattersRichard Murraynef (the new economics foundation)

richard.murray@neweconomics.org

nef’s approach to measurement

• Beyond targets - don’t just look at what is easy to count; it’s what happens afterwards that matters

• Story-telling – seek out the narrative behind the numbers in order to understand the ‘theory of change’

• Conversation – by providing a common language, measurement becomes a tool for enhancing participatory democracy

• Process – as important as the outcome; measurement can build confidence, ownership and contribute to the sustainability of an initiative

What to measure and why? • Targets and outputs: for funders and investors

• Performance against objectives: for internal strategy

• Quality of services and processes: to enhance relationships with stakeholders

• Impacts:• on people• on the economy• on the environment

…so we know that we’ve made a difference.

So it’s not just counting things…

• Proving – which is about demonstrating that change is actually taking place.

• Improving – which refers to a continuous process of learning and growth, to support the social mission and also to sustain and grow the business.

To prove and improve

1. Ask why evaluate? Know why you’re proving and / or improving.

2. Clarify your key objectives, mission and values. Know where you are going.

3. Identify your stakeholders.

4. Map out the story (the hypothesis or theory of change) and describe the milestones. This is to determine the scope of an evaluation.

5. Choose your indicators wisely (be challenging but realistic about what you can measure.)

6. Make a plan. Choose a methodology – how to go about measuring these things.

7. Collect the information:Identify existing records / Consult stakeholders / Pull together data

8. Analyse the information, and draw conclusions

9. Share it with others.

10. Learn from it and act on it.

To prove and improve

1. Ask why evaluate? Know why you’re proving and /or improving.

2. Clarify your key objectives, mission and values. Know where you are going.

3. Identify your stakeholders.

4. Map out the story (the hypothesis or ‘theory of change’) and describe the milestones. This is to determine the scope of an evaluation.

5. Choose your indicators wisely (be challenging but realistic about what you can measure.)

6. Make a plan. Choose a methodology – how to go about measuring these things.

7. Collect the information:Identify existing records / Consult stakeholders / Pull together data

8. Analyse the information, and draw conclusions.

9. Share it with others.

10. Learn from it and act on it.

The “way-in” to choosing a tool

• Understand the story (Stakeholders and Storyboard)

• Formalise the story (Impact Mapping)

• Ways of knowing (Choosing Indicators)

• Ways of measuring (Choosing from Tools)

Eight Questions for a Conversation

1. Describe “the world” in which you are working in terms of the local or wider need that the work

of the foundation is addressing?

2. What work does the foundation do (e.g. the activities and actions of staff, governors, trustees

and other stakeholders)?

3. What are the immediate and observable effects of those actions and activities?

4. What effects and changes do you expect to see in the future?

5. Where possible, describe the long-term changes for people, the environment or the economy

that:

• The foundation contributes to?

• The foundation is wholly responsible for?

6. For every immediate effect you identified in 3 above, ask “So what?” or “Why is that important?”

(Try and describe precisely how each of the immediate effects will lead to the changes in the

future.)

7. For every effect and change you identified in 4 above, ask “So what?” or ”Why is that important?”

(Try and describe how the changes will lead to the long-term changes for people, the

environment or the economy.)

8. What are the barriers that need to be overcome to make this story happen as you describe?

Building a Storyboard

8

6 7

1 5432

8.

The Impact Map

Input Activity Output Outcome Impact

•Money•Time •Premises•People

•Training•Advice•Youth Programme

•Qualifications•Received advice about x, y, z •Number of youth involved for x amount of time

•Job readiness•Job attained•Client better informed to make different choices •Increase in self-esteem •New knowledge

•Effect on people, community, economy•How are things better in the world? – your vision

Choosing tools for the job

• What are we already measuring that tells our story?

• How can we make use of what we’ve heard about today?

• What more do we need to know in order to do this?

Details on some specific tools

Social Accounting: a framework

• Multi-perspective - Reflects stakeholders’ views

• Comprehensive - Covers all activities

• Comparative - With other organisations and with the same organisation over time

• Regular - Annually? Every other year?

• Verified - By an external panel of auditors

• Disclosed - Widely circulated

Social Return on Investment

• To help organisations understand and quantify the social value that they are creating

• Developed from traditional cost-benefit analysis

• Captures the economic value of social benefits by translating social objectives into financial measures

• Stakeholder involvement is key to identifying social objectives (social and environmental accounting principles)

• Impact mapping is central to the process. Impact more precisely defined as the value of the outcome minus what would have happened anyway.

• A tool still in development- e.g. The need to build a bank of proxies

Prove It!

The original handbook – Groundwork, Barclays Sitesavers and nef

Measuring the effect of neighbourhood renewal on local people, in a way that places them at the heart of the process

Looking beyond the numbers (E.g. Of trees planted or the number of volunteers involved)

nef continuing development of Prove It! in the form of a simplified methodology to help make evaluation simple, manageable and possible.

LM3 (Local Multiplier 3)

A DIY economic indicator that enables you to measure the impact of your initiative or work on the local economy

Adapted from the Keynesian multiplier

You measure three rounds - how money enters, how it is spent, and how it is re-spent

Clear, simple, transparent, and replicable

Balanced Scorecard for S.E.s

• Measuring performance against strategic objectives

• Helps clarify how you make change and make money

• Links to your strategy– for business development and for impact on people

• Management information in a manageable form

• ‘Measures’ communicate the strategy & what needs to be done in each area

• Internal rather than external

Social Firms Performance Dashboard

• Balanced Scorecard management principles into a simple tool to help Social Firms manage and improve their performance

• Six perspectives: Financial / Employees / Customers / Internal / Environment / Trainee

• A template of typical key objectives which can be altered to suit the circumstances of your firm.

• Typical measures for each objective which can be edited and which then allow you to set targets to improve performance and record over time how you are doing.

• The software provides a range of time saving features to produce outputs for communication purposes

Assessing quality: DTA Healthcheck

• Check against Business Basics: Governance, Finance, HR

• Check against key development trust values:Engaged in economic, social and environmental regeneration of a defined area or community Independent, aiming for self-sufficiency / not for private profitCommunity based, owned, managedActive involvement in community, voluntary, private and public sectors

• There are levels of attainment in each area• Part of the start-up / planning process for development

trusts

What the Co-ops Measure

• Member economic involvement

• Member democratic participation

• Staff education & training

• Staff injury & absentee rates

• Staff profile- gender & ethnicity

• Customer Satisfaction

• Ethical procurement & investment

• Investment in community & co-operative initiatives

• CO2 emissions

• Proportion of waste recycled/ reused

Key Social and Co-operative Performance Indicators

Other impacts: creating value

Local economies– buying locally, keeping money circulating (LM3)

People– health and ‘well-being’ (well-being indicators)

Communities– social capital: networks, norms, and trust (Prove It!)

The natural environment– sustainability, reducing harmful CO2 & recycling (GRI)

Monetising impacts– social return as a narrative or “lens” (SROI)

Useful points of contact

• Quality and Impact Toolkit for Social Enterprise - www.proveandimprove.org

• Social accounting and audit - www.socialauditnetwork.org.uk

• Social Firms Performance Dashboard- www.socialfirms.co.uk

• Social Return on Investment: a primer - http://sroi.london.edu

• Socal Enterprise Balanced Scorecard - www.sel.org.uk/services.aspx

• Development Trusts Association Healthcheck- www.dta.org.uk

• Key Social and Co-operative Performance Indicators www.cooperatives-uk.coop