CSI PROFESSIONALS BRIEFING 2014Reana Rossouw

Next Generation Consultants

Session Two:Impact Investment Index

Our Context Started in 2009 – global benchmarking – 40 models currently

being applied internationally Conclusion:

Africa needs its own solution – applicable to both the funding and development sectors

Practitioners need capacity, support and knowledge Practitioners don’t need complex methodologies or approaches

requiring specialist skills, licenced software or specific hardware or expensive solutions

The industry needs a transparent, comparable, and flexible solution that contextualises and take into consideration the complexities, relationships and fundamental development principles of our specific development context

The purpose of Impact Investment Index is to create a shared performance measurement system to be utilized by all organisations in the social investment and community development sector

The current system lacks coordination, leading to added expense, limited learning, and inadequate ability to assess shared value and collective impact

Our Objective

Why the pressure to measure?

The debate on impact and return on investment are playing out in three arenas:

In private foundations and corporate CSI divisions Aiming to be more strategic about their philanthropy, grant making

and social/community investments

In nonprofit organisations in response to pressures from corporates, foundations and government To be more accountable for the resources received and program

outcomes expected

Among international development organisations such as bilateral government agencies and intermediary organisations Seeking to improve development effectiveness and lessen

dependency on grant/development aid

Funders are increasingly asking for demonstrable results – to understand the difference they make, directly and indirectly

This trend is accelerating, as the development and in particular the funding sector is increasingly looking to pay by results – to learn from what they, and those they fund, do

It is not just donors that care about impact. In a age of competition, transparency, recessionary economies, there is growing competition for resources

Why now?

Three primary applications of Impact Assessments Prospective

Looking forward to determine whether or not the projected costs and benefits indicate a favorable investment

Ongoing Testing assumptions and projections along the way in order to

aid course correction

Retrospective Looking back to determine whether or not it was a favorable

investment given the costs incurred, and therefore to inform future decisions

Variety of purposes – our school of thought One can and should use impact data to make funding

allocation decisions across program areas You can compare programs once you get in the sector of health, but

you cannot compare health vs. arts vs. education vs. sport.

One can and should use impact data to make funding decisions within program areas

It is not about building a unifying measurement across domains, but to build a conceptual framework for understanding the biggest impact across a Rand value unit. So it is not about comparing health to education to sport or the arts, but to determine which program yields the highest return for the most effective use of resources

The conundrum Funders and non-profits often use the words “evaluation”

and “impact” loosely, stretching these terms to include any type of report on the use of funds or the results they achieve

Practitioners should distinguish between measuring performance (monitoring inputs, activities, and outputs), measuring outcomes (near-term results), and evaluating impact (long-term changes that can be attributed to the investor’s activities) as well as return on investment (benefits accrued to the investor) as a result of funds and other resources invested

What is impact “Measure of the tangible and intangible effects (consequences) of

one thing's or entity's action or influence upon another. An impact evaluation/assessment assesses changes in the well-being of individuals, households, communities or firms that can be attributed to a particular project, program or policy. The central impact evaluation question is what would have happened to those receiving the intervention if they had not in fact received the program. ”

For us this means: The broad or longer-term effects of a project or organisation’s work (also referred to as the difference it makes). This can include effects on people who are direct users/beneficiaries of a project or organisation’s work, effects on those who are not direct users/indirect beneficiaries, or effects on a wider field/aspect such as government policy, processes, systems, infrastructure or support systems

What did we want to achieve?

To provide evidence To demonstrate performance To prove accountability To show program/ investment

effectiveness To demonstrate value To empower and capacitate

communities and funders Ultimately - to alleviate,

eliminate and eradicate poverty

What we have achieved Assessed R1 billion worth of investment but also considered

all types of input resources from money to hours, products and services, infrastructure and skills and volunteerism

Across both socio economic and enterprise development sectors

Including 15 focus areas 400 programmes – from flagship to donations, once off to 3-5

year programs, sponsorships and cause related marketing, social and business enterprises

Across 15 dimensions of impact and 25 dimensions of return Developed a library of more than 500 indicators

But more than that Impact across the value chain:

Outcomes of partnerships, relationships and applied resources (to be more sustainable and effective)

Outcomes of the initiative (policy change, organisations working more effectively together reducing/alleviating/ eradicating poverty)

Outcomes at the community level (are we moving the needle against the strategic objectives/mandates indicators?)

Returns for the donor/funder (are we getting value, recognition, licence to operate, etc.)

Guiding Principles of our work Impact means impact

The goal is to understand what changes as a result of the investment from funders in communities as beneficiaries and recipients of interventions

The impact is shared The goal is to understand who is impacted along the impact value chain –

including funders , intermediaries and beneficiaries Impact includes and involve all stakeholders

Analysis must be comprehensive. Instead of cherry picking something that’s working and leaving out what is not, the analysis should include all aspects of impact and those impacted

Results must be transparent Companies should report to their investors, and investors should aggregate and

report results. What is left out should be stated. Assumptions and sources should be stated. It should be possible for a third party to replicate the analysis based on the documentation of it and get the same result.

Context matters It is harder to create a stable job in a rural area than in a city. The qualitative

and quantitative context should be provided to inform the impact as well as an understanding of how much of the problem may exist or remain.

Challenges remain There are two main challenges to measuring impact

One: how to do it well Two: how to do it cheap

Getting both to happen can be done in either of two ways:

Force it, by making it a requirement Or attract it, by creating an environment in which the parties

whose efforts are necessary to make it happen want it

1

• Shared measurement system• Provides a menu of common indicators and a common platform to report on

different outcomes and indicators

2

• Comparative performance system• Involves asking participants to report on the same indicators, using the same

process and measure to compare performance

3

• Adaptive learning system• Supports ongoing collaboration and learning among organisations to align

efforts and goals, and to measure common outcomes and indicators

The breakthrough – a Comparative Performance System

The Outcome: The Impact Investment Index

Our process for impact assessment

•Design the assessment•Conduct the assessment

•Collect the information needed about impact

• Identify and calculate the difference that was made and assess how and why it was made

•Understanding the difference/change the funder wanted to make. Who the intermediaries were that was supported and who beneficiaries was that was affected

•Share and act on learning and results

•Seek to improve the investment and development practice

4 Review

1

Plan

2

Do

3

Assess

Impact Value Chain - Our FocusHierarchy Inputs Activities and

OutputsOutputs and Outcomes

Impacts Returns

Definition Resources invested (e.g. money, skills, hours, infrastructure, services, products, etc.)

The activities whose effects are to be measured.Goods and services generated by the use of inputs (short-term)

The results from activities.Expected changes in access, usage, behaviour or performance (medium-term)

Ultimate (long-term) effect of the intervention on a key dimension of the development (e.g. living standards) (long-term)

Direct or indirect business benefits/ value generated by activities/ programs/ interventions

Quantitative Indicators

Value of investment, number of hours, number of books, number of stakeholders

Number of schools, number of volunteer, number of teachers, jobs, income, etc.

% change – access to health services, education, government infrastructure, pass rates or sustainable jobs

Students finding employment, people with skills finding employed due to increased health status

Number of graduates hired by company, % change in grievance, complaints received by company

Qualitative indicators

Stakeholders satisfaction with the programme and measurable difference in their lives, environment, context

Perceptions of schools, educators, learners, community

Beneficiaries reporting benefits and application of skills, education, improved health

Quality of links to local employment opportunities, perceptions of improved socio economic status or opportunities

Changes in community / customer/ employee perceptions attributable directly/ indirectly to CSI

The Impact Investment Index - III

Inputs• How• Cash, time, in kind,

management costs• Why• Community Relations,

community investments, commercial initiatives

• Where• Geographic Location

• Who• Stakeholders Affected,

Impacted• What • Focus Areas, Investment

Portfolios

Outputs/Outcomes• Community Benefits• Qualitative• Quantitative• Dimensions of impact

(15)

• Business Benefits• Qualitative• Quantitative• Dimensions of Impact

and Return (25)

Impacts• Community Impacts• Business Impacts/Return

• Shared Value• +500 Indicators

Impact Assessment – HOW?

Impact Assessment – WHAT?

Dimensions of Impact – FOR WHOM?

How do we calculate?1. We count each and every stakeholder group2. We count each and every impact3. We distinguish between community and business impact4. Get to a figure: X:Y

Calculating Impact – The Process

Proving Impact and ReturnTHE RESULTS

Some examples of impact (1)

ED Supplier Development Program

Supplier capacity building and access to

banking services

Local Suppliers Greater output and productivity

The economic value of increased revenue and reduced costs. The increase in local

economic output could be estimated

with multiplier analysis

Employees of Suppliers

Accumulation of human capital,

empowerment and self confidence

The cost of external skills courses could

be used. The increase in wages

resulting from greater levels of human

capital. Willingness to pay for courses

Wider Local Community

Increased welfare – i.e. lower

unemployment, improved health,

education

The value of reduction in public expenditure

on health care services and

unemployment benefits

Some examples of impact (2)

Rural electrification of village

Local businesses Greater output and productivity

The economic value of increased revenue and reduced

costs. The increase in local economic output could be estimated with multiplier

analysis

Local Students Improved quality of education, better grades

Increased wages resulting from improved education levels

Local Households Increased communications, and quality of life Monetary value of savings

Investor/Donor

Increased publicity

Increased employee morale, participation, skills development

Value of increased publicity

Value of retained employees

Some examples of impact (3)

Training of local farmers

Farmers and their families

Greater output and productivity, income and

empowerment

The economic value of increased

revenue and reduced costs.

The increase of local economic output could be estimated with

multiplier analysis

Local Authorities Increased tax revenue

The value of increased income

tax and export duties

Local Consumers and Communities

Improved quality and quantity of

food

The value of increased

consumption

The value of increased savings

The value of Increased quality of life, in particular health which lead

to increased productivity

IntermediaryIncreased income,

increased employment

The value of leveraged resources

Attraction of new resources/donors

Value of salaries

Value of publicity

How much was spent – Where?

Spent per: Year, focus area, region

Testing Perceptions

Assists in a meaningful way with rural de...

Assists win a meaningful way with skills ...

Assists with bursaries and further educati...

Assists, in a meaningful way, with educat...

Assists, in a meaningful way, with improv...

Assists, in a meaningful way, with SMME ...

Assists, in a meaningful way, with the figh...

Assists, in a meaningful way, with youth ...

Contributes, in a meaningful way, to com...

Is serious about community development ...

Provides training (learnerships, skills tr...

Supports community development projects

Supports local cultural activities

Supports local development organisation...

Supports local sports development activities

Com

mun

ity D

evel

opm

ent

0% 20% 40% 60% 80% 100%

35%

32%

36%

47%

27%

34%

24%

37%

27%

49%

24%

49%

25%

49%

35%

45%

55%

43%

37%

41%

41%

48%

46%

57%

47%

49%

50%

46%

49%

47%

19%

12%

20%

16%

33%

25%

27%

17%

14%

25%

28%

18%

Perceptions - Community Devel-opment:

Strongly Agree Agree Neutral

I have the opportunity to provide feedback into or about Nedbank’s community investment and development plans and activities

I receive enough information about Nedbank’s community investment and development plans and activities

Nedbank creates long term value for the community and country’s economy

Nedbank has a positive relationship with the community

Nedbank makes a greater contribution to the community than other financial companies in South Africa

Nedbank’s communications with the community are open and honest

Nedbank’s community projects make a positive contribution to the livelihoods of people and communities

Nedbank’s social and environmental performance is better than other financial companies in South Africa

Overa

ll Perc

eption

0% 20% 40% 60% 80%100%

23%

21%

42%

43%

28%

33%

58%

28%

37%

37%

49%

50%

37%

41%

38%

44%

22%

22%

9%

7%

35%

26%

4%

26%

15%

16%

Perceptions - Nedbank at a Glance:

Strongly Agree Agree Neutral

Disagree Strongly Disagree

Testing Needs and Recommendations

Art

s

Bur

sarie

s

Bus

ines

s D

evel

opm

ent

Chi

ldre

n

Cul

ture

Edu

catio

n

Em

erge

ncie

s an

d D

isas

ters

Ent

repr

eneu

rs

Env

ironm

ent

Hea

lth

Hou

sing

Infr

astr

uctu

re

Job

Cre

atio

n

Phy

sica

lly C

halle

nged

Saf

ety

and

Sec

urity

Ski

lls D

evel

opm

ent

The

Age

d

Wom

en

You

th

Oth

er

Community Investment Areas

0%

5%

10%

15%

1%

6%

5%

7%

2%

14%

1%

7%

6%5%

2%2%

8%

5%

1%

9%

4%4%

9%

2%

Community Investment Areas - Recommended by Stakeholders:

Ba

nk

Ch

arg

es

an

d A

cce

ssib

ility

Ca

pa

city

Bu

ildin

g -

Co

un

selli

ng

Ca

pa

city

Bu

ildin

g a

nd

Fu

nd

ing

Co

mm

un

ity E

ng

ag

em

en

tE

du

catio

nE

du

catio

n a

nd

Ba

sic

Se

rvic

es

En

viro

nm

en

tF

un

din

gH

ea

lthIn

fra

stru

ctu

re D

eve

lop

me

nt

Job

Cre

atio

nJo

b C

rea

tion

, Po

vert

y A

llevi

atio

n a

nd

Yo

u...

Lo

ng

Te

rm C

om

mu

nity

De

velo

pm

en

t Su

p...

Po

vert

y A

llevi

atio

nS

kills

De

velo

pm

en

tS

kills

De

velo

pm

en

t an

d E

du

catio

nS

kills

De

velo

pm

en

t an

d J

ob

Cre

atio

nS

kills

De

velo

pm

en

t an

d S

ma

ll B

usi

ne

ss ..

.S

kills

De

velo

pm

en

t, Jo

b C

rea

tion

an

d P

ove

r...

Sp

on

sors

hip

- A

nim

al W

elfa

reS

po

nso

rsh

ip -

Ed

uca

tion

Sp

ort

s D

eve

lop

me

nt

Yo

uth

De

velo

pm

en

t

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

4%

2%2%2%

11%

2%

4%

18%

4%

5%

2%2%2%

9%

5%

2%2%2%2%2%

15%

2%2%

Stakeholder Needs' Categories:

Testing ExpectationsCommunity Investment Areas

Focus Area Number of Responses Ranking

Education Education 91 1

Top 10

Skills Development Education 62 2

Youth Community Development 56 3

Job Creation Economic Development 52 4

Entrepreneurs Economic Development 46 5

Children Community Development 44 6

Bursaries Education 42 7

Environment Environment 40 8

Business Development Economic Development 36 9

Health Health 35 10

Physically Challenged Community Development 33 11

Bottom 10

Women Community Development 29 12

The Aged Community Development 25 13

Housing Infrastructure 16 14

Infrastructure Infrastructure 11 15

Other Other 11 16

Culture Community Development 10 17

Arts Community Development 8 18

Emergencies and Disasters Disaster Relief 5 19

Safety and Security Safety and Security 4 20

Comparing and Benchmarking Across Industry Across Focus Areas Across Perceptions Including local and global benchmarks

Vision, Mission, Strategy and Policy, guidelines, frameworks

Strategic intent, objectives, mandate Focus Areas Grant making criteria Beneficiaries Intermediaries Operational alignment and integration Governance and compliance Reporting structure Budget and resources Monitoring and evaluation Employee Volunteerism Marketing, communication and

awareness Sponsorship, donations and cause

related marketing Partnerships Recognition, Awards, perceptions Future focus, GAP and SWOT analysis

Strategic and Operational ReviewTheory of change to measure impact

Co-define and co-develop indicators to measure impact

Co-Development of impact assessment guidelines

Section One: Basic information:

Project Name: Project Budget:

  Project Owner: OrganisationProject Manager:Contact Details:

  Project Description and Specific Outcomes:

 

Section Two:

Perceptions of Nedbank:

In your opinion – what do you think Nedbank gained/ benefitted by funding this program?In other words what was Nedbank’s return on investment from funding this program?

  What do you think about or is your opinion or impressions of Nedbank as a funder?

  What do you think Nedbank can do in future to be a better funder?

 

  Do you think the Nedbank Foundation engage in a respectful, honest, approachable way?

  Do you know what the Nedbank Foundation Corporate Social Investment strategy is and do you think it is applicable in the South African context?

  Do you have any other comments or feedback or suggestions for the Nedbank Foundation?

 

Section Three:Stakeholders who benefited or were impacted by the program

Qualitative Impact Qualitative Impact Qualitative Impact Qualitative Impact Qualitative Impact Quantitative Impact

Stakeholder 1            Stakeholder 2            Stakeholder 3            Stakeholder 4            Stakeholder 5            Section Four:Specific aspects of Impact:

Economic Impact   Environmental Impact   Social Impact  

  Short Term Impact   Medium Term Impact   Long Term Impact    Positive Impact   Negative Impact   Combined Impact    Intended Impact   Unintended Impact   Any other Impact  Section Five:Lessons learnt, insights, additional comments, feedback, suggestions

           

Section Six:Consultants Comments

           

Section Seven:Scores:

SG:NF:AOI:

EC:EN:SO:

S:M:L:

P:N: C:

I:UI:OI:

T

Transparency in assessmentSection One:   Basic information:

Project Name:   Project Budget:

Leliebloem House Child and Youth Care Centre   R5,6m pa

Project Owner: Organisation  Project Manager:  Contact Details:

Leliebloem House Child and Youth Care Centre   Francisco G Cornelius  021-6964947/ 0827719233

Project Description and Specific Outcomes:

Services to Vulnerable Children in Residential Child Care.  Offering various therapeutic programmes to both the children and their families that eventually allows the children to be reunified with family or community.

Section Three:Stakeholders who benefited or were impacted by the program

Qualitative Impact Qualitative Impact Qualitative Impact Qualitative Impact Qualitative Impact

Quantitative Impact

Stakeholder 1Leliebloem House Child and Youth Care Centre  

Basic care and protection, provided. The children’s basic needs are provided for.

84 children receive basic social, health, educational and developmental services. 

Less dependent on other support (government)

Operational expenses leveraged 1

Marketing and publicity exposure1

84 Children

  Leverage funds of other donors2

Jobs provided to staff2

Children cared for – behaviour and motivation changes – basic quality of life improved2

    Number of staff Number of Jobs Value of funds leveraged / generated4

Section Four:Specific aspects of Impact:

Economic Impact It was a top up to the overall budget we need to provide these services. 1

Environmental Impact None evident0

Social Impact It gave these children a second chance they may not have received under normal circumstances. 1

  Intended Impact Yes so that we can render the services to the children.1

Unintended Impact Media exposure and recognition as an organisation with good financial practices.1

Any other Impact

Support and leverage government funds/ resources & support work of government – providing basic services to communities1

Section Five:Lessons learnt, insights, additional comments, feedback, suggestions

We have a reputation of not saying no to opportunities. We pride ourselves in staying up to date with training for all staff and trying to do staff development programs 2 a year, if the funding permits. We have dedicated staff that could leave any time because of the poor salary structures or lack of funding, but they stay on and work in the life-space of these vulnerable children on a daily basis.

Section Six:Consultants Comments

In our opinion – some consideration should be given to assisting organisations to leverage funds better – i.e. capacity building (training and other assistance) for organisations to use funds not only for operational expenses but also to raise additional funds. Some organisations will never be sustainable, but similarly organisations should not be overly dependent on a single organisation as dependencies are created and it may be considered unsustainable development.

Section Seven:Scores:

SG: 1BOE: 1QL: 8ON: 4

EC: 1EN: 0SO: 1

S: 1M:1L: 1

P: 1N: 1C: 1

I: 1UI: 1OI: 1

T:25

Scorecard per project per focus area per rating

Community Impact

Business Return

Ratings and Rankings

Impact Across PortfoliosCommunity Impact - High:

• A & C Trust (215:3)• WESSA (98:6)

• Sports Trust (84:3)• PPT Housing (80:10)• Kidshaven (78:1)

• SIFE (73:4)• Childwelfare (69:1)

Business Impact - High:• PPT Housing (80:10)

• WESSA (98:6)• Hippo Roller (63:7)

Community Impact - Low:• Wylie (62:2)

• Epworth (54:1)• St Josephs (52:2)

Business Impact - Low:• Habitat (42:2)• Lebone (34:2)

• Leliebloem (28:1)• Childline (24:1)• Princess A (24:1)• WATTF (18:0)

Community Development Average - 65 : 3

Community Impact - High:• Doctors/Bor (82:3)• Leratong (38:0)

Business Impact - High:• Doctors/Bor (82:3)

Community Impact - Low:• Ramik-Gast (29:2)• ST Lukes (28:2)

Business Impact - Low:• Reakgona (20:1)• Ramik-Home (19:1)

Health Average - 36 : 2

Community Impact - High:• MyFmyCa (104:2)• Edutainer (90:2)• Sediba (90:1)

• Fort Hare (72:2)• SEED (69:1)

• Penreach (66:1)• Enke (64:1)

• Maths NPC (62:1)• SADET (59:1)• Asset (58:1)

Business Impact - High:• Matric Rev (75:5)• Pinelands (47:4)

• Mobile Labs (39:4)• Fundisa (47:3)

Community Impact - Low:• Bursaries (54:2)

• Leap (54:2)• Protec (54:2)

Business Impact - Low:• WC Essay (53: 1)• Heartlines (51:0)• Sparrow (47:1)

• Acorn/BHT (40:1)• Dominican (31:1)• Chess Kids (27:0)

• TracSA (26:1)• Spell It (24:0)

Education Average - 56 : 2

Community Impact - High:• LearntoEarn (56:2)• Siyabonga (47:1)

• Holdinghand (40:1)• Inchanga (39:1)• Inst Deaf (38:1)

Business Impact - High:• Siyazisa (31:3)

• Cape Winem (35:2)• RayMhalaba (30:2)

Community Impact - Low:• Readucate (24:1)• FurnTech (23:0)

Business Impact - Low:• Readucate (24:1)• FurnTech (23:0)

Skills Development Average - 36 : 1

Collective impact and return

Impact and return score cards

Ranking and Rating per impact

Docto

rs/B

or

Hippo

Roller

Mat

ric R

ev

Childw

elfar

eSIF

E

Sparro

w

0

2

4

6 54 4

3 3 3

Unintended

Lear

ntoE

arn

WC E

ssay

A & C

Tru

st

Childw

elfar

e

Epwor

th

Holding

hand

Mat

ric R

ev0

1

2

3

44

3 3 3

2 2 2 2 2 2 2 2 2 2

Short Term

WESSA PPT Housing A & C Trust0

1

2

3

4

5

6

76

5

4

Environment

WESSA

A & C

Tru

st

Lear

ntoE

arn

Sediba

0

2

4

6

8 7

54 4

Positive

Fundisa Matric Rev WESSA0

1

2

3

4

5

65

4

3

Negative

A & C

Tru

st

Holding

hand

Mat

hs N

PC

PPT Hou

sing

0

4

8

12

16

17

7 75

Economic

Business Return Community Impact

Some examples of indicators

Increased quality of life – 45 dimensions Increased social cohesion Increased access to services, infrastructure,

support, income, jobs – 45 dimensions Skills Development – 30 dimensions Infrastructure – transport, hospitals, schools,

electricity, water, housing – 60 dimensions Education – infrastructure, in general, specific –

early childhood, primary, secondary, tertiary Economic, Environmental, Social Health, education, housing, safety & security,

enterprise development, sport, agriculture, social justice, social participation, technology

Impact on individuals, communities, government, companies, funding partners

Quantitative, qualitative Volunteerism - dimensions

Mitigate climate change risk Reduce environmental accidents and remediation Reduce water use and management; waste

management and effluents Reduce and assist with energy management GHG emissions and air pollution Reduce biodiversity impacts Increase communications and engagement Increase customer satisfaction Increase access to tenders Increased access to new markets Increased reputation Increased sales Increased staff motivation Increased BEE Scorecard Strengthened value chain Maintained/obtained social licence to operate Increased product innovation and services

development

The impact of the impact assessment

Improved financial analysis and reporting as well as management capacity and operational efficiency

Improved operational efficiency and information management processes and systems

Improved board governance and oversight, understanding, knowledge and capacity

Improved M&E; reporting; communication; stakeholder relationships and partnerships

Improved operational processes from application requirements; due diligence processes; reporting requirements; responsiveness; knowledge and expertise of staff

Improved access to further investment

What our clients say - investors

Intermediaries We feel comfortable with the transparency of the process The process have added value to our own work – especially M&E

and reporting practices The processes have increased our effectiveness and own

performance; increased our learning and knowledge; built internal capacity; and increased our credibility

We believe we were assured independently by someone who can verify our claims – it validated our own beliefs

Beneficiaries We had an opportunity to talk without being judged – we could be

honest We learnt to document our own work and the contribution we

made We feel we are being trusted, being heard and someone asks our

opinion We had an opportunity to share and learn

What our clients say – intermediaries and beneficiaries

What we have learnt (1) One size of evaluation does not fit all

Funders should take extra time in their planning to learn which evaluation techniques will work with indigenous populations and specific communities.

Trying to define and measure empirical changes is difficult and “a slippery process,” Understanding, defining, qualifying and quantifying long-term social

change is an incremental effort and on-going process.

Although evaluations are significant for their organisational development, requiring intermediaries to perform them is a challenge. A suggested solution is to consider evaluation results as one – but

not the only – source of information and to couple it with knowledge, experience, strategy, and context to obtain a fuller picture.

What we have learnt (2) Impact assessments describes three relationships:

between evaluation and the funder’s approach; between evaluation and the funder’s strategy; and between the intermediary evaluation and the beneficiary evaluation.

Grantmakers may have to change their funding approach to better accommodate intermediaries’ capacity to conduct evaluations, or provide funding to help them develop it

Funders need to be clear about their overall goals and how individual investments fit within that model.

Intermediary and funder evaluations should be linked: Both partners face the same issues of inadequate resources,

coordination and expertise for conducting evaluations.

What we have learnt (3) ANY resources CAN be measured – books, wheelchairs,

buildings, time – cash and non-cash The same project can deliver varied results for different

funders How and on what you spend the money (inside the program)

has a direct influence on the impact and return The strategy and focus areas has to clearly define the return

and impact required - upfront Sustainability has to be clearly defined for exit and completion Indicators have to be developed, agreed, and documented

as part of the contractual phase Internal monitoring and external evaluation processes has to be

established and adhered too - Impact Assessment does not replace evaluation and monitoring

What we have learnt (4) You have to consider the impact of the impact assessment

on so many levels As a result of our work we can now categorically state that most

programs: Have only short term impact Those that have medium term impact are not necessarily

sustainable The long term impact is mostly social only as opposed to

economic impact which really contributes to poverty alleviation and eradication!

It is possible to determine impact and return The real value lies in independent, verifiable, assurance of social

investment expenditure, program results, outcomes and impact

There is so much return for business – we just need to prove it

What I have learnt (5)

Growth

New products

New markets

New customers

Innovation

Reputation

Differentiation

Return on Capital

Price premium /

market share

Capital efficiency

Human efficiency

Risk Management

Regulatory risk

Reputational risk

Licence to operate

Supply chain /security of

supply

Management Quality

Leadership

Employee development

Adaptability

Long term strategic view

What I now know… We all have impact – but it is not necessarily measurable and

sustainable impact Do we want economic or social impact? By implication social impact help people right now – but may not

help them in the future – which renders the project/our intervention UNSUSTAINABLE

Sometimes it is our own (CSI Practitioner’s) fault we don’t have higher impact as we decide what, who and how to fund/not to fund

The most sustainable projects/programs with the highest impact have social, socio economic and ECONOMIC impacts i.e. the number of jobs created

Sometimes there is negative impact – i.e. dependencies are created Mostly there is only short term impact –which makes our interventions

UNSUSTAINABLE

Moving from Strategic to Catalytic

Responsive Strategic Catalytic

Catalytic change for the good and

survival of mankind

Strategic to core

business

Responsive to society

Going Forward and Doing it better Impact assessment can help funders, intermediaries, and

beneficiaries they support to: Plan how their work will make a difference, and determine how

much of a difference they are making Understand what does or does not work and why and detect

unintended consequences Build an (scientific) evidence base to share with others, thus

influencing and informing debate, and increasing the sector’s body of knowledge

Challenge yourself and others by looking critically at your/their work in order to improve, to replicate good work, or to innovate and develop new processes, products and services

Inspire and motivate staff, trustees and other stakeholders including volunteers, beneficiaries, policy makers and other practitioners, funders and investors to build relationships with others, communicate added value and raise the profile of their work

In closing Involve stakeholders

Establish the scope and identify the key stakeholders impacted

Understand what changes Map the outcomes – identify the indicators

to measure impact Value things that matter

Look for evidence, don’t forget baseline and impact studies

Only include what is material Focus on agreed outcomes first then

incidental impact Do not over claim

Calculate the impact based on evidence Be transparent

Report the outcome – the good and BAD news

Verify the result Share the learning

Community Impact

Business Return

If you can count and define

indicators you can determine

impact and return

Questions and Discussion

How to start Where to start How to do it When to do it

Just do it

Contact Reana Rossouw Next Generation Consultants - Specialists in Development E-mail: rrossouw@nextgeneration.co.za Web: www.nextgeneration.co.za

PLEASE NOTE: THIS PRESENTATION IS PART OF A LARGER BODY OF RESEARCH! THIS INFORMATION IS COPYWRITED AND THE INTELLECTUAL PROPERTY OF NEXT GENERATION CONSULTANTS.