economic impact toolkits for archives, museums and libraries

Economic Impact Toolkits for Archives, Museums and

LibrariesALMA UK

1st December 2010

Oliver Allies & Jamie Buttrick

Introduction

Our Brief To analyse economic impact methodologies

for the sector Assess the pros and cons and thinking behind

each method Recommend methods for the creation of

toolkits, suitable for application across a range of organisations

Recommend an outline process for piloting these

Our Approach Extensive desk based review of economic

impact research (reviewed c.60 research documents)

Consultation with (19) key stakeholders from each representative sector to discuss feasibility of data capture

Progress session – (today) Workshop session with ALMA Working Group 1st

December

Economic Impact Definition

“Examines the economic effect of an activity on a specific geographical area”

OriginSporadic usage in 1980s (although said to have emerged in 1960s – John Galbraith)More commonplace in the 1990s linked to appraisal and evaluation in the public sector (HM Treasury Green Book first published in 1991)Widespread usage over the last 10 years linked to the need to identify “additionality” and return on investment (English Partnerships Additionality Guide 2001, Impact Evaluation Framework, BERR 2008)

Economic Impact Approaches Many different types of impact assessment.

Three techniques are most commonplace for the assessment of economic impact: Multipliers – calculating expenditure impacts and the

“multiplier effects” of these Contingent Valuation – (willingness to pay/willingness

to accept) Return on Investment – broadly combining the two

approaches above to generate a “ratio”

Number of Contingent Valuation Studies Identified Worldwide to Mid 2002 (Morris et al 2002)

0

1

2

3

4

5

6

7

8

9

YearNumber of Studies

Economic Impact Research in the Cultural Sector Analysed in this Study

0

2

4

6

8

10

12

14

16

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

YearResearch Studies

Economic Impact Approaches

Multiplier Seeks to capture and map expenditure

related impacts from an organisation over a given area/areas Procurement spend Spend on employees Visitor related spend

Utilises Keynesian Multipliers to assess the “indirect” and “induced” effect of this spend

Economic Impact Approaches - Multiplier

Organisation

Procurement

Wages

Visitor Spend

Secondary expenditure impacts (the multiplier)

Economic Impact Approaches - Multiplier

Pros “Relatively” straightforward and resource efficient “Benchmarks” available and considered appropriate Most widely used and recognised impact approach

Cons Relatively narrow in focus – tendency to overlook

social impacts Most suited to those activities that offer an economic

return Additionality and attribution aspects can cause

confusion – particularly visitor spend

Economic Impact Approaches

Contingent Valuation More widely associated with an assessment

of the public’s “willingness to pay” (WTP) or “willingness to accept” (WTA)

WTP - Surveys of users and non-users of services to assess what value they would pay for those services (if they were free)

WTA – the level of compensation an individual (user and non) is willing to accept for the loss of a service/good

Popular in the United States and used by the British Library

Economic Impact Approaches – Contingent Valuation

Pros A means to capture value placed on non-marketed

goods and services An effective way of capturing “intrinsic value” Could utilise benchmarks (derived from pilot or

targeted analysis) for aggregation Cons

Demands extensive survey consultation carefully worded to avoid confusion

Is a challenging concept for the public to grasp in relation to both libraries and archives in particular

The challenge of capturing non-user value

Economic Impact Approaches

Return on Investment Offers a combination of contingent valuation

(CV) and multiplier techniques Seeks to provide a ratio of return per pound

invested (e.g. £2.60 return for every £1 invested)

Is increasingly utilised to assess government spend (particularly RDA investment)

Has driven the desire to capture economic values for all activity delivered…emergence of Social Return on Investment (SROI)

Economic Impact Approaches – Return on Investment

Pros Enables the greatest breadth of activity to be captured Is equipped (where possible) to provide values

associated with social activities (through the adoption of SROI techniques)

A focus on user value would overcome some confusion

Cons Can demand extensive resources Due to diversity of socially related activity it would not

be possible to capture social returns through a toolkit approach

Demands training and knowledge of SROI

Assessing the Impact of Socially Oriented Activities

Danger of overlooking diversity of activities delivered within the sector

Capturing social value increasingly popular over the last 5-10 years

Emerged within this sector e.g. Generic Learning Outcomes and Generic Social Outcomes

Challenge of capturing this data in an effective, robust and consistent manner… e.g. 22 different approaches identified (NEF 2005)

Social Audit and Accounting and Social Return on Investment (SROI) most prominent

Stakeholder Feedback

Broadly, greater understanding and experience of impact assessment in museums sector than in libraries/archives

In general information related to the multiplier approach was not deemed too onerous

General recognition of the need for impact assessment and willingness to participate

Consensus of the need to know what to collect, when and why

Stakeholder Feedback

Require clarity on what value they will gain from the exercise

Resource (time and personnel) main barrier to participation

Alignment to frameworks or measurements or performance measurements will boost participation

Represented a broad and extremely diverse spectrum of social activity

Implications of Findings

Clarity on what are the goals Advocacy? Benchmarking performance?

Clarity on what are the returns Fine balance of an organisation or individual’s

return on investment from participating in this research

Consistency on what messages are provided

Implications of Findings – Toolkit Potential?

Is the development of a “toolkit” and its roll-out a feasible prospect?

Does a one-size fits all approach seem possible?

What type of “toolkits” are already being used?

Toolkit Review

Few toolkits exist with a specific economic impact remit

Toolkits as guides AIM Toolkit (DC Research) Contingent Valuation Toolkit (JURA)

Toolkits as “tools” Scottish Enterprise Additionality toolkit

For effective aggregation a combined guide and toolkit is required

Needs to be simple and straightforward but robust

Toolkit Options

Format Electronic favoured by stakeholders (albeit

with scope to provide hard copy) Could enable aggregation of impact yet

provide individual organisations with data (a “hook”)

Ownership and support challenges

Toolkits – Initial Proposals

Impact Approach – Museums/Archives Multiplier methodology for assessment of

Museums and Archives is fit for purpose

Options of Approach (resource implications) Low - Hard copy guide Medium - Provide a electronic/hardcopy

guide and toolkit High - Provide online resource for

download/upload of returns

Toolkits – Initial Proposals – Museums and Archives

“Medium” resource approach (ERS preferred approach) as: Facilitates organisational impact Scope for aggregation Utilise existing data where possible to minimise

resource investment (PSQG and Gift Aid for example) Potential agreement for required primary research

Piloting Suggest testing internally prior to initial roll-out Clarity of messages and promotion Managed roll-out and testing with “early adopters”

Toolkits – Initial Proposals

Impact Approach – Libraries Multiplier – whilst possible, in danger of underplaying

scale of impact Within the United States ROI increasingly utilised –

based on UK research (Economic Value of Public Libraries in the UK – Morris et al 2002)

ROI approach in US now standardised and available as an online calculator - http://www.chelmsfordlibrary.org/library_info/calculator.html

ROI approach in the UK would incorporate elements of the Museum/Archive tool (procurement and employment) alongside user value of services – derived from a survey)

Toolkits – Initial Proposals – Libraries

Proposed Option ROI – user values combined with procurement and

employment expenditure

Approach Design survey through ALMA UK to capture range of

library services that demand a user-value Survey of xxxx library users to obtain perceived value

of services Establish Calculator tool to supplement multiplier tool

to establish return on investment.

Questions?

Oliver Allies 07725 672068

oallies@ers.org.uk www.ers.org.uk