Library Outcomes: The Holy Grail

Joe MatthewsAugust 2012

Resources Outcomes

Broadcast Model

We do They utilize

Fundamental

In the information age, it is not what you know,but what you can find; genius will be measured by the speed at which you can find things.

Anonymous

Libraries are so …

1. Measure success

2. Measurement of Value

3. Innovate

4. Communicate

Define & Measure Success

Measurement of Value

Logic Model

INPUTS OUTPUTS OUTCOMES

What we invest in

What we doWho we reach

What results are achieved?

If - Then

Logic Model for Libraries

Library receives a budget to support the goals of the community

Library converts the inputs to useful collections & services

Customers use library’s physical & virtual collections & services

Individual benefits in some way due to the use of its collections & services

Community or organization benefits in some way

So that …

So that …

So that …

So that …

Inputs

Outcomes

Outputs

Process

Benefits orImpacts

Outcomes - Impact

Learning Awareness Knowledge Attitudes Skills Opinions Aspirations Motivations

Conditions Performance

Social

Economic

Environmental

Action Behavior Practice Decision- making Policies Social Action

Perspectives on Value

Benefits

Use

Nonuse

Direct

Indirect

Option – Preservation of option forfuture use by me

Existence – Perceived value and significance to the community

Legacy – Value of preservation forfuture generations

Personal

Organizational

Financial

Impacts

Personal

Organizational

Define, develop, and measure outcomes

that contribute to

institutional effectiveness

ACRL Standards for Libraries in Higher Education

If the physical proximity of

print collections had a demonstrable impact on researcher productivity,

no university would hesitate to allocate prime real estate

to library stacks.

How to Demonstrate Impact in …

Student• Enrollment• Achievement• Learning• Retention & graduation• Experiences• Career success

Faculty• Research productivity• Grants• Teaching

Institutional Reputation

Student Enrollment

Student Learning

University of Wollongong• Data into the Library Cube

• The Library Cube provides the information needed to support continuous improvement in three areas: collection development; academic relationships; and marketing.

• The Library has seen a positive correlation between borrowing activity and academic performance

University of Minnesota

Gym Bags and Mortarboards

Use Campus Recreational Facilities

At least 25 times, first-year retention increased 1%

& 5-year graduation rates

increased 2%

University of Minnesota

How scalable is library instruction?

Student Retention & Graduation

Attrition

Retention

Persistence

Completion

Graduation Rates

Australasian Survey of StudentEngagement (AUSSE)

University of Huddersfield

Dropping out!

Library Retention Studies

University of Minnesota Libraries

– 77% of undergrads made use of the libraries, 85% of grad students made use of the libraries

– Students who used the library at least once were 1.54 times more likely to re-enroll

This is important!

• Helps tie-in with institutional goals and objectives

• Helps to integrate the library with other departments & faculties

• May open increased funding opportunities

• Helps better serve the needs of our users, students

Student Career SuccessGrad School Exams

Time to First Job

Average Salary of First Job

Alumni Surveys

Alumni Giving

Ithaka Studies

• Library services not understood

• Library services not valued

• The Library is disappearing

Financial

University of Illinois, Urbana-Champaign

Connected citations to resources in the library’s collection to successful grant proposals, and the income the grants generated

ROI = 4.38:1Other studies – 0.27:1 to 15.54:1

• Comprehensive assessment of the library• ROI of the journal collection & readership• ROI for support of teaching & learning• ROI of digitized special collections• ROI of eBooks• Value of library commons

ROI instruments and calculations

do not work for academic libraries, and present naive and misinterpretedassessments of our roles and impacts

at our institutions and acrosshigher education.

James Neal

Be cautious about cause-and-effect relationships

Problems with Library Assessment

• Some studies correlate library use and retention, but no causative links

• Strength of correlations is weak – at best

• Problem with almost all existing library research are the small sample size

• Only a handful of research has been done and a lot of it is old

Collaboration

Layers of Data

Library DataCirculation, Logins, Downloads, Reference, Instruction, …

Demographic DataCollege, Level, Major, Gender, Ethnicity, Age, …

Performance DataCumulative GPA, ACT score, CLA score, ….

Other University-wide DataStudent Surveys, Faculty Surveys, Alumni Surveys, …

PrivacyUse student IDs to match records from

one data set to another and then

Strip the student ID number from thecombined record

Work with Office of Institutional Research &

the campus Institutional Review Board

(Rooney-Browne, 2009b).

Social Benefits

• Basic reading literacy

• Summer reading “gap”

• Business/career

• Information literacy

• Library as place

• Local history & genealogy

• Health & well-being

• Social cohesion

• General information

• Welcoming newcomers

Tracking ValueThe Engaged Library: Stories of Community Building

•Prove that public libraries build social capital

•Identify & connect the library’s assets to the community

•Assess & strengthen the library’s connections with and use of community assets

•Produce a toolkit for other libraries to adopt

•Mapping tools to perform an inventory services, identify areas for improvement and highlight library’s contribution to the community’s wider social, educational, cultural and economic goals.

Measurement Of Value

Cost

Use

High value

Moderate valueLow value

Little value

¢

¢

¢

¢

¢

¢

¢

Quality

Innovation

Nature of Information is Changing

Scare, controlled

Expensive

Shaped by elites

One-way, mass consumption

Slow moving

External to our worlds

All around us

Cheap or free

Shaped by consumers

Designed for sharing, participation & feedback

Immediate

Embedded to our worlds

Information was ….

Information is ….

Liberate Knowledge

Atoms to bits

Online Participation

Inactives

Spectators

Joiners

Critics

Conversationalists

Collectors

Creators

0 10 20 30 40 50 60 70 80

Expertise & influence shifts to networks

Share the stage with amateur experts

Where?

What?

How?

What for?

Social networks

Social content

Social publishing

Social shopping

Social gaming

Social search

Social impact Social currency

All products and services will be

SOCIAL

Over 303 Millionresources

Over 52,000 corrections in one day

Over 980,000 dishes transcribed

National Library of the Netherlands

Contextualization

Co-Curati

on

Where is the value?

Groups – Canadian History389

membersOntario3,677

members

843 results

65 images – most relevant

Niagara-on-the-Lake Public Library

494 results

Notice all the hyperlinks

18,100 results

Crowd Funding

User Profile

Be o

pen

Seed

com

mun

ity p

roce

sses

Devo

lve

resp

onsib

ilitie

s

Nature of Library is Changing

Lending

Distribution

Informing

Artifacts

Access

Sharing

Contribution

Conversing

Knowledge

Participation

Library was ….

Library is ….

Building Shared Valuefor a Shared Future

Communication

Stories + Stats = Success

MeasureValue

Innovate

Communicate

Thanks

Joe@JoeMatthews.Org

www.joematthews.org

Joe MatthewsLibrary Consultant