libqual+ ® : an introduction bern, switzerland july 9, 2007 presented by: martha kyrillidou...
TRANSCRIPT
LibQUAL+LibQUAL+®®: An Introduction: An Introduction
Bern, SwitzerlandJuly 9, 2007
PRESENTED BY:
Martha KyrillidouAssociation of Research Libraries
old.libqual.org
old.libqual.org
OverviewOverview
• Introduction and Overview
• LibQUAL+® in the UK
• Research Foundations – Qualitative Process
– Quantitative Evidence
• The Survey Process
• General Discussion and Q&A
old.libqual.org
old.libqual.org
old.libqual.org
World World LibQUALLibQUAL++®® Survey Survey
old.libqual.org
Rapid Growth in Other AreasRapid Growth in Other Areas
• Languages– American English– British English– Chinese (2007)– French– Dutch– Swedish– Norwegian– Finnish– Danish
• Consortia– Each may create 5 local questions
to add to their survey
• Countries– Australia, Canada, Denmark,
Egypt, Finland, France, Hong Kong, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S.
• Types of Institutions– Academic Health Sciences– Academic Law– Academic Military– College or University– Community College– Electronic– European Business– Family History– FFRDC – High School (2007)– Hospital– National Health Service England– Natural Resources– New York Public– Public– Smithsonian– State– University/TAFE
old.libqual.org
250
218
307308
204
164
13
43
152,111
176,360
151,460
113,480
78,863
4,407
20,416
128,958
0
50
100
150
200
250
300
350
2000 2001 2002 2003 2004 2005 2006 2007
Survey Year
Number ofInstitutions
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
180,000
200,000
Number ofResponses
Number of Institutions
Number of Responses
LibQUALLibQUAL++®® Participants Participants
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® First Year Participants First Year Participants
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® Surveys by Type Surveys by Type Years
2000 2001 2002 2003 2004 2005 2006
Academic law 1 25 10 6
Academic military 6 1
College or university 13 41 111 244 150 201 219
Community college 16 29 3 15 27
Electronic 1
European business 5 16
Family history library 1 2
FFRDC 5 1
Health sciences library 1 35 23 13 13 9
Hospital 10 1 1
National Health Service Eng. 10
Natural Resources 4
New York Public library 1
Public 4 1 1 3
Smithsonian 1 1
State 1 1 3
University/TAFE 2 1
old.libqual.org
LibQUALLibQUAL++®® Languages Languages
American English
Dutch EnglishFrench Canadian DutchSwedish
Swedish(British English)
Afrikaans
DanishFinnishGerman Norwegian
British English
Continental French
old.libqual.org
Surveys by Session: 2004-2006Surveys by Session: 2004-2006
Year Session 1 Session 2
2004 202 2
2005 199 56
2006 205 93
old.libqual.org
Survey methods used in the UK
West, 2004A Survey of Surveys
27
18
13
11
4
22
6
Libra
LibQUAL+
In-House
SPSS
SNAP
Perception
Excel
Others
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
The UK approach
• Coordinated on behalf of the Society of College, National & University Libraries (SCONUL) Working Group on Performance Improvement (WGPI)
• 2003 - 20 UK Higher Education (HE) institutions • 2004 -17 UK & Irish HE institutions • 2005 - 16 UK & Irish HE institutions • 2006 – 20 UK & Irish HE institutions• 2007 – 22 UK & Irish HE institutions• 62 different institutions
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
Response Comparisons
• SCONUL 2003– 20 institutions – 11,919 respondents
• SCONUL 2004 – 16 institutions– 16,611 respondents
• Increase by 4,692• SCONUL 2005
– 16 institutions– 17,355 respondents
• Increase by 744• SCONUL 2006
– 20 institutions– 19,108 respondents
• Increase by 1,753
• LibQUAL+ 2003– 308 institutions– 128,958 respondents
• LibQUAL+ 2004– 202 institutions– 112,551 respondents
• Decrease by 16,407• LibQUAL+ 2005
– 199 institutions– 108,504 respondents
• Decrease by 4,047• LibQUAL+ 2006
– 298 institutions– 176,360 respondents
• Increase by 67,856
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
SCONUL Response by User Group 2006
old.libqual.org
SCONUL Response by Discipline 2006
old.libqual.org
Respondent Comparisons
• Glasgow University
– 2006 = 1,535
– 2005 = 1,384
– 2004 = 2,178
– 2003 = 503
• London South Bank University
– 2006 = 700
– 2005 = 766
– 2004 = 568
– 2003 = 276Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
General findings
• Highly desired– Making electronic resources accessible from
my home or office– Print and/or electronic journals I require for
my work– A haven for study, learning or research
• Lowest – Library staff who instil confidence in users– Giving users individual attention– Space for group learning and group study
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
Free Text Comments Received
• Total number of comments 2005 = 8,368
• Total number of comments 2004 = 8,161
• Total number of comments 2003 = 7,342
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
Why LibQUAL+?
• Benchmarking• Cost effectiveness• Analysis compiled by LibQUAL+• Fast delivery of results• Support available, especially regarding analysis
of results • Trialling alternative survey methods• More library focused than previous in-house
method• Planned institutional survey failed to happen.
LibQUAL+ was cost effective way of doing something to fill the gap.
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
Research Foundations
old.libqual.org
Total Circulation
Note. M. Kyrillidou and M. Young. (2003).ARL Statistics 2002-03. Washington, D.C.: ARL, p.8.
old.libqual.org
Reference Transactions
Note. M. Kyrillidou and M. Young. (2003).ARL Statistics 2002-03. Washington, D.C.: ARL, p.8.
old.libqual.org
AssessmentAssessment
“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.”
Sarah Pritchard, Library Trends, 1996
old.libqual.org
Multiple MethodsMultiple Methodsof Listening to Customersof Listening to Customers• Transactional surveys*• Mystery shopping• New, declining, and lost-customer surveys• Focus group interviews• Customer advisory panels• Service reviews• Customer complaint, comment, and inquiry capture• Total market surveys*• Employee field reporting• Employee surveys• Service operating data capture
*A SERVQUAL-type instrument is most suitable for these methods
Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).
Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.
old.libqual.org
Premises
Three Seminal Quotations
old.libqual.org
PERCEPTIONS SERVICE “….only customers judge quality;
all other judgments are essentially
irrelevant”
Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.
LibQUAL+™ Premise #1
old.libqual.org
LibQUAL+™ Premise #2
“Il est plus nécessaire d'étudier
les hommes que les livres”
—FRANÇOIS DE LA ROCHEFOUCAULD
old.libqual.org
“We only care about the things we measure.”
--Bruce Thompson, CASLIN, 2006
LibQUAL+™ Premise #3
old.libqual.org
13 LibrariesEnglish LibQUAL+™ Version
4000 Respondents
QUAL
QUAN
QUAL
QUAL
QUAN
QUAL
PURPOSE DATA ANALYSIS PRODUCT/RESULTDescribe library environment;build theory of library service quality from user perspective
Test LibQUAL+™ instrument
Refine theoryof service quality
Refine LibQUAL+™ instrument
Test LibQUAL+™ instrument
Refine theory
Unstructured interviewsat 8 ARL institutions
Web-delivered survey
Unstructured interviews at Health Sciences and the Smithsonian libraries
E-mail to surveyadministrators
Web-delivered survey
Focus groups
Content analysis:(cards & Atlas TI)
Reliability/validityanalyses: CronbachsAlpha, factor analysis,SEM, descriptive statistics
Content analysis
Content analysis
Reliability/validity analyses including Cronbachs Alpha,factor analysis, SEM, descriptive statistics
Content analysis
VignetteRe-tooling
Iterative
Emergent2000
2004315 Libraries English, Dutch, Swedish,
German LibQUAL+™ Versions160,000 anticipated respondents
LibQUAL+LibQUAL+®® Project Project
Case studies1
Valid LibQUAL+™ protocol
Scalable process
Enhanced understanding of user-centered views of service quality in the library environment2
Cultural perspective3
Refined survey delivery process and theory of service quality4
Refined LibQUAL+™ instrument5
Local contextual understanding of LibQUAL+™ survey responses6
old.libqual.org
DimensionsDimensions
2000 2001 2002 2003-200741 items 56 items 25 items 22 items
Affect of Service Affect of Service Service Affect Service Affect
Library as Place Library as Place Library as Place Library as Place
Reliability Reliability Personal ControlInformation Control
Provision of Physical Collections
Self-RelianceInformation Access
Access to Information
Access to Information
old.libqual.org
Survey Structure Survey Structure (Detail View)(Detail View)
old.libqual.org
Interpreting Service Quality Data
Three Interpretation Frameworks
old.libqual.org
Benchmarking Against Peer Institutions
--1,000,000 Users; 1,000 Institutions!
NORMS! NORMS! NORMS!
Interpretation Framework #1
old.libqual.org
Score Norms
• Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.
• LibQUAL+™ norms have been created at both the individual and institutional level
old.libqual.org
Benchmarking Against Self, Longitudinally
“Nobody is more like me than me!”--Anonymous
Interpretation Framework #2
old.libqual.org
Interpreting Perceived Scores Against Minimally-Acceptable and Desired
Service Levels (i.e., “Zones of Tolerance”)
Interpretation Framework #3
old.libqual.org
old.libqual.org
old.libqual.org
“22 Items and The Box….”
Why the Box is so Important– About 40% of participants provide open-
ended comments, and these are linked to demographics and quantitative data.
– Users elaborate the details of their concerns.
– Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.
old.libqual.org
“…and Five Ancillary Items”
Either Zero or Five Ancillary items are selected to address local or consortial concerns
– Items from the initial LibQUAL+TM item pool.
– Items written by previous consortial groups.
old.libqual.org
Qualitative Grounding
old.libqual.org
Premise for Mixed-Methods
• “The underlying premise of mixed-method inquiry is that each paradigm offers a meaningful and legitimate way of knowing and understanding” (p. 7).
Note. Greene, J.C. and Caracelli, V. J. (Eds.). (1997). Advances in mixed-method valuation. San Francisco, CA: Jossey-Bass.
old.libqual.org
LibQUAL+ Process
SERVQUAL dimensions served as a priori theoretical starting point
old.libqual.org
• York University• University of Arizona• Arizona State• University of
Connecticut• University of Houston• University of Kansas
University of Minnesota University of Pennsylvania University of Washington Smithsonian Northwestern Medical
76 Interviews Conducted
old.libqual.orgLoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred
old.libqual.org
old.libqual.org
Dimensions ofLibrary Service Quality
Empathy
Ubiquity and Ease of Access
Comprehensive Collections
Reliability
Responsiveness
Symbol
Utilitarian space
Assurance
Formats
Timely access to resources
Physical location
Self-reliance Library as Place
LibraryServiceQuality
Model 1
?
Refuge
Affect of Service
old.libqual.org
“You put a search on a book and it’s just gone; it’s not reacquired. … There’s more of a problem of lost books, of books that are gone and nobody knows why and nobody’s doing anything about it.”
Faculty member
Reliability
old.libqual.org
“I want to be treated with respect. I want you to be courteous, to look like you know what you are doing and enjoy what you are doing. … Don’t get into personal conversations when I am at the desk.”
Faculty member
Affect of Service
old.libqual.org
“Over time my own library use has become increasingly electronic. So that the amount of time I actually spend in the library is getting smaller and the amount of time I spend at my desk on the web … is increasing.”
Faculty member
Ubiquity of Access
old.libqual.org
“I think one of the things I love about academic life in the United States is that as a culture…, we tend to appreciate the extraordinary importance of libraries in the life of the mind.”
Faculty member
Comprehensive Collections
old.libqual.org
“One of the cherished rituals is going up the steps and through the gorgeous doors of the library and heading up to the fifth floor to my study. … I have my books and I have six million volumes downstairs that are readily available to me in an open stack library.”
Faculty member
Library as Place
old.libqual.org
“I guess you’d call them satisfiers. As long as they are not negatives, they won’t be much of a factor. If they are negatives, they are a big factor.”
Faculty member
Library as Place
old.libqual.org
“The poorer your situation, the more you need the public spaces to work in. When I was an undergraduate, I spent most of my time in the library, just using it as a study space.”
Faculty member
Library as Place
old.libqual.org
“…first of all, I would turn to the best search engines that are out there. That’s not a person so much as an entity. In this sense, librarians are search engines [ just ] with a different interface.”
Faculty member
Self-reliance
old.libqual.org
“By habit, I usually try to be self-sufficient. And I’ve found that I am actually fairly proficient. I usually find what I’m looking for eventually. So I personally tend to ask a librarian only as a last resort.”
Graduate student
Self-reliance
old.libqual.org
Dimensions ofLibrary Service Quality
Affect of Service
Empathy
Access to Information
Personal Control
Responsiveness
Symbol
Utilitarian space
Assurance
Scope
Timeliness
Convenience
Library as Place
Refuge
Modern Equipment
Convenience
Ease of Navigation
Reliability
LibraryServiceQuality
Model 2
old.libqual.org
Dimensions of LibraryDimensions of LibraryService QualityService Quality
old.libqual.org
Core Items and DimensionsCore Items and Dimensions
22 core items (i.e., questions)
Three dimensions:
• Affect of Service – 9 questions
• Information Control – 8 questions
• Library as Place – 5 questions
old.libqual.org
Understanding a Radar ChartUnderstanding a Radar Chart
old.libqual.org
Key to Radar ChartsKey to Radar Charts
old.libqual.org
Radar Chart BasicsRadar Chart Basics
old.libqual.org
Understanding Thermometer ChartsUnderstanding Thermometer Charts
old.libqual.org
Dimension SummaryDimension Summary
old.libqual.org
Library Use Question (Unified)Library Use Question (Unified)
old.libqual.org
2006 2006 LibQUAL+LibQUAL+®® Highlights: Highlights: OverallOverall
old.libqual.org
Qualitative Analysis: Qualitative Analysis: User CommentsUser Comments• About one-half of users include comments on their
surveys
• User Comments available on the LibQUAL+ ® Web site– Download comments in Excel or text file
• Skim the comments
• Conduct Atlas.ti analysis
old.libqual.org
General Discussion General Discussion and Q&Aand Q&A
old.libqual.org
Summary and ClosureSummary and Closure
old.libqual.org
LibQUAL+LibQUAL+®® Resources Resources
• LibQUAL+® Web site:http://old.libqual.org
• Publications:http://old.libqual.org/publications
• Events and Training:http://old.libqual.org/events
• Gap Theory/Radar Graph Introduction:http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Procedures Manual:http://old.libqual.org/Publications/index.cfm
old.libqual.org
Contact Contact LibQUAL+LibQUAL+®® • Martha Kyrillidou
Director, Statistics and Service Quality [email protected]
• MaShana DavisTechnical Communications [email protected]
• Kristina JusthCustomer Relations [email protected]
• Gary RoebuckTechnical Operations [email protected]
Selena Lock, Research and Development, Cranfield University