ir 056 304 kinnell, margaret; garrod, pennydocument resume ed 405 875 ir 056 304 author kinnell,...

DOCUMENT RESUME

ED 405 875 IR 056 304

AUTHOR Kinnell, Margaret; Garrod, PennyTITLE Benchmarking and Its Relevance to the Library and

Information Sector. Interim Findings of "BestPractice Benchmarking in the Library and InformationSector," a British Library Research and DevelopmentDepartment Project.

PUB DATE [95]

NOTE 15p.; Paper presented at the NorthumbriaInternational Conference on Performance Measurementin Libraries and Information Services (1st,Northumberland, England, August 30-September 4,1995).

PUB TYPE Reports Descriptive (141) Speeches/ConferencePapers (150)

EDRS PRICE MFO1 /PCO1 Plus Postage.DESCRIPTORS Academic Libraries; Business; Evaluation Methods;

Foreign Countries; Higher Education; *InformationServices; *Libraries; Library Policy; LibrarySurveys; *Organizational Development; Private Sector;Program Effectiveness; *Quality Control; ResearchMethodology; *Total Quality Management; UserSatisfaction (Information)

IDENTIFIERS Barriers to Implementation; *Benchmarking; BritishLibrary (England); European Quality Award; QualityIndicators; Service Quality

ABSTRACTThis British Library Research and Development

Department study assesses current activities and attitudes towardquality management in library and information services (LIS) in theacademic sector as well as the commercial/industrial sector.Definitions and types of benchmarking are described, and therelevance of benchmarking to LIS is evaluated. Study methodology andinterim findings of the ongoing project are detailed. A survey foundthat the 537. of respondents have no formal policy on quality. Whilebenchmarking is used by only 7% of respondents, 81% utilize userfeedback is utilized to measure and evaluate performance. Examples ofcurrent practice reveal the problems faced by LIS in both theacademic and commercial sectors. Benchmarking is a "quality" toolwhich should form part of an overall quality program aimed atimproving services. Quality management is considered beneficial tothe library and information sector, but a model suiting the exactneeds of the sector has not yet been identified. Appendices includestudy methodology; definitions; procedure; types of benchmarking;relevance; graphical representation of survey responses; how qualitypolicies are communicated to LIS staff; the European Model for TotalQuality; and future publication information. (Contains 12references.) (SWC)

************************************************************************ Reproductions supplied by EDRS are the best that can be made

from the original document.***********************************************************************

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.

Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

Benchmarking and its Relevanceto the Library & Information Sector

by Margaret Kinnell

and

Penny Garrod

BEST COPY AVAILABLE

2

"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY

Margaret Kinnell

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC).-

SEMINAR PAPER

Benchmarking and its Relevance to the Libraryand Information Sector

Interim findings of 'Best Practice Benchmarking in the Library andInformation Sector', a British Library Research and

Development Department project

Margaret Kinnell and Penny GarrodDepartment of Information and Library Studies, Loughborough University

Abstract

This paper details the interim findings of a one-year British Library Research and Development

Department research project entitled: Best PracticeBenchmarking in the Library and Information Sec-

Definitions and types of benchmarking aredescribed, and the relevance of benchmarkingto Library and Information Services (LIS), is evalu-ated.

The aim of the project is to assess current activi-ties and attitudes to quality management in LIS inthe academic sector, and in a sample taken from thecommercial/industrial sector. Benchmarking is seenas one of a range of 'quality' tools which might beof practical use to the LIS sector. Benchmarkingtechniques are being tested so that their relevanceand utility to the information sector can be assessed.

The methodologies used throughout the researchproject, and the interim findings to date, aredetailed. The questionnaire survey found that themajority of respondents have no written formal pol-icy on quality, whilst benchmarking is used by 7%of respondents. However, user feedback is a popularmethod of measuring and evaluating performance,with 81% of respondents claiming to use it. Exam-ples of current practice, gathered at the follow-uptelephone interview phase of the project, reveal theproblems faced by LIS, in both the academic andcommercial sectors.

The paper concludes that benchmarking is a`quality' tool which should form part of an overallquality programme, aimed at improving services.Quality management is considered to be beneficialto the library and information sector, but a modelwhich is in harmony with the needs of the sector hasnot yet been identified. The self-assessment model,as developed by the British Quality Foundation(BQF) and the European Foundation for QualityManagement (EFQM), has been identified as themost appropriate model for the information sector at

the present time, although further research would benecessary before it could be considered for adop-tion.

Introduction

The concept of benchmarking has become signif-icant for library and information services (LIS)

with the growing recognition that quality manage-ment principles and practices provide the means todelivery of effective services in the current climateof increased accountability and dwindling resources(Mullen, 1993; Lawes, 1993).

Successful companies have demonstrated thatbenchmarking can be a useful tool for implementingquality management in the equally demanding oper-ating environment of the business sector. RankXerox is the company most commonly associatedwith the successful use of benchmarking techniques;in 1979 they introduced benchmarking to establishhow their competitors had achieved success, andthen emulated them. According to one leadingauthority on quality, benchmarking is now widelycredited with being one of the main factors respon-sible for improvements in company performance(Zairi and Hutton, 1995). It is against this back-ground of intensifying use of benchmarking activityin the commercial world that benchmarldng for LISis being considered.

The Project

Loughborough University of Technology is engagedon a one-year project for the British LibraryResearch and Development Department (BLRD&D)to investigate the viability of benchmarking tech-niques for the library and information sector. Theproject began in November 1994 and ends inDecember 1995. The team comprises: MargaretKinnell Evans, as Project Director, Penny Garrod,as Researcher; together with John Brockman, Quali-ty Manager at the Ministry of Defence HeadquartersLibrary, London, and Alan Gilchrist of Gavel Con-

159

3

Kinnell and Garrod: Benchmarking and its Relevance

sultancy. The latter are respectively the Secretaryand Chair of the Quality Issues special interestgroup of the International Federation for Informa-tion and Documentation (FID/QI). FID will soon bepublishing, jointly with BLRD&D, a selectbibliography of quality management items compiledby the Loughborough benchmarking team.

The aim of the project is to assess the levels ofquality management activities within the sector,with special reference to academic and special LIS.Benchmarking is one of a range of approachesavailable, with attitudes to quality management ingeneral, and to benchmarking in particular, alsobeing investigated. Barriers to change, which maybe limiting current levels of activity on qualityissues in the library and information sector, arebeing identified, whilst the beneficial effects ofimplementing quality programmes are being high-lighted as a way of overcoming the problems of lim-ited resources and customer dissatisfaction. Theproject is complemented by the Quality Manage-ment and Public Libraries study, also funded byBLRD&D, and jointly being undertaken bySheffield and Loughborough Universities. Thedata collection for this project is designed to pm-vide some basis for comparing quality managementpractices across the whole of the LIS sector.

Methodology

a) SURVEYS

A questionnaire survey was undertaken to establishcurrent levels of quality related activities in thelibrary and information sector. All library and infor-mation services in the higher education sector, and asample of 197 information units from the commer-cial/industrial sector, were surveyed. The overalltarget group was 511 library and information ser-vices. Respondents to the questionnaire were askedto state whether they were prepared to take part in afollow-up telephone interview. A sample of thosewho agreed to this has been interviewed to date.

b) LITERATURE SEARCHING

Literature searching has been an ongoing processfor the duration of the project. New itemson quality related issues are constantly being pub-lished, as the topic is highly dynamic and subject toconstant review and criticism. Literature searchinghas so far concentrated on identifying items forinclusion in the select bibliography, which is soon tobe published, and surveying previous research in thefield to supply data for an interim position paper.Searching in the area of quality management is

made difficult by the proliferation of terms used eg.performance measurement/performance indicators;quality management/total quality management;quality systems/quality standards etc. This abun-dance of jargon-laden terminology, and varied defi-nitions, represents one of the major barriers to theimplementation of quality programmes.

Coopers and Lybrand have carried out two sur-veys into the use of benchmarking: one in theUnited Kingdom in 1993, and one in Europe in1994 (Coopers . . , 1993, 1994). The first of theserevealed that out of 105 UK respondents, compris-ing directors drawn from the Times Top 1,000 list ofmanufacturing and service companies, 67% claimedcurrently to be benchmarking. The second surveyinvolved major companies from The Netherlands,Switzerland, Spain, France and the UK. Once again,benchmarking was used as a management tool by72% of respondents; for the UK alone the figurewas 78%. More importantly, 86% of UK respon-dents in the European survey stated that they hadlearned lessons from benchmarking activities. Theseactivities are important if the concept of 'TheLearning Organisation' is to become a reality. Oneof the frequently reported benefits of benchmarkingis its ability to heighten awareness of internalprocesses and communications; this in turn facili-tates identification of areas where improvementscan be made.

However, benchmarking is a difficult concept toput into practice, especially for those in theservice sector where there are no tangible products,and there are many variables, which renders com-parisons difficult. Benchmarking is now beingreferred to in government literature as a 'challenge'(Department . . , 1995). Whilst the government, theConfederation of British Industries (CBI), and otherinfluential bodies exhort all organisations, irrespec-tive of size and industry, to adopt benchmarkingtechniques, others are cautioning that it is not aneasy task, and that it cannot be achieved overnight(Bullivant, 1994, ch.1 p.53). Smaller organisationswhich have tried benchmarking have yet to confirmwhether it has been successful or not. However,they do affirm that it has been a useful learningexercise.

A 1992 report from the USA sees benchmarkingas benefiting only the higher performingorganisations (Ernst and Young . . , 1993). Theauthors of the American study relate the failure ofbenchmarking in lower and medium performingorganisations to their choice of inappropriate mod-els for comparison. These models tend to be the`best of the best,' whose practices are ineffective

160


when adopted by lower performing organisations.Instead, the authors suggest that lower performersshould concentrate their resources on their 'coreinfrastructure', rather than waste them on what theyterm 'sophisticated practices'. The extent to whichthe findings of this study has relevance for the LISsector has yet to be tested, although the present pro-ject will consider them. Identifying 'higher' and`lower' performing organisations in LIS is problem-atic, given the lack of objective criteria for success.However, certain factors can be seen as significantto ensure that LIS are compared on equal terms.Size and annual budget are two of the more obvi-ous, as is the experience of quality management inthe organisation. It seems unnecessary, for example,to point out that it would be unwise for a small HEcollege library, with no quality programme in place,to attempt a comparison with a large, universitylibrary, where total quality management (TQM) hadbeen implemented some time ago.

c) 'DEMONSTRATOR' PROJECTS

Three 'demonstrator' projects have been set up,consisting of library and information units whichhave volunteered to undertake a benchmarking exer-cise for the project. These organisations represent across-section of the academic, public and commer-cial sectors. Each 'demonstrator' project has select-ed a key process for benchmarking purposes; theseare processes which they perceive to be essential tothe success of their particular unit or organisation.Benchmarking partners are in process of being iden-tified and contacted, as the next phase of the processis entered.

The aim of this element in the methodology is toput benchmarking into practice, and therebyto assess its relevance to the library and informationsector. As well as providing valuable data for thebenchmarking project, the exercise should providean insight into the type of problems which LIS mayencounter, when trying to implement benchmarkingtechniques. The organisations involved will reporton any learning experiences they have had, andwhether the methods used were appropriate for theirparticular organisational culture. A subsequentanalysis of these data will enable the benchmarkingteam to establish the viability of benchmarking forthe library and information sector, and evaluate howit can best be adapted to this environment.

DefinitionsThere are many definitions and several types ofbenchmarking. Definitions range from the gen-

eral to the specific, and many of them originate

from well-known authorities or 'gurus' on qualitymanagement, such as Robert Camp and John Bulli-vant (Camp, 1989 and Bullivant,1994). However, adefinition which is relevant and meaningful to thelibrary and information sector is needed. As far asthe various types of benchmarking are concerned,one of the aims of this project is to evaluate thesemodels, in order to identify those which are bestsuited to the LIS sector. Two key concepts, whichshould be kept in mind when grappling with themany definitions, are: Measurement and Compari-son. The term 'benchmark' has its origins in indus-trial practice, and is synonymous with inspectionand tangible products. However, it is now widelyused to mean anything taken as a point of referenceor comparison.

The 'demonstrator' organisations are currently inprocess of testing benchmarking techniques for theproject using the concepts of measurement andcomparison in tangible ways. Their experiences willprovide valuable data on the practicalities of bench-marking for the LIS sector.

The following example provides a concreteexample of the way in which benchmarking canbe applied to a library and information service. Thedemonstrator projects are basing their activities onthese procedures:

Checklist of procedures

- Identifying a process which is critical to the suc-cess of the library and information service (LIS)eg. an enquiry or interlibrary loans service.

- Documenting or mapping the sub-processes,which are carried out as part of this process.

- Taking measurements of those factors which aredeemed critical to the success of the process, eg.the speed of document delivery, or the relevanceof a response to an enquiry.

- Analysing the results of this exercise.

- Choosing and then visiting benchmarking part-ners, in order to compare the results with otherorganisations using a similar process.

- Identifying 'best practice', ie. methods used bybenchmarking partners which can be adopted inorder to improve one's own level of service.

Types of Benchmarking

Vive main categories of benchmarking have been'cited in the literature:

1. Competitor - comparing with leading organisa-tions with similar products or services and

161


adopting their approach. This method is suitedto the with-profit sector, but the Department ofTrade and Industry (DTI) also recommend itsuse by organisations providing the same ser-vices, but where there is no competition on acommercial basis, eg. NHS trusts, governmentdepartments, and universities (Department . . ,

March 1995. p.5).

2. Generic - comparisons of business processes orfunctions that are very similar, regardless ofindustry (Oakland, 1994, p.182).

3. Internal a comparison of internal operationsby different departments within the same organi-sation (ibid., p.181).

4. Functional - comparisons to similar functionswithin the same broad industry, or to industryleaders (ibid., p.182).

5. Customer - the aim of the improvement pro-gramme is meeting and exceeding customerexpectations.

The DTI has recently suggested this latter conceptof customer benchmarking, which may be of partic-ular relevance to the library and information sector.The DTI state:

`The benchmark is customer expectations. Cus-tomers develop their own benchmarks of perfor-mance when selecting and judging suppliers. Theimprovement programme is aimed at meeting andexceeding customer expectation.' (Department . . ,

March 1995 p.5)

It is already common practice in the LIS sector touse customer feedback and customer satisfactionsurveys to measure the quality of service provision.

The findings of this project have revealed thesemethods to be widespread. However, it must beacknowledged that customer expectations can beunrealistic, and responses to them have to be tem-pered by constraints on resources. Despite thesereservations, it would be worthwhile assessing thedevelopment of customer benchmarking in thelibrary and information sector. Many LIS alreadyuse a range of techniques to obtain feedback fromtheir users, and then take action to implementchanges where possible. Customer benchmarkingwould merely formalise these activities, and couldestablish optimum levels of service on the lines ofcharters already in use in many public library ser-vices. These may or may not be considered desir-able more generally to information services andtheir customers in the academic and special libraryfields.

Relevance

THE PUBLIC SECTOR

The application of benchmarking to the public sec-tor, and in particular to service environments, hasnow been acknowledged in the literature and inpractice. The DTI argues that benchmarking is notexclusive to international corporations, but that:`many of the techniques transfer well to smallerbusinesses, the health sector, service organisationsand government departments' (ibid., p.2). However,examples of the use of benchmarking are mostlytaken from large, successful companies, despite thereiteration of its application to smaller organisa-tions. Smaller organisations may be deterred eitherby a lack of resources to expend on quality manage-ment techniques, or require hard evidence todemonstrate the utility of benchmarking to theiroperations.

The National Health Service has taken a lead rolein promoting the use of benchmarking in the publicsector through the NHS Benchmarking ReferenceCentre based in Wales. Its Director, John Bullivant,has stated that there are many models of bench-marking models from which to choose (1994, ch.3,p.81) and that it is better to adapt the approach tosuit the needs of the individual organisation, ratherthan reinvent benchmarking. He advocates learningfrom the experiences of others, and the use ofbenchmarking clubs and benchmarking visits (ibid.,ch.3, p.85). Benchmarking is seen to be part of anoverall quality programme as one method amongmany, which organisations may consider when look-ing at ways of improving performance.

ACADEMIC LIS

Throughout the literature, benchmarking is referredto as a means of improving an organisation's com-petitiveness. LIS managers in the academic sectormay find this emphasis alien to their culture andwork practices, and view themselves as remote fromthe world of commerce. This is understandable forthose who grew up viewing education, libraries andknowledge as fundamental to a civilised nation, towhich everyone, irrespective of income and status,has a right, free of charge at the point of delivery.

Times change and this ideal has largely disap-peared. Education and information are now seenas tradable commodities - especially now thatdemand is high and funding is geared to researchratings and numbers of students. Obtaining fundsfrom the Higher Education Funding Council forEngland (HEFCE) is highly competitive, and insti-tutions are having to compete to attract students and

162

6

Kinnell and Garrod. Benchmarking and its Relevance

resources. LIS have been identified as playing afundamental role in the provision of quality educa-tion in two surveys discussed in the Follett Report(Joint . . , 1993 p.37). They may wish to improveexisting services by focusing on internal processes,with the aim of identifying and eliminating non-value-adding procedures. These may have been inplace for many years; often they entail unnecessaryduplication of paper-based transactions to satisfyinternal, departmental requirements. These process-es have frequently 'always been done this way', butthe original purpose for carrying out the procedurehas long disappeared. Benchmarking offers a way offocusing on essential processes and eliminatingunnecessary tasks, through comparison with otherorganisations. Benchmarking is essentially a learn-ing experience. Participants reappraise their internaloperations, so that all activities are firmly focusedon the customer, and learn through an exchange ofinformation with other organisations.

INDUSTRIAL / COMMERCIAL LIS

Benchmarking is of particular value to LIS in theindustrial/commercial sector, as it builds on the suc-cess of others to improve performance. Benchmark-ing focuses on areas which are vital to the successof the organisation, and therefore has a positiveimpact on performance. This is critical if companiesare to succeed in today's competitive marketplace.Fewer complaints and more satisfied customers canhelp establish a good reputation, and in the case ofan LIS, it can raise its profile in the organisation.Those companies which value information as anessential resource will want to ensure that theirinformation service is applying best practice.

Project Outcomes and Disseminationof Findings

The findings for the project will be fullydescribed and evaluated in a final report to

BLRD&D. In the interim period there will be vari-ous publications, which are currently in process, orare anticipated:

SELECT BIBLIOGRAPHY OF QUALITYMANAGEMENT ITEMS

The first of these, a select bibliography, will soon bejointly published by FID and BLRD&D. This com-prises around 200 annotated items under a range ofquality related headings, and is aimed at busylibrary and information services managers interestedin a compact guide to the literature. There is nowsuch a wealth of literature on quality issues, that a

bibliography which selected from this vast poolseemed essential to support managers in their imple-mentation of quality management techniques. Itemsincluded range from books on general managementto journal articles in the information field.

FURTHER PROJECT DOCUMENTATION

A position paper is currently being drafted, whichoutlines the current status of quality managementfor all sectors. This also identifies networkingopportunities for library and information managersinterested in quality related issues. The final resultsof the project will be also be disseminated throughjournal articles and further conference presenta-tions.

Interim Project Findings to Date: theQuestionnaire Survey

(i) AIMS AND OBJECTIVES

The aim of the questionnaire survey was to establishthe levels of implementation of quality managementprogrammes in the library and information sector,and to evaluate which techniques and methodolo-gies were currently being used. The questionnaireinstrument was designed to complement that of theQuality Management and Public Libraries study, inorder to ensure comparability of data collection.

(ii) RESPONSE RATE

The overall response rate to the questionnaire sur-vey, which was addressed to heads of LIS, was56.5%. For the academic sector it was 73%, whilstfor the commercial sector it was 30%, although 28additional questionnaires were returned uncomplet-ed from commercial organisations.

The comments appended to various sections ofthe questionnaire provided the most illuminatingdata on attitudes to quality issues and barriers tochange. These were all extrapolated and recordedfor reference and analysis.

(iii) FINDINGS

1. Written Policy on quality at organisational levelThe first question aimed to establish how manyorganisations had written policies on quality. Inboth sectors (academic and commercial), 33% ofrespondents stated that they did have a writtenpolicy at organisational level, whilst 53% had nowritten policy. The remainder was made up oforganisations in process of preparing a writtenpolicy (12%) and those who failed to respond tothe question. There was little variation betweenthe two sectors. Almost 26% of commercial sector

163


respondents had a written policy on quality, and46% had no policy. Eleven per cent claimed tohave a written policy in preparation.

2. Written policy on quality at library and informa-tion services levelWe then sought to establish whether a written pol-icy on quality existed at library and informationservice level. Here, the level of those answeringpositively that LIS aid have a written policy was12% for both sectors combined. Those without awritten policy totalled 74%. In the commercialsector alone 10% had a written policy on quality,revealing that once again levels of implementationfor both sectors were similar.

3. Formal Quality programme in placeWhen asked if there was a formal quality pro-gramme in place only 14% answered in the affir-mative, although a larger number - 19% - statedthat a programme was being prepared.

4. Quality initiativesWe also asked managers if they were involved in arange of quality initiatives. These were: TQM;Quality Circles; BS 5750; Customer Contracts;The Learning Organization; Investors in People(I1P), or any other unspecified methods. Investorsin People proved the most popular initiative with19% of respondents claiming to use it. Total quali-ty management had been implemented by 13% ofrespondents, and Customer Contracts by around12.5% of respondents.

5. Management initiativesRespondents were also asked whether specificmanagement initiatives had been introduced intotheir LIS. They were presented with seven possi-ble initiatives: Management by Function; Man-agement Information Systems; Cost Centres; Flat-ter Management Structures, Performance Indica-tors; Staff Appraisal and Team Working. StaffAppraisal and Team Working proved to be themost commonly used approaches, at 61% and53% respectively.

6. TrainingOf equal importance is specific training in qualityfor staff at all levels. However, 72.5% of LIS hadno training programmes in place. Only 18% ofrespondents had training in quality for middlemanagers and paraprofessionals; whilst 14%offered training at senior manager level.

7. Measurement and evaluation of performanceOf particular relevance to this project was the useof benchmarking techniques. Benchmarking,along with usage statistics, performance indica-

tors, user feedback and cross-charging wereoffered as methods which might currently be usedto measure and evaluate performance in libraryand information services. Just over zze claimed tobe benchmarking (19 LIS in the academic sector,and one future benchmarker in the commercialsector, where they planned to introduce bench-marking in 1996). User feedback was the mostwidely used measure at 81%, closely followed byusage statistics at almost 79%.

8. Methods of communicating policies to staffThe responses to this question are significant, aseffective communication is essential to the suc-cess of any organisation, especially if the organi-sation is embarking on a quality programme. Poorcommunications are frequently the cause of thefailure of management initiatives. Respondentswere asked which methods, from a list of nine,they used to communicate policies to LIS staff.Staff meetings were the most used method ofcommunication (71% of respondents). 'On the jobtraining' was the second most used method (54%),followed by appraisal schemes and managers`walking the floor', each used by 40% of respon-dents.

Telephone interviews

Anumber of follow-up telephone interviews hasbeen conducted to date to probe the findings

further. These were divided equally between LIS inthe commercial sector and the academic sector.They have provided insight into individual practiceand cultures, and have served to highlight problemswhich hinder organisations trying to implementquality programmes.

THE ACADEMIC SECTOR: SIGNIFICANT FINDINGS

Institution A - Benchmarking was considered to beof relevance to the LIS sector by this interviewee.Informal benchmarking is carried out through analy-sis of the annual reports and reviews of other uni-versities, and through contacts. Formalising proce-dures was viewed as a natural progression. A barrierto implementation was considered to be lack oftime, and problems associated with the transitionfrom old methods to new quality approaches. Thelatter problem required cultural change, which wasdeemed difficult as there were many long-servingmembers of staff (referred to as 'platform perform-ers'), who were opposed to change, so that the insti-tution could make little progress until these peopleretired or left. A new Vice-Chancellor was about totake up appointment, with the expectation that this

164


would effect change. However, the LIS was viewedas being well-placed, with representation on all the`right' committees, and it had support from highermanagement.

Institution B the interviewee at this institution alsofelt that benchmarking was relevant to the LIS, andexpressed a keen interest in 'evaluating others'.Their current practice was to arrange visits andexchanges of staff with other institutions. The pur-pose of this is to gain knowledge of how a hostorganisation works. They had a formal twinningarrangement with a counterpart in the USA, andplanned to exploit the expertise of the US institutionin an area which was new to them. This institutionhad both a mission statement and a five-year strate-gic plan, and perceived themselves as beingadvanced in their use of performance indicators anduser satisfaction surveys. The interviewee was verypositive about looking outside of the LIS communi-ty for ideas. He stated that they had looked at theexample of Marks and Spencer when deciding toclose the library for an hour on one morning a week,in order to carry out staff training. Prior to this theyhad thought such actions were inappropriate, andwould be opposed by users. He also expressed theview that the LIS sector was already very advanced,and that the progress which had been made, forexample in the introduction of IT, had been underes-timated.

Institution C - Benchmarking was judged, by theinterviewee, to be relevant to the LIS sector with theproviso that the term meant 'finding best practiceand then emulating and comparing with this'. If, onthe other hand, benchmarking was closely tied tostandards and prescriptive then it was judged to beof no relevance or use to the LIS sector. Perfor-mance indicators, the interviewee felt, had beeneffective in generating standards in the past. How-ever, performance indicators needed to be tailoredto local aims and objectives, so that similaritieswere highlighted. This would overcome the com-mon reaction of the LIS profession - which was toclaim their organisation was 'different', which pre-cluded comparison.

THE COMMERCIAL SECTOR: SIGNIFICANTFINDINGS

There was found to be a diversity of cultures andpractices in the sample organisations interviewed inthe commercial sector. Most interviewees consid-ered benchmarking to be relevant to the LIS sector.Some expressed reservations, for example that itought to be relevant, but might not be practical, as

9

considerable time and resources needed to be allo-cated for it to be implemented. Several expressedthe view that the aims of benchmarking partnersshould be similar, if organisations had different pur-poses then their processes would differ, and com-parisons would be difficult. Specific processes, eg.document delivery and journal circulation, wereperceived as being best suited to benchmarkingexercises, as they were easy to monitor.

Organisation A - This organisation had both a writ-ten policy on quality and a formal quality pro-gramme, and had disbanded its corporate centralisedlibrary. The one remaining library professional hadbeen given a new job title, which reflected her new`support' and 'secretarial' status. Holdings had beendispersed to individual departments, according totheir relevance to the work of that department. In-house qualified librarians were no longer employed,and information requests were outsourced to infor-mation brokers.

Organisation B - The LIS in this organisation antici-pated the doubling of the level of enquiries - cur-rently around 10,000 a year - with only one newmember of staff being appointed to help deal withthe increase. They were acting as a pilot project fora Department of Trade and Industry (DTI) scheme,known as 'Technical Transfer Services'. This pro-ject involves extending the organisation's high pro-file enquiry service, with access to a collection oftechnical information resources, to non-members.The aim of this initiative is to provide small andmedium-sized enterprises (SMEs) with informationon those examples of best practice which wouldenable them to improve their services and products.The interviewee expressed concern regarding theimpact this initiative could have on quality. Theorganisation's 'bottom line' was to view all non-members as potential members, a view withwhich the interviewee agreed. However, enquiriesoutside the expertise of the organisation were antici-pated, which would require referral elsewhere. Thiscould impact on the reputation of the informationservice, if referrals proved unsatisfactory. This LIShas well-documented, formalised procedures, for arange of services, as well as a quality manual. Theyhave been involved in BS 5750 and Investors inPeople (IIP) initiatives. However, there is now agenuine concern for the quality and reputation ofthe service with the development of a new elementin service delivery, which would fall outside thedirect control of the information service.

Organisation C The LIS in this organisation can becategorised as a 'special' library. Here the organisa-

165


tion was undergoing what was termed 'soft privati-sation'. This involved plans to merge the organisa-tion with a university. Original plans for markettesting and service level agreements had been jetti-soned, due to the uncertainty of the future of theunit. Redundancies were anticipated following themerger. The interviewee stressed that the organisa-tion was driven by financial concerns alone; no con-sideration was given to customer satisfaction. Heexpressed the view that teleworking might be adopt-ed to save costs; one worker was already working inFrance and communicating with the home-base byelectronic mail. He also added that many organisa-tions in the commercial sector were only planningsix months ahead. It can be inferred that this is part-ly due to the recession, and to insecurity about thefuture. Libraries, he suggested, were commonly per-ceived to be too expensive to maintain, as well astaking up expensive space. They were thereforeviewed as prime candidates for cost-cutting exercis-es.

Organisation D - An interviewee from this organisa-tion mentioned the familiar 'what's in it for me'question, which is often posed by managers, whenevaluating new management initiatives. She felt thatbenchmarking, and quality programmes in general,had to have some easily identifiable and quantifi-able gain attached to them, in order to justify thetime and resources which would need to be allocat-ed to them.

Conclusions

(i) BENCHMARKING MODELS FOR THE LIBRARYAND INFORMATION SECTOR

The approach which has been adopted by the LIStaking part in the benchmarking exercise for thisproject is a bottom-up approach. Processes, whichhave been identified as critical to the success of theservice, have been selected for measurement andcomparison with other LIS - irrespective of sector.The two functions chosen are interlibrary loans andenquiry services. This approach is commensuratewith the comments made by interviewees thatprocesses which are easy to monitor and measureshould be chosen for benchmarking exercises. Theresults of this part of the study have not yet beenanalysed. The data are now being collected and the`demonstrator' library and information serviceshave yet to report on their experiences.

Customer benchmarking would also be worthevaluating for use by the information sector.Libraries already use customer feedback and usersurveys to a large extent. It might, therefore, be fea-

sible and cost-effective to set benchmarks based onthis feedback, with the proviso that the benchmarkbe both realistic and attainable after improvementshave been effected. Where user expectations wereunreasonably high, adjustments would have to bemade to the benchmark, and the reasons fullyexplained to users.

(ii) BENCHMARKING AND QUALITYMANAGEMENT

It is thought unlikely that benchmarking could beimplemented by any LIS where there was no qualityprogramme in place. Benchmarldng is a 'quality'tool, and the aim of this project has been to evaluateit within the context of quality management.

Quality management has been judged to be bene-ficial to the library and information sector by manylibrary and information managers, by academics andby managers in the commercial sector. However, noone `right' approach or model has yet been identi-fied. LIS serve a variety of communities and organi-sations. Each has its own culture, problems, andresource limitations. A quality model which is inharmony with the needs of most LIS is required -a considerable difficulty, given that LIS are so var-ied. It needs to be flexible and easily adapted to dif-ferent sectors. It needs to focus on the human inter-faces which are so vital to the success of most LIS.The self-assessment model, as developed by theBritish Quality Foundation (BQF) and the EuropeanFoundation for quality management, (EFQM) seemsto be the most appropriate model at the present time(British . . , 1994 p.8). This focuses largely on the`people' element of organisations and thus seemswell suited to the service environment, and to avariety of organisational settings. The BQF/EFQMmodel is shown in Appendix 14.

(iii) TRAINING

The importance of training cannot be overestimated.Managers need to be both informed and committedfor a quality programme to be successful. They arethen able to form teams, empower process owners,and motivate their staff. Front-line staff, in particu-lar, need to be motivated and convinced of the valueof quality methodologies. Training in customer careis as vital as knowledge of the information service.The image and reputation of the library is linked tothe customer's first impressions, and it is front-linestaff who constantly interact with the customer.

Training in quality tools and techniques is impor-tant for all levels of staff. They need to understandwhy certain procedures are in place, and the valueof taking a broad view of the service as a whole.

166

10


Those who deal with one process day in and day outtend to become isolated and lose contact with theoverall workings of the unit. Quality managementinvolves meetings, staff involvement, team work,and cross-functional co-operation. Above all, itrequires excellent communications at all levels. Byraising staff awareness of the impact of their contri-bution to the effectiveness of the service, theybecome more motivated and less isolated. The ques-tionnaire highlighted these issues. LIS with qualityprogrammes in place often listed the benefits ofheightened staff awareness of their role and input tothe organisation.

Whilst the project is yet to be completed, and thedata from the demonstrator projects are not yetavailable for analysis, some tentative overall con-clusions have been possible.

(iv) OVERALL CONCLUSIONS

1. Benchmarking can only be effective in those LISwhich already have a commitment to qualitymanagement, and have begun to implement quali-ty management practices.

2. Many LIS still have some way to go before quali-ty management - as practised in the commercialsector - can be implemented. There appears to belittle commitment to quality management evidentamongst many of the managers surveyed.

3. However, there is, paradoxically one might argue,evidence of continuing interest in measurementand comparison, the key elements in benchmark-ing. Performance measurement and the use ofinformal comparisons, together with the continu-ing use of comparative measures by fenders, havebeen growing in significance in recent years. Theuse of SCONUL and COPOL performance mea-sures, and the emphasis on accountability - tousers as well as fenders - have been important inshifting the perceptions of LIS managers.There appears now to be the need for a furthershift to encompass the more holistic approval of aquality management programme. This wouldenable LIS to support more fully the range ofquality management techniques, particularly bestpractice and other forms of benchmarking.

References

British Quality Foundation (1994) Towards organiza-tional excellence. London: BQF

Bullivant, J. (1994) Benchmarking for continuousimprovement in the public sector. Harlow: Long-man

Camp, R. (1989) Benchmarking. The search for indus-try best practices that lead to superiorperformance. Milwaukee: ASQC Quality Press

Coopers and Lybrand Survey of benchmarking inEurope 1994

Coopers and Lybrand and the Confederation of BritishIndustry (January 1993) Survey of benchmarking inthe UK. Coopers and Lybrand/CBI

Department of Trade and Industry (March 1995)Benchmarking the challenge. A practical guide tobusiness improvement. London: DTI

Ernst and Young and the American Quality Founda-tion (1993) Best practices report. An analysis ofmanagement practices that impact performance.New York: Ernst and Young. p.38

Joint Funding Councils' Libraries Review Group:Report. (1993) The Follett Report. Bristol: HigherEducation Funding Council for England

Lawes, A. (1993) 'The benefits of quality managementto the library and information services profession'.Special libraries 84(3) Summer. 142-146

Mullen, J. (1993) 'Total quality management: A mind-set and method to stimulate change'. Journal oflibrary administration, 18(3-4) 91-108

Oakland, J. (1994) Total quality management. Oxford:Butterworth-Heinemann

Zairi, M. and Hutton, R. (1995) `Benchmarking: aprocess-driven tool for quality improvement'. TheTQM magazine, 7(3) p.35

AppendicesContent of OHPs used at the Conferencepresentation.

Appendix 1

Methodology- Surveys: questionnaire

telephone interviews

- Literature searching

- Demonstrator projects

1.1 167

Appendix 2

Definitions

- Measurement

- Comparison

Kinnell and Garrod:

Appendix 3

Checklist of Procedures

- Identify process

- Document / map sub-processes

- Measure critical success factors

- Analyse results

Choose benchmarking partners

- Identify 'best practice'

Benchmarking and its Relevance

Appendix 4

Types of Benchmarking

- Competitive competitor to competitor compar-isons for product/function ofinterest

- Generic similar business processes andfunctions regardless of industry

- Internal internal operations of differentdepartments within sameorganisation

- Functional similar functions in same broadindustry/industry leaders

(Oakland, J. Total quality management (1994) Oxford: Butter-worth-Heinemann. 181-182)

- Customer the benchmark is customerexpectations

(Department of Trade & Industry (March 1995) Benchmarkingthe challenge. A practical guide to business improvement. p.5)

Appendix 5

Relevance

- Public sector

- Academic sector

- Commercial / Industrial sector

12

Appendix 6Question: Does your organisation have a writtenpolicy on quality?

Written Policy on Quality at OrganisationalLevel

12% inPreparation

2% No Response

53%No

.I . .. . . .e 0 I I',`,`,*,`,, 33% Yes -. . . . ..

N.. e

'4.ee, Policy Exists. . . . . I' -

II

% % . % % .1%II e I I e I

Appendix 7Question: Does your organisation have a writtenpolicy on quality?

Written Policy on Quality at Library andInformation Services Level

1.5% No Response

13% inPreparation

12% Yes - Policy inPlace

Appendix 8Question: Do you hove a formal quality programmein place?

Formal Quality Programme in Place

168

1% No Response14% Yes - Programme

in Place

19% inPreparation

.

e .

vb. z 4 i niI l I I'M ktil. Af!ll OS

_ ' a if

. . e

I

5

I II

ON.. VG. ... a.ovve....1.a .... vv. -Appendix 11Question: Are there training programmes in qualityfor staff?

Staff Training in Quality

14% SeniorManagers

18% MiddleManagers

18% Paraprofs

Appendix 12

Question: Which of the following methods does your LIS use to measure and/or evaluate performance?

Measures used to Measure and Evaluate Performance

90

80

70

60

50

40

30

20

10

0

Methods used to Measure and Evaluate Performanceie

UsageStatistics

79%

PerformanceIndicators

UserFeedback

81%

CrossCharging

Benchmarkingrn

14170

1

1

1

1

1

1

1


Appendix 13

Communication - How Policies areCommunicated to LIS Staff

Staff Meetings

- On Job Training

- Appraisal Scheme

- Managers -`Walking the floor'

- Training Courses

- Mission Statement

- Team Briefings

- Bulletins / Newsletters

- Quality Groups

71%

54%

40.5%

40%

37.5%

35.5%

30.5%

30%

9%

Appendix 15

Future Publication

Quality Management Issues: a SelectBibliography for Library and InformationServices Managers

FID Occasional Paper 10British Library R&D Report 6220

Appendix 14

UK / European Model for Total Quality

Compiled by Penny GarrodMargaret Kinnell Evans

FID: The Hague 1995ISBN 92 66 00 7102[Not yet published]

Leadership10%

PeopleManagement

9%

Processes14%

PeopleSatisfaction

9%

BusinessResults

15%

Policy & Strategy8%

CustomerSatisfaction

20%

Resources Impact on Society9% 6%

Results (50%)

British Quality Foundation: Towards organisational excellence. London: British Quality Foundation 1994. p.8

15

171

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

Educational Resources Intimation Center (ERIC)

REPRODUCTION RELEASE(Specific Document)

I. DOCUMENT IDENTIFICATION:

Title:Benchmarking and its relevance to the Library and Information sector

Author(s): Professor Margaret Kinnell Evans

Corporate Source: Publication Date:

II. REPRODUCTION RELEASE:In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced

in the monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproducedpaper copy, and electronic/optical media, and sold through the ERIC Document Reproduction Service (EDRS) or other ERIC vendors. Credit isgiven to the source of each document, and, if reproduction release is granted, one of the following notices is affixed to the document.

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following two options and sign atthe bottom of the page.

El

Check hereFor Level 1 Release:Permitting reproduction inmicrofiche (4' x 6" film) orother ERIC archival media(e.g., electronic or optical)and paper copy.

Signhere)please

The sample sticker shown below will beaffixed to all Level 1 documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL

HAS BEEN GRANTED BY

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

Level 1

4

The sample sticker shown below will beaffixed to all Level 2 documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS

MATERIAL IN OTHER THAN PAPERCOPY HAS BEEN GRANTED BY

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

Level 2

Documents will be processed as indicated provided reproduction quality permits. If permissionto reproduce is granted, but neither box is checked, documents will be processed at Level 1.

El

Check hereFor Level 2 Release:Permitting reproduction inmicrofiche (4" x 6" film) orother ERIC archival media(e.g., electronic or optical),but not in paper copy.

I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminatethis document as indicated above. Reproduction from the ERIC microfiche or electronic/optical media by persons other thanERIC employees and its system contractors requires permission from the copyright holder. Exception is made for non-profitreproduction by libraries and other service agencies to satisfy information needs of educators in response to discrete inquiries.'

Signature: 0" aOrganizationSaaress.

Dept of Information and Library StudiesLoughborough UniversityLoughborough Leics LE11 3TU UK

Printed Name/Position/Tide:

Prof M Kinnell Evans, Head of Dept

Telephone:

+ 44(0)509 223050E-Mail Address:

[email protected]

FAX:

44(0)509 223053Date:

12.8.96

(over)

III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE):

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source,please provide the following information regarding the availability of the document. (ERIC will not announce a document unless it ispublicly available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria aresignificantly more stringent for documents that cannot be made available through EDRS.)

Publisher/Distributor:

Address:

Price:

IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER:

If the right to grant reproduction release is held by someone other than the addressee, please provide the appropriate name and address:

Name:

Address:

V. WHERE TO SEND THIS FORM:

Send this form to the following ERIC Clearinghouse:ER00 D 117Center Far &um® a TechfiavyRoom 4-194Syracuse Urtzkerely

9P244.4.11a)

However, if solicited by the ERIC Facility, or if making an unsolicited-contribution. to ERIC, return this form (and the document beingcontributed) to:

ERIC Processing and Reference Facility1301 Piccard Drive, Suite 100

Rockville, Maryland 20850-4305

Telephone: 301-258-5500FAX: 301-948-3695

Toll Free: 800-799-3742e-mail: [email protected]

(Rev. 3/96/96)