counter: achievements and future challenges peter shepherd director counter april 2007
TRANSCRIPT
COUNTER: achievements and
future challenges
Peter Shepherd
DirectorCOUNTER
April 2007
Background
Understanding usage Different approaches Role of usage statistics
Usage statistics Should enlighten rather than obscure Should be practical Should be reliable Are only part of the story Should be used in context
COUNTER Achievements Current status Future challenges
COUNTERCodes of Practice
Definitions of terms used Specifications for Usage Reports
What they should include What they should look like How and when they should be delivered
Data processing guidelines Auditing Compliance Maintenance and development of the
Code of Practice Governance of COUNTER
COUNTER: current Codes of Practice
1) Journals and databases
Release 1 Code of Practice launched January 2003 Release 2 published April 2005 replacing Release 1 in
January 2006 Now a widely adopted standard by publishers and librarians 60 vendors now compliant 9000+ journals now covered Librarians use it in collection development decisions Publishers use it in marketing to prove ‘value’
2) Books and reference works
Release 1 Code of Practice launched March 2006 4 vendors now compliant Relevant usage metrics less clear than for journals Different issues than for journals
Direct comparisons between books less relevant Understanding how different categories of book are used is
more relevant
Journal and Database Code of Practice
Usage Reports
Journal Report 1 Full text article requests by month and journal
Journal Report 2 Turnaways by month and journal
Database Report 1 Total searches and sessions by month and database
Database Report 2 Turnaways by month and database
Database Report 3 Searches and sessions by month and service
Code of Practice for books
Book Report 1 Number of successful requests by month and title
Book Report 2 Number of successful section requests by month and
title Book Report 3
Turnaways by month and title Book Report 4
Turnaways by month and service Book Report 5
Total searches and sessions by month and title Book Report 6
Total searches and sessions by month and service
Journal Report 1Full text article requests by journal
Html and PDF totals reported separately
COUNTER Audit
Independent audit required within 18 months of compliance, and annually thereafter
Audit is online, using scripts provided in the Code of Practice Auditor can be:
Any Chartered Accountant Another COUNTER-approved auditor
ABCE is the first COUNTER-approved auditor Industry-owned Not-for-profit Independent and impartial Part of ABC (Audit Bureau of Circulations) Providing website traffic audits for over 150 companies and
certifying over 1400 domains Have successfully completed test audits on COUNTER usage
reports
ABCE audit fees
Year 1 (first audit) £2,500 Includes 1.5 man-days pre audit consultancy as well as 1.5 man-days audit
Max 20 million records (add 0.5 man-days per 10 million or part thereof) Max 50 reports (add 0.5 man-days per 50 reports or part thereof) Assumes 1 data source for all reported numbers, in correct format,
delivered to agreed timescale Ongoing support (technical, administrative & marketing) Reduced by £250 for COUNTER members
Year 2 (and ongoing per audit) £1,500 Includes 1.5 man-days audit
Max 20 million records (add 0.5 man-days per 10 million or part thereof) Max 50 reports (add 0.5 man-days per 50 reports or part thereof) Assumes 1 data source for all reported numbers, in correct format,
delivered to agreed timescale Ongoing support (technical, administrative & marketing)
At ABCE’s normal daily consultancy rate
COUNTER: deriving metrics from Journal Report 1
Local metrics For libraries and library consortia At journal, collection and publisher level To compare the cost-effectiveness of journal
subscriptions To assess the value of Big Deals
Global metrics For authors, funding agencies, libraries and
publishers At journal, collection and publisher level To compare quality and value
COUNTER: ‘local’ metrics
JISC (UK Joint Information Systems Committee) Funded by UK higher education funding councils Supports higher education in the use of information and
communications technologies Access to information and communication resources Advice on creation and preservation of digital archives Implications of using ICT Network services and support Research to develop innovative solutions
National overview of online journal usage Develop a reliable, widely applicable methodology Use COUNTER Journal Report 1 ‘article full-text requests’
Local metrics: an example
COUNTER data was analysed in relation to: usage range Price band Subject category
Metrics derived from this analysis Trend in number of full-text article downloads Full text article requests per title Full text article requests per publisher package Full text article requests per FTE user Most requested titles Usage of subscribed vs.. unsubscribed titles Cost per full-text article downloads Cost per FTE user
Summary report available at:www.ebase.uce.ac.uk/projects/NESLi2.htm
Local metrics: an example
Growth in full-text article downloads Publisher A: 12%- 208% Publisher B: 12%- 59% Publisher C: 23%- 154% Publisher D: 22%- 81%
Cost per full-text article download Publisher A: £0.97- £5.26 Publisher B: £0.70 - £2.91 Publisher C: £0.80 - £3.29 Publisher D: £0.45 - £2.26
COUNTER: ‘global’ metrics
Impact Factor Well-established, easily understood and accepted Endorsed by funding agencies and researchers Does not cover all fields of scholarship Reflects value of journals to researchers Over-emphasis on IF distorts the behaviour of authors Over-used, mis-used and over-interpreted
Usage Factor Usage-based alternative perspective Would cover all online journals Would reflect value of journals to all categories of user Would be easy to understood
Global metrics: UKSG Project
Assess the feasibility of developing and implementing journal Usage Factors
Level of support from author, librarian and publisher communities
Data from which UF would be derived COUNTER Journal Report 1? Article numbers Process for consolidation, calculation and reporting of UFs
Factors in the calculation Level of reporting Total usage Articles
Report in April 2007 Just completed set of 29 interviews with industry leaders Wider online survey will take place in February 2007
UKSG Project: feedback
Are the COUNTER usage statistics sufficiently robust? Frustration at lack of comparable, quantitative data on journals Should items covered by restricted to articles? Many journals still have significant usage in print Diversity of views on the factors in the calculation
Specified usage period Specified publication period
Usage data is more susceptible to manipulation Will the journal be a meaningful concept in the future? Two measures with different limitations are better than one,
and UF will be derived from a set of credible, understandable data
Usage data will be used as a measure of value, whether publishers like it or not
Current issues Interface effects on usage statistics
E.g. downloading HTML and PDF of the same article in one session
COUNTER has tested data filter solutions, but what does the duplicate downloading signify?
Reporting separately purchasable digital archive usage
Currently all usage for a journal is usually reported together
Separately purchasable archives mean we need separate reports for archival content, or a year of publication breakdown of usage
Usage in Institutional Repositories Growth in Institutional Repository (IR) content Need for credible IR usage statistics IR usage statistics already being collected, but no
standards SUSHI Improving consortial usage reports
Current usage reports inadequate New reports in XML format
Interface effects on usage statistics
COUNTER filter project: objectives Development of filters to be applied to usage data that
would dampen or compensate for the effect of certain vendor interface configurations
‘Unwanted html filter’: based on the assumption that the time that elapses between a request (click) for an html full-text journal article and the next request (click) is a measure of the value of the html document to the user.
‘Unique article filter’: this filter is based on the assumption that we can use a unique identifier for an article, irrespective of format, and an identifier for a session to derive the number of unique article requests per session – the irreducible minimum full-text usage
An assessment of current vendor practice regarding implementation of unique article identifiers, such as DOIs
Interface effects on usage statistics
‘Unwanted html’ filter: Time filter tested on EBSCO data Range of intervals applied (2sec -30sec)
At 2 sec only 4% of html views are eliminated At 8 sec only 7% of html views are eliminated At 30 sec around 60% of html views are eliminated
Results similar whether or not the ‘auto-html’ facility is active
Similar results when user starts on EBSCO site or links in from another site
Similar results for Elsevier data
Interface effects on usage statistics
Unwanted html filter: conclusions Setting a time filter <10 sec eliminates <10% of html
requests Setting a time filter of >25 sec results in over 50% of
html requests being eliminated. Unreasonable to assume that html documents still open after 30 sec are ‘unwanted’
Curve in the 10-25 sec range is so steep that it is not possible to specify a time filter that could be universally applied with confidence
COUNTER should provide guidelines for ‘best practice’ for vendor interface design rather than a new data filter
Interface effects on usage statistics
Unique article filterVendor% reduction ratio PDF/htmlA 25.14% 0.64B 25.50% 4.00C 21.40% 7.69D 35.65% 1.05E 47.36% 0.97
Note: in addition, Elsevier and EBSCO noted a 22% and 28% reduction, respectively
Interface effects on usage statistics
Unique article filter: conclusions Average reduction in usage count ca 30% In 70% of cases only one format us used per session Reasonable to assume that in a minority of cases users
want both html and PDF formats No relationship between unique article usage and
PDF/html ratio Maximum theoretical ‘inflation’ of usage statistics due
to multiple formats is 30%, probably much less COUNTER JR1 usage statistics are a reasonable basis for
publisher comparisons, but can be further improvedFinal Report:http://www.jisc.ac.uk/whatwedo/programmes/
programme_pals2/synthesis/projects/counter.aspx
Reporting separately purchasable digital archive usage
Increasingly requested by librarians Interim solution
Journal Report 1a:Number of Successful Full-Text Article Requests from an Archive by Month and Journal
Optional additional usage report Longer-term solution
Journals Report 1a? Include year-of-publication data in JR1?
SUSHI
Standardized Usage Statistics Harvesting Initiative (SUSHI)
No mechanism yet for automatically retrieving, combining, and storing COUNTER usage data from different sources
NISO-sponsored XML-based SUSHI aims to provide a means to do just this, via a standard model for machine to machine automation of statistics harvesting.
COUNTER and NISO have signed an agreement to work together on the development of SUSHI. More details of SUSHI can be found at:-
http://www.niso.org/committees/SUSHI/SUSHI_comm.html
Future challenges
Improving/extending the Codes of Practice Reliability ( audit, federated searches, prefetching) Usability (number of compliant vendors, XML format,
additional usage reports) Additional data (year of publication, article level
reports) Categories of content (Institutional Repository content)
Deriving metrics from the Codes of Practice Journals (cost per use, Usage Factor) Databases? Books?
Next steps…..
Release 3 of Code of Practice for Journals/Databases
Features: prioritisation on basis of demand and practicality Process: consultation via focus groups,etc; publication of
draft CoP Release 2 of Code of Practice for Books
Review R1 in practice Other categories of content ( eg Institutional
Repositories) Metrics derived from the COUNTER usage statistics
Cost per use Usage Factor
COUNTER Membership
Member Categories and Annual Fees (2007)
Publishers/intermediaries: £530 Library Consortia: £355 Libraries: £265 Industry organization: £265 Library affiliate: £106 (non-voting
member)
Benefits of full membership Owner of COUNTER with voting rights at
annual general meeting, etc. Regular bulletins on progress Opportunity to receive advice on
implementation
http://www.projectcounter.orghttp://www.projectcounter.org
Apply for COUNTER membership
Apply for COUNTER membership
