COUNTER: towards reliable vendor usage statistics

Progress to date (June 2004)

Peter ShepherdProject DirectorCOUNTER

SSP 26th Annual Meeting, June 2004

Background

Goal: credible, compatible, consistent publisher/vendor-generated statistics for the global information community

Libraries and consortia need online usage statistics

To assess the value of different online products/services To support collection development To plan infrastructure

Publishers need online usage statistics To experiment with new pricing models To assess the relative importance of the different

channels by which information reaches the market To provide editorial support To plan infrastructure

COUNTER: strategy

Respond to the requirements of the international librarian, publisher and intermediary communities

An open, inclusive and interactive process Representation of all three communities on COUNTER

Limit scope of Release 1 to journals and databases Systematically extend scope of the Code of Practice

Horizontally, to cover other content types, such as e-books Vertically, to provide more detailed statistics on journals

A cost effective-process for all parties involved

Code of Practice, Release 1:Main Features

Definitions of terms used Specifications for Usage Reports Data processing guidelines Auditing Compliance Maintenance and development of the Code of

Practice Governance of COUNTER

Summary

COUNTER Code of Practice released January 2003

Which vendors are now compliant?

What issues and challenges has compliance raised?

Results of library and vendor research

Auditing

COUNTER Code of Practice Release 2

Priorities for 2004 and beyond

Compliant vendor growth

Compliant vendors Jan 2003: 2 Compliant vendors Dec 2003: 12 Compliant vendors May 2004: 30

Over 50% of annual total of new articles covered by ISI now come from COUNTER compliant publishers.

Register of Compliant Vendors

Achieving compliance - hurdles and issues

Cost of development

Concern re fulltext request count reductions

Need for more guidelines e.g. searching, sessions, timeouts, overlapping IPs

Need for a “Guide to interpretation” of COUNTER reports? Like National Rail or BookScan

Testing the COP via market research

Library Focus Groups:- San Diego 14 September 2003 London 16 September 2003 Elsinore 23 October 2003 Charleston 6 November 2003

International Advisory Board Round Table, London Dec 2003

Library pilot testing programme from February 2004:- Cornell University Cranfield University GlaxoSmithKline University of California University of Leicester

le

What the market research has told us

PUBLISHERS If you want to maximise compliance, don’t

make it excessively complex, demanding, and expensive - and don’t keep changing it!

LIBRARIES Keep reports simple and basic Postpone development and implementation

of Release 2 till we have more feedback on Release 1

Put future releases in draft form on the web for a period of public comment

What the market research has told us

“Level 2 reports, especially Journal Report 3, contain too much data to be useful”

“Two levels of compliance are unnecessary” Add a “Publisher” column (helpful with

aggregator reports) Allow removal of zero usage journals in

aggregator reports Divide Table of Definitions into 2, separating

terms used in the reports from the rest One report per file please No punctuation in data ISO date format

What the market research has told us

Preconstructed reports preferred to “on the fly”. One user found only 25% of usage reports generated live actually worked.

Consortia need a way to derive an aggregated summary report cf. compiling it institution by institution.

Make clearer the protocols for measuring usage when intermediaries are involved

Source ofpage

Responsibility forrecording usageand reporting tocustomer

Reportzerousage

Comments

Direct fromvendor’sserver

Vendor Yes Delivery of content to the user is from thevendor’s own service/site, to which the userhas direct access.

Direct fromanaggregator

Aggregator No Delivery of content to the user is from anintermediary (a gateway that is also a host),using its own store of publishers’ content.Gateway is responsible for recording andsupplying usage statistics for full-textrequests direct to the customer and also,where contractually permitted to do so, tothe vendor. (In this case the vendor may notadd the ‘gateway’ usage figures to thoserecording usage of content delivered by thevendor direct to the customer)

Referral fromanaggregatoror gateway

Vendor Yes Delivery involves the gateway sending theend user from the gateway’s site to thevendor’s site for the requested content.Vendor is responsible for recording andsupplying full-text usage statistics to thecustomer. Gateway may also supply usagestatistics to the customer, but must reportthem separately from those covering itsdelivery of full-text direct to the customer

Via agateway

Gateway No Delivery of content is via a gateway, whichrequests the content from the publisher anddelivers it to the user in the context of thegateway service. Responsibility forcollecting and supplying usage statistics tothe customer is the responsibility of theGateway.

Referral toanaggregatoror gateway

One of Vendor,Aggregator orGateway

In this case an index or abstract servicerefers the customer to the gateway for full-text. In this case the full-text is deliveredaccording to one of scenarios listed above,and the recording and supplying of usagestatistics to the customer is as specified ineach of these cases.

What the market research has told us

Provide a toolkit to allow customers to combine usage statistics from different vendors automatically

Issues Role of COUNTER cf agents, library systems suppliers, and

libraries themselves. Role of XML DTD (machine readable)

Develop a separate COP for e-Books and reference works

Audit is critically important – should be credible but not so rigorous it causes publishers to raise prices

What the market research has told us

Library test sites

Monitor compliant vendors and highlight problems. Results include:-

Some differences between compliant submitted reports and actual ones (e.g. different number of columns, ISSNs with leading zeros missing)

Difficulties locating and identifying COUNTER reports amongst others

Lack of historical data for comparison limits usefulness initially

Problem of knowing when compliant data starts. Add “Compliant from” column to the register

Suggest ‘product’ rather than ‘vendor’ be ‘compliant’

Auditing Auditing by a Chartered Accountant (UK), a

Certified Professional Accountant (USA), or its equivalent elsewhere, or by another, suitably-qualified COUNTER-approved auditor, will be required to validate the usage reports

Auditing principles have now been agreed Draft test scripts have now been written and are

being discussed by the Audit Task Force with professional feedback from Deloitte & Touche and others

RFPs for other COUNTER-approved auditors ready to go out to candidates once scripts finalised.

Taken longer than anticipated – complexity revealed as scripts written

COUNTER COP Release 2

Published April 2004 in draft and placed on the website for six months for comment

Specific questions asked in an introduction Should the definition of ‘turnaways’ be broadened?

Final version Jan 2005 and valid version Jan 2006. More prescriptive re formatting Make changes minimal cost as far as possible A new Journal Report 1a, which reports usage

statistics for html and PDF full-text requests separately, but with “health warning”.

A Table of terms and definitions specifically relevant to the Usage Reports contained in Release 2.

Priorities for 2004

Publish draft of Release 2

Solicit feedback on Release 2 draft

Implement auditing

Publish e-Books draft COP mid 2004

Encourage and assist growth in compliance

Reach target of 150 members

COUNTER Membership

Member Categories and Annual Fees

Publishers/intermediaries: £500 Library Consortia: £333 Libraries: £250 Industry organization: £250 Library affiliate: £100 (non-voting

member)

Benefits of full membership Owner of COUNTER with voting rights at

annual general meeting, etc. Regular bulletins on progress Opportunity to receive advice on

implementation

Membership at March 2004

Members May 2004: 130

Vendors 35%

Libraries 28%Consortia 25%Industry Orgs 10%Lib affiliates 2%

Target for 2004: 150 Join us too and help influence the future

development of usage reporting standards

For more information……….

http://www.projectcounter.org

Thank you!

Peter Shepherd (Project Director)pshepherd@projectCounter.org