niso webinar: making better decisions with usage statistics
DESCRIPTION
As libraries move to become centers of digital collections, maintaining information on the usage of these collections is ever more critical. It's also essential to be able to maintain common measures across heterogeneous collections, in order to be able to effectively analyze how the library's collection dollars are being spent. The Project COUNTER Code of Practice and the SUSHI protocol aid in this work. This session will explore the newly-published Release 4 of the COUNTER Code of Practice for e-Resources and highlight its use in conjunction with the SUSHI (Standardized Usage Statistics Harvesting Initiative) protocol in an active library environment.TRANSCRIPT
Making Better Decisionswith Usage Statistics
June 13, 2012
Speakers: Peter Shepherd, Oliver Pesch, and Amy Lynn Fry
http://www.niso.org/news/events/2012/nisowebinars/usage_statistics/
Release 4 of the COUNTER Code of Practice for e-Resources
Peter Shepherd
COUNTER
June 2012
COUNTER Release 4 - objectives
A single, unified Code covering all e-resources, including journals, databases, books, reference works, multimedia content, etc.
Improve the database reports Improve the reporting of archive usage Enable the reporting of mobile usage separately Expand the categories of ‘Access Denied’ covered Improve the application of XML and SUSHI in the
design of the usage reports Collect metadata that facilitates the linking of usage
statistics to other datasets, such as subscription information
Release 4: main features
A single, integrated Code of Practice covering journals, databases, books, reference works and multimedia content
An expanded list of Definitions, including terms such as ‘Gold Open Access’, ‘Multimedia Full Content Unit’, ‘Record View’, ‘Result Click’, as well as different categories of ‘Access Denied’, etc. that are used for the first time in Release 4
Enhancements of the SUSHI (Standardised Usage Statistics Harvesting Initiative) protocol designed to facilitate its implementation by vendors and its use by librarians
Release 4: main features
A requirement that Institutional Identifiers, Journal DOI and Book DOI be included in the usage reports, to facilitate not only the management of usage data, but also the linking of usage data to other data relevant to collections of online content.
A requirement that usage of Gold Open Access articles within journals be reported separately in a new report: Journal Report 1 GOA: Number of Successful Gold Open Access Full-text Article Requests by Month and Journal.
A requirement that Journal Report 5 must be provided
Release 4: main features
Modified Database Reports, in which the previous requirement to report Session counts has been dropped, and new requirements, to report Record Views and Result Clicks, have been added. (Database Report 3 has also been renamed Platform Report 1).A new report, Multimedia Report 1, which covers the usage of non-textual multimedia resources, such as audio, video and images, by reporting the number of successful requests for multimedia full content unitsNew optional reports covering usage on mobile devicesA description of the relative advantages of logfiles and page tags as the basis for counting online usage Flexibility in the usage reporting period that allows customers to specify a date range for their usage reports
Release 4: Standard Usage Reports
Journal Report 1: Number of Successful Full-Text Article Requests by Month and Journal
Journal Report 1 GOA: Number of Successful Gold Open Access Full-Text Article Requests by Month and Journal
Journal Report 2: Access Denied to Full-Text Articles by Month, Journal and Category
Journal Report 5: Number of Successful Full-Text Article Requests by Year-of-Publication (YOP) and Journal
Database Report 1: Total Searches, Result Clicks and Record Views by Month and Database
Database Report 2: Access Denied by Month, Database and Category Platform Report 1: Total Searches, Result Clicks and Record Views by Month and
Platform Book Report 1: Number of Successful Requests by Month and Title Book Report 2: Number of Successful Section Requests by Month and Title Book Report 3: Access Denied to Content Items by Month, Title and Category Book Report 4: Access Denied to Content Items by Month, Platform and Category Book Report 5: Total Searches by Month and Title Multimedia Report 1: Number of Successful Full Multimedia Content Units
Requests by Month and Collection
Release 4: Journal Report 5
Release 4: Database Report 1
Release 4: Multimedia Report 1
Release 4: recording and reporting usage on mobile devices
The following optional additional reports enable usage on mobile devices to be reported separately:
Journal Report 3 Mobile: Number of Successful Item Requests by Month, Journal and Page Type for usage on a Mobile Device
Title Report 1 Mobile: Number of Successful Requests for Journal Full-text Articles and Book Sections by Month and Title ( formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices)
Title Report 3 Mobile: Number of Successful Requests by Month, Title and Page Type (formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices)
COUNTER will recognize as usage on a mobile device, which may be reported in the above reports, any usage that meets one of the following criteria:
useragents that are included in the WURFL list. WURFL is the Wireless Universal Resource FiLe, a database containing the profile of mobile devices; this database may be found at: http://wurfl.sourceforge.net/
usage via a proprietary mobile App provided by the publisher/content provider
Release 4: timetable for implementation
Deadline date for implementation of Release 4:
31 December 2013
-after this date only vendors compliant with Release 4 will be COUNTER compliant
Between now and 31 December 2013, Release 4 and the existing Releases of the Codes of Practice are valid
Release 4: compliance process
1.Agree Release 4 usage reports that are relevant to publisher content
2.Review of reports by a COUNTER library test site
3.Vendor signs Declaration of COUNTER compliance
4.Vendor added to the Register of COUNTER-compliant vendors
5.Independent audit must be carried out and passed within 6 months of being added to the Register
Release 4: independent audit
Three aspects of the audit Check report formats Check data integrity Check delivery process
Three possible audit outcomesA Pass, in which case no further action is required by the publisher as a result of the audit. In some cases the auditor may add Observations to the audit report, which are designed to help the vendor improve its COUNTER usage reports, but which are outside the scope of the audit itself.A Qualified Pass, in which the auditor deems the publisher to have passed the audit, but where the auditor raises a Minor Issue requiring further action to maintain COUNTER-compliant status. A Minor Issue does not affect the reported figures, but is one which should be resolved within 3 months of the audit to maintain COUNTER-compliant status. An example of a Minor Issue is where a report format does not conform to the COUNTER specifications.A Fail, where the auditor has identified an issue that must be resolved immediately for the vendor to maintain COUNTER -compliant status
Full details of Release 4 will be found on the COUNTER website at:
http://www.projectcounter.org/code_practice.html
COUNTER Code of Practice-Release 4
Quality Content • Resource Management • Access • Integration • Consultation
Making Better Decisions with Usage StatisticsSUSHI: More relevant than ever
Oliver PeschChief Strategist, E-Resource
Access and Management Services
Overview
• Why SUSHI• COUNTER Release 4
• Changes to schemas• Procedural changes• Applying lessons learned
• Other tools and resources available
Why SUSHI…
• Librarians doing more with less rely on usage statistics as one measure of value of their purchases
• Usage consolidation applications and related services rely on COUNTER
• To be valuable, usage collection needs to be comprehensive
• Efficiency depends on automation • SUSHI is a very scalable standard for
harvesting COUNTER reports!
COUNTER 4: Schema Changes…
SUSHI Schema (the actual SUSHI standard)
COUNTER 4: Schema Changes…
SUSHI Schema (the actual SUSHI standard)
No changes were required!
COUNTER 4: Schema Changes…
COUNTER XML Schema for reports: changes were minor and geared towards better compliance, including:• ItemIdentifier, ItemPublisher elements now
optional• PubYr attributes validate as year• PubYrFrom, PubYrTo attributes added to
support Journal Report 5
COUNTER 4: Schema Changes…
COUNTER Elements XML Schema: lists valid values for certain data elements
• DataType: added Collection and Multimedia options
• Categories: added Access_denied and removed Turnaways.
• MetricType: added several new values to support reports for mobile use, the new database reports, multimedia reports and revised access denied reports.
COUNTER 4: Procedural Changes related to SUSHI
• To be COUNTER compliant, a content provider must provide a working SUSHI server
• Testing the SUSHI implementation will be part of the audit
Lessons Learned:Perspective as a Usage Consolidation Vendor
COUNTER…• Offers the promise of consistency necessary
for consolidated reporting• Combined with SUSHI is potentially a
significant time-saver• Covers a broad range of reports providing
usage for journals, books, databases
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
UsageConsolidation
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention• SUSHI implementations falling short
UsageConsolidation
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention• SUSHI implementations falling short
UsageConsolidation
SUSHI Challenges
• Overly complex authentication methods
• Lack of understanding of market (most SUSHI clients are hosted -- one client will harvest usage for many customers)
• Support staff unfamiliar with set-up needs or operational details
Lessons Learned:Perspective as a Usage Consolidation Vendor
Challenges with COUNTER…• “Almost” compliant reports require manual
intervention• SUSHI implementations falling short• Many content providers still not COUNTER
compliant
UsageConsolidation
NISO SUSHI Standing Committee: Release 4 of the COUNTER Code of Practice
Applying the lessons learned…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Even more resources available at the NISO
SUSHI web site.
NISO
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
NISO SUSHI Web Site: Support for developers and librarians
Addressing the challenges…• COUNTER schema stricter• COUNTER SUSHI Implementation Profile
removes ambiguity• Free web-based client available for testing or
basis for development• Many resources available at the NISO
SUSHI web site.
Conclusion
• SUSHI is both relevant and necessary, but we need…• more compliant content providers• with better interoperability
• COUNTER Release 4 more comprehensive audit coupled with the COUNTER SUSHI Implementation Profile are keys to progress
SUSHI: More relevant than ever
Thank You!
Visit the NISO SUSHI web site
http://www.niso.org/workrooms/sushi
Or email me at:
Amy Fry, Electronic Resources Coordinator, Bowling Green State Universityimages by Ken Fager, http://www.flickr.com/photos/kenfagerdotcom/
Streamlining Stats:Managing Usage Statistics Efficiently and Effectively
What do I get? How do I get it?
I need it now! How do I put it all together?
Usage statistics: what libraries are thinking
Stats work today is HYBRID:part manual, part automated
Look familiar?
How about this?
Just a little manual labor…
Maybe we should just cancel everything.
Data control Organization of work Distribution of labor
Everything in its (one and only) place, all linked together.
Stats are integrated into the resource’s lifecycle.
There are jobs for librarians, staff and students.
BGSU’s three principles to streamline stats
Data controlONE place for logins and passwords:
our ERM.
Data controlONE place for logins and passwords:
our ERM.
Data controlONE place for logins and passwords:
our ERM.
Data control ONE place for downloading instructions: our wikis
dbusestats.bgsu.wikispaces.net ejusestats.bgsu.wikispaces.net
Data controlONE place for the stats files:
our R drive.
Data controlONE file for each report type
Organization of workStats are integrated into the lifecycle of each
resource.
• Reports are saved as downloaded (by calendar year)• Stats are collected once a year
Organization of workCollecting stats has been simplified
Distribution of labor
Librarian jobs:• Manage logins and passwords• Manage reporting• Manage the SUSHI table in the ERM• Manage stats through each resource’s lifecycle
Staff jobs:• Write downloading instructions• Make changes as needed in the wiki (adding, removing, and changing resources)• Convert files XML; upload XML files into the ERM
Student jobs:• Collect and save stats according to wiki instructions
Other jobs:• Serials, Acquisitions & IT staff work with order records, do coverage load, and more.
Everyone gets to help!
ReportingOur medium
– For databases: spreadsheets, for now– For e-journals: our ERM
Our philosophy– Focus on standard measures– Try to paint a picture over time– Accuracy isn’t everything
Database usage reporting:Standard measurements
Database usage reporting:Use over time
Database usage reporting:Using estimates
The “EBSCO Bump”
E-journal usage reporting: Millennium ERM JR1 Summary Report
E-journal reporting using the Millennium ERM
– Create resource records for each journal package– Relate order records with correct from/to dates for
each subscription– Download COUNTER JR1 reports in CSV format– Convert to XML using the University of Nebraska-
Lincoln script (http://statsconverter4erm.unl.edu/) – Upload each XML file to the proper resource record
Detail 1 Report for ACS Journals
Resource-Level Statistics Export for ACS Journals
With a little editing…
Coming soon…
• Innovative is expanding the report types that can be uploaded to its ERM in 2013!
Scholarly Stats– Basic stats collection service, pay per platform– Basic collated reports
SwetsWise Selection Support– Stats collection service (pay per
platform)– Options to upload data manually
360 Counter (Serials Solutions)– Unlimited platforms with annual
subscription– Options to upload data manually
Feel like outsourcing?
Open-source ERMs
• ERMes (University of Wisconsin-La Crosse)– Incorporates DB1 reports
• CORAL (Notre Dame)– Incorporates JR1 reports
• E-Matrix (NCSU)– Incorporates JR1 reports
• CUFTS (Simon Fraser)– Incorporates JR1 reports;
uses SUSHI
Remember…
You’re never, ever going to get out of collecting data altogether unless you decide to stop looking at your stats.