improving the reported use and impact of institutional ...€¦ · theme accurately assessing the...
TRANSCRIPT
![Page 1: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/1.jpg)
Kenning Arlitsch, Dean - @kenning_msu Patrick OBrien, Semantic Web Research Director - @sempob
Montana State University Library
Coalition for Networked Information Washington, D.C.
December 13, 2016
Improving the Reported Use and Impact of Institutional Repositories
![Page 2: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/2.jpg)
Theme
Accurately assessing the impact of institutional repositories
Research funded by the Institute of Museum and Library Services (IMLS) ”Measuring Up: Assessing Accuracy of Reported Use and Impact of Digital Repositories” http://scholarworks.montana.edu/xmlui/handle/1/8924
![Page 3: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/3.jpg)
Agenda
❖ A New Reporting Model ❖ Accuracy of Analytics Reporting Tools
● Page Tagging ● Log file
❖ RAMP ● New Prototype Web Service for IR reporting
![Page 4: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/4.jpg)
A New Reporting Model
Page Type Definition Examples
Citable Content Downloads
Non-HTML scholarly content that may be formally cited in the research process
● Publication (.pdf) ● Presentation (.ppt) ● Data Sets (.csv)
Item Summary HTML pages to help user decide to download the full publication
● Title & Abstract ● Item Metadata
Ancillary HTML pages that provide general information or navigation
● Search Results ● Browse by Author ● Statistics
![Page 5: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/5.jpg)
![Page 6: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/6.jpg)
![Page 7: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/7.jpg)
![Page 8: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/8.jpg)
Two Classes of Web Analytics
HTML
Analytics Service (SaaS) 1 Log Files
2 Page Tagging {JavaScript}
![Page 9: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/9.jpg)
Google Analytics used by most academic libraries
❖ Tested 279 academic library websites ● ARL ● DLF ● OCLC-RLP
❖ 90% US libraries contained Google tracking code
![Page 10: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/10.jpg)
Page Tagging
Page Tagging does not track non-HTML Citable Content Downloads
HTML
Non-HTML
![Page 11: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/11.jpg)
Show Google Scholar direct link graphic
![Page 12: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/12.jpg)
Web Analytics Accuracy Risks Risks Analytics Method
Area Types Page Tagging Log Files
OverCount
Visits Low High
Downloads Low High
Page Views Low Low
UnderCount
Visits Medium Medium
Downloads High Low
Page Views Low Low
![Page 13: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/13.jpg)
Typical Page Tagging Data Sources
Page Type Analytics
Pages Events
Citable Content Downloads - ✅
Item Summary ✅ -
Ancillary ✅ -
![Page 14: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/14.jpg)
Four IR During 2016 Spring Semester January 5 to May 17 (n = 134 days)
Study Participant IR Pla(orm URL
Montana State University ScholarWorks DSpace scholarworks.montana.edu
McMaster University MacSphere DSpace macsphere.mcmaster.ca
University of New Mexico LoboVault DSpace repository.unm.edu
University of Utah USpace CONTENTdm uspace.utah.edu
![Page 15: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/15.jpg)
Page Tagging does not track non-HTML Citable Content Downloads
Non-HTML
Search Analytics
Does!
![Page 16: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/16.jpg)
Montana Method Includes Google Search Console
Page Type Analytics Search
Console
Pages Events Search Analytics
Citable Content Downloads - ✅ ✅
Item Summary ✅ - -
Ancillary ✅ - -
![Page 17: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/17.jpg)
All four IR using Google Analytics
Page Type Analytics Search
Console
Pages Events Search Analytics
Citable Content Downloads
- 26,355 562,933
Item Summary 284,303 - -
Ancillary 201,793 - -
CCD Tracking Improvement
2,000%
![Page 18: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/18.jpg)
54.3%
46.2%
47.9%
42.5%
45.7%
53.8%
52.1%
57.5%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
MostIRActivityisUnreportedbyGoogleAnalytics01/05/16- 05/17/16(days=134)
Reported Invisible
Most IR Activity Unreported by Google Analytics.
![Page 19: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/19.jpg)
51%
54%
52%
63%
35%
27%
20%
20%
14%
19%
28%
17%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
~51%to63%ofCitableIRActivityisUnreportedbyGoogleAnalytics01/05/16- 05/17/16(days=134)
CitableContentDownloads ItemSumaryPV AncillaryPV
Most IR activity consists of Citable Content Downloads
![Page 20: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/20.jpg)
Montana Method Challenges
❖ Missing non-Google direct link CCD events ● Yahoo ● Bing ● Email
❖ Search Analytics limits time and access ● Moving 90 day window ● Granular data = programing skills to access
API
![Page 21: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/21.jpg)
RAMP - Repository Analytics and Metrics Portal
❖ Cloud Web service ❖ Currently accessible and free ● During grant period (December 2017)
❖ No training or configuration required ❖ Consistent method and terminology ❖ Benchmarking across time and
organization ❖ Request Access @ [email protected]
![Page 22: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/22.jpg)
Two New Publications Published: Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda. “Undercounting File Downloads from Institutional Repositories,” Journal of Library Administration, vol. 56, no. 7, 2016 Forthcoming: Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda. “RAMP: Repository Analytics and Metrics Portal: A Prototype Web Service that Accurately Counts Item Downloads from Institutional Repositories,” Submitted to Library Hi Tech, expected publication early 2017
![Page 23: Improving the Reported Use and Impact of Institutional ...€¦ · Theme Accurately assessing the impact of institutional repositories Research funded by the Institute of Museum and](https://reader033.vdocuments.mx/reader033/viewer/2022042220/5ec696c30eea24644d4efa56/html5/thumbnails/23.jpg)
Undercounting Research Team ❖ Montana State University
● Kenning Arlitsch, Dean @kenning_msu ● Patrick OBrien, Semantic Web Research Director @sempob ● Leila Sterman, Scholarly Communication Librarian @calamityleila ● Susan Borda, Digital Technologies Librarian @mutanthumb
❖ OCLC Research ● Jeff Mixter, Software Engineer @jeffmixter
❖ University of New Mexico ● Jonathan Wheeler, Data Curation Librarian