from reputation to citation: varying roles for scholarly metrics
TRANSCRIPT
From Reputation to Citation: Varying Roles
for Scholarly MetricsElaine M. Lasda, MLS CAS
Associate Librarian, University at AlbanySpecial Libraries Association
July 18, 2017
What I’m Going to Say:
• Types of metrics/approaches for assessing research impact
• Reputation
• Bibliometric
• Altmetric
• Appropriate Usage and Best Practices
• Grassroots Movements
• Some common sense objectives
https://www.amazon.com/Make-
Your-Bed-Little-
Things/dp/1455570249
Why “Impact”?
https://flic.kr/p/SHujQt
Maximizing Research Impact Workshop
• Author Identifiers (ORCID)•Bibliometrics (oldies, goodies and freebies)
• Open Access Scholarship
• Altmetrics
• Three Quick Hits
•ORCID
•Scholars Archive (IR)
•Author Profile in WoS/Scopus/GS
“Reputation” approach
https://flic.kr/p/JTndV5
“Squishy” Reputation Approaches
• Colleagues – word of mouth
• Colleagues – published surveys
• Ulrich’s – content indexed in what dbs
• Editorial Board composition
• Peer researchers have published there
• Published scholarly surveys of disciplinary experts
Caveats of Reputation Approach
• Subjective
• Overlook new journals/output platforms
• Rely heavily on/misuse JIF metric
• Survey methodologies vary & don’t always exist
• Info can be outdated
Traditional citation metrics
https://flic.kr/p/9XtEKC
Bibliometrics: Context and Background
Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25, 348-9.
Traditional-ish
• Citation count
• JIF
• SNIP/SJR
• h-index/e-index, g-index, etc.
• i-10
New(er) Citation-Based Tools
• Cross disciplinary metrics
• InCites
• CiteScore
• Viz!
• Manuscript Matcher
• PoP
Caveats of bibliometrics
• Self citations
• “Cartels” of scholars, journals
• Significant variation in source datasets
• Quality control vs. Inclusivity
• Static or rigid disciplinary categories
•emerging fields
•inter- and multidisciplinary fields
Altmetrics (with a small “a”)
What are altmetrics? • ImpactStory
https://impactstory.org/u/0000-0002-9498-7074
• Altmetric
• Bookmarklet: https://www.altmetric.com/products/free-tools/bookmarklet
• Scholars Archive “donut”
http://scholarsarchive.library.albany.edu/ulib_fac_scholar/11/
http://scholarsarchive.library.albany.edu/ulib_fac_scholar/18/
• PlumX
In Ebscohost databases
http://libproxy.albany.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true
&bquery=(altmetrics+AND+are+AND+awesome)&type=0&site=eds-live&scope=site
• Bought by Elseiver
Altmetrics: More of a Reputation or Citation Approach?
https://flic.kr/p/5xgQBY
If a Tweet mentions your paper,
does it make an impact?
https://flic.kr/p/5xgQBY
Role of Altmetrics
More of a quantifiable “reputation” indicator than empirical evidence of use
Can overcome caveats of more “squishy” reputation approaches
• Trendsetting/Immediacy
• Transcends disciplinary “silos”
• Can generate mainstream traction/impact
• Leads (hopefully) to increased impact in scholarly peer reviewed literature (ie, boosts one’s citation count)
Caveats of altmetric approaches
•Not Standardized•Transparency varies
• Which scholars are on what platforms?
•Who else is looking where our scholars look?
•Proprietary academic social media
•ResearchGate, Academia.edu, Mendeley, etc.
Whither “Gaming”?
https://flic.kr/p/6mzCJkhttps://flic.kr/p/6mzCJk
https://flic.kr/p/6mzCJk
Appropriate Usage of Scholarly Metrics
Rubber-Meets-the-Road Impact Schemas
Scholarly Metrics Best Practices
• Use transparent methodologies
• Supplemental to expert qualitative evaluation
• Altmetrics serve as a reputation measure (for now)
• Citation Metrics remain a more empirical measure
• Contextualize the metric to your need
• (institution, discipline, peer groups, country/region, journal, other?)
What I just talked about
• Workshop to address researcher/scholar needs on campus
• Reputation and empirical (citation) impact assessments are complementary
• Gaming happens with all measures, as does dirty data
• New approaches measure tangible outcomes of scholarly activity
• Best practices include transparent methods, contextualized interpretation, and usage that is supplemental to expert qualitative evaluation (no magic numbers!!!)
Heeeere’s
Richard!
Elaine M. Lasda
University at Albany
@ElaineLibrarian
http://slideshare.net/librarian68