Citation Metrics: Established and Emerging Tools

Linda GallowaySyracuse University Library

Syracuse, NYgalloway@syr.edu

Citation metrics -Tools to gauge scholarly impact

• Quantity– Publications (tweets, blog posts, mentions…)

• Quality– Citations (saves, downloads, likes…)

• Funding– Research funds, collaborations

• Technology transfer– Patents, start-ups, etc.

Focus today is on individual author metrics

Why should librarians care?

• Build relationships with faculty• Learn about faculty research interests• Assist in evaluation of departments, programs,

and faculty• Accreditation efforts• Marketing for academic programs

Scholars care because…

• Citation metrics are used in promotion and tenure decisions

• Prestige and influence– Within a field– Within an institution– Comparison to peer and aspirational institutions

• Vanity– Individual and institutional

Established ToolsScopusWeb of ScienceGoogle Scholar

Consider Before Beginning:• Cost of subscription databases• Ease of use• Time frames– Citations to past year’s work (2011),– Citations to author’s work in past 5 years (07-11)– Citations to author’s work in past 10 years (02-11)

• Skewed towards STEM fields• Don’t compare across databases!

Typically, assess scholarly impact by looking at:

• Works published• Citations to works published• Publication influence (h-index, journal prestige,

etc. )

Citation Metrics for Individual Faculty Members

Scopus or Web of Science?Scopus Web of Science

Web of Science consists of nine databases containing information gathered from thousands of scholarly journals, books, book series, reports, conferences, and more.

• It fully covers over 12,000 major journals.• Create a visual representation of citation

relationships with Citation Mapping • Capture citation activity and trends

graphically with Citation Report • Use the Analyze Tool to identify trends and

patterns

SU subscription: • Science Citation Index Expanded (1899-

present) • Social Sciences Citation Index (1898-

present) • Arts & Humanities Citation Index (1975-

present)

SciVerse Scopus is the world’s largest abstract and citation database of peer-reviewed literature.• Contains 47 million records, 70% with

abstracts• Nearly 19,500 titles from 5,000 publishers

worldwide• Includes over 4.9 million conference

papers• Interoperability with Engineering Village &

Reaxys • Provides 100% Medline coverageSubscription includes:• 26 million records with references back to

1996 (of which 78% include references).• 21 million records pre-1996 which go back

as far as 1823.

As of 10/2012

Cited Reference Searchin Scopus

Tips: Create an account to save your work Check within each author group for publications Merge author profiles, if necessary

Cited Reference Searchin Web of Science

Tips: Create an account to save your work If you have a very prolific author, ask for

assistance with article identification By default, citation counts are for All Years – you

must modify for your chosen parameters Read the ‘Cited Reference Search’ how-to and

follow the directions closely

Must always ‘Finish Search!

Citation Mapping in Web of ScienceVisually Demonstrate Author/Article Influence

Tips: Public profiles are available in Google Can search for an author from within

your own profile page

Google Scholar Citations

Cited Reference Searchin Google Scholar Citations

• Author needs to set up their profile (using their Google account); Google Scholar will harvest related data.

• Authors can add articles, groups of articles, edit entries, etc.

• Profiles are private unless author elects to make public

• Quality control: “To be eligible for inclusion in Google Scholar search results, your profile needs to be public and needs to have a verified email address at your university”

Times cited H-Index

Scopus 135 7

Web of Science 85 11

Google Scholar 279 10

Times cited = number of documents published from 2007-current* that have cited this author's work

H index = Number of author's articles that have been cited at least this many times (during time span indicated)

*Normally, would choose the past 5 years, i.e. 2007-2011. However, to compare more equitably to Google Scholar, used 2007- current (25 Nov 2012)

Comparison

Author DisambiguationORCID, about.orcid.org

• Not for profit• Create registry of unique

identifiers for individual researchers

• Open and transparent linking between ORCID and other ID schemes

• Many vendors, institutions are members

Identification Schemes

• Scopus – Scopus Author Identifier

• Web of Science – ResearcherID

• Google Scholar Profiles• Institutional ID• Many other identifiers…

altmetrics is the creation and study of new metrics based on the Social Web for analyzing, and informing scholarship

Altmetrics: A manifestohttp://altmetrics.org/manifesto/

What can be measured?

“Evidence of Use” – http://impactstory.org

• # of Tweets • # of “Saves” in online reference managers• Scholarly (and popular) blog interest and

activity• Activity in social networking platforms, tools• Etc!!

Tracking Mentions (or uploads/downloads, etc.) in Social Media

Mendeley is a combination of a desktop application and a website which helps you manage, share and discover both content and contacts in research.

ImpactStory is a website that makes it quick and easy to view the impact of a wide range of research output. It goes beyond traditional measurements of research output -- citations to papers -- to embrace a much broader evidence of use across a wide range of scholarly output types.

citeulike is a free service for managing and discovering scholarly references6,402,349 articles - 3,030 added today.

Zotero is a free, easy-to-use tool to help you collect, organize, cite, and share your research sources. It lives right where you do your work—in the web browser itself.

Altmetrics and STEM

Altmetric Limitations

• New• Time frame – some new tools cannot search

old mentions, tweets, etc.• Rely on user generated metadata• Should social media mentions be given the

same weight as scholarly article citations?• Can these tools be easily manipulated to raise

significance of an article?

Citation Metrics: Established and emerging tools

• Established tools – Web of Science and Scopus– Widely recognized and trusted– Pricey– Require some level of expertise to use– Google Scholar is becoming an established tool

• Emerging tools – Altmetrics– New, many promising platforms and services– Librarians & information professionals should be aware– When a few tools are established, trusted and verified

==>> acceptance

Thank you!

More information:http://researchguides.library.syr.edu/citationmetrics

References1. Eysenbach, G. (2011). Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and

Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4), e123. doi:10.2196/jmir.2012

2. Google. (n.d.). Google Scholar Citations Help. Retrieved November 19, 2012, from http://scholar.google.com/intl/en/scholar/citations.html#overview

3. ImpactStory. (n.d.). ImpactStory: faq. Retrieved November 19, 2012, from http://impactstory.org/faq4. Galloway,L. & Rauh, A. (2012). Subject Guides. Citation Metrics. Web of Science. Retrieved November

16, 2012, from http://researchguides.library.syr.edu/citationmetrics 5. ORCID Inc. (2012). ORCID. Retrieved November 6, 2012, from http://about.orcid.org/6. Priem, J., Groth, P., & Taraborelli, D. (2012). The Altmetrics Collection. (C. A. Ouzounis, Ed.)PLoS ONE,

7(11), e48753. doi:10.1371/journal.pone.0048753 7. Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2011, September 28). altmetrics: a manifesto –

altmetrics.org. Retrieved November 16, 2012, from http://altmetrics.org/manifesto/