what is your h-index and other measures of impact
TRANSCRIPT
![Page 1: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/1.jpg)
1
WHAT IS YOUR H-INDEX AND OTHER MEASURES OF ACADEMIC IMPACT Berenika M. Webster, PhD
ULS/ISchool workshop11 November 216
![Page 2: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/2.jpg)
2
Metrics are everywhere
![Page 3: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/3.jpg)
3
They are used to measure/demonstrate impact of…
• National science systems• Research-producing institutions• Research groups• Individual researchers
In order to…• Understand impact of investment• Award funding• Employment and promotion decisions• Apply for funding, job, promotion• Identify experts, collaborators, etc.
![Page 4: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/4.jpg)
4
Learning outcomes• At the end of sessions participants will
• be able to identify paper-level and author-level indicators of impact
• be able to extract these indicators from Scopus, WoS and SciVal
• Understand how individual researchers may use these indicators to demonstrate impact
![Page 5: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/5.jpg)
5
“Citizen” bibliometrics…
• Productivity (publications counts)• leads to “salami slicing”, maybe• quantity vs quality
• Impact (citation counts) • but what impact?
• Impact factor • Speaks to prestige of outlet, not quality of individual paper• 20% of papers in Nature get 80% of all citations (averages do not
work when distributions are skewed) • h-index
• arbitrary, simplistic and lacks consistency• it’s always highest in Google Scholar
![Page 6: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/6.jpg)
6
(BIBLIOMETRIC) STORY OF ONE PUBLICATIONaka article-level metrics
![Page 7: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/7.jpg)
7
Story of one publication
Kato, M., Han, TW., Xie, S., Shi, K., Du, X., Wu, LC., Mirzaei, H., Goldsmith, EJ., Longgood, J., Pei,J., and Grishin, N.V. and Frantz, DE., Schneider, JW., Chen, S., Li, L., Sawaya, MR., Eisenberg, D, Tycko, R. and S. McKnight. (2012) “Cell-free Formation of RNA Granules: Low Complexity Sequence Domains Form Dynamic Fibers within Hydrogels.”, CELL, 149 (4): 753-67. (citations 325, IF 34.242)
![Page 8: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/8.jpg)
8
Story of one publication• Journal characteristics
• JIF or SNIP or another indicator of impact in context• Acceptance rates (e.g. from Cabell's Directory of Publishing Opportunities)
• Co-author characteristics• Names, institutions, countries
• Citations• Number of citations in context (normalised baselines)
• Ratio• Percentile distribution
• Characteristics of citing papers• authors, institutions, countries, subject fields, journals
• Altmetrics • Predictor of citation impact (???)• Indicator of attention• Indicator of impact outside academic realm (e.g. citations in policy documents or clinical guidelines)
![Page 9: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/9.jpg)
9
Story of one publication: WoS
![Page 10: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/10.jpg)
10
Story of one publication: Scopus
![Page 11: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/11.jpg)
11
Story of one publication: altmetrics
![Page 12: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/12.jpg)
12
Story of one publication: more Altmetrics
![Page 13: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/13.jpg)
13
Baselines• Web of Science (WoS) (at Pitt publication and citation data going back to 1980)
• Highly Cited?; Hot?, Impact Factor value
• Essential Science Indicators (ESI) (publication and citation data going back 10 years + current year)• across 22 broad disciplines• Baselines (Averages and Percentiles)
• Journal Citation Reports (JCR)• IF values of thousands of journals and much more (citation networks, rates of obsolescence, etc.)
• Scopus (complete citation data going back to 1996, project to extend to 1970 under way to be completed by end of 2016)
• discipline adjusted citation rates (more granular disciplinary and sub-disciplinary divisions), also adjusted for all doc. types
• SciVal (complete publication and citation data going back to 1996. Based on SCOPUS-indexed publications)
• Over 20 indicators of productivity and impact
![Page 14: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/14.jpg)
14
Story of one publication
My paper has been published in Cell, 2nd top ranked journal in the field of biochemistry and mol. biology and 3nd top title in the filed of cell biology (JCR, 2015 ed.)
To date it was cited 325 times, which places it in the top 1% of all world’s 2012 biochem. and mol. biology papers. The paper has an international and multidisciplinary reach. Citations come from authors from 34 different countries and from across 95 different journals in 35 different fields (including 7 citations from Nature and 6 from Science). (WoS, 20 Sept. 2016).
My paper also attracted attention from non-scholarly audiences, …
Kato, M., Han, TW., Xie, S., Shi, K., Du, X., Wu, LC., Mirzaei, H., Goldsmith, EJ., Longgood, J., Pei,J., and Grishin, N.V. and Frantz, DE., Schneider, JW., Chen, S., Li, L., Sawaya, MR., Eisenberg, D, Tycko, R. and S. McKnight. (2012) “Cell-free Formation of RNA Granules: Low Complexity Sequence Domains Form Dynamic Fibers within Hydrogels.”, CELL, 149 (4): 753-67. (citations 325, IF 34.242)
![Page 15: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/15.jpg)
15
Exercise• Murphy, S.V. and A. Atala (2014) “3D
bioprinting of tissues and organs” NATURE BIOTECHNOLOGY; 32(8):773-785. DOI: 10.1038/nbt.2958
• Describe it using bibliometric indicators discussed today• Citation count• Normalised citation count • Percentile position• Impact of publishing journal• Characteristics of citing publications
• Use WoS or SCOPUS and Altmetric.com
![Page 16: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/16.jpg)
16
Web of Science and Essential Science Indicators
• Murphy, S.V. and A. Atala (2014) “3D bioprinting of tissues and organs” NATURE BIOTECHNOLOGY; 32(8):773-785.
• 337 citations• 337/5.31 = 63.45 (normalised citation impact)• In top 1% of all 2014 Biology and Biochemistry
publications (HICI paper)
![Page 17: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/17.jpg)
17
![Page 18: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/18.jpg)
18
![Page 19: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/19.jpg)
19
(BIBLIOMETRIC) STORY OF ONE AUTHORaka aruthor-level metrics
![Page 20: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/20.jpg)
20
Story of one author
• Source of information about publications• CV• Pitt Faculty Info System (Elements)• ORCID• Google Scholar profile
• How much output is captured in databases• Scopus Author Identifier• ResearcherID (WoS)
![Page 21: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/21.jpg)
21
Scholarly communication practicesSubject area Books and book chapters Conference papers Journal articlesHistory 45.6 3.8 50.6Politics and Policy 43.1 10.8 46.1Language 40.5 7.6 51.8Human Society 31.3 5.6 63Philosophy 29.8 5.4 64.8Economics 27.4 8 64.5Law 26.2 1.9 71.9The Arts 25.2 20.3 54.5Education 21.8 23.6 54.5Architecture 20.8 43.6 35.6Psychology 18.9 4.9 76.2Journalism, library 18.6 24.2 57.2Management 13 34 52.9Earth Sciences 8.6 9.2 82.2Medical & Health Sci 6.6 2.9 90.5Biological Sciences 6.6 2.7 90.7Agriculture 6.3 14.7 79Computing 5 62.3 32.8Mathematical Sciences 5 11.2 83.8Engineering 2.9 45.1 52Physical Sciences 2.7 7.3 90Chemical Sciences 2.3 1.9 95.7
L. Butler, 2006
And what about all other research outputs?
![Page 22: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/22.jpg)
22
Author metrics
• Overview of productivity and impact
• Should use size-dependent variables as these take into consideration volume of outputs• Publications (other outputs) counts and characteristics• Total citation counts in context (discipline, age, output type)• Excellence measures (top 1, 5, 10% of distributions)• ESI’s Highly Cited author?• h-index (if you must) – though it is inconsistent (example later)
![Page 23: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/23.jpg)
23
Story of one author
![Page 24: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/24.jpg)
24
SCOPUS view
Citation per publication rate for my 200 publications indexed in Scopus is 91.2, only 5 of my publications (2.5%) were not cited o date. (Scopus, 8 June 2016)
My research is multidisciplinary spanning across biochemistry, genetics and molecular biology, physics chemistry, materials science and engineering.
I have been publishing consistently since 1982, averaging 7 publications per year in high quality, peer reviewed outlets. These publications, collectively receive upwards of 1,000 citations per year. (Scopus, 8 June 2016)
![Page 25: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/25.jpg)
25
Story of one author
Citations to my publications in the last 10 years placed me in the group of top 1% of biochemistry researchers in the world and 5 of these publications placed in the top 1% of publications in their field. (ESI, 20 Sept 2016)
![Page 26: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/26.jpg)
26
SciVal view - overview
In the last 5 years, 68% of my publications placed in the top 10% of all similar publications (by field and age), and 32% of them were published in the top 10% of scientific journals. (SciVal, 8 June 2016)
![Page 27: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/27.jpg)
27
My publications in biochemistry consistency outperform these of NIH and USA as a whole in relation of impact and percentage of publications in top 1% in the world. (SciVal, 8 June 2016)
![Page 28: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/28.jpg)
28
Whose h-index? One author – 3 different values
137 vs. 143 vs. 160
![Page 29: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/29.jpg)
29
Author 1 Author 2 Author 3
15 150 15
10 100 10
10 50 10
5 25 5
5 5 5
4 1 0
4 0 0
3 0 0
3 0 0
1 0 0
Does not account for:
• insensitive to highly-cited publications
• citation characteristics of publication outlets
• citation characteristics of fields of science
• age of publications
• type of publications
• co-authorship
• self-citations
• scientific age of author
![Page 30: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/30.jpg)
30
More problems with h-index
Ludo Waltman and Nees Jan van Eck, JASIST 2012
If two scientists achieve the same absolute performance improvement, their ranking relative to each other should remain unchanged.
Scientist X Scientist Y
Publication Citations
1 5 6
2 5 6
3 5 6
4 5 6
5 5 3
6 2 3
7 2 3
Scientist X Scientist Y
Publication Citations
1 8 8
2 8 8
3 5 6
4 5 6
5 5 6
6 5 6
7 5 3
8 2 3
9 2 3
Scientist X h-index 5Scientist Y 4
Scientist X – 5 Scientist Y - 6
![Page 31: What is your h-index and other measures of impact](https://reader033.vdocuments.mx/reader033/viewer/2022051300/58f0fa591a28ab4a1e8b45bb/html5/thumbnails/31.jpg)
31
Exercise: create a bibliometric profile of a researcher
• Ludo Waltman, Leiden U• Krzysztof Matyjaszewski, CMU• Ervin Sejdic, Pitt• Anne-Wil Harzing, U Melbourne
• Try SCOUPS, WOS and SciVal to find out• Number of publications• Total number of citations• Normalised citations (e.g. within a discipline)• Subject distributions• Characteristics/impact of citing publications• For an entire career span/for last 5 years