Bibliometrics overview slides

Contents of this slide set• Slides 2-5

Various definitions• Slide 6

The context, bibliometrics as 1 tools to assess

• Slides 7-8Levels at which you can use bibliometrics

• Slides 9-12More detail about some of these users

• Slides 13-16The building blocks, the main products need to use more than one data source

Bibliometrics definition 1The branch of library science concerned with the

application of mathematical and statistical analysis to bibliography; the statistical analysis of books, articles,

or other publications.

[Oxford English Dictionary, http://tinyurl.com/lvq4l2, Date Accessed: 15/07/09]

Bibliometrics definition 2Bibliometrics“the discipline of measuring the performance of a researcher, a collection of articles, a journal, a research discipline or an institution”

This process involves the ‘application of statistical analyses to study patterns of authorship, publication, and literature use’. (Lancaster 1977).

Bibliometrics definition 3

• Counting of publications and citations– Measuring the output and the

impact of scientific research

• Evaluating and ranking people and institutions, countries and research outputs

Putting bibliometrics in context• Bibliometrics & citation analysis is only one quantitative indicator of

research. There are other quantitative indicators and qualitative approaches of which peer-review a key indicator.

• Bibliometric Measures:– Patterns of authorship, publication & the use of literature

• Benefits– Quantitative approaches could be argued to be fairer than qualitative

methods e.g. peer-review– Cost effective– Efficiency advantage

• Application & importance varies from field to field– tremendous controversy surrounds any assessment of the intellectual

output of academics & researchers

The varying levels of use 1• Publication strategies to ensure maximum visibility

by targeting high impact journal titles

• Assessment of individuals for promotion, tenure or grant funding

• Research Output Evaluation / Research Profiling– Micro Level – Macro Level

The varying layers of use 2• A personal context – assessing the individual

• An Institutional context:• Research Office assessing and benchmarking academic and unit

performance

• A National context:• Forfas/HEA study Research Strengths in Ireland• Department of Enterprise Trade & Employment (IRL)

Value for money review of Science Foundation Ireland (SFI)

• An International context:• Times Higher Education Supplement (THES) World University Rankings• Shanghai Jiao Tong University Academic Ranking of World Universities

Example of use to generally assess an individual

-What is Eugene Kennedy’s most highly cited work?-What is his H-Index?-What year did he get most citations in?- Is there a lot of research with no citations at all?

Example of use by individual in a CV

Example of uses to rank and assess journals• Evaluate the scholarly worth of a journal

• Rank journals within a discipline

• Help you decide where to publish your article for maximum impact

• Evaluation for promotion / tenure / grants, or in some countries, even government funding of an institution

• May be used as an evaluation source by librarians during journal cancellations or new purchases

Example of use for global rankingTHE World University Rankings

http://www.timeshighereducation.co.uk/world-university-rankings/

The Building Blocks

Dataset

ISI Citation IndexScopus

Google ScholarQuantitative Measures

Impact FactorCitation Analysis

Publication countsH-index

Eigenfactor

Metric Tools & Techniques

The “Big three”: the overlap is quite modestISI/WOS SCOPUS Google Scholar

Approx. 12,000 journalsPoor coverage of humanities e.g. monographs not includedPoor coverage of OA journals & conference papers, despite some recent additionsMajority Anglo-Saxon in origin; English language biasWeak at distinguishing between authorsOldest – 1955

Approx. 18,000 titlesGreater geographic spread than WOS – 60% is outside U.S.Better inclusion of non-journal material, e.g. conf. papersContains useful tools for author disambiguationLimited coverage, 1995

Widest range of material included although no list of journals includedGaps in the coverage of publishers’ archives; no indication of timescale coveredResults often contain duplicates of the same article (e.g. pre-prints, post-prints)No way to distinguish between authors with same initialsDifficult to search for a journal which has various title abbreviations

What are we counting?• Number of papers per individual, unit, institution

• Citation rates and averages per paper, individual, unit

• Total number of citations, and cites per paper, per journal, ranking of journals on this basis

You should use more than one data source…

• The same paper gets very different citation counts from three tools: PROLA (75); Google Scholar (48) and Web of Science (106).