bibliometric assessment of research performance in social sciences and humanities henk f. moed...
TRANSCRIPT
Bibliometric assessment of research performance in
social sciences and humanities
Henk F. Moed
Centre for Science and Technology Studies (CWTS), Leiden University, the Netherlands
2
Assessment of research performance: Basic assumptions - 1
• The concept of research quality does have a meaning in all fields of science and scholarship
• An assessment methodology should take into account the nature of the field
3
Assessment of research performance: Basic assumptions - 2
• Important contributions to scholarly progress are sooner or later communicated in scholarly publications
4
Research quality: Dimensions
• Contribution to scholarly progress
• Enlightenment of the general public
5
Science versus humanities (Price) - 1
Science Humanities
Orientation Quantitative Qualitative
Status of knowledge
Short term permanence
Permanent significance
Organisation Groups interacting at the research front
Individuals
6
Science versus humanities (Price) - 2
• Different substantive contents ask for different types of social organisation and information exchange
• Science: Citation Index
• Humanities: Normal archival library
7
Social sciences
• Social sciences constitute a heterogeneous domain, with both ‘science-like’ and ‘humanities-like’ orientations
• Even sub-disciplines may be heterogeneous (e.g., sociology)
8
Overall ISI coverage by main field
EXCELLENT (> 80%) VERY GOOD (60-80%) GOOD(40-60%)
Biochem & Mol Biol Appl Phys & Chem Mathematics
Biol Sci ~ Humans Biol Sci ~ Anim & Plants
Economics
Chemistry Psychol & Psychiat Engineering
Clin Medicine Geosciences MODERATE (<40 %)
Physics & Astron Soc Sci ~ Medicine & health
Other Soc Sci
Humanities & Arts
9
Sub-disciplines (non-exhaustive list)
Social sciences related to medicine and health
Other social sciences
Humanities
Public environm & occupat health
Nursing
Sport sciences
Substance abuse
Sociology
Anthropology
Educational sciences
Political science
Law
Literature
Linguistics
Historical sc
Philosophy
11
‘Other’ social sciences and humanities
• ‘National publication model’ is dominant
• Books play an important role
• Basic assumptions of a journal citation index are less valid
• ISI Indexes have inadequate coverage
• They also contain national journals
12
Indicator of a journal’s national orientation (INO)
• The share of the papers from the country most frequently publishing in a journal
• A purely national journal would have an INO value of 100 per cent
13
National orientation of journals (INO) for 4 fields
Discipline No. Journals
Median INO % Journals with
INO>90%
Physics & astronomy
260 37 10
Mol Biol & biochem
530 41 4
Humanities & arts
1,110 71 24
Other social sci
879 72 22
14
Policy assumption for social sciences and humanities
The extent to which research findings
• reach beyond a purely national or local viewpoint
• and are exposed to criticisms from a wide international scholarly audience
is a relevant criterion of research quality
15
The problem: ISI ≠ ‘International’
• Works exposed to an international audience are not necessarily included in the ISI Indexes
• Works included in the ISI Indexes are not necessarily exposed to an international audience
16
This book argues:
• It cannot be taken for granted that the ISI Citation Indexes provide such indicators in all subfields of these domains of scholarship
• A challenge would be to systematically explore alternative data sources and methodologies
17
4 Types of bibliometric studies
Cited/Target Citing/Source ISI coverage Field/Study
1 ISI ISI Excellent – Very Good
Astronomy
2 ISI+non ISI ISI Very Good – Good
Mathematics
3 ISI+non ISI ISI+non ISI Good – Moderate
Computer Science
4 No citation analysis at all Moderate NL Law
18
Alternative approaches
1. Expand the WoS with additional sources
2. Classification of publications and sources based on scholars’ quality perceptions
3. Library collection analysis
19
1. Expand the WoS with additional sources
• University of Granada (Spain):
• Creation of a citation index with 200 Spanish social science source journals not covered by the WoS
• 50,000 uses per year
20
2. Classifications based on scholars’ quality perceptions
• Case study on Flemish Law
• Questionnaires; No citation analysis
• Publications in Dutch: 81 %
• Publications in journals: 59 %
21
Classification of (national) journals based on a questionnaire into:
• Scholarly vs. non-scholarly
• Outstanding (A), good (B) and less good (C)
22
The ‘best’ indicator of scholarly research performance (in Flemish Law):
Count the number of
• Single and multi-authored books (first editions only)
• PhD theses
• Publications with a length > 5 pages
23
3. Library collection analysis
• Focuses on books
• Determine the number of academic library copies per book title
• Example: Use Worldcat (Linmans, CWTS, 2007)
END