THE SCIENCE COMMUNICATION RESEARCH LITERATURE MAPPING PROJECT

LOOKING BACK AT A DECADE OF PCST RESEARCH, 2000-2009

Rick BorcheltNational Cancer Institute, US National Institutes of Health

rick.borchelt@nih.gov

2

PCST at an Inflection Point

• Has PCST matured into an independent field of enquiry?

• Is there an articulated research agenda with commonly agreed provocative questions to drive research?

• Who sets/controls the PCST research agenda?3/28/2012

3

UPDATING THE TRADITIONAL LITERATURE REVIEW USING DATA VISUALIZATION SOFTWARE

• Rapid growth of science communication research over the past ten years;

• Need to understand what has been accomplished and where future research should head;

• Traditional literature reviews seek to identify important patterns in research—research fronts, emerging issues;

• “Visual representations of data take advantage of the unique ability of visual perception to detect meaningful patterns that might otherwise remain hidden.” (Few, 2007)

4

OBJECTIVES OF THE SCIENCE COMMUNICATION RESEARCH LITERATURE MAPPING PROJECT

• Use data visualization software to map the landscape of recent science communication research (2000-2009);

• Identify the active countries, major methodologies, and research topical fronts during the past decade;

• Lay the foundation for a research agenda for the next decade of science communications research

5

Two PHASES OF PROJECT METHODOLOGY

Analyze the data using IN-SPIRE data visualization software

• Upload EndNote data into

IN-SPIRE

• Data refinement in IN-SPIRE; and

• Use of IN-SPIRE analytic tools (Galaxy View, Heat Map, Time, Correlation)

Construct the Research Literature Database:

• Define the parameters of the literature review (dates, inclusion and exclusion criteria);

• Select the search engine;

• Select the search terms;

• Import citation data into EndNote; and

• Clean the citation data.

6

CONSTRUCTING THE RESEARCH LITERATURE ON SCIENCE COMMUNICATION

Defining the parameters of the review:

Focus on reports of original science communication research that

– Were published in peer-reviewed scientific journals;

– Were available in English language;

– Were published between 2000 and 2009;

– Excluded research on formal science education and teaching; and

– Excluded research on health communication.

Inclusion / Exclusion Criteria

Included: Journal articles, historical articles, evaluation studies, meta-analyses and systematic reviews.

Excluded: Abstracts, bibliographies, books, commentaries, conference or symposia proceedings, editorials, essays, introductory articles, letters to editor, narrative reviews, responses and rejoinders, reports, and narrative literature reviews.

7

SELECTION OF GOOGLE SCHOLAR AS SEARCH ENGINE

Strengths of Google Scholar:

• Covers a wide range of social science, biomedical and natural science journals and sources;

• Convenience (speed, ease of use, free);

• Generates many results; and

• Can easily import citation information into bibliographic database (EndNote).

Limitations of Google Scholar:

• Absence of a controlled search vocabulary;

• Cannot restrict output to peer-reviewed journal articles;

• Cannot view more than the first 1,000 results (“hits”);

• Duplicate citations must be removed manually from successive searches;

• Unknown scope of coverage.

8

SELECTION OF GOOGLE SCHOLAR SEARCH TERMS

Search terms were generated from three sources:

• The Science Communication project team at the National Cancer Institute;

• Members of the PCST Scientific Committee; and

• Several terms suggested by Burns, O’Connor and Stocklmayer (2003) in their article on defining science communication research.

Final Google Scholar Search Terms

Science / Technology Communication

Public Understanding of Science / Technology

Public Engagement /Participation

Science / Technology Coverage in: mass media; print media; newspapers; television; film or movies

Science or Scientific Literacy

Science Journalism

Risk Communication

Risk Perception

Science / Technology and Culture

9

STEPS IN THE DATA CLEANING PROCESS (ENDNOTE)

Once the citation data were imported into EndNote, the following process was begun:

10

OVERVIEW: CHARACTERISTICS OF LITERATURE

• 1,237 papers from 2000-2009;

• 2,462 authors (mean of about two authors per paper);

• 471 Journals; and

• 199 Key Words;

11

SCI-COMM RESEARCH ARTICLES, 2000-2009(n=1,237)

2000 2001 2002 2003 2004 2005 2006 2007 2008 20090

50

100

150

200

250

62 62

72

112

92

115

148 148

230

196

Number of Articles Published by Year

More than twice as many articles were published between 2005-2009 as between 2000-2004

12

TOP 10 JOURNALS PUBLISHING SCIENCE COMMUNICATION RESEARCH, 2000-2009

British Food Journal

Global Environmental Change

Health, Risk, and Society

New Genetics and Society

Journal of Science Communication

Journal of Risk Research

Risk Analysis

Science Communication

Public Understanding of Science

0 20 40 60 80 100 120 140 160 180 200

10

14

24

26

29

33

55

135

179

The top 10 journals accounted for 518 of 1,237 papers, or 42 percent.

13

TOP 11 MOST FREQUENT AUTHORS / CO-AUTHORS OF SCIENCE COMMUNICATION RESEARCH PAPERS, 2000-2009

Priest, S.H.

Poortinga, W.

Besley, J.C.

Siegrist, M.

McComas, K.A.

Brossard, D.

Nerlich, B.

Condit, C.M.

Rowe, G.

Frewer, L.J.

Pidgeon, N.

0 5 10 15 20 25

10

10

10

11

11

11

14

15

16

21

22

Number of Papers as Author or Co-Author, 2000-2009

404 authors (16 percent) published 2 or more papers between 2000-2009.

14

United States427

Canada77

Russia2

China14

Australia51

India11 Mexico

4

Brazil10

United Kingdom 270

Germany: 30Italy: 23

Portugal: 2

Morocco 2

Nigeria 1

Greece: 12France: 13Spain: 16

South Africa3

Japan 24

Columbia2

Netherlands: 35

New Zealand 25

Sweden: 23Denmark: 23

NUMBER OF SCIENCE COMMUNICATION RESEARCH PUBLICATIONS BY COUNTRY, 2000-2009

15

TOP FIVE COUNTRIES PUBLISHING SCIENCE COMMUNICATION RESEARCH, 2000-2009

Netherlands

Australia

Canada

United Kingdom

United States

0 50 100 150 200 250 300 350 400 450

35

51

77

270

427

16

TOP 5 MOST FREQUENT STUDY METHODS IN PUBLISHED SCIENCE COMMUNICATION RESEARCH, 2000-2009

Secondary Analyses of Surveys

Case Studies

Interviews

Content Analysis

Surveys or Questionnaires

0 50 100 150 200 250 300 350

78

149

159

290

331

17

SELECTED TOPICS IN SCIENCE COMMUNICATION RESEARCH

Nanotechnology

Natural Resources

Genetically Modified Foods

Environment

Climate Change

Public Engagement

0 20 40 60 80 100 120

62

64

83

91

93

100

Frequency of Publications on Selected Topics, 2000-2009

4/12/12

18

MAPPING OF THE SCIENCE COMMUNICATION RESEARCH LITERATURE, 2000-2009

• IN-SPIRE uses a mathematical clustering algorithm to calculate degrees of similarity and dissimilarity in most frequently occurring key words. Based on this algorithm, the software can create several visual representations of a dataset:

• Galaxy view: A plot of all of the documents contained in the dataset showing the topical relationships between them.

– Proximity implies relatedness; distant clusters are less topically related.

– Each document is represented by a dot, and the closer two dots are plotted, the more similar they are in topical content.

• ThemeView Classic (Heat Map): The ThemeView Classic is a form of Heat Map, which is a three-dimensional version of the Galaxy view.

– The height of a peak corresponds to the number of documents on a topic, and is further highlighted by a brighter color.

– Look for proximate peaks, valleys and spaces between formations as indicators of underlying topical patterns.

19

GALAXY VIEW, FULL SCI-COMM DATASET(n=1,237)

20

HEAT MAP VIEW, FULL SCI-COMM DATASET (n=1,237)

21

SCIENCE COMMUNICATIONS RESEARCH LITERATURE

FIVE-YEAR HEAT MAP VIEW, 2000-2004(n=400)

22

SCIENCE COMMUNICATIONS RESEARCH LITERATURE

FIVE-YEAR HEAT MAP VIEW, 2005-2009(n=837)

23

TWO-YEAR HEAT MAP VIEW, 2000-2001(n=124)

24

TWO-YEAR HEAT MAP VIEW, 2002-2003(n=184)

25

TWO-YEAR HEAT MAP VIEW, 2004-2005(n=207)

26

TWO-YEAR HEAT MAP VIEW, 2006-2007(n=296)

27

TWO-YEAR HEAT MAP VIEW, 2008-2009(n=426)

28

HEAT MAP VIEW, UNITED STATES STUDIES, 2000-2009

(n=427)

29

HEAT MAP VIEW,UNITED KINGDOM STUDIES, 2000-2009

(n=270)

30

HEAT MAP VIEW,CANADIAN STUDIES, 2000-2009

(n=77)

31

HEAT MAP VIEW,AUSTRALIAN STUDIES, 2000-2009

(n=51)

32

HEAT MAP VIEW,NETHERLANDS STUDIES, 2000-2009

(n=35)

33

ARTICLES ON SCIENCE COMMUNICATION RESEARCH TOPICS PUBLISHED IN TWO PRIMARY JOURNALS, 2000-

2009

Research Topic Public Understanding of Science

Science Communication

Total Articles on This Topic in

Dataset

Public Engagement 20 10 100

Climate Change 13 7 93

Environment 10 6 91

Genetically Modified Foods 12 3 83

Natural Resources 5 6 64

Nanotechnology 11 9 62

Food Safety 3 4 54

Emerging Infectious Diseases 1 5 53

Science Journalism 7 9 46

34

RESEARCH METHODS USED IN ARTICLES PUBLISHED IN TWO PRIMARY SCIENCE COMMUNICATION JOURNALS,

2000-2009

Research MethodPublic Understanding

of Science (n=179)

Science Communication

(n=135)

Total Articles Using This

Method in Dataset

Surveys or Questionnaires 17% 30% 331

Content Analysis 24% 31% 290

Interviews 13% 9% 159

Case Studies 18% 7% 149

Secondary Analysis of Surveys

7% 4% 78

Focus Groups 10% 4% 72

Evaluation Studies 3% 2% 56

35

THREE CASE STUDIES OF SCIENCE COMMUNICATION RESEARCH TOPICS

How are different Science Communication research issues investigated over time?

Are particular research methods more likely to be applied at different times in the investigation of a science communication research issue?

We selected three research issues (Public Engagement, Climate Change and Genetically Modified Foods) to examine in greater detail.

These cases met the following criteria:

– Actively investigated over entire 10-year period; and

– Reasonable number of papers published on each in the dataset.

36

FREQUENCY OF PUBLISHED ARTICLES BY YEAR FOR THREE RESEARCH CASES

2000 2001 2002 2003 2004 2005 2006 2007 2008 20090

5

10

15

20

25

30

23

45

3

6

1517

2223

9

2 23

5

98

11

18

26

2

6

3

79

11

15

11

15

4

Public Engagement Climate ChangeGenetically Modified Foods

37

HEAT MAP VIEW

PUBLIC ENGAGEMENT ARTICLES, 2000-2009(n=100)

38

HEAT MAP VIEW, CLIMATE CHANGE ARTICLES, 2000-2009

(n=93)

39

HEAT MAP VIEW:ARTICLES ON GENETICALLY MODIFIED FOODS, 2000-2009

(n=83)

40

RESEARCH METHODS IN SCIENCE COMMUNICATION RESEARCH STUDIES BY TIME PERIOD, 2000-2009

Content Anal-ysis

Surveys or Questionnaires

Case Studies Interviews Secondary Analyses

Focus Groups0%

5%

10%

15%

20%

25%

30%

25%27%

9%

13%

7%6%

23%

27%

13% 13%

6% 6%

2000-2004 2005-2009

41

PROPORTIONS OF PUBLISHED STUDIES ON PUBLIC ENGAGEMENT USING VARIOUS RESEARCH METHODS

BY TIME PERIOD

2000-2004 2005-20090%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

0

0.05

0.18

0.43

0.24

0.17

0.12 0.12

Content Analysis Case Studies Surveys or Questionnaires Interviews

42

PROPORTIONS OF PUBLISHED STUDIES ON CLIMATE CHANGE USING VARIOUS RESEARCH METHODS

BY TIME PERIOD

2000-2004 2005-20090%

10%

20%

30%

40%

50%

60%

70%

80%

90%0.81

0.31

0.05 0.03

0.52

0.29

0.05

0.15

Content Analysis Case Studies Surveys or Questionnaires Interviews

43

PROPORTIONS OF PUBLISHED STUDIES ON GENETICALLY MODIFIED FOODS USING VARIOUS RESEARCH METHODS

BY TIME PERIOD

2000-2004 2005-20090%

5%

10%

15%

20%

25%

30%

0.15

0.13

0.04

0.23

0.260.25

0.04 0.04

Content Analysis Case Studies Surveys or Questionnaires Interviews

44

PUBLIC ENGAGEMENT

HEAT MAP VIEW, 2000-2004(n=17)

45

PUBLIC ENGAGEMENT

HEAT MAP VIEW, 2005-2009(n=83)

46

CLIMATE CHANGEHEAT MAP VIEW, 2000-2004

(n=21)

47

CLIMATE CHANGE

HEAT MAP VIEW, 2005-2009(n=72)

48

GENETICALLY MODIFIED FOODS

HEAT MAP VIEW, 2000-2004(n=27)

49

GENETICALLY MODIFIED FOODS

HEAT MAP VIEW, 2005-2009(n=56)

50

Ramifications

• What do these data say about us as a research community?

• What do these data say about us as a publishing community? (and about access to research findings by practitioners)?

• What do these data say about who sets the agenda for PCST research?

3/28/2012

51

What do these data say about us as a research community?

• That we are generally more closely aligned with science/technology focus areas that with communication research per se

• Is there life beyond content analysis and attitude surveys?

3/28/2012

52

What do these data say about us as a publishing community?

• We favor publications in “gray lit” rather than peer-reviewed journals (esp. books and monographs)

• This might be okay for the small community of researchers, but puts our research out of the reach of most practitioners

• Need a more robust “open access” approach to PCST research

3/28/2012

53

What do these data say about who sets the agenda for PCST research?

• Seems primarily driven by “utilitarian” approach, emerging issue by issue

• Sustained funding is not available to address fundamental questions in PCST research not tied to utilitarian model

• PCST requires a new agenda driven by provocative questions and a funding stream to support it

3/28/2012

54

Acknowledgments

Margaret Ames, Karl PoonaiNCI Office of Science Planning and Assessment

Jack Scott, Margaret BlasinksyThe Madrillon Group, Inc.

Ben CarolloNCI Office of Public Affairs and Research Communication3/28/2012