a new measure of knowledge diffusion
DESCRIPTION
A New Measure of Knowledge Diffusion. Stephen Carley; Alan Porter Georgia Tech. Our Interests. Interest in research interdisciplinarity Possible key to more creative research Possible means to solve complex scholarly and societal problems Interest in research knowledge diffusion - PowerPoint PPT PresentationTRANSCRIPT
A New Measure of Knowledge Diffusion
Stephen Carley; Alan Porter
Georgia Tech
Our Interests
• Interest in research interdisciplinarity Possible key to more creative research Possible means to solve complex scholarly and
societal problems • Interest in research knowledge diffusion
Trans-disciplinary knowledge enrichment Translation of research to clinical practice Fostering science-based innovation
Measures• Have devised metrics:
Integration score [how diverse are the research knowledge resources upon which an article (or body of research) draws?]
Specialization score[how diverse are the venues in which a body of research (e.g., the works of one author in one time period) appears?]
• Now adding -- Diffusion score [how diverse are the fields that cite an article (or body of research)?]
Heuristics of diversity
(Stirling, 1998; 2007)(Rafols and Meyer,
2009)
Diversity:Attribute of a system whose elements may be apportioned into categories
Characteristics: Variety: Number of distinctive categoriesBalance: Evenness of the distribution Disparity: Degree to which the categories
are different.
Variety
Balance Disparity
Herfindahl (concentration): i pi2
[** Shannon & Herfindahldo not includeDisparity]
Shannon (Entropy): i pi ln pi
Dissimilarity: i di
Generalised Diversity (Stirling) ij(ij) (pipj)a (dij)b
Integration• Integration Defined:
equivalently:
• Integration Range: 0 → 1• Integration Benchmarking: ~0.42 for much of modern
scientific research (increasing moderately)
where i = row, j = column, f = frequency, SC = Cited Web of Science Subject Category;cosine similarity of SC cross-citation (based on one year of Web of Science journal cross-citation)
ji
jiij ppsI,
1
Changes in Average Integration Scores over Time
Porter, A.L., and Rafols, I. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics. 81.3 (2009): 719-745.
Diffusion Scoring Example
Paper 1 Paper 2
Cited by: Biology Genetics Math Music Forestry
• In the example above Paper 2 will likely have a higher diffusion score than Paper 1 because there is greater heterogeneity among its citing subject categories [variety & disparity; not clear on balance].
Diffusion Scoring• Diffusion Defined:
where:pi = proportion of citing references corresponding to the SC i in a given paper sij = cosine measure of similarity between SCs i & j for cross-citation in WOS
• Diffusion Range: 0 → 1• Diffusion Benchmarking: ~0.42 for much of
modern scientific research [but somewhat higher for papers “out there” longer]
Mean Diffusion scores forall benchmark years combined
Modest increase over time since publication
Diffusion Score Behavior• Diffusion score behavior is more challenging to study than
Integration:• Calculating Diffusion score requires additional “citation
searching” in Web of Science to retrieve the citing papers• A paper’s Diffusion score is not fixed; over time it will
accrue more citations• Self-citation could affect Diffusion score
• We examined – for 5 of our 6 target Subject Categories -- minimal effect
• For Math – removing self-cites increased scores for 1995 publications by 0.01-.02, until recent years [Since Math 1995 annual Diffusion scores are ~0.22, this is ~5-10% increase]
Self-Cites (1)
Self-Cites (2)
Diffusion Scores: Total vs. Annual
• Annual Diffusion may be of interest• e.g., to pursue Chen’s hypothesis that broad +
rapid diffusion may signal really impactful papers• To characterize research knowledge diffusion
patterns• Calculation issue
• We use a threshold of at least 3 Citing SCs for a paper to get a Diffusion Score[could be 1 cite by a paper whose journal is associated with 3 SCs (rare)]
• For Total score calculation this is less a factor than for Annual scores
For most of the 1995 benchmarks, Diffusion scores increase steadily with time. Mathematics is an outlier.
Mean Annual Diffusion Scores for 6 Subject Categories
Mean NeuroScience Diffusion scores for all 4 benchmark years
Diffusion Patterns
• Can we categorize research knowledge diffusion patterns? A first try here:• Steady growth• Staying power• Late-bloomer• Robust growth
• Next 4 slides illustrate for those 1995 publications in the 6 SCs that fit these patterns
Steady Growth Typology
Data Source: 1995 benchmarks
Staying Power Typology
Data Source: 1995 benchmarks
Late-Bloomer Typology
Powerhouse articles (N’s are small) tend to have high Diffusion scores
Data Source: 1995 benchmarks
Robust Growth Typology
Data Source: 1995 benchmarks
• Papers are citing more references over time -- does this exert upward pressure on Diffusion scores?
• What happens to the speed of diffusion over time? - Will electronic open source publishing boost Diffusion?
• How differently does research published in various Subject Categories diffuse?
• Chen: papers which are cited quickly and diffusely are more likely to be blockbusters
• Tool for tracking individual researchers or research groups’ influence
Diffusion Issues
Taking a Look at the Research of Robert Nerem
• Institute Professor Emeritus and Director, Georgia Tech/Emory Center for Regenerative Medicine
• 240 Web of Science (WOS) publications, in• 76 Journals, over• 47 Years, and more than• 8,000 Citations
Diffusion Scores v. Publication Age for Dr. Nerem’s Research
Increasing Diffusion score from his early papers (high Age) to more recent ones
For more on Integration see:
www.idr.gatech.edu Porter, Alan, Alex Cohen, David Roessner and Marty
Perreault. "Measuring researcher interdisciplinarity." Scientometrics. 72.1 (2007): 117-147.
Porter, Alan, David Roessner, and Anne Heberger. "How interdisciplinary is a given body of research?" Research Evaluation. 17.4 (2008): 273-282.
Porter, Alan, and Ismael Rafols. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics. 81.3 (2009): 719-745.
For more on Diffusion see: Chen, Chaomei, and Diana Hicks. "Tracing knowledge
diffusion." Scientometrics. 59.2 (2004): 199-211. Chen, C., Chen, Y., Horowitz, M., Hou, H., Liu, Z., &
Pellegrino, D. (2009). Towards an explanatory and computational theory of scientific discovery. Journal of Informetrics, 3(3), 191-209
Yu, G., Wang, M.Y., Yu, D.R. “Characterizing knowledge diffusion of Nanoscience & Nanotechnology by citation analysis.” Scientometrics, 84.1 (2010): 81-97.
Carley, S. and Porter, A.L. (2011) "A Forward Diversity Index." Scientometrics, forthcoming.
Thank you!
Mean Biotech Diffusion scores for all 4 benchmark years
Mean EE Diffusion scores for all 4 benchmark years
Mean Math Diffusion scores for all 4 benchmark years
Mean Med-R&E Diffusion scores for all 4 benchmark years
Mean Phy-AMC Diffusion scores for all 4 benchmark years