Introducción a la ‘altmetría’: fuentes, limitaciones y posibilidades

Rodrigo Costas

Center for Science and Technology Studies (CWTS-Leiden University)

Julho / 2016

Introducción a la ‘altmetría’: fuentes, limitaciones y posibilidades

Rodrigo Costas

Center for Science and Technology Studies (CWTS-Leiden University)

7 de Julho 2016

Outline

• What are altmetrics?• Concept(s)• Sources

• What do we know about altmetrics?• … a bit of theory• Coverage (time, fields and topics)• Main bibliographic characteristics• Others

• What can we do with them?

• The future?

3

Altmetrics? Concept(s)

• Altmetrics Manifesto (2010) (http://altmetrics.org/manifesto/)

• Wild west

– Simple definition: ‘any metric around research products, except

citations’

– Working definition: events on social and mainstream media platforms

related to scholarly content or scholars […] and are not the same as […]

citations (Haustein, Bowman, Costas, 2015)

– Frequently ‘defined’ by the data providers (e.g. Policy documents)

– Diversity!

• Main challenge: what do they mean?

Sources• Mendeley

• Altmetric.com

• PlumAnalytics

• ImpactStory

• PLoS – Article-level metrics

7

Plum Analytics

8

9

11

12

PloS ALM

13

15

Data inconsistencies

http://dx.doi.org/10.6084/m9.figshare.1041821

What do we know? A bit of theory

16

Altmetrics Manifesto (2010)

http://altmetrics.org/manifesto/

Framework of ‘acts’ around Research Objects

18

Otlet (1934): “set of facts or ideas

presented in the form of a text or image”

Bourdieu (1975): entities within

the scholarly community.

Access

Appraise

Apply

http://arxiv.org/abs/1502.05701

A discussion of the ‘promises’• Diversity of scientific publishing

• Strongest argument• More research is still necessary

• Broader dimensions of scientific performance• Do we agree with and understand these new dimensions?• What does a ‘tweet’ or a ‘Mendeley reader ’ mean from a

performance point of view?

• Speed• Is faster always better? Superficiality?• ‘Sleeping beauties’ or ‘Mendel syndrome’

• Openness• Openness is good• Transparency and consistency are more important

Coverage of publications per source

20

Adapted from: http://dx.doi.org/10.3145/epi.2014.jul.03

Exception to this: Mendeley (>70%)

Increasing coverage over time

Source: http://www.cwts.nl/pdf/CWTS-WP-2014-001.pdf

From Altmetric.com

Coverage by fields

22

Citations

Mendeley

http://dx.doi.org/10.1108/AJIM-12-2014-0173

• Some relevant patterns:

– Twitter: stronger in Social Sciences and General medicine,

weaker in Natural Sciences and Humanities

23

Coverage by fields

http://dx.doi.org/10.1108/AJIM-12-2014-0173

• Blogs and news media have a strong focus on

multidisciplinary journals!

24

Coverage by fields

http://dx.doi.org/10.1108/AJIM-12-2014-0173

Coverage by ‘topics’

25

Biomedical &

health

sciences

Social

sciences &

humanities

Mathematics &

Computer

sciences

Physical

sciences &

Engineering

Life &

Earth

sciences

Topics with the highest PP(tw1)

26

Main characteristics

• Moderate correlations for Mendeley and good filtering of highly cited publications

http://dx.doi.org/10.1371/journal.pone.0120495

Distribution by document types

Editorial Material:

An article that gives the opinions

of a person, group, or

organization. Includes editorials,

interviews, commentary,

discussions between individual,

post-paper discussions, round

table symposia, and clinical

conferences

Letters

News items

News, current

events, and recent

developments.

Opinions, discussions, current

events, recent developments

(‘novelty’)

http://dx.doi.org/10.1371/journal.pone.0120495

Collaboration

y = 0.4766x + 0.7498R² = 0.9706

y = 0.1323x + 0.0796R² = 0.8987

y = 0.0143x + 0.012R² = 0.9247

0

50,000

100,000

150,000

200,000

0.0

2.0

4.0

6.0

8.0

10.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Num

ber

of p

aper

s

Ave

rage

cit

atio

n or

soc

ial m

edia

rat

e

Number of authors per paper

Papers Citations Twitter Blogs Facebook Google+ News

http://dx.doi.org/10.1371/journal.pone.0120495

Number of pages

y = 0.7907ln(x) + 1.2369R² = 0.5912

y = -0.0061x + 0.8093R² = 0.3756

0

20,000

40,000

60,000

80,000

100,000

120,000

0.0

1.0

2.0

3.0

4.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Num

ber

of p

aper

s

Ave

rage

cita

tion

or

soci

al m

edia

rat

e

Number of pages

papers Citations Twitter Blogs Facebook Google+ News

http://dx.doi.org/10.1371/journal.pone.0120495

Number of references

y = 0.0553x + 0.9937R² = 0.9343

y = 0.0064x + 0.5542R² = 0.6644

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

0.0

2.0

4.0

6.0

8.0

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90 93 96 99 102

105

Num

ber o

f pap

ers

Aver

age

cita

tion

or s

ocia

l med

ia ra

te

Number of references

Papers Citations Twitter Blogs Facebook Google+ News

http://dx.doi.org/10.1371/journal.pone.0120495

Length of the titles

y = 0.4766x + 0.7498R² = 0.9706

y = -0.0039x + 1.18R² = 0.6085

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

0.0

1.0

2.0

3.0

4.0

12 17 22 27 32 37 42 47 52 57 62 67 72 77 82 87 92 97 102

107

112

117

122

127

132

137

142

147

152

157

162

167

172

177

182

187

192

197

202

Num

ber o

f pap

ers

Aver

age

cita

tion

or s

ocia

l med

ia ra

te

Title length

Papers Citations Twitter Blogs Facebook Google+ News

http://dx.doi.org/10.1371/journal.pone.0120495

35

Others: funny things

36

- Inconsistencies inter-platform (e.g. changes in

readerships counts in Mendeley, duplicates, meta

data errors, noise, etc.)

- Presence of ‘cyborgs’ and ‘bots’ among social

media users (http://doi.org/10.1002/asi.23456)

- Biases: geographic, language or even technical (e.g.

presence of DOIs, tradition of linking to papers,

etc.) (http://dx.doi.org/10.3145/epi.2014.jul.03)

37

Other data problems

What can we actually do?

• Mendeley

– Indicators! (same as with citations)

– Filtering highly cited publications

– Readership interests by types of users

• Altmetric.com

– Identifying topics with ‘social media interest’

– Altmetric landscapes

– Identification of ‘communities of attention’

Filtering highly cited publications

Mendeley

Journal Citation

Score

http://arxiv.org/abs/1507.02093

Readership activity by Mendeley users

40

http://dx.doi.org/10.6084/m9.figshare.1041819

PhDs

Professors

Twitter landscapes – E.g. African research

41

Maternal mortality

Sexually transmitted

diseases

Plant and soil

Ecology

Malaria

transmission

TuberculosisAIDS

Communities of attention

42

Communities of attention - exposure

43

http://goo.gl/kBu1li

http://sched.co/3fVP

Future?• Conceptualization of the new metrics

• Value?

• Standardization of tools and data

• Crucial problems in data quality and indicator construction

• Scalability • APIs

• Potential trivialization due to their limited use (“narcissism”)

• Manipulability

• Normalization & reliability

• Production and use of the new metrics

They must be used and accepted by the scientific community!

Opportunities

• Better understanding of scientific communication

– How are scientific publications perceived by social media users?

– Text analysis of tweets, hashtags, characterization of users

– Model types of impact (social, public attention, Public Understanding

of Science (PUS), ‘misunderstanding’, etc.)

• Possibilities of indicators

– New forms of dissemination (e.g. videos, blogs, etc.)

• Identify blogs, hastags, Twitters, news media, etc.

relevant for a discipline (e.g. to follow by new students)

• Informing scientists/policy makers of ‘communities of

attention’

• Supporting ‘Crowdfunding’ activities (finding ‘topics’,

audiences, etc.)

46

Thanks for your attention!Questions?

Practical example: visualize ‘altmetrics’ for a paper with a DOI

• Find on the web a paper with a DOI

(e.g. http://dx.doi.org/10.1073/pnas.1211286109)

• Go to http://www.altmetric.com/bookmarklet.php and drag & drop the ‘Altmetric it’ icon on the paper

• Alternatively write:

http://www.altmetric.com/details.php?doi=10.1073/pnas.1211286109

Other references

Report: Users, Narcissism and control.

http://research-acumen.eu/wp-content/uploads/Users-narcissism-and-control.pdf

The metric Tide. http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/The,Metric,Tide/2015_metric_tide.pdf

Review on Altmetrics.

http://recyt.fecyt.es/index.php/EPI/article/view/epi.2015.sep.09/21885