altmetrics for large, multidisciplinary research groups alexandra jobmann (ipn) & isabella...
TRANSCRIPT
Altmetrics for large, multidisciplinary research groups
Alexandra Jobmann (IPN) & Isabella Peters (ZBW)
Anita Eppelin (ZB MED), Christian Hoffmann (Universität St. Gallen), Sylvia Künne
(IfW), & Gabriele Wollnik-Korn (ZB MED)
A case study of the Leibniz Association
Bibliometrics
Seite 2
3
footprints
impact
Altmetrics
Seite 3
12
1
1
1
footprints
impact
Motivation for study
1. Initiatives that demand for new approaches in research evaluation (e.g.,
DORA)
2. Leibniz Association’s evaluation guidelines ask for appropriate public
outreach and engagement in public discourse how to measure?
3. Research showed significant disciplinary differences: coverage and
impact (Haustein & Siebenlist, 2011; Haustein et al., 2013; Holmberg & Thelwall, 2013; Mohammadi & Thelwall, 2013)
Seite 4
Research Questions
1. Where and to what extent are the publications of the institutions of
the Leibniz Association covered on social media platforms?
2. What impact do publications of the members of the Leibniz
Association have on users (i.e., altmetrics)?
3. What tools can be used to assess research impact? What
challenges might occur?
Seite 5
Methods
1. Webometric Analyst
for the collection of missing
DOIs via Crossref
2. Checked DOIs and retrieved DOIs
3. ImpactStory
for the collection of DOI-based
altmetrics data (e.g., Twitter
mentions, Mendeley readers)
Seite 6
• Disciplines of the Leibniz Association1. humanities and educational research2. economics, social sciences, spatial
research3. life sciences4. mathematics, natural sciences,
engineering5. environmental sciences
• 2-3 institutes of each discipline• Articles in conferences/ journals and
book chapters• Publication years: 2011, 2012
Data
Seite 7
Articles of 12 institutes 2.834
Correct DOIs 1.762 (62%)
Altmetrics 1.739 (99%)
Results
• Mendeley attracts readers across disciplines• Enviromental Science reluctantly uses Twitter
Seite 8
A: Humanities and Educational Re-search (n=128)
B: Economics, Social Sciences, Spatial Research (n=387)
C: Life Sciences (n=454)
D: Mathematics, Natural Sciences,
Engineering (n=429)
E: Environmental Sciences (n=341)
0
1000
2000
3000
4000
5000
6000
0
50
100
150
200
250
300
350
Mendeley readers tweets (Altmetrics.com)
Me
nd
ele
y r
ea
de
rs
twe
ets
Results
• Social media use is discipline-specific
Seite 9
A: Humanities and Educa-tional Research (n=128)
B: Economics, Social Sciences, Spatial Research (n=387)
C: Life Sciences (n=454) D: Mathematics, Natural Sciences, Engineering
(n=429)
E: Environmental Sciences (n=341)
0
2
4
6
8
10
12
14
16
blog Facebook Google+ f1000
Results
• Where do disciplines find their readers?
Seite 10
blog
Google+
tweets (Altmetrics.com)
Mendeley readers
f1000
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
A: Humanities and Educational Research (n=128) B: Economics, Social Sciences, Spatial Research (n=387)
C: Life Sciences (n=454) D: Mathematics, Natural Sciences, Engineering (n=429)
E: Environmental Sciences (n=341)
Results
• Altmetrics can complement missing data (e.g., life sciences)
Seite 11
0
10
20
30
40
50
60
70
0
20
40
60
80
100
120
140
160
180
PubMed Central citations Mendeley readersarticles
cit
ati
on
s
rea
de
rs
0
2
4
6
8
10
12
14
16
18
20
0
20
40
60
80
100
120
PubMed Central citations Mendeley readersarticles
cit
ati
on
s
rea
de
rs
institute 1 institute 2
Results
• Institutes from the same discipline (e.g., life sciences) find readers on
different platforms
Seite 12
C1 C20
1
2
3
4
5
6
7
8
9
0
50
100
150
200
250
300
blog Facebook Google + tweets (Altmetrics.com)
twe
ets
Lessons Learned
• Chosen tools determine quality of data• Tools and altmetrics providers change settings
• Chosen identifiers affect data• PubMedID is more popular than DOI
• Missing or erroneous identfiers in social media
• Multiple identifiers for one publication
Underestimation of real numbers
· Collection of publication data• Missing DOIs on institutes‘ websites• Double-entry of publication on websites
• Carry out data download at the same time
Seite 13
Lessons Learned
• Aggregated numbers may give wrong picture (e.g., discipline basis)• Sum html views: 2,447 (n=2) - Sum readers: 921 (n=76)
Seite 14
0
10
20
30
40
50
60
0
200
400
600
800
1000
1200
1400
1600
1800
Mendeley readers PLoS html viewsarticles of institute A1 (humanities and educational research)
read
ers
Htm
l vi
ews
Lessons Learned
• Mendeley is the platform which covers a substantial amount of
papers and shows reasonable user activity
• Look for good coverage/ usage ratio
• However, some disciplines prefer other platforms• Get to know the community preferences
• Respect reader/ community choices
• Altmetrics should not substitute, but can complement citation data
•Comparability of altmetrics not given – same situation as in traditional
citation analysis
Seite 15
Thank you!
Seite 16
Alexandra Jobmann, [email protected]
Prof. Dr. Isabella Peters, [email protected]
References
• Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of Informetrics, 5(3), 446–457.
• Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2013). Coverage and adoption of altmetrics sources in the bibliometric community. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, Vol. 1 (pp. 468-483). Retrieved from http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf
• Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, Vol. 1 (pp. 567-582). Retrieved from http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf
• Mohammadi, E. & Thelwall, M. (2013). Assessing the Mendeley readership of social sciences and humanities research. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vienna, Austria, Vol. 1 (pp. 200-2014). Retrieved from http://www.issi2013.org/Images/ISSI_Proceedings_Volume_I.pdf
Seite 17