2014 crossref annual meeting peer review panel: pre: securing trust & transparency in peer...
DESCRIPTION
Adam Etkin presents the PRE system of evaluating peer review systems at the 2014 CrossRef annual meeting on a panel on innovations in peer reviewTRANSCRIPT
Securing Trust & Transparency in Peer Review
http://www.pre-val.org
@PeerReviewEval
Adam Etkin, Founder and Managing Director
November 2014
A service that works with the publisher and journal to provide independent validation of the review process
A badge that publishers can display in various places – search results, article pages, article-level metrics – to signal to readers that quality peer review has been conducted
A window into a given journal’s peer review process accessible by end users
PRE-val answers the most basic, and important, question about scholarly works: “Has this article really been peer reviewed?”
PRE-val supports quality peer review.
What is PRE-val?
2
Why do we need PRE-val?
Traditional peer review & scholarly publishersseem to be under constant criticism.
Emergence of “predatory” publishers
High-profile cases of faulty research being published
Difficulty distinguishing peer-reviewed research from non-reviewed content in journals – publishing in a peer-reviewed journal does not equate with peer review for every article
“Publish then filter” and “Publishing is a button” attitude
3
Why do we need PRE-val?
4
Surveys show peer review is valued by researchers & authors.
Sources: Sense About Science; Taylor & Francis; CIBER Research; NPG/Palgrave Macmillan Author Insights survey
“The qualitative data also point to the fact that peer review is the central pillar of trust.”
University of Tennessee and CIBER Research Ltd, December 2013
Why do we need PRE-val?
5
Most (69%) researchers are satisfied with the current system of peer review but only a third think
that the current system is the best we can do
Most (69%) researchers are satisfied with the current system of peer review but only a third think
that the current system is the best we can do
Most (84%) believe that without peer review there would be no
control in scientific communication
Most (84%) believe that without peer review there would be no
control in scientific communication
78% of OA authors prefer traditional,
rigorous peer review
78% of OA authors prefer traditional,
rigorous peer review
Only 20% want basic check followed by post-
publication review
Almost all researchers (91%) believe that their last paper was
improved as a result of peer review
Almost all researchers (91%) believe that their last paper was
improved as a result of peer review
While many want a faster process with fewer rounds, the overwhelming majority (~70%) prefer to wait for
thorough review
While many want a faster process with fewer rounds, the overwhelming majority (~70%) prefer to wait for
thorough review
93% of science authors consider quality of
PEER REVIEW when deciding where to
publish
Less time to review more information
6
*B.-C. Björk, R. Annikki, and M. Lauri. Global annual volume of peer reviewed scholarly articles and the share available via different Open Access options*The National Science Board estimates the average annual growth of the indexes within the Web of Science to be 2.5% (See: Science and Engineering Indicators 2010, chapter 5, page 29)*The stm report: An overview of scientific and scholarly journal publishing (2009)*http://dan.corlan.net/cgi-bin/medline-trend?Q=
Total Peer Reviewed Journals
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
2006 2007 2008 2009 2010 2011 2012 2013
Year
Jou
rnal
s
28,000+ journals, and growing Total Peer Reviewed Articles Published
0
500,000
1,000,000
1,500,000
2,000,000
2,500,000
2006 2007 2008 2009 2010 2011 2012 2013
Year
Art
icle
s
2 million articles in 2013
More information than ever, less time to sort through it all.
Not just one “right” approach
The number of varied approaches to peer review is increasing, yet readers often assume it is a uniform practice
Journals with high standards have no clear way of routinely showing these standards at the article level
Trends toward “lite” peer review & critics are dominating the discussion, while quality has a harder time finding a voice
Peer review is not one thing, and it cannot be assumed.
This leads to confusion and questions about trust . . .
7
Measurements Abound
Metrics/Filters = Knowledge = Power
A Lesson From The Car Industry
Looking for a new car?• MPG• Size• Speed• Features• Safety• Cost
Evaluating a journal?• Impact Factor• Audience• Speed• Altmetrics• Peer Review (PRE) • Cost
10
Community
Different approaches, different services….one goal: Working together to educate and support
We believe in peer review
Recognizes journals with an editor-in-chief or other overseeing editor
PRE-val encourages journals to use quality reviewers
Provides transparency into iteration through review to improve the science and clarity of a paper
Helps promote use of best practices, which are markers of commitment to better peer review approaches
PRE-val creates incentives to use best practices in peer review.
11
Technical flow – Low/No work by publisher
12
First, we sit down together to learn about your process. PRE-val configuration takes into account your unique processes and preferences related to what you want to make public/transparent.
Publisher Manuscript Submission/Peer Review Tracking System
<metadata>PEER REVIEW</metadata>
PRE API
#1
Low/no development work on part of publisher or
manuscript submission/peer review system
Publisher places badge on - Journal article page - Search results - Aggregator sites - Article metrics - anywhere else a signal of peer review is important
Info about peer review process:•Type of Review•Rounds of review•Roles participating
Other measures of screening quality:•COPE member?•Plagiarism screening?•Retraction policy?
Optional info:•Reviewer comments•Reviewer names
14
Securing Trust & Transparency in Peer Review
http://www.pre-val.org
@PeerReviewEval
Adam Etkin, Founder and Managing Director
November 2014