merging trends in peer review€¦ · emerging trends in peer-review. 2 program •classical peer...
TRANSCRIPT
![Page 1: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/1.jpg)
LIBRARY
[email protected] 12 December 2017
Midi de l’info scientifique
EMERGING TRENDS IN PEER-REVIEW
![Page 2: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/2.jpg)
2
PROGRAM
• Classical peer review
• Immediate publication with no formal review
• Immediate publication with post-publication
review
• Multiphase review: mix of pre- and post-publication
review
• Portable review
![Page 3: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/3.jpg)
3
CLASSICAL PEER REVIEW
http://archive.senseaboutscience.org/pages/peerreviewed
ucation.html
![Page 4: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/4.jpg)
4
ADVANTAGES OF PEER-REVIEW
• Anonymity of the review process → free
expression of critical views
• No interaction among reviewers → no influence
or pressure
• Improves the quality of scientific publications:
filtering out low quality papers, catching errors,
improving the writing…
![Page 5: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/5.jpg)
5
KEY OBJECTIONS TO
CLASSICAL PEER REVIEW
1. Delay
2. Bias against specific categories of paper
3. Social and cognitive biases
4. Unreliability
5. Inability to detect errors and fraud
6. Lack of transparency – unethical practices
7. Lack of recognition for reviewers
Walker et Rocha da Silva, 2015
![Page 6: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/6.jpg)
6
INTERACTIONS DURING
PRE-PUBLICATION REVIEWING
• EMBO Journal: cross-peer review (referees
comment on each others’ reports)
• eLife: online consultation of other referee’s reports
• Frontiers: Collaborative Review Forum (feedbacks
between reviewers, authors and editors)
![Page 7: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/7.jpg)
7
• a website and free service for academics to
track, verify and showcase their peer review
• Launched in 2012. In 2017: 200,000
researchers, with over 1 million reviews
![Page 8: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/8.jpg)
8
NEW PEER-REVIEWING IMPLIES NEW
REVIEWERS
● Editors-selected Reviewers
● Author-selected Reviewers
● Volunteered Reviewers (Open Peer Commentary)
● Community/Public Review
● What kind of rewards a reviewer can get?
Jubb, 2016 & Walker et Rocha da Silva, 2015
![Page 9: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/9.jpg)
Immediate publication with
no formal review
![Page 10: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/10.jpg)
10
NON-COMMERCIAL PRE-PRINT SERVERS
• Papers available to readers
• No prior review or minimal «access review»
• Example: ArXiv since 1991
![Page 11: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/11.jpg)
11
JOURNAL PUBLISHERS PREPRINT SERVERS
• Nature Precedings (stopped in 2012)
• PeerJ Preprints
![Page 12: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/12.jpg)
Immediate publication with
post-publication review
“Publication and Peer Review are two
independent concepts”
F1000Research
![Page 13: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/13.jpg)
13
POST-PUBLICATION REVIEW
F1000Research
![Page 14: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/14.jpg)
14
POST-PUBLICATION REVIEW
• Distinguish the publication and the Peer
Review
• Shorten the processes
• Crossed reviews increase quality
• Reviews considered as mini publications
Kriegeskorte, 2012
![Page 15: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/15.jpg)
15
POST-PUBLICATION REVIEW
https://f1000research.com/articles/6-1515/v4
![Page 16: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/16.jpg)
16
POST-PUBLICATION REVIEW
https://f1000research.com/articles/6-1515/v4
![Page 17: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/17.jpg)
17
POST-PUBLICATION REVIEW
https://www.atmos-chem-phys.net/17/14675/2017/acp-17-14675-
2017.html
![Page 18: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/18.jpg)
18
POST-PUBLICATION REVIEW
https://www.atmos-chem-phys.net/17/14675/2017/acp-17-14675-
2017.html
![Page 19: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/19.jpg)
19
OPEN REVIEW
● A review procedure in which author names AND
reviewer names are revealed
○ But reviewer identities are not necessary
revealed
○ Sometimes only an option
● Open Reports (Full reports are published)
● Increases interaction between authors and reviewers
which facilitates constructive feedback et dialog
Jubb 2016, Ross-Hellauer, 2017 & Walker et Rocha da Silva, 2015
![Page 20: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/20.jpg)
Multiphase review:
mix of pre- and post-publication review
![Page 21: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/21.jpg)
21
FRONTIERS IN NEUROSCIENCE
• Tiering system: «democratic» selection of
outstanding research
• «Tier 1»: publication after normal pre-
publication review
• «Tier 2»: invite authors with top viewed and
downloaded papers to write a «Focused
Review»
![Page 22: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/22.jpg)
22
PRE-PUBLICATION REVIEW
+ IN CHANNEL COMMENTARY
• Out of a sample of 53 journals using formal review, 24
provide the option to readers to comment on the article
• Only 17% of articles published in PLOS ONE had
comments
![Page 23: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/23.jpg)
23
PRE-PUBLICATION REVIEW
+ OUT OF CHANNEL COMMENTARY
Academic social networks and repositories for
documents published elsewhere:
• F1000 prime
• PubMed Commons
• Pubpeer.com
• ResearchGate
![Page 24: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/24.jpg)
Portable review
![Page 25: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/25.jpg)
25
PORTABLE REVIEW: «RECYCLING» OF
PREVIOUS REVIEWING REPORTS
• Reports from the review by a 1st journal
(which rejected the paper) are passed on to a
2nd journal
• Often for journals from the same publishers
(Nature Publishing Group, Biomedcentral...)
![Page 26: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/26.jpg)
26
COMMERCIAL PORTABLE REVIEW
• 1st step: Authors submit their papers for
review by a commercial review service
• 2nd step: authors (or the commercial service)
submit the paper along with the reports to a
journal
• Axios/Rubriq: charge authors a fee
• Peerage of science: receives revenue only
from journals and publishers
![Page 27: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/27.jpg)
27
TAKE HOME MESSAGE
• 2 major trends:
- Repository with only access review (e.g. ArXiv)
- Publishers with non-selective review (e.g. PLoS
ONE, Frontiers…)
• New innovative forms, still rare:
- Open review, which remove reviewer anonymity
- Interactive review between reviewers, authors and
editors
- Informal reader commentary
![Page 28: MERGING TRENDS IN PEER REVIEW€¦ · EMERGING TRENDS IN PEER-REVIEW. 2 PROGRAM •Classical peer review •Immediate publication with no formal review •Immediate publication with](https://reader030.vdocuments.mx/reader030/viewer/2022040409/5ec8652e965bed248e27e444/html5/thumbnails/28.jpg)
28
BIBLIOGRAPHY
• ACP. Consulté le 12 décembre 2017. https://www.atmospheric-chemistry-and-
physics.net/index.html.
• F1000Research. Consulté le 12 décembre 2017. https://f1000research.com/.
• Hunter, J. (2012). Post-Publication Peer Review: Opening Up Scientific
Conversation. Frontiers in Computational Neuroscience 6.
• Jubb, M. (2016). Peer review: The current landscape and future trends: Peer
review landscape. Learned Publishing 29, 13–21.
• Kriegeskorte, N. (2012). Open Evaluation: A Vision for Entirely Transparent Post-
Publication Peer Review and Rating for Science. Frontiers in Computational
Neuroscience 6.
• Ross-Hellauer, T. (2017). What is open peer review? A systematic review.
F1000Research 6, 588.
• Sense about Science. Consulté le 12 décembre 2017.
http://archive.senseaboutscience.org/index.html.
• Walker, R., and Rocha da Silva, P. (2015). Emerging trends in peer reviews - a
survey. Frontiers in Neuroscience 9.