alternative peer review : quality management for 21 st century scholarship
DESCRIPTION
Alternative Peer Review : Quality Management for 21 st Century Scholarship. http://www.public.iastate.edu/~gerrymck/APR.ppt. Gerry McKiernan Science and Technology Librarian and Bibliographer Iowa State University Library Ames IA USA. [email protected]. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/1.jpg)
Alternative Peer Review:Quality Management
for 21st Century Scholarship
Gerry McKiernan
Science and Technology Librarian
and Bibliographer
Iowa State University Library
Ames IA
http://www.public.iastate.edu/~gerrymck/APR.ppt
![Page 2: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/2.jpg)
Workshop on Peer Review in the Age of Open Archives
International School for Advanced StudiesInterdisciplinary Laboratory
Trieste, ItalyMay 23-24, 2003
![Page 3: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/3.jpg)
THANK YOU!Workshop Advisory Board [(Marco Fabbrichesi (INFN/SISSA Italy), Stevan Harnad (University of Southampton, UK), Stefano Mizzaro (University of Udine, Italy) and Corrado Pettenati (CERN Library, Geneva, Switzerland)]Iowa State University, Faculty Senate, Committee on Recognition and DevelopmentEuropean CommissionIowa State University LibraryHeike Kross, Ph.D.
![Page 4: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/4.jpg)
DISCLAIMER (1)The screen prints selected for this presentation are for educational
purposes and their inclusion does not constitute an endorsement of an associated product, service,
place, or institution.
![Page 5: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/5.jpg)
DISCLAIMER (2)The views and opinions expressed
in this presentation are those of the presenter and do not constitute
an endorsement by Iowa State University or its Library.
![Page 6: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/6.jpg)
NOTICE
No editors, authors, or referees were harmed in the preparation of
this presentation.
![Page 7: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/7.jpg)
http://www.sciencecartoonsplus.com/
![Page 8: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/8.jpg)
Giuseppe De Nittis (1846-1884)The Macchiaioli / Italian Impressionists
![Page 9: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/9.jpg)
Campo di Biche (1875)
![Page 10: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/10.jpg)
PEER REVIEW: DEFINITION
“Peer review is the assessment by an expert of material submitted for publication.”
Carin M. Olson, “Peer Review of the Biomedical Literature,” American Journal of Emergency Medicine 8 no.4 (July 1990): 356-358.
![Page 11: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/11.jpg)
PEER REVIEW: PURPOSES
Carin M. Olson, “Peer Review of the Biomedical Literature,” American Journal of Emergency Medicine 8 no.4 (July 1990): 356-358.
Peer review helps to ensure that published research is:
Important Original
Timely Technically-reliable
Internally-consistent Well-presented
Benefited from guidance by experts
![Page 12: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/12.jpg)
PEER REVIEW: STRENGTHS
Anne C. Weller, Editorial Peer Review: Its Strengths and Weaknesses. (Medford, NJ: Information Today, 2001).
The underlying strength of peer review is“…the concerted effort by large numbers of researchers and scholars who work to assure that valid and valuable works are published and conversely to assure that invalid or non-valuable works are not published … .”
![Page 13: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/13.jpg)
“Houston, We Have a Problem!”
![Page 14: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/14.jpg)
PEER REVIEW: PROBLEMSSubjectivity
Bias
Abuse
Detecting defects
Fraud and Misconduct
DelayFytton Rowland, “The Peer-Review Process,”
Learned Publishing 15 no. 4 (October 2002): 247-258.
Report version: http://www.jisc.ac.uk/uploaded_documents/rowland.pdf
![Page 15: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/15.jpg)
SUBJECTIVITYSummary rejections by editor without sending the paper to referees
Choice of referees by the editor (choosing for example, a known harsh referee for a paper the editor wishes to see rejected)
![Page 16: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/16.jpg)
BIASDiscrimination against authors because of their nationality, native language, gender or host institution
Situations where author and referee are competitors in some sense, or belong to warring schools of thought
![Page 17: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/17.jpg)
ABUSEToo many articles out of one piece of research, or duplicate publicationIntellectual theft: omission or downgrading of junior staff by senior authorsPlagiarism (stealing others yet unpublished work that has been sent for review)Delaying publication of potentially competing research
![Page 18: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/18.jpg)
DETECTING DEFECTSIdentification of factual errors within submission
![Page 19: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/19.jpg)
FRAUD and MISCONDUCTFabrication of results
Falsification of data
False claim of authorship for results
![Page 20: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/20.jpg)
DELAY“There is much muttering about publication delay, a real enough problem, especially in paper publication, but peer review itself is often responsible for as much of the delay as the paper publication and distribution process itself.”
Stevan Harnad
Stevan Harnad, “Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals, in Scholarly
Publication: The Electronic Frontier, edited by Robin P. Peek and Gregory B. Newby (Cambridge MA: MIT Press, 1996).
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html
![Page 21: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/21.jpg)
http://www-marketing.wharton.upenn.edu/ideas/pdf/peerpaper.pdf
![Page 22: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/22.jpg)
“Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias,
easily abused, poor at detecting gross defects, and almost useless in detecting fraud.”
Richard Smith, “Opening Up BMJ Peer Review,”
BMJ 318 (7175) (January 2 1999): 4-5
Richard SmithEditor, BMJ
![Page 23: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/23.jpg)
Stephen Lock, A Difficult Balance: Editorial Peer Review in Medicine (Philadelphia, PA: ISI Press, 1986).
![Page 24: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/24.jpg)
![Page 25: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/25.jpg)
EXAMPLE
Jan Hendrik Schön
![Page 26: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/26.jpg)
http://archive.salon.com/tech/feature/2002/09/16/physics/
-
![Page 27: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/27.jpg)
Science
Nature
http://physicsweb.org/article/news/6/9/15/
![Page 28: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/28.jpg)
http://agenda.cern.ch/fullAgenda.php?ida=a01193
![Page 29: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/29.jpg)
RECOMMENDATIONSWorkshop on the Open Archives Initiative (OAI)
and Peer Review Journals in Europe, CERN, Geneva Switzerland, March 22-24, 2001
“The participants were unanimous in their belief that the certification of scholarly work remains a fundamental part
of a system for scholarly communication.”
“It was [also] generally believed that the electronic environment allows for novel approaches to accord a
stamp of quality to scholarly works.”
Alison Buckholtz, Raf Dekeyser, Melissa Hagemann, Thomas Krichel, and Herbert Van de Sompel, “Open Access: Restoring Scientific Communication to Its Rightful Owners,” European Science Foundation Policy Briefing 21 (April 2003): 1-8.
http://www.arl.org/sparc/SPB21_OAI.pdf
![Page 30: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/30.jpg)
“Let us be imaginative in exploring the remarkable possibilities of this
brave new medium.”
Stevan Harnad, “Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals, in Scholarly Publication: The Electronic
Frontier, edited by Robin P. Peek and Gregory B. Newby (Cambridge MA: MIT Press, 1996).
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad96.peer.review.html
![Page 31: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/31.jpg)
![Page 32: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/32.jpg)
![Page 33: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/33.jpg)
![Page 34: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/34.jpg)
![Page 35: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/35.jpg)
“Let us be more imaginative in exploring the remarkable possibilities of this
brave new medium.” With Apologies to Stevan Harnad
![Page 36: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/36.jpg)
![Page 37: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/37.jpg)
http://lockss.stanford.edu/
TM
![Page 38: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/38.jpg)
LOCKSSFor centuries libraries and publishers have had stable roles: publishers produced information; libraries kept it safe for reader access. There is no fundamental reason for the online environment to force institutions to abandon these roles.
The LOCKSS model capitalizes on the traditional roles of libraries and publishers. LOCKSS creates low-cost, persistent digital "caches" of authoritative versions of http-delivered content.
![Page 39: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/39.jpg)
LOCKSSThe LOCKSS software enables institutions to locally collect, store, preserve, and archive authorized content thus safeguarding their community's access to that content.
The LOCKSS model enforces the publisher's access control systems and, for many publishers, does no harm to their business models.
![Page 40: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/40.jpg)
LAMPSS
Lots of Alternative Models Provide Sensible Solutions
![Page 41: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/41.jpg)
![Page 42: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/42.jpg)
DISCLAIMER
The alternative peer review models profiled are for informational and
educational purposes only and do not
necessarily constitute an endorsement.
ON
![Page 43: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/43.jpg)
ALTERNATIVE PEER REVIEWNeo-ClassicalCertification-basedOpen Peer ReviewCommentary-basedCollaborately-filtered
Institution-basedCitation-basedIndex-basedMetadata-basedComputer-assistedNO Peer ReviewModerator-basedTier-based
![Page 44: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/44.jpg)
NEO-CLASSICAL PEER REVIEW
![Page 45: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/45.jpg)
Neo-Classical Peer Review
![Page 46: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/46.jpg)
Neo-Classical Peer Review
![Page 47: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/47.jpg)
Neo-Classical Peer Review
![Page 48: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/48.jpg)
CERTIFICATION-BASED
![Page 49: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/49.jpg)
Certification-Based“The process of pre-publication peer
review could be improved and become a more reliable indicator of manuscript quality if reviewers were trained in, and subsequently applied systematically, critical skills and use of a hierarchy of evidence to classify submitted articles being reviewed.”
Stephen Pritchard , “Peer Review - a Proposal for Change,”Paper presented at Thinking Globally - Acting Locally:
Medical Libraries at the Turn of an Era,8th European Conference of Health and Medical Libraries,
September 16-21, 2002, Deutsche Zentralbibliothek für Medizin, Köln, Germany.http://www.zbmed.de/eahil2002/abstracts/pritchard.pdf
![Page 50: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/50.jpg)
OPEN PEER REVIEW
![Page 51: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/51.jpg)
Open Peer ReviewIDENTIFICATION OF REVIEWERS / SIGNED REVIEWS
BMJ bmj.com
BioMed Centralbiomedcentral.com
eMJA (Medical Journal of Australia)www.mja.com.au/public/information/project.html
![Page 52: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/52.jpg)
COMMENTARY-BASED
![Page 53: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/53.jpg)
Commentary-basedReaders can comment before and/or after classic peer
review, or instead of classic peer review
Electronic Transactions on Artificial Intelligence (www.etaij.org)
OPEN REVIEW / REFEERING
Journal of Interactive Media in Education (www-jime.open.ac.uk)
PRE- AND POST- COMMENTARY
Psycoloquy
(psycprints.ecs.soton.ac.uk)
POST PEER REVIEW COMMENTARY)
![Page 54: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/54.jpg)
Electronic Transactions on Artificial Intelligence
![Page 55: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/55.jpg)
Journal of Interactive Media in Education
![Page 56: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/56.jpg)
http://psycprints.ecs.soton.ac.uk/
![Page 57: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/57.jpg)
![Page 58: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/58.jpg)
![Page 59: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/59.jpg)
COLLABORATIVELY-FILTERED
![Page 60: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/60.jpg)
Collaboratively-Filtered
“DEFINITION: “Guiding people's choices of what to read, what to look at, what to watch, what to listen to (the filtering part); and doing that guidance based on information gathered from some other people (the collaborative part)."
Paul Resnick
http://www.cni.org/Hforums/cni-announce/1996/0031.html
![Page 61: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/61.jpg)
ResearchIndex / CiteSeer
http://www.researchindex.com
![Page 62: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/62.jpg)
INSTITUTION-BASED
![Page 63: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/63.jpg)
Institution-BasedINSTITUTIONAL INITIATIVES
DSpace™ (MIT)www.dspace.org
eScholarship (University of California)escholarship.cdlib.org
Glasgow ePrint Service (University of Glasgow)eprints.lib.gla.ac.uk
![Page 64: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/64.jpg)
CITATION-BASED
![Page 65: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/65.jpg)
Citation-Based Citations to Open Access / OAI-compliant
documents are indicators of document importance
http://citebase.eprints.org/
![Page 66: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/66.jpg)
![Page 67: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/67.jpg)
![Page 68: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/68.jpg)
![Page 69: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/69.jpg)
![Page 70: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/70.jpg)
INDEX-BASED
![Page 71: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/71.jpg)
Index-Based INDEXING OF EPRINTS BY COMMERICAL ABSTRACTING AND
INDEXING SERVICE
Chemical Abstracts (American Chemical Society) (CAS) indexes select appropriate e-prints from the arXiv.org eprint server as well from the Chemical Preprint Service (Elsevier)Its “selection criteria for this kind of electronic document are essentially the same as for the traditional printed documents: they must report new information of chemical or chemistry-related interest and must be original publications. Also, the electronic publication must be publicly available and have some relative permanence ….”
Eric Shively / Chemical Abstracts Service
![Page 72: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/72.jpg)
Index-Based “CAB Abstracts doesn’t currently include Eprints or Preprints, but we are looking at the implications and possible mechanisms for accessing and indexing Eprints and/or Preprints related to the applied life sciences.”
Tracy Shaw / CAB International
![Page 73: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/73.jpg)
METADATA-BASED
![Page 74: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/74.jpg)
Metadata-Based<oai-quality>
<category>internal</category>
<process>
peer review
</process>
<organization>
CERN
</organization>
<policies>
http://www.cern.ch/policies/review.html
</policies>
</oai-quality>
William Y. Arms, “Quality Control in Scholarly Publishing On The Web. What Are the Alternatives to Peer Review?” PowerPoint presentation given at the Workshop on the Open
Archives Initiatives (OAI) and Peer Review Journals in Europe, March 22-24, 2003, CERN, Geneva, Switzerland.http://www.cs.cornell.edu/wya/papers/CERN-2001.ppt
![Page 75: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/75.jpg)
COMPUTER-ASSISTED
![Page 76: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/76.jpg)
Computer-Assisted (1)SOFTWARE THAT ASSISTS IN THE EVALUATION OF A
SUBMITTED MANUSCRIPTA Software Program to Aid in Peer ReviewAlvar Loria and Gladys Faba
Objective: To characterize a personal computer-based software program developed as an aid to peer review of medical papers. Design: The software is a Windows-based application that records automatically a numeric score to a series of questions related to 8 sections of scientific papers (introduction, methods, results, and discussion, plus 4 other sections). The questions and sections vary according to type of paper (original reports, case reports, or reviews), and the final output is a score with a maximum of 100 for a "perfect" paper. The software was tested using a single reviewer to judge 289 papers (169 original reports, 50 case reports, and 70 reviews) from 44 Mexican medical journals. All statistical analysis of scores were done with nonparametric tests.
![Page 77: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/77.jpg)
Computer-Assisted (2)Results: The paper scores ranged from 29 to 97 with slightly higher median and less dispersion of scores for reviews as compared with original reports and case reports, but these differences did not reach significance. Two observations suggest that the software operated reasonably well: a) there were some differences in the section scores by type of paper that agreed well with differences in their complexity; b) the journal scores showed an association with their number of original papers and their percentage of original papers (Kruskal-Wallis test, P=.06 and 0.07, respectively). Conclusions: The software operated reasonably well when used to compare the relative quality of 289 papers. The validity of the program is restricted in this study to the experience of 1 reviewer. An analysis of the raw scores helped in detecting some ambiguous and redundant questions that have been modified in an improved version. The program has a potential as a training tool for inexperienced reviewers or as a scorekeeper for experienced peer reviewers.
Alvar Loria and Gladys Faba, “A Software Program to Aid in Peer Review,” Abstract of paper presented at the Third International Congress on Biomedical Peer Review and Global Communications ,
September 18-20, 1997, Prague, Czech Republic. http://www.ama-assn.org/public/peer/arev.htm
![Page 78: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/78.jpg)
NO PEER REVIEW
![Page 79: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/79.jpg)
NO Peer Review
http://xxx.arXiv.cornell.edu
![Page 80: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/80.jpg)
MODERATOR-BASED
![Page 81: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/81.jpg)
Moderator-based (1)The intent of this model is to allow the widest range of scientific manuscripts to be archived, searched, and distributed electronically at the lowest possible cost.This would be accomplished through very minimal filtering and subsequent placement of eprints on a non-commercial archival server by a subject-specific Moderator appointed by a society (or consortia of societies).A society-appointed Editorial Board (with double-blind peer review approved by the non-profit Peer Review Inc. organization) would then the identify the most important materials from among these archived items, and the stamp of approval for these items would be included in a secondary Virtual Collection.
![Page 82: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/82.jpg)
Moderator-based (2)There are no direct submissions to the Editorial Board; manuscripts would be directed to the Editorial Board in one of three ways:
1. nominated by the eprint Moderator upon receipt for the archival server,
2. notification sent to the Editorial Board when a threshold number of hits are generated by any one manuscript on the archive server, and
3. nominated by readers of material from the archive; this process requires a letter of support outlining the importance of the work to the Editorial Board.
![Page 83: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/83.jpg)
Moderator-based (3)The Virtual Collection could be produced as a variety of products:
enhanced abstracts email threads (with comments) virtual reviews of sub-disciplines SDIs (selective dissemination of information) current awareness tools
This process: reduces the load on the Editorial Boards, which results in a faster review process; differentiates those items worthy of higher recognition from those worthy of archiving, making it easier for a reader to filter material, based upon a society and discipline authority (rather than commercial reasoning);provides for search/browse/sdi from the Virtual Collection for filtered info, reducing this more expensive option for only those items recognized as of the highest quality.
![Page 84: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/84.jpg)
David Stern, “The eprint Moderator Model,” Newsletter on Serials Pricing Issues no. 214 (February 8, 1999).
http://www.lib.unc.edu/prices/1999/PRIC214.HTML#214.5
1
2 3
![Page 85: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/85.jpg)
TIER-BASED
![Page 86: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/86.jpg)
Tier-basedTwo separate domainsStandard Tier Any and all submissions would be accepted after a cursory examination of or other pro forma certification. The review process could be “minimally labor-intensive, perhaps relying primarily on an automated check of author institutional affiliation, prior publication record, research grant status, or other related background; and involve human labor primarily to adjudicate incomplete or ambiguous results of an automated pass.”
![Page 87: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/87.jpg)
Tier-basedUpper Tier “At some later point (which could vary from article to article, perhaps with no time limit), a much smaller set of articles would be selected for the full peer review process. The initial selection criteria for this smaller set could be any of a variety of impact measures, to be determined, and based explicitly on their prior widespread and systematic availability and citability: e.g., reader nomination or rating, citation impact, usage statistics, editorial selection, ... .”
Paul Ginsparg, “Can Peer Review be Better Focused?,” Science & Technology Libraries 22 No. 3/4 (In press).
http://arxiv.org/blurb/pg02pr.html
![Page 88: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/88.jpg)
DISCLAIMER
Presented for Your Consideration
OFF
![Page 89: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/89.jpg)
FREEDOM OF IDEAS
http://www.nrm.org/exhibits/current/four-freedoms.html
![Page 90: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/90.jpg)
‘FUTURE OF IDEAS’ The explosion of innovation we have seen
in the environment of the Internet was not conjured from some new, previously
unimagined technological magic; instead, it came from an ideal as old as the nation. Creativity flourished there because the
Internet protected an innovation commons.
![Page 91: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/91.jpg)
‘FUTURE OF IDEAS’
The Internet’s very design built a neutral platform upon which the widest range of
creators could experiment.
The legal architecture surrounding it protected this free space so that culture and information –
the ideas of our era–could flow freely and inspire an unprecedented breadth of expression.
Lawrence Lessig, The Future of Ideas: The Fate of the Commons in a Connected World
. (New York: Random House, c2001).
![Page 92: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/92.jpg)
“IT’S
NOT
ABOUT
PUBLICATION;
IT’S
ABOUT
IDEAS.”
![Page 93: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/93.jpg)
The Association of Learned and Professional Society Publishers (ALPSP) Survey
Authors and Electronic PublishingScholarly research communication has seen far-reaching developments in recent years. Most journals are now available online as well as in print, and numerous electronic-only journals have been launched; the Internet opens up new ways for journals to operate. Authors have also become conscious of alternative ways to communicate their findings, and much has been written about what they ought to think.
![Page 94: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/94.jpg)
ALPSP felt that it would be timely to discover what they actually thought and what they actually did. This survey aimed to discover the views of academics, both as authors and as readers. Some 14,000 scholars were contacted across all disciplines and all parts of the world, and nearly 9% responded; their detailed comments make thought-provoking reading.
Alma Swan and Sheridan Brown. Authors and Electronic Publishing: The ALPSP Research Study on Authors' and Readers’ Views of Electronic Research Communication. (West Sussex, UK: The Association of Learned
and Professional Society Publishers, 2002).http://www.alpsp.org/pub5.htm
![Page 95: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/95.jpg)
“When asked to predict what would be the most common form of quality
control in five years time, only a bare majority answered
‘traditional peer review’.”
Fytton Rowland, “The Peer-Review Process,” Learned Publishing 15 no. 4 (October 2002): 247-258.
Report version: http://www.jisc.ac.uk/uploaded_documents/rowland.pdf
![Page 96: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/96.jpg)
FURTHERMORE …16% said that the referees would no longer be anonymous27% said that traditional peer review would be supplemented by post-publication commentary 45% expected to see some changes in the peer-review system within the next five years
Fytton Rowland, “The Peer-Review Process,” Learned Publishing 15 no. 4 (October 2002): 247-258.
Report version: http://www.jisc.ac.uk/uploaded_documents/rowland.pdf
![Page 97: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/97.jpg)
Importance of the Peer Review Process
0102030405060708090
100Peer-reviewed
Refs' commentspublished
Referees identified
Public commentary oneprints
Post-publication publiccommentary
Ability to submitcomments
http://www.alpsp.org/pub5.ppt
![Page 98: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/98.jpg)
What is Gray/Grey Literature ?Papers are often written to inform funding bodies
about the results of research projects, to support grant applications, to inform rapidly a specific scientific community, to present preliminary results at conferences or as dissertations.
Such material is disseminated quickly, often in limited numbers, before or without the formal publication process. Such documents are called non-conventional or grey literature.
http://www.kb.nl/infolev/eagle/what_is_gl.htm
![Page 99: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/99.jpg)
The Value of Grey LiteratureGrey literature is really a type of informal
communication, which on a scale of formality, fits in somewhere between conversation and
normal publication. A formal publication may follow later but in many cases - contrary to the
common assumption - these papers may not been made publicly available at all.
http://www.kb.nl/infolev/eagle/what_is_gl.htm
![Page 100: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/100.jpg)
Nevertheless, grey publications may contain comprehensive, concrete and up-to-date information on research findings, and investigations have shown, that even when grey documents are published officially at a
later stage, detailed information on techniques, methods, measured values and details of experiments are
frequently omitted.
For these details of importance for further research, the non-conventional literature is then the first and only
source of information.
http://www.kb.nl/infolev/eagle/what_is_gl.htm
![Page 101: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/101.jpg)
Veterinary Medicine12 Major Veterinary Medicine JournalsOverall, 6.38% of cited literature was Gray/Grey Literature
The figures for individual journals ranged from about 2.5 % to 10% gray/grey literature
Research journals cited a higher percentage of Gray/Grey Literature than did Clinical titles
William H. Weise and Nancy Pelzer, “Bibliometric Study of Grey Literature in Core Veterinary Medicine Journals,” Journal of the
Medical Library Association 91 no. 4 (October 2003): In press.
![Page 102: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/102.jpg)
Indexing and Abstracting Services
SIGLE: System for Information on Grey Literature
National Technical Information Service (NTIS)
PsychINFO (Psychological Abstracts)
![Page 103: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/103.jpg)
SIGLESystem for Information on Grey
LiteratureGrey literature documents covered
by SIGLE are technical or research reports, preprints, committee reports, working papers, dissertations, conference papers, discussion and policy papers, government reports, market surveys, etc.
http://www.kb.nl/infolev/eagle/frames.htm
![Page 104: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/104.jpg)
SIGLESystem for Information on Grey
LiteratureNo. of Records | Category
4,158 | Aeronautics
17,044 | Agriculture, plant & veterinary sciences
17,668 | Environmental pollution, protection & control
256,657 | Humanities, psychology & social sciences
81,269 | Biological & medical sciences
25,089 | Chemistry
http://www.kb.nl/infolev/eagle/frames.htm
![Page 105: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/105.jpg)
NTIS (National Technical Information Service)
NTIS Database provides bibliographic data and abstracts of unclassified and publicly
available information from research reports, journal articles, data files, computer programs and audio visual products, from U.S. and non-
U.S. governmental, organizational, and commercial sources
The NTIS Database produced by the National Technical Information Service, is the preeminent resource for
accessing the latest U.S. government-sponsored research and worldwide scientific, technical, engineering, and
business-related information.
http://www.csa2.com/csa/factsheets/ntis.shtml
![Page 106: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/106.jpg)
Subject CoverageAdministration and Management
Aeronautics & Aerodynamics
Agriculture Behavior & Society
Business Chemistry Communications Computer Science
Education Energy Engineering Environmental Sciences
Health Care International Trade
Library & Information Science
Materials Sciences
Mathematical Sciences
Natural Resources & Earth Sciences
Nuclear Science Physics
Regulations Technology Tele-communications
Transportation
![Page 107: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/107.jpg)
PsycINFOPsycINFO provides access to international literature in
psychology and related disciplines. Unrivaled in its depth of psychological coverage and respected worldwide for its high
quality, the database is enriched with literature from an array of disciplines related to psychology such as psychiatry, education, business, medicine, nursing, pharmacology, law, linguistics, and
social work.
http://www.csa2.com/csa/factsheets/psycinfo.shtml
PsycINFO includes psychological research and its applications; the database is of prime relevance to many industries and research
establishments worldwide. The sources include over 1,400 professional journals, chapters, books, reports, theses and
dissertations, published internationally.
![Page 108: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/108.jpg)
Subject CoverageApplied Psychology
Communication Systems
Developmental Psychology
Educational Psychology
Experimental
Psychology
Personality Psychological and physical disorders
Professional personnel issues
Physiological Psychology and
Neuroscience
Psychometrics
And Statistics
Social Psychology
Treatment and Prevention
![Page 109: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/109.jpg)
Database Coverage Size
SIGLE 1976 –
Present
781,410 records(November 2002)
NTIS 1964 - Present 2,168,400
records(October 2001)
PsycINFO 1872-
Present
1,870,180
records(September 2002)
DATABASE COVERAGE AND SIZE
![Page 110: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/110.jpg)
http://www.neci.nec.com/~lawrence/papers/online-nature01/
![Page 111: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/111.jpg)
Conference papers are typical gray/grey literature!
![Page 112: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/112.jpg)
EPrints are Gray/Grey Literature
Daniela Luzi (1998) “E-Print Archives: a New Communication Pattern for Grey Literature,”
Interlending & Document Supply 26 no. 3 (1998): 130-139.
![Page 113: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/113.jpg)
Gray/Grey Literature“It’s good enough,
it’s smart enough,
and
doggone it, people use it!”
With apologies to Stuart Smalley
![Page 114: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/114.jpg)
http://www.openarchives.org/Register/BrowseSites.pl
![Page 115: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/115.jpg)
![Page 116: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/116.jpg)
![Page 117: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/117.jpg)
![Page 118: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/118.jpg)
http://software.eprints.org/
June 2003
![Page 119: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/119.jpg)
http://eprints.anu.edu.au/
![Page 120: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/120.jpg)
http://caltechcstr.library.caltech.edu/
![Page 121: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/121.jpg)
http://eprints.lub.lu.se/
![Page 122: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/122.jpg)
http://ndltdpapers.dlib.vt.edu:9090/
NETWORKED DIGITAL LIBRARY OF THESES AND DISSERTATIONS
![Page 123: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/123.jpg)
http://dspace.org/index.html
![Page 124: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/124.jpg)
http://rocky.dlib.vt.edu/~etdunion/cgi-bin/browse.pl
![Page 125: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/125.jpg)
http://www.ncstrl.org/
![Page 126: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/126.jpg)
![Page 127: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/127.jpg)
Quality Control in Scholarly Publishing on the Web
"Most of the high quality materials on the Web are not peer-reviewed and much
of the peer-reviewed literature is of dubious quality.”
William Y. Arms, "What Are the Alternatives to Peer Review? Quality Control in Scholarly Publishing On The Web."
Journal of Electronic Publishing, 8 no. 1 (August 2002).
http://www.press.umich.edu/jep/08-01/arms.html
![Page 128: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/128.jpg)
http://www.update-software.com/Cochrane/MR000016.pdf
![Page 129: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/129.jpg)
Cochrane Methodology ReviewDespite its widespread use and costs, little hard evidence exists that peer review improves the quality of published biomedical research.
There had never even been any consensus on its aims and that it would be more appropriate to refer to it as ‘competitive
review’.Caroline White, “Little Evidence for Effectiveness of Scientific Peer Review,”
BMJ 326 (February 1, 2003): 241http://bmj.com/cgi/reprint/326/7383/241/a.pdf
![Page 130: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/130.jpg)
Cochrane Methodology ReviewOn the basis of the current evidence, ‘the practice of peer review is based on faith in its effects, rather than on facts,' state the authors, who call for large, government funded research programmes to test the effectiveness of the [classic peer review] system and investigate possible alternatives.
Caroline White, “Little Evidence for Effectiveness of Scientific Peer Review,”BMJ 326 (February 1, 2003): 241
http://bmj.com/cgi/reprint/326/7383/241/a.pdf
![Page 131: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/131.jpg)
Cochrane Methodology ReviewThe use of peer-review is usually assumed to raise the quality of the end-product (i.e. the journal or scientific meeting) and to provide a mechanism for rational, fair and objective decision-making. However, these assumptions have rarely been tested.
Tom O. Jefferson, Phil Alderson, Frank Davidoff, and Elizabeth Wager, Editorial Peer-review for Improving the Quality of Reports of
Biomedical Studies. (Middle Way, Oxford:Update Software Ltd, 2003).
http://www.update-software.com/Cochrane/MR000016.pdf
![Page 132: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/132.jpg)
Cochrane Methodology ReviewThe available research has not clearly identified or assessed the impact of peer-review on the more important outcomes (importance, usefulness, relevance, and quality of published reports)
… [G]iven the widespread use of peer-review and its importance, it is surprising that so little is known of its effects
Tom O. Jefferson, Phil Alderson,Frank Davidoff, and Elizabeth Wager, Editorial Peer-review for Improving the Quality of Reports of
Biomedical Studies. (Middle Way, Oxford:Update Software Ltd, 2003).
http://www.update-software.com/Cochrane/MR000016.pdf
![Page 133: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/133.jpg)
http://www.biomedcentral.com/news/20030203/04/
Royal Society
![Page 134: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/134.jpg)
UNCITEDNESS
David P. Hamilton, "Publishing by and for? -- the numbers,” Science (New Series) 250 (4986) (December 7 1990): 1331-1332.http://www.garfield.library.upenn.edu/papers/hamilton1.html
![Page 135: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/135.jpg)
David P. Hamilton,“Research Papers: Who’s Uncited Now?,” Science (New Series) 251 (4989) (January 4, 1991): 25
http://garfield.library.upenn.edu/papers/hamilton2.html
![Page 136: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/136.jpg)
David P. Hamilton,“Research Papers: Who’s Uncited Now?,” Science (New Series) 251 (4989) (January 4, 1991): 25
http://garfield.library.upenn.edu/papers/hamilton2.html
![Page 137: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/137.jpg)
REJECTED CITATION CLASSICSNOBEL PRIZE RESEARCH
Severo Ochoa
Polynucleotide phosphorylase
Hans Krebs
Citric acid cycle
Rosalind Yalow
Radioimmunoassay
Harmut Michel
Photosynthetic processesJuan Miguel Campanario, “Commentary: On Influential Books and Journal Articles
Initially Rejected Because of Negative Referees’ Evaluations, Science Communication 16 no. 3 (March 1995): 306-325
![Page 138: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/138.jpg)
PEER REVIEW: PURPOSESPeer review helps to ensure that published research is:
Important ? Original ?
Timely ? Technically-reliable ?
Internally consistent ? Well-presented ?
Benefited from guidance by experts ?
Carin M. Olson, “Peer Review of the Biomedical Literature,” American Journal of Emergency Medicine 8 no.4 (July 1990): 356-368.
![Page 139: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/139.jpg)
FILTERING (1)UpStream / DownStream
“Researchers look at … [certain types] of electronic publications because, despite being tentative, may be relevant to their
work. Researchers are expected to do their own ‘downstream-filtering’ of relevant
information, which in the electronic world can be facilitated by providing meta-
information.”
![Page 140: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/140.jpg)
FILTERING (2)UpStream / DownStream
Some have expressed the concern that having non-peer reviewed documents with peer-
reviewed documents on the same server would ‘contaminate’ the latter and compromise its
quality: Readers could have trouble in distinguishing
the different sections ‘Making non-peer-reviewed as well as peer-
reviewed material will confuse both scientists and the public … .’
![Page 141: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/141.jpg)
FILTERING (3) UpStream / DownStream
“‘However, this perhaps belittles the ability of scientists to recognize different levels of evidence and to be able to interpret [quality] labels that could make clear that certain materials is non-peer-reviewed content’ after all this is
the age of transparency rather than paternalism … .”
Gunther Essenbach, “The Impact of Preprint Servers and Electronic Publishing on Biomedical Research,”
Current Opinion in Immunology 12 no. 5 (October 2000): 499-503.
http://yi.com/home/EysenbachGunther/scans/Eysenbach2000e_CurrOpImmunol_preprint_servers.pdf
![Page 142: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/142.jpg)
INVISIBLE HAND OF CLASSICAL PEER REVIEW
“The refereed journal literature needs to be freed from both paper and its costs, but not from peer review, whose ‘invisible hand’ is what maintains its quality.”
Stevan Harnad
http://www.presidentmoron.com
![Page 143: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/143.jpg)
INVISIBLE HAND OF CLASSICAL PEER REVIEW
![Page 144: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/144.jpg)
INVISIBLE HAND OF CLASSICAL PEER REVIEW
“Human nature being what it is, it cannot be altogether relied upon to police itself. Individual exceptions there may be, but to treat them as the
rule would be to underestimate the degree to which our potential unruliness is vetted by
collective constraints, implemented formally.”
Stevan Harnad, “The Invisible Hand of Peer Review,” Exploit Interactive no. 5 (April 2000).
http://www.exploit-lib.org/issue5/peer-review/
![Page 145: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/145.jpg)
INVISIBLE HAND OF CLASSICAL PEER REVIEW
“The system is not perfect, but it is what has vouchsafed us our refereed journal literature to
date, such as it is, and so far no one has demonstrated any viable alternative to having experts judge the work of their peers, let alone
one that is at least as effective in maintaining the quality of the literature as the present imperfect
one is.”
![Page 146: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/146.jpg)
INVISIBLE HAND OF CLASSICAL PEER REVIEW
“Remove that invisible constraint -- let the authors be answerable to no one but the general users of the
Archive [arXiv.org] (or even its self-appointed "commentators") -- and watch human nature take its
natural course, standards eroding as the Archive devolves toward the canonical state of unconstrained
postings: the free-for-all chat-groups of Usenet … , that Global Graffiti Board for Trivial Pursuit -- until
someone re-invents peer review and quality control.”
Stevan Harnad, “The Invisible Hand of Peer Review,” Exploit Interactive no. 5 (April 2000).
http://www.exploit-lib.org/issue5/peer-review/
![Page 147: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/147.jpg)
INVISIBLE HANDS
![Page 148: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/148.jpg)
INVISIBLE HANDS
Personal reputation
Institutional reputation
Pride
Self-respect
Professional respect
Peer pressure
‘Critical Peer Response’
Invisible College
Self-Archiving-Process-Itself
Open access
Common Sense
Self-correcting dynamics
![Page 149: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/149.jpg)
RECOMMENDATIONSWorkshop on the Open Archives Initiative (OAI)
and Peer Review Journals in Europe, CERN, Geneva Switzerland,March 22-24, 2001
“It was [also] generally believed that the electronic environment allows for novel
approaches to accord a stamp of quality to scholarly works.”
Alison Buckholtz, Raf Dekeyser, Melissa Hagemann, Thomas Krichel, and Herbert Van de Sompel, “Open Access: Restoring Scientific Communication to Its Rightful
Owners,” European Science Foundation Policy Briefing 21 (April 2003): 1-8.
http://www.arl.org/sparc/SPB21_OAI.pdf
![Page 150: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/150.jpg)
Examples of new metrics that can be extracted from a fully electronic communication system are:
Usage counts of a work
Automatically extracted citation information with a scope beyond the ISI- core journals
Amount of discussion generated by a paper submitted in a system with open peer review and peer comment
Etc.Alison Buckholtz, Raf Dekeyser, Melissa Hagemann, Thomas Krichel, and Herbert Van de Sompel, “Open Access: Restoring Scientific Communication to Its Rightful
Owners,” European Science Foundation Policy Briefing 21 (April 2003): 1-8.
http://www.arl.org/sparc/SPB21_OAI.pdf
![Page 151: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/151.jpg)
Scientific Publishing as RhetoricThe problems with peer review become evident once the fact that science has a rhetorical element is accepted. On the one hand, the traditional mode of peer review obscures the problems of reference and the rhetorical dimension of science. The rhetorical process which is at the heart of science and peer review conveniently disappears with the final publication of the manuscript. In its place is an ideal typical representation (the scientific paper) of the realist assumptions about empirical reference. All the academic world sees is a polished manuscript where the personal involvement of the researcher and reviewers has been systematically eliminated.
Mike Sosteric, “Interactive Peer Review: A Research Note,” Electronic Journal of Sociology 2 no. 1 (1996).
http://socserv.socsci.mcmaster.ca/EJS/vol002.001/SostericNote.vol002.001.html
![Page 152: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/152.jpg)
‘IDEAL SPEECH SITUATION’
A theoretical construct that describes the ideal type of interpersonal interaction that should exist in a rhetorical situation.
Jürgen Habermas
![Page 153: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/153.jpg)
IDEAL SPEECH SITUATIONthe ideal speech situation permits each interlocutor an
equal opportunity to initiate speech;there is mutual understanding between interlocutors;there is space for clarification; all interlocutors are equally free to use of any speech
act; there is equal power over the exchange.
Applied in the context of peer, the Ideal Speech Situation ‘would permit unimpeded authorial initiative, endless rounds of give and take, [and] unchecked openness among authors, editors, and referees.’
Mike Sosteric, “ Interactive Peer Review: A Research Note,” Electronic Journal of Sociology 2 no. 1 (1996).
http://socserv.socsci.mcmaster.ca/EJS/vol002.001/SostericNote.vol002.001.html
![Page 154: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/154.jpg)
CONTINUAContinuum of PUBLICATION
(‘Scholarly Skywriting’)
WEAK
MEDIUM
Continuum of REVIEW (‘Scholarly Skyreading’)
STRONGSTRONG
![Page 155: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/155.jpg)
OPEN ACCESS and
OPEN RETRIEVAL without
OPEN USEIncongruent
Contradictory Ironic
Paradoxical
Cognitively Dissonant
![Page 156: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/156.jpg)
Three-Legged Stool
ACCESS -- RETRIEVAL -- USE
![Page 157: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/157.jpg)
![Page 158: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/158.jpg)
ACCESS
OPEN ACCESS INITIATIVES
![Page 159: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/159.jpg)
RETRIEVAL
OPEN ARCHIVES INITIATIVE FOR METADATA HARVESTING
![Page 160: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/160.jpg)
USE
OPEN SCHOLARSHIP
![Page 161: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/161.jpg)
ACCESS -- RETRIEVAL -- USEACCESS -- RETRIEVAL -- USE
![Page 162: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/162.jpg)
Lots of Alternative ModelsProvide Sensible Solutions
![Page 163: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/163.jpg)
Four-Legged Stool
ACCESS -- RETRIEVAL -- USE -- NAVIGATION
![Page 164: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/164.jpg)
INFORMATION OVERLOAD
![Page 165: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/165.jpg)
OAIster
A search engine for freely available, difficult-to-access, academically-oriented digital resources that are OAI -compliant
http://oaister.umdl.umich.edu/
![Page 166: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/166.jpg)
![Page 167: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/167.jpg)
University of Michigan Digital Library Production Service
institutional repositories
departmental repositories
e-Journal collections
technical reports
dissertations and theses
discipline eprint collections
working papers
Internet resources
audio
video
images
![Page 168: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/168.jpg)
![Page 169: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/169.jpg)
Cognitive Psychology
![Page 170: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/170.jpg)
![Page 171: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/171.jpg)
Open Archives Initiative Protocol for Metadata Harvesting
(OAI-PMH)
METADATA ELEMENT for QUALITY
ADD
![Page 172: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/172.jpg)
“Within the framework of OAI,
there is a need for a new protocol for certification.
There was strong support
for the extension of the usage of the
OAI protocol beyond discovery-related
metadata. Given the focus of the [1st OAI]
workshop on peer review, concrete
actions were suggested to address the
exchange of certification-related
metadata using the OAI protocol in a
trusted environment.”
http://www.arl.org/sparc/SPB21_OAI.pdf
Alison Buckholtz, Raf Dekeyser, Melissa Hagemann, Thomas Krichel, and Herbert Van de Sompel, “Open Access: Restoring Scientific Communication to Its Rightful Owners,” European
Science Foundation Policy Briefing 21 (April 2003): 1-8.
![Page 173: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/173.jpg)
QUALITY METADATA (1)<oai-quality>
<category>internal</category>
<process>
peer review
</process>
<organization>
CERN
</organization>
<policies>
http://www.cern.ch/policies/review.html
</policies>
</oai-quality>
William Y. Arms, “Quality Control in Scholarly Publishing On The Web. What Are the Alternatives to Peer Review?” PowerPoint presentation given at Workshop on the Open Archives
Initiatives (OAI) and Peer Review Journals in Europe, March 22-24, 2003, CERN, Geneva, Switzerlandhttp://www.cs.cornell.edu/wya/papers/CERN-2001.ppt
![Page 174: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/174.jpg)
QUALITY METADATA (2)<oai-quality>
<category>internal</category>
<rating>
*****
</rating>
<organization>
42
</organization>
<policies>
http://www.cern.ch/policies/review.html
</policies>
</oai-quality>
![Page 175: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/175.jpg)
![Page 176: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/176.jpg)
CERTIFICATION SERVICESFaculty of 1,000,000
![Page 177: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/177.jpg)
CERTICATION SERVICESNew roles for Indexing and Abstracting Services
Expanded Role for Learned and Professional Societies
Establishment of Formal/Commercial Reviewing Services
![Page 178: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/178.jpg)
PEER REVIEW: STRENGTHSThe underlying strength of peer review is“…the concerted effort by large numbers of researchers and scholars who work to assure that valid and valuable works are published and conversely to assure that invalid or non-valuable works are not published … .”
Anne C. Weller, Editorial Peer Review: Its Strengths and Weaknesses. (Medford, NJ: Information Today, 2001).
![Page 179: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/179.jpg)
Faculty of 1000 / BioMed CentralBioMed Central (biomedcentral.com) publishes Faculty of 1000 (F1000), the leading literature evaluation service and “new online research tool that highlights the most interesting papers in biology, based on the recommendations of over 1000 leading scientists.” F1000 is managed “by scientists for scientists” …. [and] provide[s] a rapidly updated consensus map of the important papers and trends across biology.”
www.facultyof1000.com
![Page 180: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/180.jpg)
Faculty of 1000 / BioMed CentralAmong its many benefits, F1000:systematically organizes and evaluates the mass of information within scientific literature; provides scientists with a continuously updated insider's guide to the most important papers within any given field of research;highlights papers on the basis of their scientific merit rather than the journal in which they appear;offers the researcher a consensus of recommendations from well over 1000 leading scientists; and,offers an immediate rating of individual papers by the authors' peers, and an important complement to the indirect assessment provided by the journal impact factor.
![Page 181: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/181.jpg)
Faculty of 1000 / BioMed CentralWithin the F1000, the entire field of biology
is divided into 16 subject areas (‘Faculties’)(e.g., ‘Biochemistry,’, Cell Biology, ‘Microbiology’). Each ‘Faculty’ is subdivided into three (3) to twelve (12) ‘Sections,’ (e.g., Biochemistry: Biocatalysis, Molecular evolution, Protein folding), with each section comprised of between 10 to 50 faculty members. F1000 seeks to invite the best internationally known scientists in each represented field and to involve both experienced and younger investigators.
![Page 182: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/182.jpg)
“Peer review is a quality-control and certification (QC/C) filter necessitated by the vast scale of learned research
today. Without it, no one would know where to start reading in the welter of new work reported every day, nor
what was worth reading, and believing, and trying to build one’s own further research upon.”
Stevan Harnad, “Free at Last: The Future of Peer-Reviewed Journals,”
D-Lib Magazine 5 no. 12 (December 1999)
http://www.dlib.org/dlib/december99/12contents.html
![Page 183: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/183.jpg)
SEIZE THE E!
Embrace the potential of the digital environment to facilitate access, retrieval,
use, and navigation of electronic scholarship.
![Page 184: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/184.jpg)
OPEN NAVIGATION
![Page 185: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/185.jpg)
New Age Navigation:Innovative Interfaces for
Electronic Journals
Gerry McKiernan
The Serials LibrarianFall 2003
![Page 186: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/186.jpg)
SUMMARY. While it is typical for electronic journals to offer conventional search features similar to those
provided by electronic databases, a select number of e-journals have also made available higher-level access options as well. In this article, we review several novel
technologies and implementations that creatively exploit the inherent potential of the digital environment to
further facilitate use of e-collections.
Gerry McKiernan, “New Age Navigation: Innovative Interfaces for Electronic Journals,”
The Serials Librarian, Fall 2003.
http://www.coleonline.us/serialslibrarian/
![Page 187: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/187.jpg)
http://www.highwire.org
![Page 188: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/188.jpg)
Topic Map
![Page 189: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/189.jpg)
Topic Map
![Page 190: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/190.jpg)
Topic Map
![Page 191: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/191.jpg)
Topic Map
![Page 192: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/192.jpg)
Topic Map
Secondary Screen
![Page 193: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/193.jpg)
Topic Map
![Page 194: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/194.jpg)
Topic Map
http://www.inxight.com
TopicMap is based on the Hyperbolic Tree SDK for Java, licensed from Inxight
Software, Inc., a spin-off company from Xerox PARC, and leading provider of
Unstructured Data Management solutions for accessing, analyzing and delivering
information.
![Page 195: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/195.jpg)
Semio
Automated categorization software technologyhttp://www.entrieva.com/
![Page 196: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/196.jpg)
Kohonen Self-Organizing Maps
http://websom.hut.fi/websom/milliondemo/html/root.html
![Page 197: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/197.jpg)
Kohonen Self-Organizing Maps
http://websom.hut.fi/websom/milliondemo/html/1_cx5.html
![Page 198: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/198.jpg)
Kohonen Self-Organizing Maps
http://websom.hut.fi/websom/milliondemo/html/2_gx10.html
![Page 199: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/199.jpg)
Kohonen Self-Organizing Maps
http://websom.hut.fi/websom/milliondemo/getnd.cgi?32323
![Page 200: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/200.jpg)
Kohonen Self-Organizing Maps
http://websom.hut.fi/websom/cgi-bin/getfile.cgi/comp.ai/39340
![Page 201: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/201.jpg)
http://www.springer.de/books/toc/3540679219-c.pdf
![Page 202: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/202.jpg)
“There are some excuses, but at the bottom it will be seen to be the
sluggishness of human nature and its superstitious cleavage to old habits.”
Stevan Harnad, “Free at Last: The Future of Peer-Reviewed Journals,”
D-Lib Magazine 5 no. 12 (December 1999)
http://www.dlib.org/dlib/december99/12contents.html
![Page 203: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/203.jpg)
“… [In] the digital world, the evaluation process stands ready to be reinvented in a clear, rational
way by the relevant research communities themselves.”
Jean-Claude Guédon,
In Oldenburg’s Long Shadow: Librarians, Research Scientists, Publishers,
and the Control of Scientific Publishing.
(Washington, D.C.: Association of Research Libraries, •2001), 54.
http://www.arl.org/arl/proceedings/138/guedon.html
![Page 204: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/204.jpg)
</ENDQUOTE>
“The Medium is the Message …
And
the Method.”
With apologies to Marshall McLuhen
![Page 205: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/205.jpg)
THANK YOU FOR YOUR ATTENTION!
![Page 206: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/206.jpg)
Alternative Peer Review:Quality Management
for 21st Century Scholarship
Gerry McKiernanScience and Technology Librarian
and Bibliographer
Iowa State University Library
Ames IA
![Page 207: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/207.jpg)
OPEN MIND
![Page 208: Alternative Peer Review : Quality Management for 21 st Century Scholarship](https://reader033.vdocuments.mx/reader033/viewer/2022061409/568148e5550346895db5fee8/html5/thumbnails/208.jpg)
REVISED VERSION 1.01
July 27, 2003
11:30 AM