3 js devdoc (peer review)

University of New MexicoUniversity of New Mexico

OLIT 593-008OLIT 593-008Spring 2008Spring 2008

John John Jason Jason JenJen

Collaborative Knowledge CreationCollaborative Knowledge CreationDevelopment PresentationDevelopment Presentation

forforUniversity ReviewUniversity Review

April 10, 2008April 10, 2008

Background: Peer-ReviewBackground: Peer-Review

Peer-review is a collaborative knowledge creation process. The intended product is an accurate and significant scientific contribution in the form of a paper. The authors and reviewers engage in an iterative editing process mediated by a journal editor to develop the paper to publishable quality.

Background: Peer-ReviewBackground: Peer-Review

A journal exists so as to streamline and centralize the repetitive tasks involved in publishing a work. To readers, the journal must offer credible

and significant work in the research area that is executed, organized, and displayed in a pleasing manner.

To the authors, the journal must offer peer-review services and credence to the work.

To the reviewers, the journal must provide an environment that encourages their voluntary and unpaid labor.

ProblemProblem

The current implementation of peer-review at University Review does not take advantage of collaborative knowledge creation techniques.

TaskTask

We will restructure the peer-review process of University Review by developing a comprehensive workflow plan to more effectively and efficiently produce academic knowledge.

Knowledge Enablers

1. Instill A Vision

2. Manage Conversations

3. Mobilize Knowledge Artifacts

4. Create Right Content

5. Globalize Local Knowledge

Enabler 1

Instill a Vision

The new vision of University Review is:

"honoring knowledge and academic integrity through iterative review, grammatical standards, and layout design."

Criteria of aCriteria of aGood Knowledge VisionGood Knowledge Vision

1. Commitment to a Direction

2. Generativity

3. Specific Style

4. Restructuring the Knowledge System

5. Restructuring the Task System

6. Communication of Values

7. Shaping Competitiveness

Criterion ICriterion I

Commitment to a DirectionCommitment to a Direction

Our knowledge vision places the journal as a steward of knowledge

“Honoring Knowledge”

Criterion IICriterion II

GenerativityGenerativity

The phrase "honoring knowledge" has the power to stimulate new organizational imagination.

Criterion IIICriterion III

Specific StyleSpecific Style

The phrasing of the vision encompasses a role for every person involved in work for the journal. Iterative review Grammatical standards Layout design

Criterion IVCriterion IV

Restructuring the Knowledge SystemRestructuring the Knowledge System

The vision emphasizes team spirit, iteration, and speed, which are three criteria for increasing the effectiveness of the knowledge creation process.

Criterion VCriterion V

Restructuring the Task SystemRestructuring the Task System

The vision is a guide for the new workflow

Criterion VICriterion VI

Communication of ValuesCommunication of Values

The phrase "honoring knowledge" is a clear delineator of University Review from the rest of the publishing world.

The inclusion of “academic integrity” in the parsimonious vision statement places its realization as a priority.

Criterion VIICriterion VII

Shaping CompetitivenessShaping Competitiveness

Authors want to publish in a journal that has a fast turnaround.Reviewers want to feel a part of the process.Readers want a high quality product.

This vision incorporates a commitment to all three desires.

Conversations at University Review will have a number of unique attributes:

they will be asynchronous they will be text-based they will be (often) held between people who

do not know each other the editor will be the only person who knows

who all the other discussants are they will have a clear purpose: to improve the

existing paper they will be on a deadline

Enabler 2

Manage Conversations

Principles of Good Principles of Good ConversationsConversations

1. Actively Encourage Participation

2. Establish Conversational Etiquette

3. Edit Conversations

4. Foster Innovative Language

Principle IPrinciple I

Actively Encourage ParticipationActively Encourage Participation

Communication from the editor will be courteous.

Expectations for participation will be stated on the journal website.

Principle IIPrinciple II

Establish Conversational EtiquetteEstablish Conversational Etiquette

Sample documents will demonstrate proper tone and commenting procedure.

Expectations for participations will be stated on the journal website.

Principle IIIPrinciple III

Edit ConversationsEdit Conversations

It will be the responsibility of the editor to decide what comments to pass along to the author and to make the final decision as to the paper's fitness for publication.

Principle IVPrinciple IV

Foster Innovative LanguageFoster Innovative Language

Innovative language is not desired.

Innovative language must be clearly defined and placed in reference to existing language with appropriate reference to the literature.

Enabler 3

Mobilize Knowledge Activists

Empowerment for authors and contributors

Tacit to Explicit Knowledge: shared on the web for the authors and contributors

Enabler 4 Create the Right Context

Virtual and mental spaces created to help authors, editors, reviewers, and readers honor knowledge Virtual space: SharePoint site Mental space: focused with job aids

Enabler 4 Create the Right Context

Individual Interaction Collective Interaction

ORIGINATING

Paper submission

CONVERSING

Exchange of comments amongst author, editor, and reviewers

INTERNALIZING

readers who will be able to become authors once the knowledge is internalized.

DOCUMENTING

Interactions with the website amongst author, editor, and reviewers

Enabler 5

Globalize Local Knowledge

Submission

Review Process

Reader Internalization

Triggering

Re-packaging

Re-creating

Knowledge Creation Steps

1. Share Tacit Knowledge

2. Create a Concept

3. Justify a Concept

4. Build a Prototype

5. Cross-Leveling Knowledge

Step 1

Share Tacit Knowledge

There will be two sources to embed explicit knowledge:

1. The job aids included in the journal website compiled by the editor.

2. The conversation taking place between the reviewers, editor, and author.

Step 1

Share Tacit Knowledge The role defines the incentives to share tacit knowledge:

Authors: create a submission that quickly and smoothly passes the peer-review process and is published

Editors: build a more efficient system, help authors better prepare submissions, and help reviewers better prepare their comments

Reviewers: be given the privilege of reviewing next time, which includes access to developing works before publication and a professional association with the journal

Step 2

Create a Concept

As peer-review is an iterative process, the concept is created while the tacit knowledge is shared. The externalized knowledge will include: past publications of the journal to give

everyone a feel for the journal checklists to serve as job aids at point of

entry for particular stages in the process the paper itself and associated comments

Step 3

Justify a Concept

The entire peer-review process is a process of justifying a concept. It includes:

Authors: review submission job aid and submits work Editors: check for appropriateness of paper and

assigns reviewers Reviewers: check details of paper and provide

feedback Authors: rewrite or justify each feedback item Editors: approve paper for publication

Step 4

Build a Prototype

Throughout the entire process, the author is in control of the prototype construction

The prototype begins as a submission and may require a resubmission to address reviewers’ comments

Proofs of the final paper must be accepted by the author before publication

Step 5

Cross-Leveling Knowledge

It is the goal of every journal to cross-level knowledge. The journal itself is the documented knowledge and its sale circulates the knowledge. However, this new system will:

increase efficiency and thereby shorten the time from submission to publication

document every step of the process by requiring that communication be electronic and then organizing and storing it

Collaboration Process Elements

1. Major steps

2. Knowledge product

3. Performance objective

4. Knowledge required

The Major Step

There is only one major step in the peer-review process and that is to produce a publishable paper

The Knowledge Product

There is only one knowledge product that will result from peer-review and that is a publishable paper.

The Performance Objectives

The objectives are as follows: The subject matter of the paper aligns with

that of University Review The paper is formatted to reflect the

standards of University Review The paper contains enough information to

replicate the results reported The paper represents a significant advance in

the field

The Knowledge Required

The knowledge required to produce the knowledge product can be analyzed by role:

Authors: must have the specific knowledge to complete the work as well as to write an academic paper

Editor: must have command of the workflow Reviewers: must have domain knowledge of the

paper’s subject matter as well as of how to review an academic paper

The Knowledge Required

Peer-review at University Review is a Wikinomy:

“a way of producing goods and services that relies … on self-organizing, egalitarian communities of individuals who come

together voluntarily to produce a shared outcome” (Tapscott & Williams, 2006, p. 67)

Workflow Model

1.1. IntroductionIntroduction2.2. RoutingRouting3.3. TriggersTriggers4.4. Petri-netPetri-net

Embedding learning into the workflow modelEmbedding learning into the workflow model Documents Embody Factual Knowledge Instruction Embodies Conceptual Knowledge Examples Embody Procedural Knowledge Expert Advice Embodies Metacognitive Knowledge

Workflow ProcessWorkflow Process

Submission An author submits a paper (email), the paper is

received by the journal, the journal sends an acknowledgment of receipt (email), the editor reviews the paper for appropriateness, if not appropriate, the editor emails the author, if appropriate, the peer review process starts.

Peer-Review Three reviewers are identified, the paper is sent to

them (email), each reviewer agrees (or not) to review it (email), each sends it back in a timely manner or a reminder email is sent by the journal, once all three reviews get back to the editor, the editor emails his/her decision to the author.

AuthorAuthor SubmissionSubmission

Visit Journal Visit Journal

websitewebsite

Review Review criteriacriteria

Submit paperSubmit paper

Editor Editor

Initial ReviewInitial Review

Confirm receiptConfirm receipt

Decide if appropriateDecide if appropriate

If yes - reviewIf yes - review

If no - notify authorIf no - notify author

Peer Peer

ReviewReview

Notify authorNotify author

Invite reviewersInvite reviewers

Review feedbackReview feedback

Editor decisionEditor decision

Editor Editor

Final ReviewFinal Review

Positive review - publishPositive review - publish

Negative review – do not Negative review – do not publishpublish

Notify authorNotify author

Routing Required Sequential Selective - for example, the reviewer accepts

or declines to review Iterative - for example, the author resubmits

the paper with suggested changes

Routing Not Required Parallel

Triggers Required Time Signal - for example, if the reviewer is

taking too long to reply, they will receive a reminder email

Resource Initiative - for example, all actions of the author, editor, and reviewers in the system will be triggers

External Event - for example, receiving feedback from a reviewer or a resubmitted paper

Petri NetsPetri Nets

Background:Petri nets are visual presentations showing workflow management.

The process has one entry point and one end point with a series of conditions and tasks. (p 49)

Petri NetsPetri Nets

Symbols and Functions:Symbols and Functions:Conditions (or “cases”) are shown as places and represented by

the circle symbol.

Tasks are shown as transitions by the square symbol.

Tokens correspond with particular cases and are shown by a dot.

The decision process involves joining and/or splitting at transition points.

Transitions are eager to fire as soon as they are enabled through triggering.

Arrows demonstrate the directional flow.

Petri NetsPetri NetsWork Items & ActivitiesWork Items & Activities

WORK ITEMSWORK ITEMS

CaseCase TaskTask

Case 1Case 1 review_criteriareview_criteria

Case 2Case 2 reviewer_decisionreviewer_decision

Case 3Case 3 select_notificationselect_notification

ACTIVITIESACTIVITIES

CaseCase TaskTask ResourcesResources

Case 1Case 1 review_criteriareview_criteria JenJen

Case 2Case 2 reviewer_decisionreviewer_decision JasonJason

Case 3Case 3 select_notificationselect_notification JohnJohn

Work Items involve cases and tasksWork Items involve cases and tasks

Activities include cases, tasks and resources. (p 84)Activities include cases, tasks and resources. (p 84)

Scholarly Journal Review Scholarly Journal Review ProcessProcess

Author: Author:

SubmissionSubmission

Editor: Editor:

Initial ReviewInitial ReviewPeer: ReviewPeer: Review Editor: Editor:

Final ReviewFinal Review

StartStart

visitvisit

websitewebsite

review criteriareview criteria

submitsubmit

paperpaperconfirm receiptconfirm receipt

editoreditor

reviewreview

inviteinvite

reviewersreviewers

acceptaccept

declinedecline

conductconduct

reviewsreviews

receivereceive

feedbackfeedback

publish?publish?

receivereceive

submissionsubmission

acceptaccept

declinedecline

editoreditor

reviewreview

yesyes

notifynotify

authorauthor

selectselect

notificationnotification

selectselect

reviewersreviewers

publishpublish

evaluationevaluation

endend

reviewerreviewer

decisiondecision

SharePoint siteSharePoint site

Please visit the SharePoint site for Please visit the SharePoint site for University Review University Review at:at:http://129.24.38.60:10652/olit509/jwstigre/GroupProjDesign/UniversityReview/default.aspxhttp://129.24.38.60:10652/olit509/jwstigre/GroupProjDesign/UniversityReview/default.aspx

Also available as a link on the homepage of Also available as a link on the homepage of University University

Review Review found on the class SharePoint sitefound on the class SharePoint site

Comments Welcome!Comments Welcome!

3 js devdoc (peer review)

Entertainment & Humor

peer review 1

peer review presentation

peer review workshop

peer js + clmtrackrでつくるモテるアプリ

aicpa peer review program...

state plan peer review criteria peer review notes...

is peer review any good? a quantitative analysis of peer...

peer review board open session materials peer ... -...

state plan peer review criteria peer review notes...

practicing peer review - how excellence is constructed in...

peer review

nih peer review - eunice kennedy shriver national ... ·...

peer review 3

effective peer review

transparent and open peer review - ariessys.com ·...

enhancing peer review initiative survey of peer reviewers...

peer review manual

intercultural peer review

reggienet: group and peer review assignments adding a peer...

peer review - a value addition.ppt · 2019. 12. 13. ·...