How to Survive Peer Review© BMJ Books 2002

Fiona GodleeEditorial Director for Medicine, BioMed Central, London, UKTom JeffersonDirector, Health Reviews Ltd, Anguillara Sabazia, Roma, Italy

Peer vb intr. 1. to look intently with or as if with difficulty. n. 1. a person who is an equal in social standing, rank,age,

etc.

The term is most often used to describe a formal system whereby a piece of academic work is scrutinized by people who were not involved in its creation but are considered knowledgeable about the subject.

it is also used to describe professional appraisal processes used to assess the performance of an individual, team, or department.

Peer review of conference abstracts Peer review of grant proposals Book proposals Peer review within the Cochrane

collaboration Most use a “top-down” approach--receive

many more submissions than they can publish, and use peer review to cream off the most interesting

Reject 80– 90% of submissions and use peer review to identify those of greatest relevance to their readers

Use peer review to distinguish sound and ethical research

To identify the findings most likely to interest their readers.

Single editor, all externally reviewed The reviewers will be asked whether they

think that the submission should be published, and a detailed review that includes suggestions for improving the submission.

Editorial board with occasional further review

The board is selected to provide a range of expertise, and other reviewers will rarely be used

The in-house editors act like an editorial board. They have a good understanding of research methodology, the journal’s aims, and its readers’ interests.

They review all submissions and are responsible for rejecting 30–50% without external review. rapid decision

If you are rejected in this way, it usually means that you chose the wrong journal in terms of its scope or prestige.

Read the instructions to find out what you are being asked to do and why.

If you receive no instructions and are not clear about what you are being invited to do, ask for more information or decline the request.

Review the work not the person (unless you have been asked to do this), and don’t try to be clever.

Admit your limitations. Be as objective as possible and take

account of (and declare) any conflicts of interests.

Is the manuscript within my field of expertise?

Am I happy with the journal’s peer review process? (open peer review)

Do I have time to do this review? Can I meet the deadline? Do I have any conflicts of interest?

Tell the journal immediately so that the editors can look for alternative reviewers

Suggest alternative reviewers if you can. Finding the right reviewers is one of the most difficult aspects of editorial peer review, so most editors will thank you for this.

Let the journal know and confirm the deadline. Ask for instructions to authors and reviewers.

do everything you can to submit your report on time. If you cant let the journal know as soon as possible.

Keep it confidential. Don’t contact the authors except with the

journal’s permission Aim to be as objective, constructive,

conscientious, and systematic as possible. These attributes separate the best reviewers from the

rest.

Why was the study done? Have the authors adequately reviewed

existing research? Was there a clearly defined question? What study design was used? Was the design right for the question? Was the study ethical? Are the conclusions justified?

Is the research question or objective clearly stated? Is it clear from the manuscript why the authors did the study? Do the authors summaries the existing literature

Is the research question interesting and important? the question matters more than the answer. if the question has been clearly stated and is important, the answer is important whatever it is (positive,negative, or neutral).

To check this, you may need to do a literature search. The term “original” means different things in different contexts, it includes the reporting of new data, ideas, or methods, or the reanalysis of existing data.

If the question has been addressed before, does this manuscript add enough new information to justify publication?

give references to previous work: don’t just say “It’s not original”. If you know of important studies that the authors don’t refer to, provide the references.

Is the study design right for answering the study’s main question?

Were the subjects sampled correctly? Were the controls appropriate and adequate?

Was a power calculation required and, if so, was it done

Are the methods adequately described? Were the analyses done correctly? Do the numbers add up?

Are the conclusions supported by the data?

Is the work well presented? Is the writing clear and coherent? Is the manuscript structured appropriately? the text should tell the story, the tables should provide the detailed data, and the figures should illustrate the story. Make a note of important spelling mistakes but leave detailed copy editing to the technical editor.

Are there any ethical problems?

Should the journal publish the work?

Should the journal commission any accompanying commentaries?

you may want to alert the editors to a particularly important and relevant piece of work, and suggest names of people (including yourself if appropriate) to write a commentary

The aim of the report is twofold: to help the editors decide what to do

with the paper, and to help the authors improve it

before publication.

Begin with a brief outline of the paper. This shows the authors and editors that you have understood the paper.

Number your comments. Indicate which comments relate to which parts of the manuscript.

Don’t submit handwritten edits on the margins of the paper.

Stick to what you know. Don’t feel you have to cover all aspects of a paper. Make clear to the editors where your expertise ends so that they will know when to consult additional reviewers.

Acknowledge help from others.

Focus on the paper not the author.

Be courteous and constructive. An important aim of peer review is to improve manuscripts before they are published. Remember to identify strengths as well as weaknesses.

The study may not be perfect. If the data are important but the study is flawed, it may still be useful to publish the paper. The authors should be asked to acknowledge weaknesses in their study, and the journal may wish to commission a commentary using the paper to highlight problems as a lesson in research methodology.

Mention all conflicts of interest.

Send your report in on time.

Did the authors search adequately and without bias for all relevant studies?

Did they use appropriate criteria to decide which studies to include in the review (based on study design, interventions, outcome measures, populations, sample size)?

Was selection and methodological assessment of studies done in a reproducible and unbiased way

Were the studies comparable on clinical grounds (interventions, outcomes, population)?

If the authors combined the studies for statistical analysis,

were the right statistical methods used (fixed effects or random effects model)?

Did the authors perform sensitivity analyses to see whether excluding or including different studies, or performing alternative statistical tests, made a difference to the results?

Is the difference between the groups statistically significant? If so, is it clinically significant?

Were the outcomes clinically important (for example, survival or mobility rather than results of blood tests or x rays)?

Was a power calculation performed to determine the sample size needed to detect a clinically important difference?

Was allocation to treatments randomised?

Except for the intervention being tested, were the groups treated exactly the same?

Was compliance to treatment (adherence) assessed?

Were the outcomes assessed by people who were blinded?

Were all patients properly accounted for?(inclusion &exclusion criteria)

Was follow-up adequate (more than 85%)?

Is the difference between the groups statistically significant? If so, is it clinically significant? If not statistically significant, was the study large enough to detect a clinically significant difference?

Can you tell from the report how many people need to be treated with the new treatment rather than the old to achieve one additional positive outcome (number needed to treat) or to cause one additional adverse effect (number needed to harm)?

Did the authors gather an inception cohort (patients identified at an early, uniform, unbiased and well-defined point in the course of the disease)?

Was complete follow up achieved? Were outcome criteria objective,

reproducible, and accurate, and were they assessed blind?

• Did the authors adjust for extraneous prognostic factors?

Was the new test compared with the current gold standard test?

Were both gold standard and new tests performed on all participants?

Do the authors adequately describe the setting for the study and the criteria for deciding which patients to include?

Did the sample include a full range of people with mild and severe, treated and untreated disease, as well as people with other disorders that fall within the differential diagnosis?

Do the authors describe the new test in sufficient detail to allow others to replicate it?

Does the study assess whether the test is reproducible (precision) and whether different observers agree on interpretation (observer variation)?

Was the term “normal” defined sensibly? Does the study assess whether the

patients were really better off as a result of having the test

Did the authors use the best possible study design (prospective cohort study, or case control study for rare diseases)?

Were the opportunities for and the determination of exposure free from bias?

Is the relative risk/odds ratio significantly greater than 1? If so, is the increased risk clinically significant? If not statistically significant, was the study large enough to detect a clinically significant difference?

Is it clear how the cohort was recruited? Did the authors consider factors that

might influence the type of people included in the cohort (for example, reasons why more severely ill people might have been excluded)?

If the study used hard endpoints such as death, were all events identified?

If the study used “softer” endpoints, were all measurement tools (such as questionnaires) properly validated?

Were severity of disease and presence of other diseases taken into account in the analysis?

Does the study adequately describe how cases were defined?

Were the controls appropriate? Did the controls match the cases in

every necessary way except the disease/risk factor under study?

Were data collected in the same way for cases and controls?

Were measurements free from bias? Do the authors take account of all

possible sources of confounding?

Are the aims clearly stated? Was the selection of subjects (sampling)

unbiased and adequately described? Was the questionnaire validated in

terms of intra- and inter-rater reliability? Does the questionnaire measure the

things that matter? Was the sample size justified? Was the response rate adequate? Did the authors look for important

differences between responders and non-responders?

Were the alternative healthcare programmes adequately described?

Has their effectiveness been assessed in RCTs? If not, do the authors clearly identify the sources of their effectiveness estimates?

Were all important costs and effects identified?

Did the study use credible measures of these costs and effects?

Were the analyses appropriate?

Look at checklist--usually comment on:Scientific rigorStudy Design—MethodsAdequacy of dataImportance and originality of resultsValidity of conclusion reached Completeness of literature citedClarity of writing

Many ask reviewers to give a quality or priority ranking to various aspects

If two disagree--sent to arbiter reviewer Arbiter reviewer comments on reviewers

comments and your paper

شما زمستانی زیبای صبح سالمخیر به

To get published To have research work acknowledged Get degree Publish or Perish

Choose the right journal. Spend time considering the implications of your research, your intended audience, and the message you want to communicate

Keep the journal and your intended audience in mind as you write. Ask yourself, “Why would these people want to read my paper?” Consider the aspects of your findings that would particularly interest them: focus on these and cut down on everything else. Check

Consult the journal’s instructions to authors and other useful sources of information.

Uniform Requirements for Submission to Biomedical Journals prepared by the International Committee of Medical Journal Editors (available at www.icmje.org).

CONSORT, STROBE,STARD,… When you’ve finished writing, read the

instructions to authors again

Abstract: Does it fit the journal’s maximum length and format? Does it accurately reflect the manuscript?

Keywords: Check if these are required and, if so, whether they need to conform to NLM MeSH headings.

Title: Is it concise and informative? Do you need to supply a short title (running head) for use on the header of subsequent pages?

?

Layout: Have you double spaced everything – even tables, figure captions, and references? Have you indicated the text position of tables and figures?

Acknowledgements: Have you acknowledged the source of funding?

Conflicts of interest: Have you declared all of these? (It is good

Ethics: Have you mentioned ethics committee (review board) approval

Full names, qualifications and affiliations of all authors , Full contact details of (phone, fax, email, full postal address)

Authors’ signatures and statement of contributorship (describing who did what, which some journals ask for); these can be separate from the covering letter and prepared well in advance

Copyright form (which may have to be signed by all authors)

Consent to reproduce copyright material or patients’ photographs or medical details

Signed agreement from anyone mentioned in the acknowledgements(sometimes required by American journals)

Conflict of interest form

Evidence of “in press” citations (for example, a copy of the acceptance letter from the journal)

Find most appropriate journal for research

Read Editorial Objectives or Notes to Contributors

Write paper to journal guidelines & objectives.

Write to the highest standards—including logical focus and structure

Make reviewing process more pleasant for both reviewers and editors

Send in a well written paper State aim and contribution to research

early to hook reviewer Respond positively and promptly to

reviewer comments

Title contains key words—describe position or contribution to research

Title is within length limits—8 words Abstract contains key ideas in your paper

Short theme sentence to orient readerMain aim or problem statementWhy research carried out & why

importantWhat was done and what happened

(methods)Results/findings from studyValue of work in terms of conclusions or

contributions to research that filled existing gaps and/or

Implications for theory and practice

Introduction (4 parts)

1. Establishes field background & asserts significant position in theory & practice

2. Summarizes previous research (about 2 paragraphs)

3. Indicates gaps, inconsistencies & or controversies & why they are important

4. States the aim/purpose of article (to address gaps, inconsistencies, controversies above), states briefly the final position & outlines structure of article (roadmap)

You have not used too many quotations marks—indicates writer is lazy—put things in your own words instead

Check numbers in tables & figures Most numbers should be in graphics and

explanations in text Make sure numbers precise—not

confusing

Put manuscript away a week Reread it for content one last time Read instructions to authors once more

before submitting Want to come across as careful, well

organized Your work deserves publication

Check to Make Sure

Use headed paper to indicate where you work.

Get the editor’s name right (check a recent edition of the journal – sending a letter to the previous editor or misspelling the editor’s name does not inspire confidence.)

Get the journal’s name right (if the manuscript has been rejected by one journal, make sure you change the journal name on the covering letter when submitting elsewhere.)

Describe, very briefly, what you found and why this is relevant to readers. Don’t just use a neutral description of your research

Briefly explain the key message and implications of your findings but don’t oversell your work or claim that it will change the face of medicine if it won’t.

Tell the editor why you are submitting to that particular journal

Show an understanding of the journal’s readership and/or previous related publications. If possible, give the editor a reason for publishing your paper (for example, if it complements an earlier piece published by the journal).

Consult the instructions to authors for necessary wording

this research has not been published before and this paper is not being considered for publication by any other journal;

• all of the undersigned authors have approved the final version;

• all authors fulfil the authorship criteria.

The data included in this manuscript have not been published previously and are not under consideration by any other journal. A consent to publication form signed by the authors is enclosed. All authors have read this final manuscript and have given their approval for the manuscript to be submitted in its present form.

Some journals accept electronic submission but may still require paper

If electronic, follow journal instructions:File formats and how to separate paper into separate electronic files

Usually text, table and figures are separate

Label any graphics, tables or figures on reverse with name and title

Exact electronic and paper copies of what you submitted includingCorrespondenceManuscriptFigures and tablesPhotographs

Acknowledgement of paper receipt within one month

Letter from editor on status within 4 months

Papers sometimes get lost in main or in system

Do not hear, contact editor

Store all data and documentation surrounding data analysis in durable and appropriately referenced form

Store the original data—questionnaires, data collection sheets, CD’s

Keep as long as readers may reasonably expect to ask questions where you need to reference this data

Store 5-10 years--normal All references to where data held and

how archived logged in handbook Include all details so study can be

repeated Names, locations of electronic data files Data bases, data recodes, data analysis

Once paper submitted now property of journal

Editor has total discretion over who has access

Peer editors sometimes pass on papers to colleagues for review

Confidentiality not always maintained

Picks ones he/she feels worthy Supposed to be confidential Most believe anatomy is kept You should hear back in 4-6 months Be patient but okay to send polite

inquiry after 4 months

Is acceptable for publication Is acceptable for publication following

minor revisions Is acceptable for publication following

major revision May be reconsidered for publication

following major revisions May be considered for publication as a

letter or a short report Is unacceptable for publication

Rare to get “acceptable” without some form of revision but if you did CELEBRATE!

Revisions extensive may get sent back to external reviewers one more time for further comments

You usually have 3 to 6 months to make all revisions

After this time may be considered first submission all over again

You can withdraw at any time Must be formally accepted by editor

before you can submit to another journal

Send to another journal gets you another set of reviewers

Delays publication

Do not ignore comments Be calm and objective Deconstruct each of the messages into

individual items Respond to each item thoughtfully Make responses clear

Make most if not all of changes Don’t have to fully accept suggestions

but must give reasons that will convince editor your opinion is reasonable

Be pragmatic and not dismissive of reviewer’s work

Use a table where you list each comment and your response

1 The term “active group” may not be the best term

Through-out paper

The term has been changed to “interview group”

2 Figure 3 could be deleted

Fig 3 Fig 3 defines the . . . We have retained the figure but are happy to delete it if the editor wishes

3 Setting the type I error at 0.05 does not avoid the possibility of a type I error, it just controls the error rate as not greater than 5 %

Pg 3 We have replaced the phrase “to avoid the possibility of a type II error” with “to control the type I error rate”

4 A summary paragraph would be helpful

Pg 4 A summary paragraph has been added

5 If sampling was by urban area, then there is a potential statistical issue to do with cluster designs

Pg 2

We apologize for unintentionally being misleading. This study was not a cluster design and we have altered our working accordingly. Business were selected blah blah blah. It is now clear.

Make sure all reviewer’s comments have been addressed

Always be polite even if reviewer’s comments were harsh

Explain just enough to enable you to survive

Benefit from it, and learn how to be a competent reviewer.

No guarantee will be published Editor will consider new version and

your replies to comments Editorial process can be subjective Reviewer’s comments only one factor May lean heavily to accepting papers

where journal is cited regularly

Editor may reject paper even if reviewer’s comments were minor

Editor may accept one that reviewer’s suggested fundamental changes

Editor has absolute discretion Be prepared to accept decision and

move on

Nothing is Certain– Until paper is published

Paper can be formally accepted by regional editor but then rejected by later by editor-in-chief

If you think reviewer’s overlooked or misunderstood something important

Appeal by writing a letter stating your case

Rare decision overturned but it does happen

If appealing—send new copy of paper—rejected papers do not remain on file

Usually arrive much quicker—editors don’t send out all papers to reviewers

Usually within a month becauseInsufficiently originalNo interest to journal’s readersScientifically flawedToo longIncomprehensible

Swallow pride—be optimistic and objective

Decide what to do Maybe needs major attention You selected wrong journal Once receive rejection—free to submit it

to another journal Without changes or a total rewrite!

Reviewers critical of basic methods—may need to rethink study and do further data analysis

Reviewers critical on style and presentation—fix problems before resubmitting to another journal

Three rejections—completely reassess entire approach

Receive page proofs—typeset copy of work—how looks in journal

May take several months to receive Time for final check Journal usually send proof reading

instructions you must follow Usually standard proofreading marks

Read every word carefully Any errors in final publication are your

responsibility Sometimes copy editor can reword your

sections

Check galleys or page proofs for:TypographicalPrinting or reporting errorsTable errors—check formatting and content—check numbers carefully

Table and figures usually retyped—main errors occur here

Can only make simple changes Make sure words and numbers are

totally correct Punctuation changes are typical Some journals add more—some take

them out Can’t change punctuation unless justify

to editor

Law protects writers from having work copied without permission

Worldwide to protect authors If you are a researcher, allowed to copy

any copyright material for purpose of research

Ok if you do not infringe copyright and use it fairly in research

Copyright automatically assigned to journal once you have submitted paper

Your paper under strict copyright restrictions once you submit

Do not give copies to anyone who is not coauthor

Write in heading: This article is confidential and is under strict copyright restrictions—do not copy under any circumstances

Once published Journal may allow copies on personal

website 3 years after publicationCannot be scanned or downloaded version

from publicationCannot post it on department, university or

corporate websiteAlways ask for clarification

Totally redo study Restate problem? Use different method? Get help with writing? Try electronic journal?

Often use bottom up approach Take anything that meets their

minimum standards Weed out submissions that are

incomprehensible or that report research that is ethically or methodologically unsound

Let readers select the items that interest them by means of electronic searching and alerts

Professional appraisal systems, especially those for nurses, are often referred to as peer review. They may involve assessment by peers (that is, colleagues at the same level) and by bosses and supervisors or even junior colleagues. Unlike the processes used to review submissions to journals, conferences, or funding bodies, these assessments take place face to face. The aim of the process is to review performance and identify both

strengths and problems.

When you asked a colleague to read through something that you have written

I had my six good serving men, they taught me all I knew, Their names were, how and why and what, and when and where and who.