education of others. Thank you also to Kathy Cable, MLS, Infor-mation Services Reference and Liaison Librarian.

At the time of submission, the authors reported no competinginterests.

Letters of Recommendation in ResidencyTraining: What Do They Really Mean?

Letters of recommendation (LOR) are required as partof the selection process for hiring fellows and faculty.LOR contain information regarding attitudes, aptitude,leadership ability, behaviors, and character of graduatingresidents, based on personalized testimony from someonewith rank and adequate exposure to judge his or her qual-ities (1). Some believe LOR are the optimal criterion usedwhen evaluating applicant materials (2). Despite the ex-pectations of LOR as a qualitative measure of success,their accuracy and interpretation have been questionedbecause of letter writers’ insufficient information (3).Studies have reported LOR to be unreliable and the leastvalid of all selection measures (4). Also, letter construc-tion and interpretation are affected by low interrater reli-ability between LOR writers (3, 5) and conscious andunconscious motives and biases (5). The value of LOR isdependent on the reader and writer’s understanding andinterpretation of complex prose and descriptions that makeup the content of the letter (3).

A 34-item online survey was created and administerednationally to educators who were members of the Amer-ican Association of Directors of Psychiatric ResidencyTraining to gather information regarding the writing andinterpretation of LOR for psychiatry residents. Questionsincluded respondent demographics, professional informa-tion, views on writing and interpreting LOR, letter source,resident’s technical and clinical skills, and personalitycharacteristics. Negative statements, honesty, perceivedpressure to write LOR, legal repercussions, and factorscontributing to writer bias were also assessed. Descriptivestatistics were used to report response rates. Survey re-sponses were evaluated by chi-square, Fisher’s exact test,or Kruskal-Wallis tests to determine the association be-tween subgroup characteristics and responses. A p value of0.05 was used to assess significance.

Of the 103 respondents (17%), 72% were program di-rectors; 61% were men. Nearly all (90%) reported “some”or “a lot” of pressure to write a positive letter; 22% re-ported never writing negative letters; and nearly half(43%) feared legal repercussions for writing a negative

letter. One-third (33%) reported they have omitted orwould omit an area of concern; 30% reported they wouldspecifically indicate a resident’s weak technical or clinicalskills, whereas 50% reported they would “hint at it.” Ofthose remaining, 20% reported they would not indicate theweakness, with a higher percentage of men indicating theywere likely to omit the weakness (N�16, 25% men; N�4,10% women). Similarly, 35% (N�36) reported that theywould indicate poor interpersonal skills, with half (51%,N�52) hinting at it, and 14% omitting the weakness; nogender differences were identified. When reading LOR,nearly half (44%) reported difficulty deciphering residentcompetency and interpreting LOR (45%) “most of thetime.” When asked to compare, the majority (68%) re-ported that personality characteristics were more impor-tant than technical/clinical skills, whereas 32% reportedthe opposite. A letter omitting descriptions of technical orclinical skills was interpreted as negative by 75%; theremaining 25% indicated “it depends.” When asked “Howimportant is personal information about the applicant inthe LOR?” nearly half (44%) indicated “important,” and20% indicated “very important.” Half (50%) interpretedLOR negatively when personality characteristics wereomitted, whereas nearly all the remaining respondents(45%) indicated “it depends.” When asked if writer biaswas considered when reading LOR, three-quarters (75%)responded positively. Over half (55%) indicated that letterlength had “hidden meaning.” Written comments indicatedthat shorter letters were interpreted as more negative; ex-amples included “short letters indicate poor candidates,”and “short letters always raise concern about why therewas so little to say.”

Inconsistency in reporting and interpretation of LORrevealed biases and limited usefulness in decision-making.Both inflation and evasion of descriptions were reported,and intentions of the writer were often unclear, confirmingreports that important information is frequently missing orworded in a manner that is subject to broad interpretation(3). Nearly all reported some degree of pressure to write apositive letter; the vast majority reported never refusing towrite LOR. One-quarter reported omitting negative ele-ments in LOR, which implies that “letter-inflation” mayresult in a ceiling effect, making it difficult to discriminateamong applicants (6). Overall, survey results verified theimportance and challenges to interpretation of LOR. Anal-ysis also illustrated the competing needs to find goodplacements and awareness of contentious legal environ-ments, as indicated by the reported hesitation to includenegative comments about personality and technical skills,

LETTERS

342 http://ap.psychiatryonline.org Academic Psychiatry, 35:5, September-October 2011

omissions, and misrepresentations of abilities. Clearly, in-complete communications compromise the reader’s abilityto decipher competency. LOR that are inconsistent with anindividual’s abilities may hurt the applicant, future pro-gram graduates, and ultimately discredit the writer. A cul-ture change is needed from writers of LOR to clarify andprovide more accurate and unbiased descriptions of can-didates and to improve meaning of key phrases for thereaders. If the required LOR is to remain a valuable com-ponent of the application process, consistency of languageand full, interpretable description of clinical and person-ality factors are essential.

Lisa M. MacLean, M.D.

Dept. of Behavioral Health SciencesHenry Ford Health System

Gwen Alexander, Ph.D.

Nancy Oja-Tebbe, B.S.

Dept. of Public Health SciencesHenry Ford Health System

Detroit, MIE-mail: lmaclea1@hfhs.org

At the time of submission the authors reported no competinginterests.

References

1. Aamodt MG, Williams F: Reliability, validity, and adverseimpact of references and letters of recommendation. Sympo-sium conducted at the 25th annual conference of the Societyfor Industrial and Organizational Psychology, Los Angeles,CA, April 2005

2. Sheehan EP, McDevitt TM, Ross HC: Looking for a job as apsychology professor? factors affecting applicant success.Teach Psychol 1998; 25:8–11

3. O’Halloran CM, Almaier EM, Smith WL, et al: Evaluation ofresident applicants by letters of recommendation: a compar-ison of traditional and behavior-based formats. Invest Radiol1993; 26:274–277

4. Aamodt MG, Bryan DA, Whitcomb AJ: Predicting perfor-mance with letters of recommendation. Public Pers Manage1993; 22:81–90

5. Baxter JC, Brock B, Hill PC, et al: Letters of recommen-dation: a question of value. J Appl Psychol 1981; 66:296 –301

6. Miller RK, Van Rybroek GJ: Internship letters of recommen-dation: where are the other 90%? Prof Psychol Res Proc1988; 19:115–117

Resident Physicians and Peer Review

The American Journal of Psychiatry Residents’ Journalwas created in 2006 to teach residents how to read original

research reports and also to highlight findings from TheAmerican Journal of Psychiatry (1). Since then, the Resi-dents’ Journal has expanded into a resident-authored, edited,and peer-reviewed monthly publication (2). The editors (Cer-imele and Fayad) began peer-reviewing manuscripts in July2010, and opened the process to other residents in October2010. Some residents volunteered to serve as a peer reviewer;other residents were recruited by the editors. The residentsselected as reviewers were required to have previously au-thored a manuscript for any journal.

Before reviewing a manuscript, resident reviewers areasked to read articles describing the peer-review process(3). The editors then send a manuscript to the residentreviewer, offer guidance, and usually receive a thoroughreview within 1 week. Authors appreciate the well-writtenreviews, and revise their manuscripts on the basis of thepeer review. As editors, our experience with the peer-review process has been positive. We have found thatresidents are capable of assessing a manuscript’s clinicalapplication, the writing style, how interesting the piecewill be to other residents, and the relevance and compre-hensiveness of the references. Reviewers generally requiresome guidance in giving their opinion on the manuscript’soutcome (e.g., “major modify”), and in providing com-ments reserved only for the editor. As resident editors,implementing a review process required that we teachourselves to correspond with reviewers about their com-ments, incorporate their impressions into our decision-making, and identify a thoughtful review.

To identify previous reports of manuscript peer-reviewtraining in residency, we searched the following term com-binations in PubMed: residency, resident, peer review,manuscript review. We found three articles that have sys-tematically described the peer-review process for the pur-pose of educating young physicians (3–5), but found noreports of residency training programs implementing anyformal curriculum on manuscript review. Furthermore, wetalked with training directors at our residency programs tofind out whether they have ever taught residents to review.In one program, the director delivered a 1-hour lecture anddiscussed articles about peer-review to a group of PGY-1residents in an adult-psychiatry program. Overall, the directorfelt that these residents were receptive to learning about peer-review at that time, although there has been little evidencethat this technique led to residents’ serving as reviewers,possibly because of the few such opportunities available forresidents. Although some journals actively seek out juniorreviewers (6), the peer-review process at many journals isreserved for senior faculty, leaving early-career physicians

LETTERS

343Academic Psychiatry, 35:5, September-October 2011 http://ap.psychiatryonline.org