A Longitudinal Study of ClinicalPeer Review's Impact on Quality andSafety in U.S. HospitalsMarc T. Edwards, MD, president and CEO, QA to QI, LLC: A Patient SafetyOrganization, West Hartford, Connecticut

E X E C U T I V E S U M M A R YGlinical peer review is the dominant method of event analysis in U.S. hospitals. Itis pivotal to medical staff efforts to improve quality and safety, yet the quality assur-ance process model that has prevailed for the past 30 years evokes fear and is funda-mentally antithetical to a culture of safety. Two prior national studies characterized aquality improvement model that corrects this dysfunction but failed to demonstrateprogress toward its adoption despite a high rate of program change between 2007and 2009. This study's online survey of 470 organizations participating in either ofthe prior studies further assessed relationships between clinical peer review programfactors, including the degree of conformance to the quality improvement model (theQI model score), and subjectively measured program impact variables. Among the300 hospitals (64%) that responded, the median QI model score was only 60 ona 100-point scale. Scores increased somewhat for the 2007 cohort (mean pair-wisedifference of 5.9 [2-10]), but not for the 2009 cohort. The QI model is expanded asthe result of the finding that self-reporting of adverse events, near misses, and hazard-ous conditions—an essential practice in high-reliability organizations—is no longerrare in hospitals. Self-reporting and the quality of case review are additional multi-variate predictors of the perceived ongoing impact of clinical peer review on qualityand safety, medical staff perceptions of the program, and medical staff engagementin quality and safety initiatives. Hospital leaders and trustees who seek to improvepatient outcomes should facilitate the adoption of this best practice model for clini-cal peer review.

For more information about the concepts in this artide, contact Dr. Edwards atmarc@QAtoQI.com.

369

JOURNAL OE HEALTHCARE MANAGEMENT 58:5 SEPTEMBER/OCTOBER 2013

I N T R O O U C T I O NClinical peer review has long beenpivotal to physicians' efforts to improvequality and safety. Considering itsimportance to the profession andsociety, surprisingly little research hasbeen directed toward optimizing theprocess. In the United States, the pre-vailing process model—quality assur-ance (QA)—emerged as an unintendedconsequence of Joint Commissionstandards promulgated in 1979 thatcalled for an organized program of QA(Sanazaro & Mills, 1991). In order tocomply, hospitals abruptly abandonedthe clinical audit model, which hadpreviously been promoted by profes-sional standards review organizations.The resulting QA model has beenwidely criticized as being at best inef-fective and at worst antithetical to truequality improvement (Berwick, 1990;Dans, 1993). By focusing narrowly onquestions of individual competenceto meet the standard of care, the QAmodel evokes fear of censure and lossof livelihood. It thereby contributes tothe persistent culture of blame docu-mented in the 6-year trend in Survey ofHospital Safety Culture data, in whichthe composite measure "nonpunitiveresponse to error" has run consistentlyat an abysmal 44% positive (AHRQ,2012). The associated unstructuredmethods of evaluation have low reli-ability (Goldman, 1994). Moreover, theuse of generic screens for adverse eventsto identify cases for review is inefficient,particularly compared to the self-reporting process adopted by airlinesand other types of high-reliabilityorganizations (Sanazaro & Mills, 1991;O'Neil et al., 1993; Helmreich, 2000).

A 2007 study was the first to reportnormative data about clinical peerreview practices on a national scale(Edwards & Benjamin, 2009). Using ahigh-level process framework (see Figure1), it collected data on 39 items from339 institutions including 61 majorteaching hospitals. The study identifiedwide variation in the scope of activitiesencompassed by peer review programsbeyond the essential core process of ret-rospective medical record review (casereview).

The findings from this study suggestthat medical staff peer review may bethe dominant method of adverse eventanalysis in U.S. hospitals. The mediancase review volume (1-2% of hospitalinpatient volume) is an order of magni-tude closer to the known rate of prevent-able adverse events than is formal rootcause analysis, the mandated methodfor the most serious occurrences. A highrate of program change was observeddue to dissatisfaction with the QAmodel and changes in Joint Commis-sion requirements. Most importantly,the 2007 study revealed a set of practicesthat are strongly associated with thebelief that the program has a significantongoing impact on the quality andsafety of care. Such practices includestandardization of process, recognitionof clinical excellence, attentive pro-gram governance, trustee involvement,integration with other hospital qualityimprovement activity, timely perfor-mance feedback, and identification ofclinician-to-clinician issues and otherprocess problems during case review(Edwards & Benjamin, 2009).

These practices define a newapproach to clinical peer review best

370

CLINICAL PEER REVIEW'S IMPACT ON QUALITY AND SAFETY IN U.S. HOSPITALS

F I G U R E 1Clinical Peer Review Process Framework

co

caSPos

Interdependent Hospital Processes

Governance Oversight

Inputs

Peer ReviewProcess

n/Outputs y

CA

Out

com

e

Administrative Support

• 2 0 0 8 Marc T. Edwards

Source: Edwards (2009). Reproduced with permission of the American College of Physician Executives.

described as the quality improvement(Ql) model. The Ql model contrastssharply with the QA model (see Table1) and can be understood as the prod-uct of consistent application of qualityimprovement methods to peer reviewprogram process, structure, and gover-nance. It highlights the extent to whichthe QA model has disconnected theends and means of peer review activity.

In 2009, the Ql model was validatedin a separate cohort of hospitals againstboth subjective and objective measuresof quality and safety (Edwards, 2010,2011). The validation study demon-strated that important differencesamong clinical peer review programscan predict up to 18% of variation on32 standardized performance measures.It also suggested that organizational fac-tors, such as leadership and openness tochange, influence program effectiveness.In addition, it showed a persistently

high rate of change, wherein many pro-grams were shifting from clinical depart-ment-based review to multispecialtycommittee-based review. However, the2009 study yielded no clear evidence ofadvancement of the Ql model.

These studies raised important newquestions. Does the Ql model betterserve to engage physicians in qualityand safety? Has any benefit been derivedfrom the trend toward the multispecialtycommittee process? Is self-reportingof adverse events being encouraged inhospitals? If so, what is its influence onpeer review program outcomes? Thepresent study was undertaken to answerthese questions and to further assess theevolution of peer review practices andtheir impact on quality and safety.

The above questions acquireadded importance in the context oftheories of quality improvement andpatient safety. The pioneering work of

371

JOURNAL OF HEALTHCARE MANAGEMENT 58:5 SEPTEMBER/OCTOBER 2013

TABLE 1Comparison of QA and Ql Models for Clinical Peer Review

Dimension QA Model Ql Model

Focus

Identify

Determine

Reference point for

physician assessment

Case finding

Goal of review

Process

Reliability

Data capture

Relation to hospitalQl process

Ultimate process

outputs

Govemance

Hospital trustee

involvement

Accountability for

improvement

Cultural impact

Rehabilitate or remove outliers

Substandard care

Competence

Single case

Generic screen triggers

Judgment

Variable

Low

"Leveling" against the

"standard of care"

Disconnected

Individual physician-focusedcorrective actions

Laissez-faire

Limited to adverse action

approval

Low

Fear, blame, punishment

"Shift the curve" of group

performance

Learning opportunity

Performance

Multiple cases

Self-reporting

Balanced performance evaluation

Standardized

Good

Multiple elements of performance

Highly interdependent

System improvement

Performance feedback

Recognition of clinical excellence

Attentive

Program performance monitoring

High

Trust, collegiality, mindfulness of

safety, organizational leaming

Source: Adapted from Edwards (2009). Reproduced with permission of the American College of Physician Executives.

Juran (1989), Deming (1982), Crosby(1979), and others launched the hey-day of total quality management in the1980s and contributed to the conceptof the leaming organization (Senge,1990). While the methods and toolsfor quality improvement have con-tinued to proliferate, all share a focuson business processes in the context

of the organizational system in whichthey occur. They also emphasize theessential responsibility of leadershipto shape organizational culture andperformance by communicating thevision and the performance standard,organizing for quality improvement,and providing adequate supports (atten-tion, resources, tools, and training).

372

CLINICAL PEER REVIEW'S IMPACT ON QUALITY AND SAFETY IN U.S. HOSPITALS

Defects are most often a property of thesystem and are infrequently the faultof individual workers. Therefore, theblame commonly cast on individualsfound at the "sharp end" of an adverseevent is demoralizing to staff, leavingthe organization powerless to improveperformance.

More recently, attention has beengiven to identifying the factors that havefostered high reliability and safety incomplex, dynamic, high-risk environ-ments (Reason, 2000; Weick & Sutdiffe,2001). Healthcare leaders have begunto champion high reliability as a worth-while goal, if not a mandate (Chassin &Loeb, 2011; Denham et al., 2009). Highreliability demands a culture of safetycharacterized by trust; the imperative toreport and improve; and the collectivemindfulness of the high cost of processfailure, the inevitability of human error,and the need to identify, contain, andrecover from errors as early as possible.It combines the systematic pursuit ofquality improvement with constantvigilance for the unexpected. The skill ofmanaging the unexpected well is under-developed in most organizations, largelybecause it requires a counterintuitiveact: a strong response to a weak signal(Weick & Sutdiffe, 2001).

METHODSThe present survey was conducted underthe auspices of a patient safety organiza-tion listed by the Agency for HealthcareResearch and Quality. The sample wasconstructed ftom the records of the twoprior national studies cited earlier. The2007 study was sponsored by the Ameri-can College of Physician Executives(ACPE); the University HealthSystem

Consortium; Premier Inc.; and hospitalassociations in Arkansas, California,Florida, Michigan, Missouri, South Car-olina, and Wisconsin. Each organizationused its own process for inviting thosewith working knowledge of their hos-pital's dinical peer review program toparticipate in an anonymous online sur-vey. Qn the basis of voluntarily providede-mail and/or phone contact informa-tion, 158 organizations participatingin the 2007 study could be positivelyidentified retrospectively. Qne of theseorganizations subsequently merged withanother participating facility. The ACPEalone sponsored the 2009 study. From362 complete responses, 330 uniqueorganizations were identified, two ofwhich have since closed; 15 had alsoparticipated in the 2007 study. Thus, thecombined data sets yielded 470 uniqueorganizations broadly representative ofU.S. hospitals.

The survey instrument incorpo-rated the QI model and those itemspreviously used that had the greatestexpected value for comparative pur-poses. It also collected information onadditional factors potentially relatedto program effectiveness, such as theperceived quality of case review. Severalphysician leaders assessed the draft forclarity and completeness. The survey isreproduced in the online supplement tothis article (www.ache.org/Publications/SubscriptionPurchase.aspx).

The degree of conformance to theQI model (the QI model score) was cal-culated on a 100-point scale on the basisof the responses to specific survey itemsaccording to the method previouslydeveloped (Edwards & Benjamin, 2009)(illustrated in the online supplement).

373

JOURNAL OF HEALTHCARE MANAGEMENT 58:5 SEPTEMBER/OGTOBER 2013

Key program outcome variablesincluded the following:

• Study participants' perceptions oftheir program's ongoing impact onquality and safety

• Medical staffs perceptions of the peer-review process

• Physician engagement in quality andsafety improvement

• Overall physician-hospital relations.

A total of 40 items were captured viafour web-based forms. Prior participantsreceived e-mail solicitations. Phone callswere made to nonresponding organiza-tions to identify alternative contacts.Data were collected from September15, 2011, through January 26, 2012, aspatient safety work product under theterms of the federal Patient Safety andQuality Improvement Act of 2005. Inaccord with its statutory protections,only aggregate, nonidentifiable data aredisclosed in this article.

A response was considered completeif all four pages of the survey were sub-mitted and partial if two or three pageswere submitted. A response with fewerthan two pages was considered a break-off. Both complete and partial responseswere included in the analyses. In the fewsituations in which multiple responseswere received from a given facility, oneset of responses was chosen according toorganizational titles of respondents.

The longitudinal change in QImodel scores was characterized by themean difference using a two-samplet-test. Ordinal logistical regressionwas used to define the primary factorscontributing to program impact onquality and safety, medical staff per-ceptions, medical staff engagement.

physician-hospital relations, andreviewer participation. Alternative mod-els were developed that either allowedfor or excluded other outcome variablesas predictors. For comparative purposes,at least one model was developed foreach outcome that included the QJmodel score as a predictor. Outlier val-ues were retained. A regression equationwas accepted only if all the factor coeffi-cients and intercepts were significant at p< .05 and if the goodness-of-fit test metp > . 1. Response levels were selectivelycollapsed as guided by this goal. Theestimate of variance explained by eachmodel was taken ftom the equivalentlinear regression model R^. Statisticalanalysis was carried out using Minitabversion 15 (2007).

RESULTSThe survey process yielded 297 completeresponses, three partial responses, andfour break-offs. The response rate was64%. Among these were 259 acute carehospitals, 21 critical access hospitals,16 children's hospitals, and 4 specialtyhospitals. The informants were generallyeither senior leaders (47%) or mid-levelmanagers (46%). The majority (71%)were physicians. There were no signifi-cant differences between respondentand nonrespondent hospitals on thebasis of prior survey QI model scores,perceived quality impact, and reportedmedical staff perceptions.

As shown in the online supplement,the tabulated responses for survey itemsindicate that dinical peer review contin-ues to be a key component of hospitalquality and safety improvement activ-ity: 40% believe it is very likely that theprogram makes a significant ongoing

374

CLINICAL PEER REVIEW'S IMPACT ON QUALITY AND SAEETY IN U.S. HOSPITALS

contribution to quality and safety, and87% report that it ranks at least 4 ona 6-point scale of importance relativeto all QI activity. Nevertheless, 29% ofinformants did not know their program-level case review volume. Comparableevidence of lack of attention to basicprogram metrics was present in the priorstudies.

Table 2 highlights changes in thescope of activities encompassed by peerreview programs over the past 4 years.The biggest change has been to addphysician health program administra-tion to the mix, but disruptive behaviormanagement, which was not previouslydifferentiated, is now included by evenmore facilities (77% vs. 56%). Both ofthese activities relate to Joint Commis-sion standards. About 70% of hospitalsseparate the peer review program ftomthe credentialing process but use theresults of peer review in credentialingdecisions including ongoing profes-sional practice evaluation (OPPE).

Among the aiteria used to identifycases for peer review, generic screens stillpredominate. Other criteria, includingphysician or hospital staff concerns andunexplained deviation ftom protocols,pathways, or specified clinical stan-dards, are applied much less ftequently.Although focused professional practiceevaluation (FPPE) for new privileges isincluded within the scope of the peerreview program at 77% of facilities,it infrequently translates to the casereview process. A variety of sources andmethods are used to apply these crite-ria, but it remains unusual to rely onself-reporting, reporting by residents, orintegration with quality improvementstudies. Even so, at 15 study hospitals.

self-reporting is among the top threesources for case identification. Discus-sions with leaders at several of the 15hospitals revealed that this was achievedthrough deliberate efforts. Moreover,30% of respondents agree or stronglyagree that "Medical staff membersfrequently report adverse events, nearmisses and/or hazardous conditionsaffecting their own patients for peerreview."

Table 3 shows changes in datacaptured during case review. About 80%of programs specifically document thecategory of event. More than half record10 additional data elements, includingratings of overall quality of care ren-dered by individual clinicians. Only onethird document findings of excellence incare. A sizable increase (18%) was seenin the identification of system-of-careissues, which is associated with the QImodel, but also in the use of appropri-ateness/standard-of-care ratings (28%),which are more closely associated withthe QA model.

About two thirds of programs recruitvolunteer reviewers, and one quartermandate participation as a role expec-tation for service members or heads.About 40% routinely provide training tonew reviewers. This training most com-monly covers review forms, documenta-tion, role expectations, and policy andprocedures. Some programs also train tochart review methods and legal or riskmanagement issues. Interpersonal skillsand QI methods are not commonlyaddressed. As was the case in 2007,only one in five programs compensatesreviewers.

Sixty percent make peer review deci-sions primarily in a committee setting.

375

JOURNAL OF HEALTHCARE MANAGEMENT 58:5 SEPTEMBER/OCTOBER 2013

TABLE 2Changes in Clinical Peer Review Program Scope 2007 to 2011

2007 2011 Change p-Value»

Activity

Retrospective medical record review

Focused individual review of quality when

serious concems are raised

Ongoing professional practice evaluation

Comparative evaluation of performance

measures (e.g., complication rates, core

measures, patient satisfaction)

Case-specific, individually targeted recom-

mendations to improve performance

Focused professional practice evaluation for

new privileges

Disruptive behavior management

Root cause analysis

Proctoring for new privileges

Comparative evaluation of aggregate datafrom peer review

Development and/or review of clinical N/A 62(187)

policies, order sets, etc.

Morbidity and mortality case conferences

Benchmarking to normative data (e.g.,

NSQUIP, STS, UHC, Premier)

Physician health program administration

Conducting quality improvement studies

and/or projects

Concurrent medical record review

Produdng educational programs for groups

of clinicians

Other forms of direa observation 23(79) 25(76) 2 N/S•Fisher's exact test for two proportions.

Note; N/A = not applicable; N/S = not significant; NSQUIP = American College of Surgeons National Surgical Quality

Improvement Program; STS = Society of Thoracic Surgeons; UHC = University HealthSystem Consortium

«=337

96 (324)

N/A

N/A

74 (249)

N/A

N/A

N/A

66 (223)

47 (160)

50(167)

«=300

97 (292)

91 (272)

82(247)

82 (245)

78 (234)

77 (232)

77 (232)

77 (231)

63 (188)

62(186)

0/.

1

8

11

15

13

N/S

.02

.003

<.OO1

.002

58(196)

N/A

15 (51)

41 (139)

54(181)

34 (116)

57 (170)

57 (170)

56(166)

54 (162)

50 (149)

43 (130)

-2

40

13

-4

9

N/S

<.OO1

.001

N/S

.02

376

CLINICAL PEER REVIEW'S IMPACT ON QUALITY AND SAFETY IN U.S. HOSPITALS

In about two thirds of these (39% over-all), the committee has multispecialtyphysician representation. In 2009, noprograms reported having representa-tion ffom nursing or other disciplines.In the present study, 37 hospitals specifi-cally indicated that nursing leaders orstaff are included. While these commit-tees are truly multidisciplinary in termsof composition, they do not conduct afully integrated peer review process inwhich all involved disciplines evaluateeach other's performance on an equalfooting. Controlling for other factors,multispecialty committee process hasno significant relationship to programeffectiveness. Committee discussion,however, is a significant predictor ofthe perceived impact on quality andsafety, with an odds ratio (95% confi-dence interval) of 1.8 (1.1-3.0) for ashiff to the highest level of committeediscussion.

The trend since 2009 has been forapproximately 20% of programs tomake major modifications in programstructure, process, and/or governanceeach year. These changes have com-monly involved multispecialty commit-tees or modification of the OPPE/FPPEprocess.

Ql model scores ranged ffom 1 to100 with a median of 60. A 10-pointincrease in the score was associatedwith an increase in reported qualityand safety impact with an odds ratio of2.2 (1.9-2.7). Only 16% of programsscored >75, a level that indicates sub-stantial conversion to the Ql model.For the 2007 cohort, Ql model scoresincreased. The mean difference was 5.9(2-10). Perceived quality impact andreported medical staff perceptions of the

program also improved. For the 2009cohort, a small increase in perceivedquality impact was seen, but no changein Ql model scores or medical staff per-ceptions was noted.

Table 4 presents a summary of 11regression models relating program andorganizational variables to outcomes.The Ql model score remains a primefactor explaining impact on quality andmedical staff perceptions. Self-reportingand the quality of case review appear inregression models that account for 47%of the variation in the perceived ongoingprogram impact on quality and safety,37% of the variation in medical staffperceptions of the program, and 25% ofthe variation in physician engagerrientin quality and safety. A shiff to the high-est level of self-reporting is associatedwith a 6-fold (1.8-21.6) greater likeli-hood of higher quality impact. Medi-cal staff perceptions are highest whenphysicians are engaged and the programembraces the Ql model. Not shown inTable 4, 43% percent of the variancein reviewer participation is explainedby reported medical staff perceptionsof the program, the extent to whichleadership is available to support anyneeded program change, the quality ofcase review, and the degree of processstandardization.

D I S C U S S I O NThis study is the first to report longitudi-nal data on clinical peer review practicesin the United States. Furthermore, itrefines our understanding of the key fac-tors contributing to program effective-ness; expands the Ql model to includeself-reporting and the quality of casereview; confirms the supporting role

377

louRNAL OF HEALTHCARE MANAGEMENT 58:5 SEPTEMBER/OCTOBER 2013

TABLE 3

Changes in Data Captured During Case Review 2007 to 2011

2007 2011 Change p-Value'

Data Element

/V=335 /V=293

% (n) % (n)

Categorization of an event type (e.g.,

mortality, readmission)

Identification of process of care issues

involving other disciplines, information

systems, organizational policy/

procedures, etc.

Rating of appropriateness or deviationfrom standard of care

Identification of dinician-to-dinicianissues (e.g., gaps in communication,call coverage, supervision, coordinationamong dinicians)

Overall quality-of-care rating for an indi-

vidual clinician

Any recommendations for improved per-

formance of an individual clinician

Other recommendations or actions for

improvement (e.g., group educational

program, correction of system or pro-

cess problem, initiation of a QI study,

extemal review)

Identification of contributory factors to an

adverse event (e.g., high-risk patient or

procedure)

Rating of the degree of any associatedpatient harm

Categorization of type of error made (e.g.,

diagnosis, treatment, performance)

Rating of whether an adverse event was

preventable

Overall rating of completeness of medical

record or quality of documentation

Categorization of reason for error (e.g.,

knowledge, skill, habits, situational

factors)

81 (272) 81 (238)

53 (178) 71 (208) 18

42 (142) 70 (204)

62 (207) 69 (203)

57(190) 67(197)

70 (234) 65 (189)

58(194) 62(183)

51(172) 62(182)

65 (218) 59 (174)

N/A 56 (163)

53 (179) 53 (156)

N/A 48 (140)

N/A 42 (122)

<.OO1

28 <.OO1

.05

10

-5

.008

N/S

N/S

11

-6

.008

N/S

378

GLINIGAL PEER REVIEW'S IMPACT ON QUALITY AND SAFETY IN U.S. HOSPITALS

TABLE 3 continued

Rating of whether an individual dinician

could have prevented an adverse event

Written case analysis

Identification of excellence in dinical care

Structured radngs of spedfic elements of

individual performance (e.g., legibility,

quality of history and physical exam,

differential diagnosis, orders)

Radng the likelihood that another pro-

vider would have handled the case

differently

None of the above

'Fisher's exact test for two proportions.

Note: N/A = not applicable; N/S = not significant.

37(123) 40(118)

39(131)

N/A

38(127)

N/A

1(2)

39 (115)

34 (99)

24 (71)

22 (64)

1(4)

3 N/S

0

-14 <.OO1

of leadership, openness to change, andaccess to resources; offers a balancedperspective on the potential value ofmultispecialty committee process; anddemonstrates important interrelation-ships between the peer review process,the perceived program impact on thequality and safety of care, physicianengagement in quality and safety, andphysician-hospital relations. The studypresents stark implications for patientcare quality and safety that need theattention of all hospital leaders, not justphysician executives.

Despite the high rate of programchange, hospitals have been slow toadopt the QI model. A small increase inQI model scores was observed for the2007 cohort. This finding likely reflectsthe length of the follow-up intervals (4years vs. 2 years) more than intrinsicdifferences between cohorts. Substantialopportunity for improvement remainsfor more than 80% of programs. TheQA model perpetuates the toxic culture

of blame, which continues to dominateU.S. hospitals and hampers progress increating a culture of safety, while the QImodel seeks to maximize organizationallearning ftom defects in the system ofcare. Thus, hospital leaders and trusteesshould be concerned about the continu-ing harm that will result ftom accep-tance of the status quo.

This study demonstrates the viabil-ity and importance of self-reporting ofadverse events, hazardous conditions,and near misses by members of themedical staff. Although the experiencein industries that have pursued the pathof high reliability strongly attests to thevalue of nonpunitive self-reporting, onlyone healthcare program was previouslyknown to have sustained high rates ofself-reporting (90% of cases reviewed,70% of events identifiable by all means)and excellent clinical outcomes (Katz &Lagasse, 2000). With this study, we nowhave evidence that a growing number ofhospitals have not only encouraged the

379

JOURNAL OE HEALTHCARE MANAGEMENT 58:5 SEPTEMBER/OCTOBER 2013

TABLE 4

Summary of Key Predictors in Regression Models for Clinical Peer Review Effects'

Effects of Clinical Peer Review

Regression Model Predictors

(2) (3)(1) Medical Staff Medical Staff

Quality Impact Perception Engagement

(4)Physician-

HospitalRelations

Quality impactMedical staff perception

Physician engagement

Physician-hospital relations

QI model score

Self-reporting

Quality of case review

Relative importance of program

Openness to change

Resource availability

Leadership

Committee discussion

Model

RM%)

—

A,

A,

A,

A,

A,55

—

B,

B,

B,

B,

B,47

- A3

— — — A,

A3

B3 C3

C3 A, B,4

40 42 37

A3 B3 ^3 ..^ „^ ^^

38 25 18 24 18

These 11 regression models for the four main effects either allow for other outcome variables as predirtors (A) or exclude them

(B, C). Model C,, the weakest for physician-hospital relations, includes only one predictor (QI model score) but still explains

11% of variation. Complete statistical detail for all models is available from the author.

'Substitution of these variables gives equivalent models.

'By ANOVA, F = 7.01, p = .009.

practice but are realizing the expectedimprovement in quality and safety.

Self-reporting can only thrive inthe presence of trust. At a minimum,organizational policy must offerimmunity from sanctions for good-faith self-reporting (Edwards, 2012).Self-reporting also requires leadershipsupport, dear communication aboutthe program, ease of reporting, a blame-free review process, and resulting visibleimprovement. Intent to promote self-reporting entails the management of

these interdependencies and therebyaccelerates progress toward peer reviewprogram effectiveness and a cultureof safety. While this study focused onphysicians, the lessons are applicableto nurses and other disciplines regard-less of whether their performance isreviewed by peers or managers.

Self-reporting also solves the effi-ciency problem in hazard identification.By comparison, generic screens (andtheir derivative, global trigger tools) aresensitive but not specific. They are labor

380

CLINICAL PEER REVIEW'S IMPACT ON QUALITY AND SAFETY IN U.S. HOSPITAL;

intensive to apply and produce manyfalse-positives. Self-reporting can elimi-nate this waste.

Over the past few years, hospitalexecutives in the United States havebeen attracted by the concept of physi-cian engagement (Gosfield & Reinertsen,2007). The Affordable Care Act hasstimulated a resurgence of investment inprimary care physician practice manage-ment as a means of buying that engage-ment and the associated accountabilityfor cost control (Kocher & Sahini, 2011).Physician engagement in quality andsafety is predicted by perceived peerreview program impact on quality andsafety, medical staff perceptions of theprogram, and overall physician-hospital relations (38% of variance). Inturn, if we ask what factors most con-tribute to the program impact on qualityand safety, we find that about half thevariation can be explained by somecombination of the Ql model score,physician engagement, self-reporting,the quality of case review, and the pro-gram's importance relative to all otherQl activity at the hospital. These interre-lationships are highly relevant to physi-cian and hospital leaders. They shouldbe interpreted in light of the organiza-tion's specific needs for improvement.From a management perspective, ifthe organization does not have strongphysician engagement, adopting the Qlmodel peer review process and encour-aging self-reporting will likely increaseit. Organizations that already have itwill reap the other benefits of transition-ing to the Ql model by strengthening allactivity that supports quality and safety.

This study also considered physi-cian-hospital relations as a program

outcome, recognizing that peer reviewis only one factor among many withpotential influence. Even so, I amfamiliar with medical staff who initiatedimprovements in the peer review processwith precisely this objective in mind. Bytaking full responsibility to produce aresult of value to the hospital, the physi-cian leadership offered quid pro quo forhospital management to respond withchanges that were important to them.Much can be gained by initiating a virtu-ous cycle of cooperation.

The strongest regression model forphysician-hospital relations includesmedical staff perceptions of the peerreview program, physician engagementin quality and safety, and openness tochange, explaining 24% of the variance.The Ql model score itself explains 11%of the variance. While the link betweenpeer review and physician-hospital rela-tions is weaker than that with qualityand safety, clinical peer review programimprovement can still be beneficial.

In recent years. Joint Commissionrequirements for professional practiceevaluation and dissatisfaction with theQA model have been the primary driversof peer review program change. Multi-specialty committee-based review hasbeen embraced as a panacea. The con-cept has successfully challenged the iso-lated (and sometimes self-serving) silosof departmentalized review. The inclu-sion of other disciplines may increasethe sense of team, which is essential forthe pursuit of high reliability and use-ful in assessing the system and processissues that contribute to adverse events.It is not, by itself, sufficient to correctthe dysfunction of the QA model andbring the culture of blame to an end.

381

JOURNAL OF HEALTHCARE MANAGEMENT 58:5 SEPTEMBER/OCTOBER 2013

Much of the benefit for quality andsafety improvement comes ffom processstandardization and the use of com-mittee discussion per se. In my experi-ence, it is the discussion that generatesthe leaming and camaraderie that is soprofessionally satisfying to reviewers,particularly when the process is non-punitive and produces visible improve-ments in clinical performance. About40% of facilities still conduct most casereview without committee discussion.

In general, physicians will be morelikely to participate actively as peerreviewers when the program is wellregarded for contributing to improve-ments in quality and safety, the processis standardized, the quality of casereview is high, and leadership is evident.Professionalism is still important. Physi-cians willingly donate time when theprogram has a noble and effective aim,but they balk even if compensated whenit is perceived as punitive.

LimitationsThis study has several limitations. Itrelied on a single informant at eachinstitution. External validation ofresponses was minimal. Survey contentand response options varied across thethree studies. Quality and safety impactmight be assessed with objective perfor-mance measures or by program-specificindicators, such as the number ofimprovement opportunities realized inrelation to patient volume. Hospi-tal respondents may have been overlyoptimistic in their ratings for medicalstaff satisfaction and physician-hospital relations, which might be moreaccurately measured through survey ofmedical staffs. If so, the ratings should

still be strongly correlated with that goldstandard and of valid proxy value inregression analysis. The other issues arepartially mitigated by the robustness ofthe observed multivariate relationships,their internal consistency, and theirpersistence through the three studies.The high response rate minimizes thepotential for nonresponse bias. The lackof difference in prior survey data forrespondents and nonrespondents offersfurther reassurance in this regard.

C O N C L U S I O N SOne can confidently conclude that theclinical peer review process has muchroom to improve in most hospitals. Itremains all too common for it to beconducted haphazardly, without refer-ence to performance metrics and with-out diligent oversight. Such practice isnow an anomaly. Professional manage-ment has become the norm in hospitals.Organization leaders embrace qualityimprovement methods. They pursueimprovements in quality of care andpatient safety. They measure, manage,and improve all core processes. Thus,they should no longer turn a blind eyeto peer review.

Leaders need practical measures ofprogram performance to complementthe Ql model score. While a measure ofpreventable harm might be desirable,no practical option is widely available.Hospital event reporting captures onlya small proportion of adverse events(OIG, 2010). Even with statisticalsampling, global trigger tools demandsignificant physician review time thatmight be better directed toward identify-ing improvement opportunities throughthe Ql model. Other metrics, such as the

382

CLINICAL PEER REVIEW'S IMPACT ON QUALITY AND SAFETY IN U.S. HOSPITALS

hospital standardized mortality ratio,are incomplete and do not capture thepreventable portion. In any event, theonly acceptable goal is zero preventableharm (Denham et al., 2009), which theexisting system design is incapable ofachieving. Qnce self-reporting is estab-lished, the measurement problem willvirtually solve itself, because the harmwill be made visible, the preventabilitywill be estimated through case reviewand other methods, and all provid-ers and leaders will be engaged in thereduction effort. In the meantime, thebest measures may be simple countsof self-reported cases, improvementopportunities identified, and individu-als recognized for excellent performancein proportion to total cases reviewed,along with the ratio of cases reviewed tohospital admissions.

The best-practice QI model stronglysupports a culture of safety and hasthe potential to significantly improvepatient outcomes. The journey can beinitiated by promoting self-reportingand by changing the primary questionasked during case review ftom "Wasthe standard of care met?" to "Whatcan we learn to improve clinical perfor-mance?" Hospital leaders and trusteesshould hold the medical staff account-able to conduct effective peer reviewand provide adequate support to do so.Program change without measurableimprovement should not be accepted.

ACKNOWLEDGIVIENTSThe author gratefully acknowledgesthe contributions of Julie Felt, MD;Craig Montgomery, MD; and RichardWeinberg, MD, who assessed the draftsurvey for darity and completeness.

and of Larry Tonberg Edwards, PsyD,and Anthony Fotenos, MD, PhD, whoaitiqued the draft manuscript.

REFERENCESAgency for Healthcare Research and Qual-

ity (AHRQ). (2012). Hospital survey onpatient safety culture: 2012 user compara-tive database report. Retrieved from http://www.ahrq.gov/legacy/qual/hospsurveyl2/

Berwick, D. M. (1990). Peer review and qualitymanagement: Are they compatible? Qual-ity Review Bulletin, 16{7), 246-251.

Chassin, M. R., & Loeb, I. M. (2011). The ongo-ing quality improvement journey: Nextstop, high reliability. Health Affairs, 30(4),559-568.

Crosby, P. B. (1979). Quality is free. New York,NY: McGraw-Hill.

Dans, P. E. (1993). Clinical peer review:Burnishing a tarnished image. Annals ofInternal Medicine, 118{7), 566-568.

Deming, W. E. (1982). Out of the crisis, Cam-bridge, MA: MIT Press.

Denham, C. R., Angood, P., Berwick, D.,Binder, L, Clancy, C. M., Corrigan, I. M.,& Hunt, D. (2009). Chasing zero: Canreality meet the rhetoric? Journal of PatientSafety, 5(5), 216-222.

Edwards, M. T. (2009). Peer review: A new toolfor quality improvement. Physician Execu-tive Journal, 35(5), 54-59.

Edwards, M. T. (2010). Clinical peer reviewprogram self-evaluation for US hospitals.American Journal of Medical Quality, 25(6),474-480.

Edwards, M. T. (2011). The objeaive impactof clinical peer review on hospital qualityand safety. American Journal of MedicalQuality, 26(2), 110-119.

Edwards, M. T. (2012). Engaging physiciansin patient safety through self-reporting ofadverse events. Physician Executive Journal,38(4), 46-52.

Edwards, M. T, & Benjamin, E. M. (2009). Theprocess of peer review in US hospitals.Journal of Clinical Outcomes Management,16(10), 461-467.

Coldman, R. L. (1994). The reliability of peerassessments: A meta-analysis. Evaluationand the Health Professions, 17(1), 3-21.

Cosfield, A. C , & Reinertsen, 1. L. (2007). Shar-ing the quality agenda with physicians.Trustee, 60(9), 12-17.

383

JOURNAL OF HEALTHCARE MANAGEMENT 5 8 : 5 SEPTEMBER/OGTOBER 2 0 1 3

Helmreicb, R. L. (2000). On error manage-ment: Lessons from aviation. British Medi-cal Journal, 320, 781-785.

Juran, J. M. (1989). Juran on leadership for qual-ity: An executive handbook. New York, NY:Free Press.

Katz, R. I., & Lagasse, R. S. (2000). Factorsinfluencing the reporting of adverse ped-operative outcomes to a quality manage-ment program. Anesthesia & Analgesia, 90,344-350.

Kocher, R., & Sabini, N. R. (2011). Hospitals'race to employ physicians—Tbe logicbebind a money-losing proposidon. NewEngland Journal of Medicine, 364(19),1790-1793.

Minitab. (2007). Minitab 15. [Statistical soft-ware]. College Stadon, PA: Minitab Inc.

Office of Inspector General (OIG). (2010,March). Adverse events in hospitals: Methodsfor identifying events. No. OEI-06-08-00221.

Retdeved from bttp://oig.bbs.gov/oei/reports/oei-06-08-00221 .pdf

O'Neil, A. C, Peterson, L. A, Cook, E. F., Bates,D. W., Lee, T H., & Brennan, T A. (1993).A compadson of physicians self-reportingto medical record review to idendfymedical adverse events. Annals of InternalMedicine, 119, 370-376.

Reason, J. (2000). Human error: Models andmanagement. British Medical Journal, 320,768-770.

Sanazaro, P. J., & Mills, D. H. (1991). A cddqueof tbe use of generic saeening in qualityassessment. Journal of the American MedicalAssociation, 265(15), 1977-1981.

Senge, R M. (1990). The fifth discipline: The art& practice of the leaming organization. NewYork, NY: Doubleday.

Weick, K. E., & Sutdiffe, K. M. (2001). Man-aging the unexpected. San Francisco, CA:Jossey-Bass.

P R A C T A P P L I C A T

Maire O. Simington, PhD, FACHE, director. Care Management Services,Banner Health, Phoenix, Arizona

A s Edwards notes in his artide, "Glinical peer review is the dominant method ofevent analysis in U.S. hospitals." Traditional peer review, though useful in identi-

fying outlier practitioners, has not always been useful in improving dinical processes.This system is designed to identify only those physicians who fail to meet the mini-mum standard of care, a definition that is purposely broad to accommodate somelegal protections. The application of these ratings to an individual physician is inher-ently subjective and biased—as well as inconsistent. Multidisciplinary peer reviewpanels tend to be more objective than those with members ftom a single discipline,but both lack robust mechanisms for quality improvement. In recent years, a shifthas occurred toward increased transparency regarding data sharing for improvementpurposes, though self-reporting of inddents is still slow.

Some organizations, induding Banner Health, have a hybrid model in placethat incorporates elements of both the quality assurance (QA) model and the qual-ity improvement (QI) model. While aeating a culture of leaming is important, sois accountability. The QI model sets the stage for a culture of accountability thatrequires compliance with quality measures and participation in dinical processimprovement. It thus offers a more robust approach to improvement than simplyremoving a few "bad," or outlier, practitioners.

384

Copyright of Journal of Healthcare Management is the property of American College ofHealthcare Executives and its content may not be copied or emailed to multiple sites or postedto a listserv without the copyright holder's express written permission. However, users mayprint, download, or email articles for individual use.