1

CNACA Comparison of Medical Error Reports Submitted to a Voluntary Patient Safety Reporting System by Different Types of Reporters: A report from the ASIPS Collaborative

Daniel M. Harris, PhD, The CNA Corporation,Wilson Pace, MD & Doug Fernald, MA, Univ of CO DFM,

Linda Marr, MS, The CNA Corporation

AcademyHealth Annual Research MeetingJune 6, 2004

N

2

Presentation outline

BackgroundResearch objectiveStudy design FindingsConclusions & implications

3

Background Applied Strategies for Improving Patient Safety

(ASIPS) is a 3-yr AHRQ-funded demonstration project based at the Univ of CO Dept of Fam Med

Developed patient safety reporting system (PSRS) Collect voluntary narrative reports of medical errors from

providers, other clinical staff, and non-clinical staff at primary care practices in two CO PBRN

Anonymous or confidential-to-anonymous reports Instructed to report “any event you don’t wish to have

happen again that might represent a threat to patient safety”

Overall purpose of project is to test ability of the PSRS to collect incident reports and to use them to design interventions to improve patient safety

4

Research objective Medical error reporting systems are advocated as a

strategy for improving patient safety Learning from errors requires receiving reports on repre-

sentative range of errors Limited evidence exists re: nature of events report-

ed to such systems by different reporter types, especially in ambulatory primary care settings Do different types tend to report different types of events?

Our objective is to (1) analyze reports submitted to ASIPS PSRS to ascertain “who reports what,” (2) identify similarities and differences in reports, and (3) characterize differentiating event attributes by reporter type

5

Study design Error incident reports are received from 34 primary care

practices throughout Colorado Reports are consensually coded into multi-axial taxonomy of

400+ attributes by teams of coders Attributes characterize events by type of participants & their

contribution, complexity, setting, clinical intent, activities, under-lying causes, patient outcomes, interventions, and “discoverer”

Attributes are coded “1” if present in report; “0” if not present Reporters self-classified as provider, other clinical

staff, or non-clinical staff 522 (85.8%) of 608 reports received thru Aug 2003 identified a

reporter type Report content compared by reporter type using cross

tabs, ANOVA, and discriminant analysis

6

Findings

Analysis of report Reports by reporter type Nature of report Event “discoverer”

Analysis of event reported Participants Patient harm Communication errors Other event characteristics Discriminant analysis

7

Reports by reporter type

Most reports (68.6%) from providers Unchanged from yr1 to yr2

About 25% from other clinical staff Increased from yr1 to yr2

Fewest reports (6.9%) from non-clinical staff Deceased from yr1 to yr2

despite project effort to increase participation

Percent of Reports within Year, by Reporter Type

0

20

40

60

80

Prov Oth Clin Non-Clin

Total Yr 1 Yr 2

8

Nature of report

No difference by reporter type: Patient gender Perceived prevent-

ability Perceived pt knows Mean taxonomy,

event activity, and “don’t know” codes

Differences by type:Patient age

• Mean lowest for non-clin; highest for provider

At least 1 “insufficient information” code

• Non-clin most likelyNoted in Med Rec’d

• Non-clin least likelyPerceived pt harm

• Oth clin least likely

9

Event “discoverer”Providers & other clinical

staff likely to report events discovered by caregivers like themselves

Non-clinical staff likely to report events discovered by office staff like themselves

Providers somewhat less likely to report events discovered by patients or patient’s families

Percent of reports discovered by various parties, w/in reporter

type

0

20

40

60

80

100

Caregvr Ofc Staff Pt/Fam

Prov Oth Clin Non-Clin

10

Event participants Each reporter type is

more likely to report an event with a participant of the same type All relationships statistically

significant by 2

No significant association by reporter type for other types of participants: Patient/family or 3rd party Number of participants or

number of participant types (by ANOVA)

Percent of reports with selected type of participants,

within reporter type

0102030405060

Prov Oth Clin Non-Clin

Prov Oth Clin Non-Clin

11

Patient harmOverall, 26.4% of report-

ed events coded w/some form of patient harm Additional 7.7% of reports,

coded w/patient may have been harmed, but too early to tell

Significant differences in type of harm (by 2): Provider reports most

likely coded w/clinical harm

Non-clinical staff reports most likely coded with non-clinical harm

Other clinical staff reports least likely coded with any harm

Percent of reports coded with patient harm, w/in reporter

type

05

101520253035

Clin Harm Non-ClinHarm

Any Harm

Prov Oth Clin Non-Clin

12

CommunicationOver half (57%) of reports

involve a communication error between parties Each reporter type likely to

report communication involving their own type

Oth clin staff less likely to report within office communication errors

Overall, non-clin staff less likely to report communication errors

Providers more likely to report communication involving patients

Percent of reports with communication errors

involving selected parties, w/in reporter type

0

10

20

30

40

50

Att ProvParty

Non-PhyParty

Ofc StaffParty

Prov Oth Clin Non-Clin

13

Other event characteristics No difference by

reporter type: Missing information General proc issue Medication error Supervision error Intervention following

error detection Judgement error Clinical knowledge or

skill error Resource function or

availability error

Differences by type:Documentation

• Oth clin most likelySpecific proc issues

• Delay in Dx and Tx – prov most likely

• Delay in testing – oth clin staff most likely

• Dx testing error – non-clin least likely

Pt mgmt error• Non-clin most likely

Distraction/inattention• Prov least likely

System issue• Prov least likely

14

Discriminant Analysis (1) Identify event attributes that differentiate be-

tween reports submitted by each reporter type Stepwise method: attributes enter that maximize

distance between 2 closest groups (16 of 32 entered) Resulting 2 discriminant functions differentiate

between the 3 reporter types• Group centroids are significantly different• Provider reports are most different from other 2 types

Canonical correlations (~ANOVA Eta) for the functions are .455 & .346; jointly account for 30% of variance in group scores

Discriminant scores correctly classify 63% of reports Classify unknown reports in similar proportion

15

Discriminant Analysis (2) The analysis identified attributes that best discriminate

between events reported by each reporter group Provider reports most differentiated by:

provider participant, communication involving a provider, disclosure to a patient, delay in diagnosis, diagnostic testing error, and problem with resource (availability or function)

Other clinical staff reports most differentiated by: Non-physician provider participant, third party participant,

communication involving a provider, communication within the office, delay in performing a test, diagnostic testing error, and misuse of a system

Non-clinical staff reports most differentiated by: Non-clinical staff participant, non-clinical harm, error in patient

management, distraction/inattention error, misuse of a sys-tem, and malfunction of a system

16

Conclusions

Different types of reporters tend to submit reports of different kinds of medical errors to a voluntary PSRS Each type of reporter tends to report errors

involving the kinds of activities and participants they can be most expected to observe and believe should be reported

For example:• Providers tend to report clinical harm events while non-

clinical staff tend to report non-clinical harm events• Providers tend to report events involving clinical proce-

dures while non-clinical staff tend to report patient management and system issue events

17

Implications To be effective in identifying a full and repre-

sentative range of errors and threats to patient safety, a voluntary PSRS needs to assure that it receives incident reports from all types of staff (and patients?) who are in a position to observe errors Provider exclusive or dominated reporting systems

will restrict this range Our experience demonstrates the difficulty of

obtaining full participation of non-providers More effort necessary to achieve their participation

18

Questions?

The CNA Corporation