11

The Use of Control Charts in The Use of Control Charts in Health Care Monitoring and Public Health Care Monitoring and Public

Health SurveillanceHealth Surveillance

William H. WoodallWilliam H. Woodall Department of StatisticsDepartment of Statistics Virginia TechVirginia Tech Blacksburg, VA 24061-0439Blacksburg, VA 24061-0439 bwoodallbwoodall@@vt.eduvt.edu

22

Topics to be covered:Topics to be covered:

Some general issuesSome general issues Risk adjustmentRisk adjustment Examples of some of the plots used for Examples of some of the plots used for

monitoringmonitoring Detection of clusters of chronic disease Detection of clusters of chronic disease League tables, control charts, and funnel League tables, control charts, and funnel

charts for cross-sectional datacharts for cross-sectional data ConclusionsConclusions

33

In 1999 the Institute of Medicine reported the In 1999 the Institute of Medicine reported the number of deaths due to medical errors in number of deaths due to medical errors in U.S. hospitals to be 44,000 to 98,000 per year. U.S. hospitals to be 44,000 to 98,000 per year.

Some prefer the term “preventable adverse Some prefer the term “preventable adverse events.”events.”

44

Examples of health care variablesExamples of health care variables Lab turnaround timeLab turnaround time Days from positive mammogram to definitive Days from positive mammogram to definitive

biopsybiopsy Patient satisfaction scoresPatient satisfaction scores Medication error countsMedication error counts Emergency service response timesEmergency service response times Infection ratesInfection rates Mortality ratesMortality rates Number of patient fallsNumber of patient falls Post-operative length of stayPost-operative length of stay ““Door-to-needle” time ……and many others…Door-to-needle” time ……and many others…

55

Control charts are used to Control charts are used to understand variation over time and understand variation over time and to detect unusual events and trends.to detect unusual events and trends.

They are most effective in a hospital They are most effective in a hospital when used as a part of its organized when used as a part of its organized quality improvement program, such quality improvement program, such as Six-Sigma. as Six-Sigma.

66

Observation

Indiv

idual V

alu

e

5045403530252015105

3.0

2.5

2.0

1.5

1.0

0.5

0.0

_X=1.334

UCL=2.744

LCL=-0.077

I Chart of Transformed Time

77

Sample

Pro

port

ion

5045403530252015105

0.40

0.35

0.30

0.25

0.20

0.15

0.10

_P=0.2016

UCL=0.2999

LCL=0.1033

1

P Chart of Percentage

88

Sample

Sam

ple

Mean

24222018161412108642

102.5

102.0

101.5

101.0

100.5

100.0

99.5

99.0

__X=100.492

UCL=102.137

LCL=98.847

Xbar Chart of QC Variable

99

Sample

Cum

ula

tive S

um

24222018161412108642

2

1

0

-1

-2

0

UCL=2.194

LCL=-2.194

CUSUM Chart of QC Variable

1010

Some Types of Control ChartsSome Types of Control Charts Shewhart X-bar and R charts (Shewhart X-bar and R charts (nn>1) >1) NormalityNormality Shewhart I and MR charts (Shewhart I and MR charts (nn=1) =1) NormalityNormality ShewhartShewhart p p-charts for proportions -charts for proportions

Binomial Binomial Shewhart Shewhart cc- and - and uu-charts for counts-charts for counts

PoissonPoisson Cumulative sum (CUSUM) chartsCumulative sum (CUSUM) charts

1111

Our focus will be on the Our focus will be on the monitoring of chronic monitoring of chronic diseases, congenital diseases, congenital malformations, and mortality malformations, and mortality rates over time.rates over time.

1212

Health care quality data are oftenHealth care quality data are often

attribute (yes/no) data with 100% attribute (yes/no) data with 100% inspection.inspection.

counts or times to a “failure” with an counts or times to a “failure” with an assumed underlying Bernoulli, assumed underlying Bernoulli, geometric or exponential distribution.geometric or exponential distribution.

1313

Suppose one counts the number of Suppose one counts the number of births between successive cases of a births between successive cases of a specific type of congenital malformation.specific type of congenital malformation.

The sets method of Chen (1978, The sets method of Chen (1978, JASAJASA))signals an increase in the rate if a signals an increase in the rate if a specified number of consecutive counts specified number of consecutive counts are all less than a specified value. are all less than a specified value.

For example, signal if 5 consecutive For example, signal if 5 consecutive values are less than 1000. values are less than 1000.

1414

Risk-adjustment is often essential in Risk-adjustment is often essential in health care applications, where a health care applications, where a logistic or other model is used to logistic or other model is used to predict the probability of “failure.”predict the probability of “failure.”

…oooooooooooooooooooo…oooooooooooooooooooo

…oD8…oD899ppkkeejj589589CCv0v0223838&*&*&&%%#$#$

1515

Examples of Risk FactorsExamples of Risk Factors Down’s SyndromeDown’s Syndrome: Age of mother: Age of mother Heart SurgeryHeart Surgery: Age, gender, hypertension, : Age, gender, hypertension,

diabetic status, renal function, left ventricular diabetic status, renal function, left ventricular mass. (Parsonnet score)mass. (Parsonnet score)

Heart Surgery (Europe)Heart Surgery (Europe): Age, gender, chronic : Age, gender, chronic pulmonary disease, extracardiac arteriopathy,pulmonary disease, extracardiac arteriopathy, neurological dysfunction,neurological dysfunction, previous cardiac previous cardiac surgery,surgery, creatinine > 200 µmol/ L, active creatinine > 200 µmol/ L, active endocarditis,endocarditis, critical preoperative state.critical preoperative state. (euroSCORE)(euroSCORE)

1616

Much of the focus and work on Much of the focus and work on mortality rate monitoring for mortality rate monitoring for physicians is being done in the physicians is being done in the UK and Canada.UK and Canada.

1717

1818

1919

Shipman Inquiry July 2002:

215 definite victims,

45 probable

2020

Cumulative excess death certificates signed by Shipman: age >64 and death in home/practice

-200

20406080

100120140160180200

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

Year

Cu

mu

lati

ve e

xce

ss m

ort

ali

ty

Male

Female

(Shipman Inquiry: total of definite or probable victims: 189 female > 65, 55 male over 65)

2121

Sequential probability ratio test (SPRT) for detection of a Sequential probability ratio test (SPRT) for detection of a doubling in mortality risk: age >64 years and death in doubling in mortality risk: age >64 years and death in home/practice for Dr. Harold Shipman. (Spiegelhalter et home/practice for Dr. Harold Shipman. (Spiegelhalter et al. (2003))al. (2003))

2222

Resetting sequential probability ratio test (RSPRT) Resetting sequential probability ratio test (RSPRT) for detection of a doubling in mortality risk, age for detection of a doubling in mortality risk, age >64. (From Spiegelhalter et al. (2003).>64. (From Spiegelhalter et al. (2003).

2323

RSPRT charts have a problem with RSPRT charts have a problem with building up “credit”. building up “credit”.

An increase in the mortality rate can An increase in the mortality rate can occur when the SPRT value is below occur when the SPRT value is below zero. zero.

This phenomenon is referred to as This phenomenon is referred to as “inertia” in the industrial SPC “inertia” in the industrial SPC literature.literature.

2424

Cumulative risk adjusted mortality (CRAM) chart Cumulative risk adjusted mortality (CRAM) chart with 99% control limits for change in mortality in with 99% control limits for change in mortality in last 16 expected deaths. (From Poloniecki et al. last 16 expected deaths. (From Poloniecki et al. (1998))(1998))

2525

Example of a two-sided risk-adjusted CUSUM Example of a two-sided risk-adjusted CUSUM chart (provided by Stefan H. Steiner)chart (provided by Stefan H. Steiner)

0 500 1000 1500 2000 2500 3000 3500

0

2

4

6CU

SUM

Xt+

0 500 1000 1500 2000 2500 3000 3500-6

-4

-2

0

Number of Patients

CUSU

M X

t-

2626

The CUSUM chart is the best option.The CUSUM chart is the best option. It can be risk-adjusted.It can be risk-adjusted. It has optimality properties in detecting It has optimality properties in detecting

sustained shifts in the process.sustained shifts in the process. It has good inertial properties.It has good inertial properties. It can be designed based on It can be designed based on

meaningful performance measures meaningful performance measures such as average run length (ARL).such as average run length (ARL).

It can be used in the background with It can be used in the background with CRAM charts.CRAM charts.

2727

Control charts can be used to identify Control charts can be used to identify physicians or hospitals with unusually high physicians or hospitals with unusually high (or low) mortality rates.(or low) mortality rates.

The Society of Cardiothoracic Surgeons of The Society of Cardiothoracic Surgeons of Great Britain and Ireland interprets giving the Great Britain and Ireland interprets giving the benefit of the doubt to physicians as 9999:1 benefit of the doubt to physicians as 9999:1 odds of adverse outcomes being due to odds of adverse outcomes being due to chance alone before any alarm. chance alone before any alarm.

2828

The Centers for Disease Control and The Centers for Disease Control and Prevention use CUSUM and other Prevention use CUSUM and other control charting methods in their control charting methods in their Early Aberration Reporting System Early Aberration Reporting System (EARS). (EARS).

www.bt.cdc.gov/surveillance/ears/index.aspwww.bt.cdc.gov/surveillance/ears/index.asp

2929

Virtually all methods for the Virtually all methods for the detection of clusters of disease are detection of clusters of disease are retrospective, based on historical retrospective, based on historical spatial data.spatial data.

There are some new methods for There are some new methods for detecting clusters prospectively, i.e., detecting clusters prospectively, i.e., as they are forming.as they are forming.

3030

3131

Detection of clusters of chronic diseaseDetection of clusters of chronic disease

Aggregation of data by time and locationAggregation of data by time and location

Raubertas (1989, Raubertas (1989, Statistics in MedicineStatistics in Medicine))

Rogerson and Yamada (2004, Rogerson and Yamada (2004, Statistics in MedicineStatistics in Medicine) )

Aggregation of data by locationAggregation of data by location

Rogerson (1997, Rogerson (1997, Statistics in MedicineStatistics in Medicine)) No aggregationNo aggregation

Rogerson (2001, Rogerson (2001, JRSS-AJRSS-A))

3232

It is often useful to compare It is often useful to compare units, e.g., institutions or units, e.g., institutions or physicians, using cross-sectional physicians, using cross-sectional data.data.

3333

3434

Example of a League Table from Adab et al. (2002).Example of a League Table from Adab et al. (2002).

3535

Example of Proposed “Control Chart” by Adab et al. (2002)

3636

"Funnel plot" of emergency re-admission rates following "Funnel plot" of emergency re-admission rates following treatment for a stroke in large acute or multi-service treatment for a stroke in large acute or multi-service hospitals in England and Wales in 2000–1. Exact 95% and hospitals in England and Wales in 2000–1. Exact 95% and 99.9% binomial limits are used. (From Spiegelhalter 99.9% binomial limits are used. (From Spiegelhalter (2002))(2002))

3737

Harold ShipmanHarold Shipman Killed his patients using morphine overdoses.Killed his patients using morphine overdoses. Was caught after carelessly revising a patient’s Was caught after carelessly revising a patient’s

will, leaving all her assets to himself.will, leaving all her assets to himself. His office typewriter was used to type the His office typewriter was used to type the

revised will.revised will. His computer records were doctored to show His computer records were doctored to show

his patients had needed morphine just after the his patients had needed morphine just after the patients had been killed. The computer patients had been killed. The computer software, however, recorded the dates of these software, however, recorded the dates of these modifications.modifications.

He hung himself in prison, never confessing to He hung himself in prison, never confessing to his crimes. his crimes.

3838Baker, R. et al. British Medical Journal 2003;326: pp. 274-276

3939

Highly recommended reference:Highly recommended reference:

Michael L. Millenson (1999). Michael L. Millenson (1999). Demanding Demanding Medical Excellence: Doctors and Medical Excellence: Doctors and Accountability in the Information AgeAccountability in the Information Age, , The University of Chicago Press.The University of Chicago Press.

4040

Recommended ReferencesRecommended References Sonesson, C. and Bock, D. (2003). “A Review and Sonesson, C. and Bock, D. (2003). “A Review and

Discussion of Prospective Statistical Surveillance in Public Discussion of Prospective Statistical Surveillance in Public Health”. Health”. Journal of the Royal Statistical SocietyJournal of the Royal Statistical Society A 166, pp. A 166, pp. 5-21.5-21.

Grigg, O. A.; Farewell, V. T.; and Spiegelhalter, D. J. (2003). Grigg, O. A.; Farewell, V. T.; and Spiegelhalter, D. J. (2003). “Use of Risk-adjusted CUSUM and RSPRT Charts for “Use of Risk-adjusted CUSUM and RSPRT Charts for Monitoring in Medical Contexts”. Monitoring in Medical Contexts”. Statistical Methods in Statistical Methods in Medical ResearchMedical Research 12, pp. 147-170. 12, pp. 147-170.

Grigg, O. and Farewell, V. (2004a). “An Overview of Risk-Grigg, O. and Farewell, V. (2004a). “An Overview of Risk-Adjusted Charts”. Adjusted Charts”. Journal of the Royal Statistical SocietyJournal of the Royal Statistical Society A A 167, pp. 523-539.167, pp. 523-539.

Steiner, S. H.; Cook, R. J.; Farewell, V. T.; and Treasure, T. Steiner, S. H.; Cook, R. J.; Farewell, V. T.; and Treasure, T. (2000). “Monitoring Surgical Performance Using Risk-(2000). “Monitoring Surgical Performance Using Risk-Adjusted Cumulative Sum Charts”. Adjusted Cumulative Sum Charts”. BiostatisticsBiostatistics 1, pp. 441- 1, pp. 441-452.452.

4141

My paper is available at My paper is available at

http://filebox.vt.edu/users/bwoodall/http://filebox.vt.edu/users/bwoodall/

4242

ConclusionsConclusions There are many important applications of There are many important applications of

control charts in health care.control charts in health care. Improvement of health care is a life-or-death Improvement of health care is a life-or-death

matter.matter. There are many interesting SPC research There are many interesting SPC research

opportunities in public health surveillance.opportunities in public health surveillance. There needs to be a greater transfer of There needs to be a greater transfer of

knowledge between the medical and knowledge between the medical and industrial application areas. industrial application areas.