m20: beyond statistical process control: advanced...

11/29/2012

1

API and CT Concepts, 2012

M20: Beyond Statistical Process

Control: Advanced Charts for Healthcare

Lloyd P. Provost, Associates in Process Improvement

Sandra Murray, CT Concepts

1

These presenters have nothing to disclose


2

L17: Beyond Statistical Process Control: Advanced Charts for Healthcare

This session will explore some of the more advanced uses for statistical process control (SPC) charts for health care data. Some issues to be discussed include how to tell if anything has improved when “yucky” events are already rare; why some control charts have such narrow limits with all of the data outside the limits; and how to factor in seasonal impacts on data. Our journey will include the use of T and G charts for rare events data, Prime charts for dealing with over dispersion in data, adjustments for autocorrelation, using changing center lines on control charts, and the use of the CUSUM control chart.

Session Objectives:

• Select the appropriate SPC chart for rare events data

• Identify when it is most appropriate to use a CUSUM control chart

• Describe when to use a prime chart for over‐dispersion in data

• Identify autocorrelation and describe how to deal with it using a Shewhart control chart

11/29/2012

2


SPC Tools to Learn from Variation in Data for Improvement

Frequency Plot Pareto Chart Scatter Plot

3


SHEWHART CHART:What Is It?

• The Shewhart chart is a statistical tool used to distinguish between variation in a measure due to common causes and variation due to special causes. • Commonly called “control chart.” • A more descriptive name might be “learning charts” or “system performance charts”

• Format:•Data is usually displayed over time•Data usually displayed in time order•Shewhart chart will include:

•Center line (usually mean)•Data points•Statistically calculated upper and lower 3-sigma limits

4

11/29/2012

3


Parts of a Shewhart Chart

X Axis: Sequence of data‐leave blank identifiers

Y Axis: Scale for the data.

Leave white space!

Data Points (each dot is “subgroup”)3 sigma limits

3 sigma limits

5


Introduction to Shewhart Chart• Statistical tool used to distinguish special from common cause variation

6

11/29/2012

4



Why 3 Sigma Limits?

8

11/29/2012

5


What do we use a Shewhart chart for?

• Learn how much variation exists in process

• Assess stability and determine improvement strategy (common or special cause strategy)

• Monitor performance and correct as needed

• Find and evaluate causes of variation

• Tell if our changes yielded improvements

• See if improvements are “sticking”

9


Shewhart’s Theory of Variation

• Common Cause: causes that are inherent in the process, over time affect everyone working in the process, and affect all outcomes of the process– Process stable, predictable– Action: if in need of improvement must redesign

• Special cause: causes that are not part of the process all the time, or do not affect everyone, but arise because of special circumstances– Process unstable, not predictable– Action: go investigate special cause and take appropriate action

– May be evidence of improvement (change we tested working)

10

11/29/2012

6



Note: Ties between two consecutive points do notcancel or add to a trend.

Note: A point exactly on the centerline does not cancel or count towards a shift

Note: A point exactly on a control limit is not considered outside the limit

When there is not a lower or upper control limit

Rule 1 does not apply to the side missing limit

When there is not a lower or upper control limit

Rule 4 does not apply to the side missing limit

Rules or detecting a special cause

The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.

12

11/29/2012

7



Good

Change A v1/v2v3/v4/v5

v 6/7 Impl.

14

11/29/2012

8


Is LOS for DRG XXX Improving

Sequential Cases

LO

S i

n D

ays

4 6 7 5 4 6 4 8 3 6 7 5 6 7 8 7 7 8 6 8 9 6 7 8 6Data

Individuals

Good

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250

2

4

6

8

10

12

14

UL

Mean

LL Change 1Ch 2

Ch 3 Ch 5Ch 4

15


Good

ID Dr.

Whiteboards

Hand off huddles

16

11/29/2012

9


Type of Data

Count or Classification (Attribute Data)

Qualitative data in terms of an integer (# errors, # nonconformities or # of items that passed or failed) Discrete: must be whole number when originally collected (can’t be fraction or scaled data when originally collected)This data is counted, not measured

Count (nonconformities)1,2,3,4, etc.

Classification (nonconforming)either/or, pass/fail, yes/no

Equal area ofopportunity

Unequal area of opportunity

Unequal or equal subgroup size

Continuous (Variable Data) Quantitative data in the form of a measurementTime, money, scaled data i.e. length, height, weight, temperature, mg. and throughput (volume ofworkload/ productivity)

Subgroup size of 1 (n=1)

Unequal or equal subgroup size (n>1)

C Chart U Chart P ChartI Chart (also known as X Chart) X Bar and S

Number of nonconformities

Nonconformities per unit

Percent nonconforming

Individual measurement Average and standard deviation

Shewhart Chart Selection Guide

Each subgroup is composed of a single data value

Each subgroup has more than one data value

-

17


Continuous Data: Characteristics• Quantitative data ‐uses some type of measurement scale

• Don’t need to be a whole number when collected (may include decimal places)

• Typically are biological measures, time, money, physical measures, perception data recorded on a scale (for example, Likert scale) or throughput (workload, productivity).

• NOTE: In health care, workload or productivity often use a numerical scale (e.g. number of clinic visits). Although the numerical scale for workload data and Likert scales for perception data usually yield discrete data (data that are whole numbers when collected), they are still best treated as continuous data.

• Basic Charts for continuous data: Individuals or X bar and S18

11/29/2012

10


Attribute Data: Characteristics• Qualitative data such as number of errors, occurrences, or # of

items that passed or failed

• Must be a whole number when collected (discrete data)

• Two types:

– Classification data: conforming units/nonconforming units, go/no‐go decision, on‐time/late appointment, in compliance/not in compliance.

• Tips: numerator can never be larger than the denominator, it is possible to count both items that passed and that failed.

– Count data: we do not focus on a unit. Instead we count incidents that are unusual or undesirable, such as the number of mistakes, medication errors, complications, infections, patient complaints, or accidents.

• Tips: the number can be larger than the denominator, it is possible to count the number of errors, but not the non errors.

• Basic Charts for attribute data, P, C, U 19


11/29/2012

11


Is LOS for DRG XXX Improving

Sequential Cases

LO

S i

n D

ays

4 6 7 5 4 6 4 8 3 6 7 5 6 7 8 7 7 8 6 8 9 6 7 8 6Data

Individuals

Good

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250

2

4

6

8

10

12

14

UL

Mean

LL Change 1Ch 2

Ch 3 Ch 5Ch 4

21


11/29/2012

12


The X bar S Chart

• For measurement (variables) data

• A pair of charts

• Subgroup size can be equal or unequal

• Each dot on an X bar S chart is the average of multiple pieces of data

– The X bar chart graphs the average from subgroup to subgroup

– The S chart graphs the variation around that average (the standard deviation within the subgroup)

23


11/29/2012

13



11/29/2012

14



Shewhart Charts for Attribute DataAttribute Data

• Qualitative data such as number of errors, occurrences, or # of items that passed or failed

• Must be a whole number when collected (discrete data, cannot be a fraction when originally collected)

• Is counted, not measured

Two types: Classification and Count• Classification (binomial distribution)• Count (Poisson distribution)• To select correct Shewhart chart must identify

whether data are classification or count

28

11/29/2012

15



P Chart• For classification data (yes/no, pass/fail, either/or)

• Tips:

– Numerator may never be greater than denominator

– Can count both those that pass and those that failed, both the either and the or

• Subgroup size may be equal or unequal

– If equal limits will be straight line

– If unequal limit will vary

• Plots the percent (P chart) conforming or non conforming

30

11/29/2012

16



11/29/2012

17



11/29/2012

18



Funnel Limits

36

11/29/2012

19



11/29/2012

20



C and U Charts

• Shewhart charts for count data

• Based on Poisson distribution

• We count each defect, error, nonconformity in the area of opportunity

• Tips:

– Can count each defect or nonconformity but can not count the non‐defects (e.g. can count each error in a medical record but not the # of non‐errors)

– Numerator may be larger than denominator (may have more errors than medical records)

40

11/29/2012

21


Areas of Opportunity

41


Assumptions Associated with C Charts

•Formula:

42

11/29/2012

22



11/29/2012

23


Assumptions Associated with U Chart

45


A C Chart or a U Chart??

46

11/29/2012

24



11/29/2012

25


Working with Rates per 1000, 10,000, etc.•Rate per 1000 or other base useful when comparing to others•Also when rate calculation would result in such a small number is hard for people to grasp •When attribute rate data always select U chart

49


11/29/2012

26


Clinic Complaint Data

51


Clinic Complaint U Chart

52

11/29/2012

27


Clinic Complaint U Chart with Funnel Limits

53


T and G Charts for Rare Events

• The problem and examples

• Intro to template

• Work Case Study – (10)

• Debrief ‐ Tips, issues, references

54

11/29/2012

28


The Problem: How do we tell if we are improving when

undesirable events are rare anyway?

Month# Med

Errors# Doses #Doses/1000 Notes

N 06 3 24222 24.222

D 1 23616 23.616

J 07 4 23072 23.072

F 2 19439 19.439

M 2 24568 24.568

A 3 21020 21.02 Chg 1 test and Imp, Ch 2 test

M 2 28754 28.754 Chg 2 Test

J 2 23390 23.39 Chg 2 Imp

J 1 28475 28.475 Chg 3 Lg Test

A 2 22079 22.079 Chg 3 Imp

S 0 29206 29.206

O 1 23390 23.39

N 0 28475 28.475

D 1 23390 23.39

AverageBefore=2.4

Average After=1.3

55


The Problem: when undesirable events are rare even a standard run or Shewhart control chart may not be very helpful

0.0 56

11/29/2012

29


How Do You Know You Need to View Data Differently?

• Have too many zeros (>25% of data points 0)

• Have no lower limit

Percent C‐Sections

Sequential Months

Percent

p‐chart (%) Mean=15.84

Mo

95/1

95/2

95/3

95/4

95/5

95/6

95/7

95/8

95/9

95/10

95/11

95/12

96/1

96/2

96/3

96/4

96/5

96/6

96/7

96/8

96/9

96/10

96/11

96/12

97/1

97/2

97/3

97/4

97/5

97/6

97/7

97/8

97/9

97/10

97/11

97/12

98/1

98/2

98/3

98/4

98/5

98/6

98/7

98/8

98/9

98/10

98/11

98/12

99/1

99/2

99/3

99/4

99/5

99/6

99/7

99/8

99/9

99/10

99/11

99/12

00/1

00/2

00/3

00/4

00/5

00/6

00/7

00/8

00/9

0

10

20

30

40

50

60

70

80

90

100

UCL

Mean

57


The Problem: when undesirable events are rare even a standard run or Shewhart control chart may not be very helpful

0.0 58

11/29/2012

30


What is a Rare Event?

• Rare events do not always occur when the process is observed, so a value of 0 for the number of incidences of the event is often recorded.

• So the concept or “rare” depend on how often we want to observe our process (hourly, daily, weekly, monthly, etc.)

59


When Will Rare Events Happen?

Classification Data: Guidelines for Selecting Subgroup Size for an Effective P chart

Average PercentNonconforming

Units (pbar)

Minimum Subgroup Size (n) Required to Have

< 25% zero for p's

Minimum Subgroup Size Guideline

(n>300/pbar)

Minimum Subgroup Size Required to Have

LCL > 0

0.1 1400 3000 9000

0.5 280 600 18001.0 140 300 9001.5 93 200 6002 70 150 4503 47 100 3004 35 75 2205 28 60 1756 24 50 1428 17 38 104

10 14 30 8112 12 25 6615 9 20 5120 7 15 3625 5 12 2830 4 10 2240 3 8 1450 2 6 10

Note: for p>50, use 100‐p to enter the table (e.g. for p=70% use table p of 30%, for p=99% use table p of 1%, etc.) Source: The Health Care Data Guide: L Provost and S. Murray, Jossey‐Bass, 2011 60

11/29/2012

31


When Will This Happen?For Count Data (C or U chart)

• If C chart center line less than 9 will be no lower limit

• If C chart center line less than 1.4 will have too many 0’s

• For U chart, divide 9 (or 1.4) by the CL to get the minimum number of “standard areas of opportunity”)

• What Can We Do?

– Increase opportunity (time, cases, etc.) to get larger subgroup size

61


One Way to Transform Data: Limits: Appropriate Vs. Too Small Subgroup SizeAggregate Level: Total C‐Section Rate By Quarter

Sequential Quarters

95/1

95/2

95/3

95/4

96/1

96/2

96/3

96/4

97/1

97/2

97/3

97/4

98/1

98/2

98/3

98/4

99/1

99/2

99/3

99/4

00/1

00/2

0

10

20

60

70

80

90

100

Mean

%

30

40

50

UCL

p‐chart (%)

C‐Section Rate

%

Sequential Months

Mo

95/1

95/2

95/3

95/4

95/5

95/6

95/7

95/8

95/9

95/10

95/11

95/12

96/1

96/2

96/3

96/4

96/5

96/6

96/7

96/8

96/9

96/10

96/11

96/12

97/1

97/2

97/3

97/4

97/5

97/6

97/7

97/8

97/9

97/10

97/11

97/12

98/1

98/2

98/3

98/4

98/5

98/6

98/7

98/8

98/9

98/10

98/11

98/12

99/1

99/2

99/3

99/4

99/5

99/6

99/7

99/8

99/9

99/10

99/11

99/12

00/1

00/2

00/3

00/4

00/5

00/6

00/7

00/8

00/9

0

10

20

30

40

50

60

70

80

90

100

UCL

Mean

p‐chart (%)

%

% C‐Sections By Month % C‐Sections Quarter

62

11/29/2012

32


Alternative: Plot Time or Count Between Occurrences of Rare Events

Instead of plotting the number of incidences each month, plot the time (or number of cases, patients, visits, etc) between incidences.

Plot a point each time an incidence occurs

63


Use Special Shewhart Charts for Rare EventsType of Data

Count or Classification (Attribute Data)‐Qualitative data such as # errors, # nonconformities or # of items that passed or failed) ‐ Discrete: must be whole number when originally collected (can’t be fraction or scaled data when originally collected)‐This data is counted, not measured

Count (Nonconformities)1,2,3,4, etc.

Classification (Nonconforming)Either/Or, Pass/Fail, Yes/No

Equal Area of Opportunity

Unequal Area of Opportunity

Unequal or Equal Subgroup Size

Continuous (Variable Data) ‐Quantitative data in the form of a measurement‐Requires a measurement scale ‐Time, Money, Scaled Data (i.e. length, height, weight, temperature, mg.) and Throughput (volume of workload/ productivity)

SubgroupSize of 1(n=1)

Unequal Or Equal SubgroupSize (n>1)

C Chart U Chart P ChartI Chart (also known as an X chart)

X‐Bar and S

Number ofNonconformities

Nonconformities

Per Unit PercentNonconforming

Individual Measures Average and StandardDeviation

Other types of control charts for attribute data:1. NP (for classification data)

2. T-chart [time between rare events]3. Cumulative sum (CUSUM)4. P’, C’, and U’

5 G chart (number of opportunities between rare events) 6. Standardized control chart

Other types of control charts for continuous data:7. X‐bar and Range8. Moving average9. Median and range10. Cumulative sum (CUSUM)11. Exponentially weighted moving average (EWMA)12. Standardized control chart

64

11/29/2012

33


g‐Chart for Rare Occurrences

• Alternative to p‐chart, c‐chart, or u‐chart for count or classification data when the measure is the number of opportunities between the incident or non‐conformity of interest.

• The number of opportunities (surgeries, insertions, admissions, etc.) between an incidence can be modeled by the geometric distribution.

• This chart allows the evaluation of each occurrence to be evaluated rather than having to wait to the end of a time period before the data is plotted. – The g chart is also particularly useful for verifying improvements (such as reduced SSIs) and for processes with low rates.

65


g‐Chart for Rare OccurrencesThe number of opportunities (surgeries, insertions, Shape of Geometric Distribution

admissions, etc) between an incidence can be modeled by the geometric distribution.

Calculation of control limits:

g = number opportunities between incidencesg‐bar= average of g’sCL = 0.693* g‐bar (theoretical median of g’s ) UL = UL = gbar + 3 * square root [gbar * (gbar +1)]LL = there is no lower control limit

Notes:

1. The UL is approximately 4 times gbar (or 5.7 times the CL) for quick visual analysis.

2. Since the count data on a g chart are usually highly skewed, the plotted data will not be symmetric around the average (gbar). The theoretical median = 0.693 * mean for the geometric distribution should be used for the center line when it is desirable to apply the shift rule (8 consecutive points above or below center line).

g

66

11/29/2012

34


G –Chart Example

67


ComparisonOf G chart and U chart for infections in the ICU

68

11/29/2012

35


T‐Chart for Time Between Rare Occurrences

Alternative to p chart, c chart or u chart for count or classification data when the measure is the time (continuous variable) between the incident or non‐conformity of interest.

This chart allows the evaluation of each non‐conformity or non‐confirming unit to be evaluated rather than having to wait to the end of a time period before the data is plotted.

The only information required is the time that the rare event occurs .

Time can recorded in years, months, weeks, days, hours, minutes, seconds. Pick a time unit so that you cannot have two rare events in the same time period.

Source: The Data Guide69


T‐Chart for Time Between Rare Occurrences

The times between occurrences can be modeled using the exponential distribution. Since the exponential distribution is highly skewed, the times are first transformed to a symmetric Weibull distribution by raising the time measure to the 1/3.6 = 0.2777 power or

Shape of Exponential Distribution

[y = t 0.2777].

Calculation of control limits:t = time between incidences, y = transformed timeMR’ = average moving range of y’sŷ = average of y’s (center line)

UL = ŷ + 2.66 * MR’ LL = ŷ ‐ 2.66 * MR’

Transform the limits back to time scale before plotting chart: t = y3.6 with the original times

Notes: 1. Conduct the usual screening of MR’s to calculate the average moving range.

2. Cannot have 0’s in the data (increase precision (hrs vs. days) for recording data if expect 0.3. If calculated LL < 0, there is no LL for this measure.4. Since the I chart is the basis, desirable to have > 20 events to calculate limits.

Source: The Data Guide

t

70

11/29/2012

36


What Does a T Chart Look Like?

71

Example of T chart showing improvement in the incidents of retained foreign objects during procedures


How Should a T Chart be Presented?

72Page 232

I chart on Transformed time data

T chart on time scale

T chart on log time scale

11/29/2012

37


T Chart for FallsChris McCarthy: Time Between [email protected]

Sample

Sam

p le

Coun

t

2421181512963

5

4

3

2

1

0

_C=1.385

UCL=4.915

C C ha r t o f # of F a lls pe r M onth

73


T chart

U chart

74

11/29/2012

38


T Chart with Recalculated Limits

0

10

20

30

40

50

60

70

80

90

1001/

7/06

2/6/

063/

1/06

4/5/

065/

1/06

6/2/

067/

5/06

8/23

/06

9/18

/06

10/4

/06

10/3

0/0

612

/15

/06

1/14

/07

2/13

/07

4/20

/07

5/25

/07

6/2/

076/

9/07

7/6/

078/

1/07

8/18

/07

9/3/

079/

12/0

79/

26/0

710

/4/0

710

/14

/07

11/1

/07

11/2

0/0

712

/27

/07

1/14

/08

1/22

/08

2/21

/08

3/7/

083/

18/0

84/

4/08

5/11

/08

6/7/

086/

15/0

87/

1/08

8/24

/08

9/19

/08

10/2

6/0

812

/2/0

81/

9/09

2/18

/09

4/16

/09

7/5/

09

Day

s B

etw

een

Dea

th

Date of Maternal Death

Days Between Maternal Deaths

Improvement Team Formed

Average time between deaths has doubled (from 10.3 to 22.4 days)

75


Event occurs ~every:

# Weeks before mature t chart

# Months before mature t chart

# Quarters before mature t chart

1 day 3 0.75 0.25

1 week 20 5 1.7

1 month 80 20 6.7

3 months 240 60 20

6 months 480 120 40

Subgroups considerations with t charts

•You still need data to establish a baseline (Voice of the Process)!

•Need ~ 20 data points for “mature” control chart (Shewhart, 1931)

•Table below shows approx. time needed to get mature t chart (can still learn from t chart with < 20 data points)

Rare is relative!

If it is important to monitor a process every hour, then events that occur only a few times a day will be rare!

Source: Rocco Perla, PhD.

76

11/29/2012

39


Debrief• Both t chart and g chart can use the five rules for detecting special cause

– Both charts use a theoretical median for the center line to get about ½ of data above and below the CL

• Resources:• The Health Care Data Guide: Learning from Data for Improvement. Provost and

Murray, Jossey Bass, 2011. Chapter 7.

• Jackson, J. E. , “All Count Distributions are not Alike”, Journal of Quality Technology, Vol 4 (2), pp 86‐92, 1972.

• Yang, S., et al, “On the Performance of Geometric Charts with Estimated Control Limits” Journal of Quality Technology, Vol 34, No.4, pp 448‐458, October, 2002.

• Wall, R., et al, “Using real time process measurements to reduce catheter related bloodstream infections in the ICU, Qual. Saf. Health Care 2005;14;pp. 295‐302.

• Nelson, L., “A Control Chart for Parts‐Per‐Million Nonconforming Items”, Journal of Quality Technology, Vol 26, No. 3, pp. 239‐240, July, 1994.

77


Cumulative Sum Control Charts

• Problem

• Examples, Intro to template



78

11/29/2012

40


Some Advanced Control Charts

• Shewhart control charts plot information from only the last subgroup.

• The sensitivity of the chart can be improved by incorporating previous data in each plotted point.

• The moving average and moving range are two examples of using previous data points.

• Other effective alternatives to the Shewhart control charts are the cumulative sum (CUSUM) control chart and the exponentially weighted moving average(EWMA) control chart.

• Especially useful when it is important to detect small, persistent shifts in the measure of interest.

79


Warning: With these Advanced Control Charts, Can Not Use

Standard Rules 2‐5 for Determining a Special Cause

SPC-

OK

80

11/29/2012

41


EXAMPLE: Data Used to Calculate Plotted Point(after 5 data points)

x1 x2 x3 x4 x5 x6 x7 x8 x9 x10

Run Chart or Control Chart

0 0 0 0 1.0 0 0 0 0 0

Cusum .2 .2 .2 .2 .2 0 0 0 0 0

Moving Average (3)

0 0 .33 .33 .33 0 0 0 0 0

Moving Average (5)

.2 .2 .2 .2 .2 0 0 0 0 0

EWMA λ = 0.2 .08 .10 .13 .16 .2 0 0 0 0 0

EWMA λ = 0.3 .07 .10 .15 .21 .3 0 0 0 0 0


Data Used to Calculate Plotted Point (After 8 Data Points)

x1 x2 x3 x4 x5 x6 x7 X8 x9 x10

Run Chart or Control Chart

0 0 0 0 0 0 0 1 0 0

Cusum .125 .125 .125 .125 .125 .125 .125 .125 0 0

Moving Average (3)

0 0 0 0 0 .33 .33 .33 0 0

Moving Average (5)

0 0 .0 .2 .2 .2 .2 .2 0 0

EWMA λ = 0.2 .04 .05 .065 .08 .10 .13 .16 .2 0 0

EWMA λ = 0.1 .02 .03 .05 .07 .10 .15 .21 .3 0 0

11/29/2012

42



CUSUM

The cumulative sum statistic (S) is the sum of the deviations of the individual measurements from a target value, for example:

Si = S i‐1 + (Xi ‐ T),

Xi = the ith observation,

T = Target (often from historical average)

Si = the ith cumulative statistic.

84

11/29/2012

43


CUSUM Calculation – Patient Satisfaction Data

Month %Sat(X) Target (Average) X-Target (Si) CUSUM CUSUM + Target

J-02 82 88.296 -6.296 -6.296 82

F 79 88.296 -9.296 -15.592 72.704

M 84 88.296 -4.296 -19.888 68.408

A 82 88.296 -6.296 -26.184 62.112

M 92 88.296 3.704 -22.48 65.816

J 80 88.296 -8.296 -30.776 57.52

J 94 88.296 5.704 -25.072 63.224

A 78 88.296 -10.296 -35.368 52.928

S 83 88.296 -5.296 -40.664 47.632

O 84 88.296 -4.296 -44.96 43.336

N 92 88.296 3.704 -41.256 47.04

D-02 84 88.296 -4.296 -45.552 42.744

03, 04

M 04 95 88.296 6.204 0.008 88.304


Run Chart of Patient

Satisfaction DataProcess Changes

Cusum Chart of Patient Satisfaction Data Target = Avg = 88.296

CUSUM Chart

86

11/29/2012

44


Target = average = 88.29CUSUM graph

sensitive to target selected

The Slope is the Focus of the Interpretation

Target = 85%

Target = goal = 85%

87


Control Limits for CUSUM Chart – V‐Mask

88

11/29/2012

45


Moving the V‐Mask (h=5, k=0.5)


Alternate Form of CUSUM Control Chart

Called Tabular Form

• Do not need to use V‐mask

• Need to plot two statistics for each measure

• Procedure:– Let xi be the ith observation on the process

– Estimate using screened MR’s of series

– Accumulate derivations from the target 0 above the target with one statistic, C+

– Accumulate derivations from the target 0 below the target with another statistic, C—

– C+ and C‐‐ are one‐sided upper and lower cusums, respectively.

90

11/29/2012

46


The Tabular CUSUM ‐ Alternate Form

The statistics are computed as follows:The Tabular Cusum

starting values are

K is the reference value (or allowance or slack value)

If either statistic exceed a decision interval H, the process is considered to be out of control. Often taken as a H = 5

91


Cusum Control Chart for Patient Satisfaction Data

Ci+

Ci-

92

11/29/2012

47


‐40.0

‐30.0

‐20.0

‐10.0

0.0

10.0

20.0

30.0

40.0

Jan‐08

Feb‐08

Mar‐08

Apr‐08

May‐08

Jun‐08

Jul‐08

Aug‐08

Sep‐08

Oct‐08

Nov‐08

Dec‐08

Jan‐09

Feb‐09

Mar‐09

Apr‐09

May‐09

Jun‐09

Jul‐09

Aug‐09

Sep‐09

Oct‐09

Nov‐09

Dec‐09

Jan‐10

Feb‐10

Mar‐10

Apr‐10

May‐10

Jun‐10

Jul‐10

Aug‐10

Sep‐10

Oct‐10

Cusum statistic

Cusum Chart of Average Wait Time

Cplus CL UL

LL Cminus

0

10

20

30

40

50

60

1/1/

08

2/1/

08

3/1/

08

4/1/

08

5/1/

08

6/1/

08

7/1/

08

8/1/

08

9/1/

08

10/1

/08

11/1

/08

12/1

/08

1/1/

09

2/1/

09

3/1/

09

4/1/

09

5/1/

09

6/1/

09

7/1/

09

8/1/

09

9/1/

09

10/1

/09

11/1

/09

12/1

/09

1/1/

10

2/1/

10

3/1/

10

4/1/

10

5/1/

10

6/1/

10

7/1/

10

8/1/

10

9/1/

10

10/1

/10

Ave

rag

e D

ays

I Chart Average Wait Time (days)

93


Debrief ‐ Cusum

Resources:• The Health Care Data Guide: Learning from Data for Improvement. Provost and Murray,

Jossey Bass, 2011. Chapter 7.

• Roberts, S. W., “A Comparison of Some Control Chart Procedures”, Technometrics, Vol 8, No. 3, p. 411‐430, August, 1966.

• Lucas, J., “The Design and Use of V‐mask Control Schemes,” Journal of Quality Technology, Vol 8, No. 1, pp 1‐12, January, 1976.

• Banard, G. A. “Control Charts and Stochastic Processes”, Journal of the Royal Statistical Society B21, pp 230‐271, 1959

• Evans, W. D. , “When and How to Use Cu‐Sum Charts”, Technometrics, Vol. 5, pp. 1‐22, 1963

• Sibanda, T. and Sibanda N., “The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data”, BMC Medical Research Methodology 2007, 7:46 doi:10.1186/1471‐2288‐7‐46

• Noyez, Luc, “A review of methods for monitoring performance in healthcare Control charts, Cusum techniques and funnel plots”, Interact Cardiovascular Thoracic Surg 2009; 9:494‐499;

• Biau, D., “Quality control of surgical and interventional procedures: a review of the CUSUM”, Qual Saf Health Care 2007;16:203–207. 2006

94

11/29/2012

48


Over‐dispersion and Prime Charts

• The problem and examples

• Intro to template



95


Problem of Over dispersion with a P chart

“Are all these points outside the limits due to special causes, or is something else going on here?”

Month J‐07 F‐07 M‐07 A‐07 M‐07 J‐07 J‐07 A‐07 S‐07 O‐07 N‐07 D‐07 J‐08 F‐08 M‐08 A‐08Members 8755 9800

17000

16400

19500

19800

21200

22300

21600

20500

18700

18900

14300

14800

14500

14600

Mng by Phone 3852 4100 7083 7339 9406 9310 7250

10400 9250 9950 9846 9854 8034 8162 8122 8200

percent 44.0 41.8 41.7 44.8 48.2 47.0 34.2 46.6 42.8 48.5 52.7 52.1 56.2 55.1 56.0 56.2

96

11/29/2012

49


Over dispersion: Prime Charts

• Sometimes Shewhart charts look weird!

• This can happen when subgroup sizes large

– Limits on charts for attribute data impacted by subgroup size

– Larger subgroup size means tighter limits

– May be issue when subgroup > 5000

• We have an alternative: Prime charts

P’ or U’

97


Alternative to the P chart: the P’‐Chart

98

11/29/2012

50


U’ Chart for Medications Errors

Data obtained from a screening of computer order entry of prescriptions in the hospitals for one month.

Subgroup sizes (number of prescription entries for the month) ranges from 4,467 to 27,203.

Hospitals on the u chart from the smallest to largest denominator (e.g. funnel plot format).

The points for one-half of the hospitals were outside the limits.

Based on conversation with the subject matter experts (and the very large subgroup sizes), the quality analyst created the u’.

*Hospitals rationally ordered from smallest to largest

*

*99


Guidance for Attribute Charts• First develop the appropriate chart (p or u).

• If the limits “appear too tight” and very large subgroup sizes are involved:

1. Look for ways to further stratify the data• monthly into weekly or daily subgroups• organization data into department subgroups• Overall clinic data subgrouped by clinician

2. If you still end up with large subgroup sizes and a chart that is full of special causes, spend time with the subject matter expert trying to identify and understand the special causes.

3. If you are not able to learn from the special causes, then constructed a modified attribute chart (P’ or U’ ).

100

11/29/2012

51


Guidance for Attribute ChartsStep 1 – calculate the p chart (getting pi and σpi for each subgroup).

Step 2 – convert the individual p values to z-values using zi = [ pi – p-bar] / σpi.

Then use the I chart calculation of moving ranges to determine the sigma for Z-values: σzi = screened MRbar divided by 1.128.(note: as with any I chart, it is very important to screen the moving ranges for special caused prior to calculating the average moving range).

Step 3: Transpose the z-chart calculations back to p values to get the limits for the p' chart by multiplying the theoretical sampling sigma (σpi) by the between subgroup sigma (σzi ) as follows:

CL = pbar (same as original p chart)UCL = pbar + 3 σpi σziLCL = pbar - 3 σpi σzi 101


Let’s Practice

102

11/29/2012

52


P’: Percent ANC

103


Debrief: Cautions in using the Prime modification to p and u charts

1. Don’t consider the adjustment unless the subgroup sizes are very large.

2. In cases where average subgroup sizes are very large (>5000), first try using different subgrouping strategies to the stratify the data into smaller rational subgroups.

3. Spend time with subject matter experts trying to understand and learn from the initial indications of special causes on the attribute chart before considering the modification to these charts.

104

11/29/2012

53


Improper use of P’ Chart

Very important to not just automatically switch to a these modified charts until the source of the over-dispersion has been thoroughly investigated.

Only when subgroup sizes are above 2000 should the adjustment be even considered.

The purpose of Shewhart’s method is to optimize learning, not get rid of special causes.

See problem Example. Subgroup sizes range from 180 to 1845.

P'‐chart: Percent of Diabetic Patients with Self‐Management Goals

% of patients with goals

0

50

100

I B H L K C N M E A J D F G

CL

Clinic

*Clinics rationally ordered from smallest to largest

*

*

*105


References for Over dispersion with Attribute charts

1. Provost and Murray, Jossey-Bass ,The Health Care Data Guide: Learning from Data for Improvement, 2011, Chapter 8.

2. Heimann, P.A., “Attributes Control Charts with Large Sample Sizes”, Journal of Quality Technology, ASQ, 1996, Vol 28, pp 451-459.

3. Spiegelhalter D. “Handling overdispersion of performance indicators”. Journal of Quality and Safety in HealthCare, BMJ, 2005;14:347–51.

4. Laney DB. “Improved Control Charts for Attribute Data”. Quality Engineering, 2002; Vol. 14, p. 531–7.

5. Mohammed, M. A. and D Laney,” Over dispersion in health care performance data: Laney’s approach”, Journal of Quality and Safety in Health Care, 2006; Vol. 15, pp.383–384.

Debrief

106

11/29/2012

54


107

Autocorrelation

• Auto correlated

– When using registry data

• Problem

• Examples, Intro to template




Autocorrelation

A basic assumption in determining the limits for Shewhart charts is that the data for each subgroup are independent, that is the data from one subgroup does not help us predict another period.

The most common way this assumption is violated is when special causes are present.

Then subgroups associated with the special causes tend to be more alike than subgroups affected only by special causes.

The result is that these subgroups show up as “special”, exactly what the chart was designed to do.

108

11/29/2012

55


AutocorrelationBut sometimes process operations or data collection procedures result in data affected only by common causes that is not independent from subgroup to subgroup.

This phenomenon is called autocorrelation.

With a positive autocorrelation, successive data points will be similar. For time ordered data, subgroups close together will tend to be more alike then subgroups far apart in time.

With negative autocorrelation, successive points will tend to be dissimilar, resulting in a saw tooth pattern.

109


Effect of Autocorrelation

For all Charts:

This relationship between the plotted points would make all the additional rules used with all Shewhart charts invalid.

For Continuous Data Charts:

• For the I chart and Xbar and S chart the limits will not be accurate expressions of the common cause variation

• the autocorrelation will result in an increase in false signals of special causes.

For Attribute Charts: calculated limits are ok

110

11/29/2012

56


Autocorrelation with Registry Data

• The “autocorrelation problem” occurs because the data for most patients is not updated each month; only the patients who come in for a visit.

• If one‐third of the patients come in during the current month and have their data updated, the monthly summary will use the same data as the previous month for two‐thirds of the patients.

• This creates a statistical relationship between the monthly measures – it creates autocorrelation.

111


Autocorrelation in Registry Data

• I chart for the average glycated hemoglobin test (HbA1c value) from a registry of about 130 adult patients with diabetes. Patients are scheduled to visit the clinic every three months, so about one-third visit each month and their registry values are updated.

• Special causes: 10 points outside limits, runs below the center line, and numerous points near the limits.

• Are these special causes or the impact of autocorrelation due to the use of the registry values? Because of the way the data are collected for this chart, autocorrelation was expected.

112

11/29/2012

57


Examine Autocorrelation using a Scatter Diagram – Point i vs. Point i‐1

113


Dealing with Autocorrelation

• Identify the source of the autocorrelation and take appropriate actions to learn from it and incorporate it into improvement strategies.

• If the autocorrelation is due to the sampling or measurement strategy, modify the data collection to reduce its impact.

• Continue to learn from and monitor the process as a run chart (using only visual analysis, not using run chart rules).

• Use time series analysis to model the data series and analyze the residuals from the time series using a Shewhart chart.

• Make adjustment to the control limits to compensate for the autocorrelation . The recommended adjustment is to increase the limits by multiplying by the factor:

____1 / √ 1‐r2 or use sigma = Rbar/ [d2 * sqrt(1‐r

2)]

114

11/29/2012

58


I Chart with Autocorrelation Adjustment

Control limits adjusted to compensate for the autocorrelation. The limits are increased by multiplying by the factor:

____ 1 / √ 1-r2 or sigma = Rbar/ [d2 * sqrt(1-r2)]

115


References for Autocorrelation on Control ChartsThe Health Care Data Guide: Learning from Data for Improvement. Provost and

Murray, Jossey Bass, 2011. Chapter 8..Montgomery, D. C. and Mastrangelo, C. M., “Some Statistical Process Control Methods For Autocorrelated Data”, Journal of Quality Technology 23, 1991, pp. 179–193.

Wheeler, D. 1995, “Advanced Topics in Statistical Process Control”, SPC Press, Knoxville, TN, Chapter 12. (adjustment factor)

Nelson, C. R., Applied Time Series Analysis for Managerial Forecasting, Holden-Day, Inc. San Francisco, 1973

Debrief

Note: Don’t overreact to autocorrelation. Most of the time, special causes cause the detection of autocorrelation. Spend time identifying the special causes.

116

11/29/2012

59


Caution: Do Not Over‐react to Autocorrelation

117


Examining Autocorrelation for Visit Time I Chart

• Scatterplot prepared to look at autocorrelation.

• The high value of r2 (autocorrelation = .905) could indicate autocorrelation that must be dealt with in order to use the limits on the chart.

• Receptionist noted that “it was pretty clear which of the specialists were in the office each day”.

• Aware of different average cycle times for each of the doctors.

• The QI team prepared a chart to show times for each of the three specialists..

118

11/29/2012

60


I Chart Clarifying Special Causes

119


Standardized Shewhart Charts• Shewhart charts with variable subgroup sizes (Xbar and S Chart, P chart, U chart) result in variable control limits.

•Sometimes this complexity in the appearance of the chart results in them not being used.

• An alternative is called the standardized Shewhart Chart. To construct the chart, the data is transformed using:

Z = (X-u) / σ where z is the standardized value, X is the original data value; µ is the mean and σ the standard deviation of the original data.

• Using this transformation, data for all the types of Shewhart charts can be transformed so that the resulting chart has limits that are always:

CL = 0 UL = 3 LL = -3

120

11/29/2012

61


Use of Standardized Chart

121


Control Charts with Slanted Center‐line

122

11/29/2012

62


Seasonal Effects on a Shewhart Chart

123


Seasonal Effects on a Shewhart Chart

124

11/29/2012

63


125

Making Shewhart Charts More Effective

• Guidance regarding limits on Shewhart Charts

– When to make limits

– When to revise limits

• Tips for good graphical display


Establishing Limits• For continuous data, less than 12 data points (subgroups) use

run chart rather than Shewhart chart

• May establish trial limits with 12 or more data points– Freeze and extend these limits until 20‐30 data points obtained

• Revise limits to make initial limits when 20‐30 data points are available– Freeze and extend limits into future. This will result in earlier detection

of special cause in future data.

• For attribute data (especially with large subgroup sizes) can establish limits with fewer data points.

126

11/29/2012

64



Technique matters!-Obtain baseline mean/limits from stable period and freeze them-Minimum baseline 12, preferred 20-30

The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.128

11/29/2012

65



11/29/2012

66



11/29/2012

67

API and CT Concepts, 2012Source: Promed133

API and CT Concepts, 2012Source: Trendstar134

11/29/2012

68


Designing Effective Shewhart Charts

• Tip 1: Select Appropriate Subgroup Size

– Too small

• Classification data: see guidelines for P chart

• Count data: U chart if center line less than 9 will be no lower limit. If center line less than 1.4 will have too many 0’s

– Too large

• Over‐dispersion issue. Consider Prime charts

135


Guidelines for Selecting Subgroup Size for an Effective P chart

Average PercentNonconforming

Units (pbar)

Minimum Subgroup Size (n) Required to Have

< 25% zero for p's

Minimum Subgroup Size Guideline

(n>300/pbar)

Minimum Subgroup Size Required to Have

LCL > 0

0.1 1400 3000 9000

0.5 280 600 1800

1.0 140 300 9001.5 93 200 6002 70 150 450

3 47 100 3004 35 75 220

5 28 60 175

6 24 50 1428 17 38 104

10 14 30 8112 12 25 66

15 9 20 5120 7 15 36

25 5 12 2830 4 10 22

40 3 8 14

50 2 6 10

Note: for p>50, use 100‐p to enter the table (e.g. for p=70% use table p of 30%, for p=99% use table p of 1%, etc.) Source: The Data Guide: L Provost and S. Murray, 2009

136

11/29/2012

69



• Tip 2: Rounding Data: have choices– When using computer system, always err on the side of maintaining too many decimal place

– Rounding used for compliance may not be useful for learning

• E.G LOS in days for compliance may be better in minutes for learning

– Rounding center lines/ limits: keep one more decimal than the statistic plotted on the chart.

• E.g. data 11.2…center line 9.79

137



• Tip 3: Formatting Charts

– Put related graphs on same page

138

11/29/2012

70


Good

Good

Good

Good

139



• Tip 3: Formatting Charts


– Presentation:

• Shape matters

– ratio of horizontal to vertical of 5:2

140

11/29/2012

71




Tip 3: Formatting Charts


– Presentation:

• Shape matters– ratio of horizontal to vertical of 5:2

• Vertical Scale

– include the limits in the middle 50%.Other 50% of graph space as “white space” on either side of limits. Don’t force scale to include 0 unless important to learning.

– If the data can’t go below 0 or exceed 100% don’t scale beyond these

142

11/29/2012

72


Just Right

Too Small a Scale

Too Large a Scale

143


InappropriateScale

AppropriateScale

144

11/29/2012

73


InappropriateScale

AppropriateScale

145



• Tip 3: Continued

– Presentation:

• Labels – include user friendly labels on axes, centerline, limits, and other key values (targets, baselines, requirements, etc) on chart

• Annotations: Integrate key annotations

146

11/29/2012

74




• Tip 3: Continued

– Presentation:

– Labels – include user friendly labels on axes, centerline, limits, and other key values (targets, baselines, requirements, etc) on chart

– Annotations: Integrate key annotations

– Gridlines‐ keep gridlines and other lines/colors/markings to a minimum

– Data‐May be helpful to display –if legible!

– Points‐ connecting the points is optional

» If in time order may

» If data not time order do not connect points 148

11/29/2012

75


m20: beyond statistical process control: advanced...

Documents