cusum control charts based on likelihood ratio for preliminary...

Science in China: Series A Mathematics ?? Vol. ?? No. ? ??–?? 1

DOI:

Cusum Control Charts Based on Likelihood RatioFor Preliminary Analysis

Dai Yi & Wang Zhaojun & Zou Changliang

LPMC and Department of Statistics, School of Mathematical Sciences, Nankai University, Tianjin 300071,ChinaCorrespondence should be addressed to Wang Zhaojun (email: [email protected])Received ?? ??, 2006; accepted ?? ??, 2006

Abstract To detect and estimate a shift in either the mean or deviation or both for thepreliminary analysis, statistical process control (SPC) tool, the control chart based on thelikelihood ratio test (LRT), is the most popular method.

Ref. [1] pointed out the test statistic lrt(n1, n2) is approximately distributed as χ2(2)

as the sample size n, n1 and n2 are very large, and the value of n1 = 2, 3, · · · , n − 2 andn2 = n − n1. So it’s inevitable that n1 or n2 is not large. In this paper the limit distributionof lrt(n1, n2) for fixed n1 or n2 is figured out, and the exactly analytic formula for evaluatingthe expectation and the variance of the limit distribution are also obtained.

In addition, the property of the standardized likelihood ratio statistic slr(t1, n) wasdiscussed in this paper. Although slr(n1, n) contains the most important information,slr(i, n)(i 6= n1) also contains lots of information. The cumulative sum (CUSUM) con-trol chart can obtain more information in this condition. So we proposed two CUSUMcontrol charts based on likelihood ratio statistics for preliminary analysis of individual ob-servations. One focuses on detecting the shifts in location in the historical data and theother is more general can detect a shift in either the location or the scale or both.

Moreover, the simulated results show that the proposed two control charts are, respec-tively, superior to their competitors not only in the detection of sustained shifts, but also inthe detection of some other out-of-control situations considered in this paper.

Keywords: preliminary analysis, false alarm probability, Cusum chart, likelihood ratio test.

1 Introduction

Statistical Process Control (SPC) has been widely used to monitor various industrialprocesses. Most of research in SPC focuses on the charting techniques. Some controlcharts, such as Shewhart chart, CUSUM chart and exponentially weighted moving average(EWMA) chart are used in detecting the shifts in the process mean and/or variance. It’ssaid that Shewhart control chart is very effective to detect the large shifts, CUSUM andEWMA control chart are good at detecting the small shifts.

In the literature there are Phase I and Phase II control charts need to distinguish. InPhase II the process distribution is assumed to be completely known. However, the processdistribution or the process parameters are often unknown in practice. So, it’s necessary

2 Science in China: Series A Mathematics

to establish that a process is statistically in control and estimate the process parameters,referred as “preliminary”, “retrospective” or “Phase I” analysis. In Phase I analysis, thehistorical data is used to decide if the process is statistically in control and to estimate theparameters of the process.

In the literature, there are a lot of control charts were used in the preliminary analysisfor individual or grouped observations, such as X and MR control charts (see ref. [2]-[7]), the Q chart (see ref. [8] and [9]), the control chart based on the likelihood ratio test(LRT) (see ref. [1]) and the CUSUM control chart for detecting the linear trend (CUSUM-LT), which was developed from the point of view of uniformly most powerful test (see ref.[10]). Ref. [1] pointed out the LRT control chart is more powerful than the combinedX and MR control charts in detecting a shift in either the mean or deviation or bothfor the preliminary analysis. Ref. [10] showed that CUSUM-LT control chart has betterproperties to detect the trend and step shifts in the mean of the data than the LRT chart andQ chart. Comparing the LRT and CUSUM-LT charts, we know the CUSUM-LT can onlybe used to detect the shift in the mean, but, the LRT chart can be used to detect a shift ineither the location or the scale or both. Another fine property of LRT chart is its abilitynot only to point out the position, where the shift happened, but also to specify whetherthe shift comes from the mean or variance.

As ref. [1] pointed out the test statistic lrt(n1, n2) (see ref. [1] for details) is approx-imately distributed as χ2(2) as the sample size n, n1 and n2 are very large. But, it is nottrue for fixed n1 or n2. The limit distribution of lrt(n1, n2) for fixed n1 or n2 is discussedin this paper, and the analytic formula for evaluating the expectation and the variance ofthe limit distribution are obtained in this paper.

Two CUSUM control charts based on the maximum likelihood ratio statistic, CUSUM-M and CUSUM-MS control chart, are proposed in this paper. The CUSUM-M chart fo-cuses on detecting the shifts in the mean for the individual observations and CUSUM-MSchart is used to detect the shifts in mean and/or standard deviation. The simulated resultsshow that CUSUM-M chart is not only superior to the CUSUM-LT chart for detecting themean step-shifts but also has competitive performance for the trend-shifts in mean, theCUSUM-MS chart performs significantly better than the LRT chart in terms of detectingthe shifts of process mean and/or deviation except the slightly disadvantage in detectingvery large shifts. The robustness of the proposed control charts for other kinds of shifts isalso considered in this paper.

2 The New Preliminary Control Charts

Two CUSUM control charts based on the likelihood ratio statistics, the CUSUM-Mand CUSUM-MS charts are proposed in this section. For the reasons that the CUSUM-M chart is a special case of CUSUM-MS chart, we first discuss the construction of theCUSUM-MS chart in detail and then describe the CUSUM-M chart briefly. The designsof the CUSUM-M and CUSUM-MS are also discussed in this section.

Cusum Control Charts Based on Likelihood Ratio For Preliminary Analysis 3

2.1 CUSUM-MS Control Chart

Suppose there are n independent observations that are assumed to be normally dis-tributed as

xi ∼ N(µi, σ2i ), i = 1, · · · , n.

If the process is in control then µi = µ and σ2i = σ2 for all i. Assume that a step shift in

the mean or variance or both occurs after n1th observations, i.e. the mean and variance ofthe first n1 observations is (µa, σ

2a), and the last n2 = n−n1 observations have the same

mean and variance (µb, σ2b ).

The logarithm of the likelihood function of x1, x2, . . . , xn is given by

−12

n∑i=1

[log(2πσ2

i ) +(xi − µi)2

2σ2i

].

If the process is in control, the maximum value of the logarithm of the likelihood functionis

l0 = −n

2log(2π)− n

2log(σ̂2

n)− n

2,

where σ̂2n = 1

n

n∑i=1

(xi − x̄)2 and x̄ = 1n

n∑i=1

xi. While there is a step shift after the n1th

observation, the corresponding maximum value is

l1 = −n

2log(2π)− n1

2log(σ̂2

n1)− n2

2log(σ̂2

n2)− n

2,

where

σ̂2n1

=1n1

n1∑i=1

(xi − x̄n1)2, x̄n1 =

1n1

n1∑i=1

xi,

σ̂2n2

=1n2

n∑i=n1+1

(xi − x̄n2)2, x̄n2 =

1n2

n∑i=n1+1

xi.

The classical likelihood ratio statistic is defined by

lr(n1, n) = −2(l0 − l1) = n log[σ̂2n(σ̂2

n1)−

n1n (σ̂2

n2)−

n2n ]. (1)

For fixed i = 2, 3, · · · , n−2, let EH0 [lr(i, n)] and VarH0 [lr(i, n)] denote the expecta-tion and variance of lr(i, n) under in-control conditions, define the standardized lr(i, n)as

slr(i, n) =lr(i, n)− EH0 [lr(i, n)]√

VarH0 [lr(i, n)]. (2)

Define a CUSUM statistic based on slr(i, n) as

Si(n) = max{0, Si−1(n) + slr(i, n)}, i = 2, 3, · · · , n− 2. (3)

where the initial value S1(n) = 0 .

Our proposed CUSUM-MS chart is constructed by plotting the statistics Si versus i. Ifa Si exceeds the given decision interval h, an out of control signal is triggered.


Although as n, i, n−i →∞, lr(i, n) L→χ2(2), it’s not true for fixed i (see the simulatedresults of ref. [1]). So, it’s necessary to calculate lr(i, n)’s expectation and variance. Forgiven n = 30, the simulated expectation and variance of lr(i, n) is shown in Table 1 (By700,000 simulations). Note that for fixed n, if the process is in control, the distributionof lr(i, n) are symmetric about i, that is to say, lr(i, n) and lr(n − i, n) are identicallydistributed. However, as mentioned in ref. [1], the EH0 [lr(i, n)] and VarH0 [lr(i, n)]depend on the value of i and n, the CUSUM statistic in equation (3) is not easy to use inpractice.

For fixed i, the limit null distribution of lr(i, n), and its expectation and variance aregiven in appendix of this paper. The expectation and variance of this limit distribution aregiven by

EH0 [lr(i,∞)] = i

[log

(i

2

)− ψ0

(i− 1

2

)], (4)

VarH0 [lr(i,∞)] = i2ψ1

(i− 1

2

)− 2i, (5)

where ψ0(·) and ψ1(·) are, respectively, the digamma and trigamma function. The recur-sive formula for calculating ψ0 and ψ1 are shown in the appendix of this paper.

Table 1 The expectation and variance of lr(i, 30) and lr(i,∞)i EH0 [lr(i, 30)] EH0 [lr(i,∞)] VarH0 [lr(i, 30)] VarH0 [lr(i,∞)]2 3.93 3.93 15.74 15.743 2.96 2.95 8.78 8.804 2.64 2.63 7.01 6.965 2.48 2.47 6.18 6.126 2.39 2.37 5.73 5.657 2.33 2.31 5.44 5.358 2.29 2.27 5.24 5.149 2.26 2.23 5.12 4.9910 2.24 2.21 5.03 4.8711 2.22 2.18 4.97 4.7812 2.21 2.17 4.91 4.7113 2.21 2.17 4.88 4.6414 2.20 2.14 4.87 4.5915 2.20 2.13 4.87 4.55

From Table 1, we can see there is a slight difference between the simulated and asymp-totic value, and the differences get smaller as the value of n get larger. We also comparedthe performance of the control charts using these two kinds of values. Although the chartsbased on the simulated value perform a little better than that of the asymptotic values, werecommend to use the asymptotic expectation and variance in practice for simplicity.

So, in the rest of this paper, the standardized likelihood ratio is defined by

slr(i, n) =lr(i, n)− EH0 [lr(i,∞)]√

VarH0 [lr(i,∞)]. (6)

Theorem 1 If the process is out of control (µa 6= µb or σ2a 6= σ2

b ), then E[slr(t1,∞)]increase with t1(t1 6 n1) and E[slr(t1,∞)] > 0.


Proof From Appendix A we can obtain the expectation of slr(t1,∞) as follows:

E[slr(t1,∞)] =

�σ2

aσ2

b

−1−logσ2

aσ2

b

�+ 1

σ2b

(µa−µb)2q

ψ1( t1−12 )− 2

t1

. (7)

we can define two functions as follows:

h1(x) =x− 1− log x, x > 0,

h2(t) =ψ1

(t− 1

2

)− 2

t, t > 1,

It is easy to check that ∂∂x

h1(x) < 0 for 0 < x < 1 and ∂∂x

h1(x) > 0 for x > 1. Thush1(x) > 0 for 0 < x and x 6= 1, so that E[slr(t1,∞)] > 0 .

Because∂

∂th2(t) =

12ψ2

(t− 1

2

)+

2t2

=2t2−

∞∑k=0

8(t− 1 + 2k)3

<2t2−

∞∑k=0

8(t− 1 + 2k)(t + 2k)(t + 1 + 2k)

=2t2− 4

t2 − 1= − 2(t2 + 1)

t2(t2 − 1)< 0,

E[slr(t1,∞)] increases with t1.

By (7), the following corollaries can be obtained.

Corollary 1 If the process is out of control (µa 6= µb or σ2a 6= σ2

b ), then

E[slr(t1,∞)] = ∞, as t1 →∞.

Corollary 2 If the process is in control (µa = µb and σ2a = σ2

b ), then

E[slr(t1,∞)] = 0.

Corollary 3 If the process is out of control (µa 6= µb or σ2a 6= σ2

b ), then

E[St1(∞)]− E[St1−1(∞)] > E[slr(t1,∞)] > 0.

For the given false alarm probability (FAP) α, let hα denote the decision interval ofthe CUSUM-MS chart. For various given combinations of α and n, the simulated hα ofCUSUM-MS are shown in Table 2. From Table 2 we observed that hα increases as n

increases and α decreases.

Table 2 Simulated hα of CUSUM-MSn h0.05 h0.04 h0.03 h0.02 h0.01 h0.0075 h0.005 h0.0025 h0.001

30 32.2 35.4 40.2 45.0 53.6 56.0 61.5 70.0 84.045 50.1 54.0 60.2 67.7 80.5 87.0 93.2 110.5 123.560 67.1 72.4 81.0 93.7 113.7 119.4 131.0 145.5 174.075 82.3 89.1 98.2 112.2 133.9 143.0 158.2 184.5 212.090 102.5 111.2 121.5 139.0 163.5 174.2 190.5 222.0 254.0

From Figure 1 we can see there is a linear relationship between hα and n for fixedFAP α. As pointed out in ref. [1], it is not important for the preliminary application to find


0 10 20 30 40 50 60 70 80 90 1000

50

100

150

200

250

300

n

h

Figure 1. The linear relationship between hα and n

exact control limits that correspond to a specific FAP. For simplicity, we take the followingline to approximate the decision interval hα of CUSUM-MS:

hα = A(α) · n. (8)

The simulated slopes, A(α), are shown in the Table 3.

Table 3 The simulated slopes A(α)

α 0.05 0.04 0.03 0.02 0.01 0.0075 0.005 0.0025 0.001

A(α) 1.1182 1.2112 1.3370 1.5281 1.8178 1.9339 2.1190 2.4559 2.8309

From Figure 2 we can see there is an exponential relationship between A(α) and α.Through the least square fitting to the A(α), we obtain the following approximated deci-sion interval hα for CUSUM-MS chart:

hα = −(0.4401 log α + 0.2043) · n. (9)

The simulated results show that the approximated decision interval in equation (9) per-forms very well, and it can be used for the practical application of the preliminary analysis.

2.2 CUSUM-M Control Chart

Under the condition that the process variance is stable, ref. [10] proposed a newCUSUM control chart for detecting the shift in the mean only. As they proved this newCUSUM chart is constructed with the most uniformly powerful test for the linear trend inthe process mean. The results in ref. [10] show their CUSUM performs better than theLRT chart in detecting the shift of process mean. But this CUSUM chart is useless fordetecting the shift in the process standard deviation. Under the same condition as in ref.[10] (shift in the mean only), the CUSUM-M chart is proposed in this subsection.


0 0.01 0.02 0.03 0.04 0.05 0.061

1.5

2

2.5

3

3.5

4

alpha

slope

Figure 2. The exponential relationship between A(α) and α

Suppose there are n independent observations that are assumed to be normally dis-tributed xi ∼ N(µi, σ

2), i = 1, · · · , n. If the process is in control then µi = µ for alli. Assume that a step shift in the mean occurs after n1th observation, i. e. the first n1

observations have the same mean µ, while the other n2 = n−n1 observations have meanµa. Similar to the formula (1), the maximum logarithm likelihood ratio statistic is givenby

lr(n1, n) =1σ2

[nσ̂2n − iσ̂2

n1− n2σ̂

2n2

]. (10)

It can be easily shown that this statistic is distributed as a χ2(1) for any n and n1. So, inthis case the standardized statistic slr(i, n) is lr(i,n)−1√

2. The CUSUM-M chart is defined

by equaion (3) through replacing the slr(i, n) by lr(i,n)−1√2

.

Theorem 2 If the process is out of control (µa 6= µb ), then E[slr(t1, n)] increasewith t1(t1 6 n1) and E[slr(t1, n)] > 0.

Proof It can be easily shown that nσ2n

σ2 and t2σ2

t2σ2 are distributed as noncentral chi-

squared distribution χ2(n − 1, n1n2n

(µa − µb)2) and χ2(t2 − 1, (n1−t1)n2

t2(µa − µb)2)

(t2 = n− t1) for any n and n1.

Because

E[slr(t1, n)] =n2

2√2n

(n

n− t1− 1)(µa − µb)2, (11)

E[slr(t1, n)] > 0 increase with t1(t1 6 n1) and E[slr(t1, n)] > 0.

By (11), the following corollary can be obtained.

Corollary 4 If the process is out of control (µa 6= µb ), then E[slr(t1, n)] =1√2t1(µa − µb)2, as n2 →∞, and E[slr(t1, n)] = ∞, as n2 →∞, t1 →∞.


Corollary 5 If the process is in control (µa = µb ), then E[slr(t1, n)] = 0.

Corollary 6 If the process is out of control (µa 6= µb ), then

E[St1(n)]− E[St1−1(n)] > E[slr(t1, n)] > 0.

From the above theorems and corollary we observed

• If the process is in control , then E[slr(t1, n)] = 0.

• If the process is out of control , then E[slr(t1, n)] > 0, max E[slr(t1, n)] =E[slr(n1, n)] , E[slr(t1, n)] increase with t1 → n1 and the expectation of CUSUMstatistic St1(n) is strictly increased.

Although slr(n1, n) contains important information, slr(i, n)(i 6= n1) also containslots of information. The CUSUM chart can obtain more information, which is the basedidea of this paper.

However, in many practices, the σ2 is usually unknown and must be estimated. Inthis paper, the minimum-variance unbiased estimator (when the process is in control),

σ̂2 = 1n−1

n∑i=1

(xi − x̄)2, is used to estimate the unknown process variance. Although the

performance could be improved with the other robust estimator, such as σ2DLM given by

ref. [11], but for simplicity and convenient use, we only consider the use of σ̂2 in thispaper.

Using the same method as previous, the approximated decision interval hα of CUSUM-M chart is given by

hα = −(0.5446 log α + 0.5060) · n. (12)

3 Illustrative Examples

In this section, two illustrative examples are given to introduce the implementation ofCUSUM-MS and CUSUM-M control charts. In these examples, the approximate controllimits given by (8) and (10) are used.

The observations of the Example 1 are from the Table 2 in ref. [1]. There are totaln = 30 observations, which are normally distributed with mean µ and variance 1. Notethat the mean µ has been shifted from 0 to 1 after observation 15. The Table 4 presents theobservations Xi and the standardized likelihood ratio statistics slr(i, 30) and CUSUM Si

for CUSUM-M and CUSUM-MS charts respectively. Also tabulated in Table 4 are thevalues of Nlrt given by (7) in ref. [1] and SH,i, SL,i given by (6) in Koning and Does(2000). Given FAP α = 0.05, the control limits for the four charts are respectively 33.8,33.4, 1.0 and 11.09. It can be clearly seen that all of the four control charts can detectthe change. Moreover, observe that the maximum values of slr(i, 30) in column 3 and5 are both the slr(15, 30) which indicates the location of the shift when CUSUM-M andCUSUM-MS control charts are used.


The data for Example 2, shown in Table 5, consist of 30 observations normally dis-tributed with mean zero and standard deviation σ which shifts from 1 to 2.5 after obser-vation 20. Only the CUSUM-MS chart signals and an accurate estimate of the location ofthe shift is the observation 20 because the slr(20, 30) = 3.06 is the maximum values ofslr(i, 30).

Table 4 Data for Example 1 with a shift in the mean.i Xi CUSUM-M CUSUM-MS LRT CUSUM-LT

slr(i, 30) Si slr(i, 30) Si SH,i SL,i

1 -0.692 0.56 -0.53 0.00 -0.61 0.00 0.07 0.00 0.063 -0.96 0.08 0.08 -0.06 0.00 0.16 0.08 0.004 -0.11 0.20 0.28 0.63 0.63 0.28 0.05 0.035 -0.25 0.44 0.72 1.54 2.17 0.43 0.04 0.046 0.45 0.23 0.95 1.73 3.90 0.46 0.00 0.217 -0.26 0.50 1.45 2.73 6.63 0.63 0.03 0.198 0.68 0.22 1.66 2.64 9.27 0.62 0.00 0.469 0.22 0.23 1.89 3.43 12.70 0.75 0.00 0.56

10 -2.10 1.78 3.67 2.02 14.72 0.51 0.84 0.0011 0.65 1.45 5.13 1.98 16.70 0.50 0.43 0.4012 -1.49 2.93 8.06 2.68 19.38 0.62 1.09 0.0013 -2.49 6.00 14.06 4.51 23.90 0.92 2.29 0.0014 -1.11 7.89 21.95 6.67 30.57 1.28 2.68 0.0015 0.23 8.05 30.00 6.87 37.44 1.31 2.23 0.4616 2.16 5.67 35.67 4.61 42.06 0.94 0.46 2.2217 1.95 3.92 39.59 3.20 45.26 0.70 0.00 3.8418 1.54 2.84 42.42 2.23 47.49 0.54 0.00 5.1419 0.67 2.66 45.08 1.85 49.34 0.48 0.00 5.7320 1.09 2.14 47.22 1.27 50.60 0.38 0.00 6.6821 1.37 1.45 48.67 0.63 51.23 0.28 0.00 7.8822 0.69 1.34 50.01 0.43 51.66 0.24 0.00 8.4423 2.26 0.19 50.20 -0.30 51.36 0.12 0.00 10.5624 1.86 -0.41 49.79 -0.68 50.68 0.05 0.00 12.2725 0.62 -0.46 49.33 -0.83 49.85 0.03 0.00 12.6326 -1.04 0.40 49.74 -0.07 49.78 0.16 1.52 11.1127 2.30 -0.54 49.19 -0.18 49.60 0.14 0.00 13.5228 0.07 -0.31 48.88 -0.32 49.27 0.12 0.31 13.2129 1.49 0.00 14.6930 0.52 0.00 14.91

4 Performance Comparisons

As ref. [1] pointed out, the average run length (ARL) can not be used as the criterionof performance in the preliminary analysis. Therefore, similar to ref. [1] and [10], theFAP and the true signal probability (TSP) are used to compare the performance of controlcharts for the preliminary analysis. A control chart is said to be better than other one if itsTSP is larger than the other’s when they have the same FAP.

In this section, our proposed CUSUM-MS and CUSUM-M control charts, the LRT chartof ref. [1] and CUSUM-LT chart of ref. [10] are compared. Because the CUSUM-M andCUSUM-LT chart are based on the assumption that the process variance is stable, hence,they have advantage in detecting the mean shift when there is only a change in location in


Table 5 Data for Example 2 with a shift in the variance.i Xi CUSUM-M CUSUM-MS LRT CUSUM-LT

slr(i, 30) Si slr(i, 30) Si SH,i SL,i

1 -1.932 0.36 -0.46 0.00 -0.77 0.00 0.04 0.00 0.073 1.00 -0.70 0.00 -0.82 0.00 0.03 0.00 0.194 -0.04 -0.70 0.00 -0.49 0.00 0.09 0.00 0.215 0.63 -0.70 0.00 -0.13 0.00 0.15 0.00 0.316 -0.21 -0.70 0.00 0.41 0.41 0.24 0.04 0.277 0.08 -0.70 0.00 1.08 1.50 0.35 0.01 0.298 0.71 -0.65 0.00 1.64 3.13 0.45 0.00 0.469 2.03 -0.29 0.00 1.16 4.29 0.36 0.00 0.97

10 -0.35 -0.37 0.00 1.62 5.91 0.44 0.19 0.7811 -0.24 -0.42 0.00 2.21 8.12 0.54 0.34 0.6312 0.05 -0.40 0.00 2.97 11.09 0.67 0.39 0.5813 -2.69 -0.70 0.00 1.01 12.10 0.34 1.51 0.0014 -0.54 -0.71 0.00 1.52 13.61 0.42 1.72 0.0015 -0.43 -0.70 0.00 2.11 15.73 0.52 1.88 0.0016 -0.14 -0.70 0.00 2.77 18.50 0.63 1.90 0.0017 -2.46 -0.46 0.00 2.01 20.52 0.50 3.13 0.0018 -1.23 -0.21 0.00 2.48 22.99 0.58 3.67 0.0019 0.30 -0.28 0.00 2.98 25.98 0.67 3.32 0.3520 1.11 -0.51 0.00 3.06 29.04 0.69 2.47 1.2021 -3.04 0.35 0.35 2.13 31.17 0.53 4.32 0.0022 2.18 -0.33 0.01 1.29 32.46 0.39 2.60 1.7223 -0.47 -0.19 0.00 1.82 34.28 0.48 2.78 1.5424 0.36 -0.26 0.00 2.26 36.54 0.55 2.34 1.9925 -2.72 1.08 1.08 1.97 38.51 0.51 4.30 0.0226 3.03 -0.27 0.81 0.51 39.02 0.26 1.58 2.7427 1.34 -0.60 0.21 0.48 39.50 0.25 0.29 4.0428 -4.05 2.53 2.75 2.07 41.57 0.53 3.75 0.5729 2.63 1.10 3.2230 2.29 0.00 5.55

the data. Though for a meaningful comparison, the CUSUM-MS and CUSUM-M chartsshould be compared with their competitors, the LRT and CUSUM-LT charts respectively.For simplicity, the case for 30 observations and FAP 0.05 is discussed only in this paper.The results in this section is evaluated by 10,000 simulations.

Sustained (step) shift is most commonly used to compare the performance of controlcharts, the four charts are compared to detect the step-shift in mean and/or variance. Aswe known, the position of shifts affects the performance of the charts greatly. ref. [1]compared the performance of charts assuming the step shifts occur after 5, 15, or 25 of30 observations. In general, we do not know where the shift should be occurred. If it’sreasonable to assume the position of shifts is distributed as F (k), k = 1, 2, · · · , n − 1,the average TSP (ATSP)

ATSP =n−1∑k=1

F (k)TSPk, (13)

could be used to as the criterion for comparing the performance of control charts, whereTSPk denotes the TSP of control chart when the shift occurs after kth observation. In thispaper, we assume the position of shift is uniformly distributed, so, the average TSP in this


paper is evaluated by 1,000 simulations.

The in-control model is given by

xi = µ + σεi, εiiid∼ N(0, 1), i = 1, 2, · · · , n.

The out-of-control model for one step- shift of size δ1 at position t1 in mean and varianceare respectively defined as

xi = µ + δ1I[t1+1,n](i) + σεi,

xi = µ + σεiI[1,t1](i) + δ1σεiI[t1+1,n](i),

where IA(·) is the indicator function of set A.

The TSP’s for one step-shift in mean occurring after 5, 15, and 25th of 30 observationsand the ATSP are shown in Figure 3(a-d). From Figure 3 we observed

• Except that the difference in Figure 3(b) is very little, the CUSUM-M chart performsclearly better than CUSUM-LT chart in detecting one step shifts in the process mean.

• The CUSUM-MS chart performs uniformly better than LRT chart.

• In terms of ATSP, the CUSUM-MS and LRT charts even have a little advantagein detecting large shifts compared with CUSUM-M and CUSUM-LT charts. Thisresult can be explained by studying the robustness of the variance estimator σ̂2. Inref. [11], the author concluded that σ̂2 dominates other ”robust” estimator for thesituations that there is no or small shifts in the process mean, on the other hand forthe large shift case, it is dramatically inferior than the others.

Figure 3. The TSP’s for one step mean shift occurringafter 5, 15, and 25th observation and ATSP.


The simulation results for one step-shift in variance occurring after 5, 15, and 25th of30 observations and the ATSP are summarized in Figure 4(a-d). The Figures 2 show

• If there is only few observations remain stable, such as Figure 4(a), the LRT chartperform better than our proposed CUSUM-MS chart especially for the large shift.Whereas, in the case that shift occurs after 15 or 25th of 30 observations, the CUSUM-MS chart has significant advantage.

• In terms of ATSP, the CUSUM-MS chart performs better than LRT chart in detectingsmall or moderate shifts in the process variance and has the slight disadvantage indetecting rather large shifts.

• The CUSUM-M and CUSUM-LT charts almost cannot detect any magnitude ofshifts in variance.

Figure 4. The TSP’s for one step variance shift occurringafter 5, 15, and 25th observation and ATSP.

For other sample size, the similar results could be obtained.

5 Robustness of the proposed Chart

Basing on the most uniformly powerful test for a trend shift in the mean, ref. [10] pro-posed the CUSUM-LT chart. Note that the CUSUM-LT chart is uniformly most powerfultest only when the shift occurs from the beginning of the data. The trend shift of size δ1 atposition t1 is defined as

xi = µ + δ1

i− t1n− t1

I[t1+1,n](i) + σεi.


The TSP’s for trend shift occurring after 5, 15, and 25th of 30 observations and theATSP are plotted in Figure 5(a-d), which shows the CUSUM-M chart performs betterthan the other three charts, uniformly.

Figure 5 The TSP’s for trend shift occurringafter 5, 15, and 25th observation and ATSP.

In recent years, the multiple shifts in the mean of the process have been discussed bysome papers, such as ref. [1], [12] and [13]. As we known, our proposed charts and LRTchart are constructed with the likelihood ratio of two samples. If the process has multiplechange points, the assumption of these charts is not satisfied, so their detecting abilitiesshould be weaken. For a robustness study, the charts are compared in the following fourkinds of two step-shift models:

[1] Two step mean shifts δ1 and δ2 at positions t1 and t2 respectively (M-M shift). Theout-of-control model is

xi = µ + δ1I[t1+1,t2](i) + δ2I[t2+1,n](i) + σεi, i = 1, 2, · · · , n.

[2] Two step variance shifts (S-S shift). The out-of-control model is

xi = µ + σεiI[1,t1](i) + δ1σεiI[t1+1,t2](i) + δ2σεiI[t2+1,n](i), i = 1, 2, · · · , n.

[3] The first shift δ1 in the mean occurs after t1th observation and the second shift δ2

in the variance occurs after t2th observation (M-S shift). The out-of-control modelis

xi = µ + σεiI[1,t2](i) + δ1I[t1+1,t2](i) + δ2σεiI[t2+1,n](i), i = 1, 2 · · · , n.


[4] The first shift δ1 in the variance occurs after t1th observation and the second shiftδ2 in the mean occurs after t2th observation (S-M shift). The out-of-control modelis

xi = µ + σεi(I[1,t1](i) + I[t2+1,n](i)) + δ1σεiI[t1+1,t2](i) + δ2I[t2+1,n](i).

Figure 6(a) shows that the ATSP comparisons for M-M shift with n = 30, δ1 = 0.5,δ2 ∈ [−3.6, 4.0] and t1, t2 are randomly generated from the discrete uniform distributionU(2, 28) by 1000 times repetitions. From this Figure we can see that the CUSUM-Mand CUSUM-MS charts are more robust for this kind of shift pattern. The ATSP com-parisons for S-S shift with n = 30, δ1 = 2.0, δ2 ∈ [1.0, 5.75] are shown in Figure6(b). The CUSUM-MS chart performs significantly better than the LRT chart. The sim-ulated ATSP’s for M-S and S-M shift with δ1 = 0.5, δ2 ∈ [1.0, 5.75] and δ1 = 2.0,δ2 ∈ [0.0, 3.8] are, respectively, summarized in Figure 6(c) and (d). These figures showthat our proposed charts are more robust for the multiple change-points.

Figure 6 The ATSP’s for M-M, S-S, M-S and S-M shifts of 30 observations.

6 Conclusions and Considerations

Basing on the likelihood ratio statistic, two new preliminary control charts, CUSUM-M and CUSUM-MS charts, were introduced to detect mean/deviation shifts in Phase 1analysis. These two CUSUM charts perform very well in detecting the traditional sus-tained shifts and they are more robust than the LRT and CUSUM proposed by ref. [10]for some other kinds of shifts. Moreover, an approximated analytic formula for evaluatingthe decision interval is given in this paper.


A possible further work would to develop a chart based on the idea of the CUSUM-MS chart, which is used to detect shifts in the multivariate observations. Furthermore, aninteresting problem is how to generalize the method to the Phase 1 case of linear profile.

Acknowledgements This paper was supported by The Natural Sciences Foundation of Tianjin(033603111). The au-thors are grateful to the two anonymous referees and the Editor for several constructive comments that have improvedthis article.

Appendix A.

The limit distribution of lr(t1, n) as n →∞ (for fixed n1 and t1 6 n1)

Assume that a step shift in the mean or variance or both occurs after n1th observations, i.e. the mean and variance

of the first n1 observations is (µa, σ2a), and the last n2 = n − n1 observations have the same mean and variance

(µb, σ2b ). Note that t2 = n − t1, σ̂2

n =t1σ̂2

t1+t2σ̂2

t2n

+ t1t2n2 (x̄t1 − x̄t2 )2, and σ̂2

nP→σ2

b , σ̂2n2

P→σ2b , x̄t2

P→µb,t2n→ 1, as n →∞. So, we have the following conclusion:

lr(t1, n) =t1(log bσ2n − log bσ2

t1) + t2(log bσ2

n − log bσ2t2

)

=− t1 logbσ2

t1bσ2n

+ t2 log

"1 +

t1

n

bσ2t1bσ2n

− 1

!+

t1t2

n2

�x̄t1 − x̄t2bσ2

n

�2#

=− t1 logbσ2

t1bσ2n

+ t2

"t1

n

bσ2t1bσ2n

− 1

!+

t1t2

n2

�x̄t1 − x̄t2bσ2

n

�2

+ op(n−1)

#P→− t1 log

bσ2t1

σ2b

+ t1bσ2

t1

σ2b

− t1 + t1(x̄t1 − µb

σb)2 (as n, n2 →∞)

=− t1 logbσ2

t1

σ2a

− t1 logσ2

a

σ2b

+ t1σ21

σ2b

bσ2t1

σ2a

− t1 + t1σ21

σ2b

(x̄t1 − µa

σa)2

+t1

σ2b

(µa − µb)2 +

2t1(µa − µb)σa

σ2b

(x̄t1 − µa

σa)

D=− t1 log

x

t1− t1 log

σ2a

σ2b

+σ21

σ2b

x− t1 +σ21

σ2b

z2

+t1

σ2b

(µa − µb)2 +

2t121 (µa − µb)σa

σ2b

z,

where x = t1bσ2

t1σ2

a∼ χ2(t1−1), z = t

121 (

x̄t1−µa

σa) ∼ N(0, 1), x and z are independent. If the process is in control

then lr(t1,∞) = −t1 log xt1

+ x + z2 − t1.

The expectation and variance of lr(t1,∞) under in-control

Because nbσ2n ∼ χ2(n− 1), we have

E[n log bσ2n] =

Z ∞

0n(log

x

n)

1

2n−1

2 Γ(n−12

)x

n−12 −1e−

x2 dx

=n

Γ(n−12

)

Z ∞

0(log t)t

n−12 −1e−tdt− n(log

n

2)

1

Γ(n−12

)

Z ∞

0t

n−12 −1e−tdt

=n(ψ0(n− 1

2)− log

n

2),

from which we can obtain the expectation of lr(t1, n) as follows:

EH0 [lr(t1, n)] =E[n log bσ2n]− E[t1 log bσ2

t1]− E[t2 log bσ2

t2]

=nψ0(n− 1

2)− t1ψ0(

t1 − 1

2)− t2ψ0(

t2 − 1

2) + log

tt11 tt22nn

,


where ψ0(·) is the digamma function.

Therefore, we have

EH0 [lr(t1,∞)] = E[x− t1 − t1 logx

t1+ z] = t1

�log

t1

2− ψ0(

t1 − 1

2)

�.

To obtain the variance of lr(t1,∞), we first evaluate the second moment of x−t1−t1 log xt1

, where x ∼ χ2(t1−1).

E[x− t1 − t1 logx

t1]2

=

Z ∞

0(x− t1(log

x

t1))2

1

2t1−1

2 Γ( t1−12

)x

t1−12 −1e−

x2 dx

=

Z ∞

0(2t− t1(log

2t

t1))2

1

Γ( t1−12

)t

t1−12 −1e−tdt

=t21

Γ( t1−12

)Γ(2)(

t1 − 1

2)− 4t1

Γ( t1−12

)Γ(1)(

t1 + 1

2)− 2t21 log t1

2

Γ( t1−12

)Γ(1)(

t1 − 1

2)

+4

Γ( t1−12

)Γ(

t1 + 3

2) +

4t1 log t12

Γ( t1−12

)Γ(

t1 + 1

2) +

t21 log2 t12

Γ( t1−12

)Γ(

t1 − 1

2)

=t21(ψ20(

t1 − 1

2) + ψ1(

t1 − 1

2))− 2t1(t1 − 1)ψ0(

t1 + 1

2)− 2t21(log

t1

2)ψ0(

t1 − 1

2)

+ (t21 − 1) + 2t1(t1 − 1) logt1

2+ t21(log

t1

2)2

=t21(ψ1(t1 − 1

2)) + (t1(ψ0(

t1 − 1

2)) + 1− t1)2 − 2t21(log

t1

2)ψ0(

t1 − 1

2)

+ (t1 logt1

2+ t1 − 1)2 − t21 − 3.

So, the variance of lr(t1,∞) is given by

V arH0 [lr(t1,∞)] = V ar[x− t1 − t1 logx

t1] + 2

= E[x− t1 − t1 logx

t1]2 − (E[x− t1 − t1 log

x

t1])2 + 2

= t21ψ1(t1 − 1

2)− 2t1,

where ψ1(·) is the trigamma function.

The expectation of lr(t1,∞) under out of control (EH1 [lr(t1,∞)])

EH1 [lr(t1,∞)]

=E[−t1 logx

t1− t1 log

σ2a

σ2b

+σ21

σ2b

x− t1 +σ21

σ2b

z2 +t1

σ2b

(µa − µb)2 +

2t121 (µa − µb)σa

σ2b

z]

=t1

�log

t1

2− ψ0(

t1 − 1

2)

�− t1 log

σ2a

σ2b

+σ2

a

σ2b

(t1 − 1)− t1 +σ2

a

σ2b

+t1

σ2b

(µa − µb)2

=t1

�log

t1

2− ψ0(

t1 − 1

2)

�+ t1(

σ2a

σ2b

− 1− logσ2

a

σ2b

) +t1

σ2b

(µa − µb)2.

Appendix B.The calculation of ψ0 and ψ1

The recursive formula for evaluating the digamma function ψ0 and trigamma function ψ1 is given by

ψ0(z + 1) = ψ0(z) +1

z, ψ0(1) = −γ, ψ0(

1

2) = −γ − 2 log 2,


ψ1(z + 1) = ψ1(z)− 1

z2, ψ1(1) =

π2

6, ψ1(

1

2) =

π2

2,

where γ(γ = 0.577215664 · · · ) is the Euler-Mascheroni constant.

Therefore, it’s easy to get the following formula:

ψ0(n) = −γ +

n−1Xk=1

1

k, ψ1(n) =

π2

6−

n−1Xk=1

1

k2,

ψ0(n +1

2) = −γ − 2 log 2 + 2

nXk=1

1

2k − 1,

ψ1(n +1

2) =

π2

2− 4

nXk=1

1

(2k − 1)2,

where n is an integer.

References

1. Sullivan, J. H., Woodall, W. H., A control chart for preliminary analysis of individual observations, J. QualityTechnol., 1996, 28: 265–278.

2. Nelson, R. H., Control charts for individual measurements, J. Quality Technol., 1982, 14: 172–173.3. Ryan, T. P. Statistical methods for quality improvement. New York: John Wiley & Sons, 1989.4. Montgomery, D. C. 3rd ed. Introduction to statistical quality control. New York: John Wiley & Sons, 1997.5. Wadsworth, H. M., Stephens, K. S., Godfrey, A. B. Modern methods for quality control and improvement New

York: John Wiley & Sons, 1986.6. Rigdon, S. E., Cruthis, E. N., Champ, C. W., Design strategies for individuals and moving range control charts, J.

Quality Technol., 1994, 26: 274–287.7. Roes, K. C. B., Does, R. J. M. M., Schurink,Y., Shewhart-type control chart for observations, J. Quality Technol.,

1993, 25: 188–198.8. Quesenberry, C. P., SPC Q charts for start-up processes and short or long Runs, J. Quality Technol., 1991, 23:

213–224.9. Quesenberry, C. P., On properties of Q charts for variables, J. Quality Technol., 1995, 27: 184–203.

10. Koning, A. J., Does, R. J. M. M., CUSUM control charts for preliminary analysis of individual observations, J.Quality Technol., 2000, 32: 122–132.

11. Boyles, R. A.Estimating common-cause sigma in the presence of special cause, J. Quality Technol., 1997, 29:381–395.

12. Sullivan, J. H., Detection of multiple change points from clustering individual observations, J. Quality Technol.,2002, 34, 371–383.

13. Hawkins, D. M., Qiu, P., Kang, C. W., The changepoint model for statistical process control, J. Quality Technol.,2003, 35: 355–366.

cusum control charts based on likelihood ratio for preliminary...

Documents