the orthogonalization of undesigned experiments

This article was downloaded by: [New York University]On: 15 October 2014, At: 10:49Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

TechnometricsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/utch20

The Orthogonalization of Undesigned ExperimentsOtto Dykstra aa American Cyanamid CompanyPublished online: 30 Apr 2012.

To cite this article: Otto Dykstra (1966) The Orthogonalization of Undesigned Experiments, Technometrics, 8:2, 279-290

To link to this article: http://dx.doi.org/10.1080/00401706.1966.10490347

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purposeof the Content. Any opinions and views expressed in this publication are the opinions and views of theauthors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should notbe relied upon and should be independently verified with primary sources of information. Taylor and Francisshall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, andother liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relationto or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/utch20

http://dx.doi.org/10.1080/00401706.1966.10490347

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

TECHNOMETRICS VOL. 8, No. 2 MAY 1966

The Orthogonalization of Undesigned Experiments”

OTTO DYKSTRA,JR. American Cyanamid Company

Methods are proposed for specifying additional runs to be included with the data of a non-orthogonal, undesigned experiment, in such a way as to increase the statistical power of the total experiment and to decrease the statistical dependence among the controllable variables. The methods are illustrated.

In the analysis of the results of an undesigned experiment one usually would use the general linear regression model

(1) Ya = P + 2 rC,(xis - 3,) + em , ff = 1, . ..n.

where there are p variables and n runs, and where the e, are assumed to be independent and normally distributed with mean 0 and variance g2.

If X is the n by p matrix of deviations z;, - 3i and if Y is the vector of n observations, then the vector of least squares coefficients is obtained by

(2) /3 = (X'X)~'X' Y.

In most designed experiments the matrix X’X is diagonal or can be made diagonal, and the estimation and interpretation of the coefficients 8, is straight- forward, because of the balance inherent with designed experiments.

Even if the variables on the right hand side of (1) are independently controllable, an undesigned experiment is apt to be unbalanced to the point where most statisticians would be wary of even attempting an analysis.

One reason for trepidation is the poor sensitivity of an unbalanced experiment. If aii is the i-th diagona1 element of the matrix A = X’X and cii the i-th diagonal element of C = (X/X)-‘, then the squared multiple correlation of variable x, with the other x variables, rcprescnted simply as RT , is

(3) R? = 1 - l/aiicSi .

If this value is high (approaching 1, say) , then the vurkkle xi is virtually, if not completely, useless as a predictor. Furthermore, the variance of the regression coefficient, given by

(4) V(& = cip2 = a2/u,,(l - R:), Received Fcbrllary 19G.5; rcvistc 1 September 1965. * An earlier version of the minimum regression method proposed here was developed while

the author was with the Technical Center of Gcncral Foods Corp:)ration. The minimum regression method was refined and tile first draft of this paper was written whi!e the author was with Ford Motor Company.

279

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014

280 TECHNOMETRICS, VOL. 8, NO. 2, MAY 1966

is at a minimum when RT = 0. The test of the null hypothesis H, : pi = 0 could have low power if Rg is high.

In addition to the poor sensitivity of unbalanced experiments, there is also the problem of interpretation of the coefficients. If two variables are completely confounded, the inverse of X’X does not exist, so that the regression coefficients cannot be determined. If one of these variables is eliminated from the analysis, and if an effect is found for the other variable, it is impossible to determine to what extent each variable contributes to the effect which was found. A proper analysis would reveal the confounding between the two variables and should lead to additional experimentation to isolate the effects of the two variables. A worse situation could exist: the variables might have equal effects of opposite signs, so that no effect would appear in the analysis.

Hoer1 has stated that, when two variables are highly, but not completely confounded, there is a tendency for their least squares coefficients to be un- realistically large and possibly to have incorrect signs. Hoerl’s approach (1962) is to abandon the least squares criterion. Another approach, unpublished but mentioned by Box (1959), is “PARC” analysis, which is somewhat like step-wise regression.

The methods proposed in this paper attack the problem by specifying additional runs which will combine most effectively to reduce Var (8,) and/or Rf . Daniel (1958) has proposed a method for augmenting incomplete two- level factorials so that the net result is one of the fractional factorials.

THE SOLUTION IN p DIMENSIONS

One approach to balancing an undesigned expcrimcnt is to add one run at a time, each run being optimum according to some criterion. While a solution can be found using this approach, there is no guarantee that there will be orthogonal blocking, attained only when fi for the additional runs equals zi for the existing data, for all i. A more tractable solution is obtained when pairs of additional runs are specified. There is also the question of the criterion to be used in selecting the “optimum” pair of runs. One criterion is to determine the pair of additional runs so that the conditions for variable xi , given the conditions for the other variables, minimize the regression sum of squares of 5; with the other variables, for all i. This approach minimizes Var (fii) and R: indirectly. A second criterion is to select the pair of additional runs so that the residual sum of squares is maximized (which implies maximizing the total sum of squares while minimizin, v the regression sum of squares), resulting in direct minimization of Var (p,). A third criterion is to minimize R: directly, for all i. Solutions have been found for all three criteria.

Run n + 1 will be represented by the vector V and run n + 2 by the vector W. Initial values v, and wi must be selected as elements of V and W, which will subsequently be modified by iteration. In order that there be orthogonal blocking WC sclcct the element,s of the vectors V and W such that

(5) (Vi + WJP = zi , i = 1,2, ... ,p,

where 5; is based on the n existing runs. Initial values could be chosen so that

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014

ORTHOGONALIZATION OF UNDESIGNED EXPERIMENTS 281

03 vi = zi + R;&,,/(n - 1)

wi czz zi - Rf l/a,& - 1)

This rule ensures that some of the differences di = vi - wi be non-zero, which is necessary for each criterion.

In the following text we will endeavor to improve the values vk and wk , where k is one of the values 1 to p, according to one to the criteria. In the frame- work of multiple regression we will treat the variable xk as the “dependent” variable, and we will treat the other 2 variables as the “independent” variables, in order to obtain the sum of squares for the regression of xk on the other CC variables. We let A be the p - 1 by p - 1 matrix of the sum of squares and cross-products among the levels of the ‘(independent” variables, for n + 2 runs, so that the elements of A are

(7) U,i = V;Vi + W;Wj + C XiXi

- (Vi + wi + c XJ(Vi + wi + c Xi>/@ + 8, i, j = 1, 2, . . . , p; i, i # k,

TABLE 1 Illustrative Data (Constructed)

Variable

Run 1 2 3 4 5

1 1 2 7 4 2 2 1 3 6 6 4 3 1 4 5 2 2 4 2 1 4 3 2 5 2 2 7 5 3 6 2 3 6 7 5 7 2 4 5 1 1

8 3 1 4 5 4 9 3 2 3 7 7

10 3 3 7 3 3 11 3 4 6 4 3 12 3 5 5 1 1

13 4 1 4 6 6 14 4 2 5 4 4 15 4 3 6 2 2 16 4 5 2 4 4 17 4 6 3 2 3 18 4 7 4 6 5 19 5 3 3 7 6 20 5 4 2 4 5

21 5 5 1 3 4 22 5 6 5 1 3 23 5 7 4 5 5 24 6 4 3 7 7 25 6 5 2 3 r

26 6 6 1 5 i

27 6 7 4 1 2 28 7 4 3 4 6 29 7 5 2 6 6 30 7 6 1 2 4

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014


TABLE 2 Correlutions and Addifional Runs for the Minimum Regression Criterion

Simple Correlations

Totnl Xumber of Rims

30 32 34 36 35 40

n2 0.5417 0.5361 0.4314 0.3772 0.2952 0.2762 rl3 -0.7292 -0.7347 -0.4655 -0.4795 -0.48SO -0.2'979 I.14 0.0000 0.0532 0.0489 0.1070 0.0571 0.0534 r16 0.5173 0.4061 0.3733 0 2358 0.2480 0.2747 733 -0.4167 -0.4124 -0.4314 -0.4051 -0.3193 -0.2075 734 -0.3279 -0.3043 -0.3012 -0.3235 -0.1474 -0.1474 r25 0.0000 0.0000 0.0000 0.0543 0.0000 0.0000

924 -0.0772 -0.1240 -0.1140 -0.1411 -0.0877 -0.0818 r35 -0.52s3 -0.4160 -0.3324 -0.3000 -0.3115 -0.2427 r45 0.7048 0.5160 0.5160 0.3699 0.3020 0.2096

Multiple Correlations

R: R2 2 R2 3 1:: R;

0.7754 0.6334 0.3543 0.2934 0.2738 0.1775 0.4156 0.4097 0 38”8 I 0.3650 0.1651 0.1642 0.6246 0.5615 0.3638 0.3481 0.3177 0.1767 0.8797 0.3611 0.3611 0.3014 0.1151 0.1148 0.9180 0.4285 0.42GO 0.2101 O.lS97 0. 15‘1-1

Standard Deviation of Regression

Coefficients (/u)

Conditions for Additional Rims

Variable/Run 1 2 3 4 5

.2154 .1668 .1155 ,106s

.1335 .I325 .1286 .12x

.1666 .1525 .1164 .1140

.2725 .1097 .1007 ,101s

.3766 .1297 .1295 .I019

31 32 33 34 35 36 37 38

3 5 7 1 2 G 3 5 4 4 3 5 5 3 7 1 5 3 71 5 3 5 3 1 7 4 4 2 6 71 7 1 4 4 7 1 3 5

.1045

.lOOS

.1105 OS51

.0995

30 40

71 4 4 7 1 4 4 5 3

.0919

.1007

.003Y

.0851

.09SG

-

where the summntions extend from 1 to n. It is assumed that A is positive definite, so thnt its inverse exist’s. We let B be the vector of the sums of cross- products of the “dependent” variable xk wit,11 the ‘,independent” variables, so thnt the p - 1 elements of 13 zlre

(8) b, = v,v,, + w;wk + c X,Xk

-(vi+wi+ ~x;)(vk+Wk+ ~rJ/(n+2>, i= 1,2;.. ,p; iflc.

The regression sum of squares for vari:lble k with the other vnrinbles then is S, = B’A-‘Ii’, the residual sum c;f syuares is E,: = aiik - B’il-‘B, where

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014

ORTHOGONALlZATlON OF UNDESIGNED EXPERIMENTS 283

ukk is the sum of squares for variable k, and the multiple correlation of variable k with the ot,her variables is R: = B’A-‘B/a,, .

The following sub-s&ions develop the method for specifying additional pairs of runs according to each of the criteria. Each criterion is illustrated, using the constructed data given in Table 1. In the 30 runs each of the five variables range from 1 through 7, and bounds of 1 and 7 were specified for each variable. The additional runs and certain measures associated with the data are given in Tables 2, 3, and 4.

TABLE 3 Correlations and Additional Runs for the Maximum Residual Criterion

Simple Correlations 30

Total Number of Runs -

32 34 36 3s 40

7-12 .5417 .2982

r13 - .72X - .f719 7.14 .oooo .1479 rl6 .5173 .2663 r23 - .4167 - .1930 r24 - .3279 - .4271 r25 0000 .1653 r34 - .0772 - .2136 7.36 - .5283 - .2755 r45 .7948 .5160


.7754 .6233 .4893 lS20 .I832 ,100s

.4156 .4589 .2596 .1051 .0096 .0292

.6246 .6066 .5459 .1634 .1772 11?88

.8797 .5642 .4182 .0594 .0626 0442

.9180 .4494 .3227 .1043 .0667 .0645


Coefficients (/u)

d\/1/El

d1/E1

dijz .__

.2154 .1526 ,121s .0903 .0854 .0773

.1335 .1273 .1012 .0863 .0775 .0744

.1666 .1493 .1292 .0593 .0851 .0786

.2725 .1329 ,107s OS00 .0761 .0720

.3766 .1322 .llOO .0593 .0824 .0779 dl/Ea d\/1/ES

Conditions for Additional Runs

Variable/Run

1 2 3 4 5

31 32

1 7 7 1 7 1 1 7 7 1

.1212 .2267 - .5303 - .3467

.oooo .1141

.3iO4 .2001 - .3030 - .I467 - ,2433 - .1014

.oooo --.1242 - .3148 - 1648 - .0946 - .2070

.3126 .1574

.0952 - ,010s - .4167 - 2796

0000 .0929 .0675 - .0387

- ,023s --.1183 .0114 - ,0525 0000 .0995

- .0455 .0516 - .0737 -.I653

.2463 .1273

33 34 35 36 37 38 39 40

7 1 7 1 7 1 7 1 1 7 7 1 1 7 1 7 7 1 7 1 1 7 7 1 1 7 7 1 1 7 7 1 7 1 1 7 1 7 1 7

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014


TABLE 4 Correlations and Additional Runs for the Minzmum Correlation Criterion

Simple Correlations


R:

;i

Ri

RZ


Coefficients (/u)

GE eE

m m d\/1/Es

Conditions for Additional Runs

Variable/Run

1 2 3 4 5

Total Number of Runs

30 32 34 36 38 40

.5417 .4971 - 7292 - .7190

0000 .1479 5173

-:4167 .2663

- .4124 - ,3279 - .3043

0000 0000 - ,0772 - .1240 - 5283 - .4160

.7948 .5160

.3414 .4287 - .4687 - ,282l

0000 .1141 .3704 .3006

- .4734 - .2659 - .1773 - .0281 - .1065 - 1463 - .2442 - .0939 - .2017 - .2327

.3126 .2509

.2608 .1688 - .3367 -.1971

.1078 .0679 ,1489 .1415

- .1560 - ,2209 - ,0262 -.1220

0000 0000 - .0912 - ,122o - ,126s -.I194

.2345 .2331

.775*

.4156 6246

.8797

.9180

.5964 .3642 ,325l .1741 .0708

.4019 .3556 3076 .0779 .0675

.5877 4296 .1537 .1266 .0929

.4061 .2356 ,065s .0646 .0642

.4007 .2731 .2576 .0763 ,0762

.2154 .1474 .1092

.1335 .1320 .1222

.1666 .1573 .1229

.2725 .1138 .0940

3766 .1267 .I062

.0994 .0849 .0761

.1088 .0880 .0851

.0939 .0898 .0830

.0803 .0803 ,079s

.1042 .0873 .0873

31 32 33 34 35 36 37 38 39 40

1 7 7 1 7 1 7 1 7 1 4 4 2 6 7 1 1 7 2 6 5 3 7 1 7 1 2 6 7 1 1 7 1 7 7 1 4 4 3 5 7 1 7 1 3 5 1 7 4 4

Criterion I-Minimize Regression Sum of &pares

Since all values in (7) and (8) are either known or have been chosen, the regression sum of squares for variable k on the other variables can be changed only by changing vk and wk . A value c is added to vk and subtracted from w,+ , so that the average of vk and wk is not changed. The vector B becomes a vector B* with elements

b: = b< f C(V< - Wi) = bi + cdi ,

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014

ORTHOGONALIZATION OF UNDESIGNED EXPERIMENTS 285

where the di are elements of a vector D. The regression sum of squares becomes

(10) S, = (B*)‘A-‘B* = (B + cD)‘A-‘(B + CD).

Sk is minimized by setting dXk/dc = BB’A-‘D + 2cD’A-‘D to zero, which gives

(11) c = -B’A-’ D/D’A-’ D

Since d2Sk/dc2 = 2D’A-lD, which is positive, c does minimize X.

While vk + C and Wk - c minimize the regression sum of squares for variable k on the other variables, the solution is dependent on the other values vi and wi . Since we want to minimize the regression sum of squares for each variable on the other variables, it is necessary to apply (11) to each variable k = 1, . . . , p, and to repeat the entire process until the elements in V and W do not change. In order that the conditions given by the new runs be in the realm of practicality, it is advisable to impose lower and upper bounds on the vi and wi . A secondary benefit of such limitation is the hastening of convergence.

Additional pairs of runs may be determined by expanding the data matrix to n + 2 runs, i.e., acting as if the conditions specified by the vectors V and W had been run, and then finding new vectors V and W.

The minimum regression criterion is illustrated in Table 2. The limiting values 1 and 7 are obtained for the variable with the largest RT for each of the five pairs of additional runs. The limiting values also are obtained for the variable with the second largest RY for three pairs. In runs 35 and 36 the limiting values appear only with variable 5. Here the multiple correlations for the other four variables are about equal. In runs 37 and 38 limiting values are observed for the variable with the third largest multiple correlation. We also note that where the conditions for two variables are at their extremes, the same variables have the largest pair-wise correlation.

Criterion 2-Maximize Residual Sum of Squares

We follow the earlier approach of adding a value c to vk and subtracting c from wk . The total sum of squares for variable k becomes

(159 azk = akk + 2c(vk - wk) + 2c2,

where akk is given by (7) with i = j = k. The residual sum of squares is (12) - (10) or

(13) E, = akk + 2c(vk - WJ + 2c2 - (B + cD)‘A-‘(B + CD).

The first and second derivatives of E with respect to c are

(14) clE,/dc = 2(v, - Wk - B’A-‘D) + a(2 - D’A-‘D)

and

(15) d2E,/dc2 = 2(2 - D’A-‘D).

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014


It can be proved (Appendix A) that (15) is positive, so that the value of c obtained by equating (14) to zero leads to minimum residual. The desired solution of c is in the opposite direction, so that

(16) c = [B’A-‘D - (vk - w,)]/[D’A-‘D - 21

leads to maximum residual sum of squares. This, of course, forces the additional runs to the corners of the experimental region.

The maximum residual criterion is illustrated in Table 3. We see that every additional run consists of combinations of the bounds imposed on the variables. An additional study showed that the additional pairs of runs are in the corners of the hypercube where the variance of a predicted response is the highest, i.e., where the information, c”/V(Q), is lowest.

Criterion S-Miminize Multiple Correlation

The solution for two variables is covered in Appendix B. For more than two variables, we again determine a value c to add to vk and subtract from wk . Using (10) and (12) the multiple correlation of variable k with the other variables becomes

(17) R,2 = (B + O’A-‘(B + CD)

akk + 2c(v, - w,) + 2~”

Setting dR:/dc to zero yields

(18) c’[(v, - w,)D’A-‘D - 2B’A-‘D] + c[a,,D’A-‘D - 2B’A-‘B]

+ [a,,B’A-‘D - (vk - w,)B’A-‘B] = 0.

One of the two roots of (18) gives the value c to minimize the multiple correlation. Since the change c is intended to reduce Ri , we would use each root in (17) to determine which root causes (17) to become less than B’A-‘B/a,, .

The minimum multiple correlation criterion is illustrated in Table 4. We note that the additional pairs of runs are more extreme than those given by the minimum regression criterion (Table 2), but they are not so extreme as those given by the maximum residual criterion (Table 3). One should expect thatrthe minimum multiple correlation criterion should eventually lead to a completely orthogonal set of data.

UNIQUENESS OF THE SOLUTION When the computer program for the minimum regression criterion was

being developed, it was found that pairs of runs different from those in Table 2 were occasionally specified. In each case the differences first appeared for runs 35 and 36, and they seemed to be due more to round-off rules Can the initial settings. The three diffcrcnt solutions for run 35 are documented in Table 5. In each case iterations continued until 110 changes could be made, and the program was instructed to round the final results to whole numbers. The computational rules were:

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014

ORTHOGONALIZATION OF UNDESIGNED EXPERIMENTS 287 TABLE 5

Three Different Solutions for Run 35, Minimum Regression Criterion

Initial Values Method 1 Method 2 htethod 3

Xl 4.6643 4.6643 4.5021 X2 4.6596 4.6596 4.5861 x3 4.6822 4.6822 4.5295 X4 4.7167 4.7167 4.5216 X5 7.0000 7.0000 4.7289

Results of First Iteration

Xl 1.0000 1.0000 1.0000 X2 6.6340 6.6000 7.0000 X3 5.2510 5.2000 5.1000 24 3.4220 3.4000 4.8000 X5 7.0000 7.0000 5.2000

Final Values

Xl 2 3 3 x2 5 4 7 X3 5 5 7 X4 2 1 5 X5 7 7 5

No. of Iterations 5 10 16 Mult. Corr. (N = 36)

R? 0.2934 0.3317 0.3375 Ri 0.3650 0.3827 0.1835

0.3481 0.3560 0.2550 ;I 0.3014 0.2285 R%

0.3462 0.2101 0.1993 0.4041

1. Xtarting values: The initial values are given by (6) for Methods I and 2, except that the variable (CC,) with the largest multiple correlation was set to its bounds; the starting values for Method 3 were (1 f Rf)& for all variables.

2. Biassing the results: The initial Vi (run 35) were all above and the initial wi (run 36) were an equal amount lower than their respective averages. While it is necessary that the differences, vi - wj , be nonzero, the direction of the differences is arbitrary. In an attempt to overcome the arbitrariness of the initial values, the variable with the largest multiple correlation (2,) was not iterated on in the first iteration cycle. This was done in Methods 1 and 2, but not in Method 3.

3. Round-08: In Method 1 three extra digits (i.e., three decimals) were allowed in the first, iteration cycle, two in the second, one in the third, and no extra digits thereafter. In Methods 2 and 3 all iterations allowed one extra

digit.

For all three methods the results were comparable after 40 runs. For this reason the author perfers the procedure which minimizes computer time, here identified as Method 1. Furthermore, the author advocates performing the

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014


TABLE 6 Comparison of the Three Criteria: Average Ratio of the Multiple Correlations

and Average Ratio of the Standard Deviations of the Regression Coejkients (Base = SO Runs)

A. Average Ratio of the Multiple Correlations


Criterion 32 34 36 38 40

Minimum regression .7159 .5672 .4771 .3193 .2477 Maximum residual .8020 .5913 .1861 .1374 .1054 Minimum correlation .7150 .5157 .3522 .1542 .1117

B. Average Ratio of the Standard Deviations of the Regression Coefficients


Criterion 32 34 36 38 40

Minimum regression .6863 .5889 .5529 .4962 .4636 Maximum residual .6794 .5573 .4265 .3972 .3718 Minimum correlation .6743 .5574 .4823 .4238 .4027

iterations according to the rank order of the R: , largest to smallest. This seems to impose less restriction on the variables with the larger multiple correlations.

COMPARISON OF THE THREE CRITERIA

Table 6, based on Tables 2, 3, and 4, provides a means for comparing the effectiveness of the three criteria in reducing Rq and Var (Bi). The minimum regression criterion, which indirectly minimizes Rf and Var (bi), is readily seen to be inferior to the minimum correlation criterion. As expected, the maximum residual criterion reduced the standard deviations of the regression coefficients proportionately more than did the minimum correlation criterion, although it required 34 total runs to realize this expectation. It was surprising (to the author, at least) that the maximum residual criterion, after 36 runs, also reduced the multiple correlations more than did the minimum correlation criterion. One should not expect this apparent superiority of the maximum residual criterion to hold indefinitely, since the minimum correlation criterion, but not the maximum residual criterion, will eventually lead to a completely orthogonal experimental plan. This suggests a procedure in which some additional runs are specified according to the maximum residual criterion and then some more runs are specified according to the minimum correlation criterion.

ACKNOWLEDGEMENT

The author is deeply indebted to Dr. John W. Gormnn for his very constructive criticism and useful recommendations, which resulted in substantial improve-

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014

ORTHOGONALlZATlON OF UNDESIGNED EXPERIMENTS 289

ment of the paper. Any faults and errors in this paper, of course, remain the responsibility of the author.

The author also wishes to draw attention to a paper in the February 1966 issue (Gorman, J. W. and Toman, R. J., Selection of variables for fitting equations to data. Technometrics, 8, 27-51), which illustrates the most recent development in the analysis of unbalanced experiments.

APPENDICES

Appendix A. Proof That the Residual Sum of Squares Attains a Minimum.

In the discussion of the maximum residual criterion it was claimed that setting equation (14) to zero and solving for c leads to a minimum residual, rather than a maximum residual. In this Appendix we prove that D’A-‘D < 2, in order to justify our claim that (15) is positive.

For convenience we subtract their respective averages from the elements of the vectors V and W and the matrix A. Then di = vi - wi , as before, wi = -vi , from (5), and

(Al) A = X’X + VV’ + WW’,

where X’X is positive definite. We now define a matrix F, where

and the 2 by 2 matrix

(A3) R = F(X’X)-‘F’ = a

the extreme right-hand side resulting from the relationships between V, W, and F.

Adapting Hackett’s result (1950), we note that

(A4) A-’ = (X’X)-’ - (X’X)-‘F’(1 + R)-‘F(X’X)-‘,

where I is a 2 by 2 identity matrix. From (A3) and (A4),

(A5) FA-‘F’ = R[I - (I + R)-‘RI = *

so that D’A-‘D = 4a/(l + 2a) < 2.

Appendix B. A Minimum Correlation Solution in Two Dimensions.

The correlation coefficient between x1 and xZ is

(Bl) rlz = c (Xl - &)(x2 - *J/d c (Xl - 3x c (x2 - CE,)“,

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014


where the summations extend over the n runs. This measure of the statistical dependence between x1 and x2 must be set to zero by adding two more runs. Letting

we see that the corre!ation coefficient based on n + 2 runs equals zero when

2ab + c (XI - $(xz - 3,) = 0, which, by rearranging terms and using (Bl), becomes

(B4)

One may choose a equal to any value but zero and solve for b, so that there is an infinite number of solutions. However, a unique solution may be obtained by requiring that

(B5) c (XC xx = c (+p’- xJ2 ’ which will give a pair of runs which are at a minimum distance from the center of the data, in standardized units.

REFERENCES

Box, G. E. P., 1959. Discussion of the papers of Messrs. Sstterthwaite and Budne. Techno- metrics, 1, 174-180.

DANIEL, C., 1958. On varying one factor at a time. Biometrics, 430-431. GORMAN, J. W. and TOMAN, R. J., 1966. Select,ion of variables for fitting equations to data.

Technomelrics 8, 27-51. HOERL, A. E., 1962. The application of ridge analysis to regression problems. Chem. and &rg.

Prog., 68, 3, 54-59. PLACKETT, R. L., 1950. Some theorems in least squares. Bzometrika, 149-157.

Dow

nloa

ded

by [

New

Yor

k U

nive

rsity

] at

10:

49 1

5 O

ctob

er 2

014

the orthogonalization of undesigned experiments

Documents