classification of three-level strength-3 arrays

Contents lists available at SciVerse ScienceDirect

Journal of Statistical Planning and Inference

Journal of Statistical Planning and Inference 142 (2012) 794–809

0378-37

doi:10.1

n Corr

E-m

journal homepage: www.elsevier.com/locate/jspi

Classification of three-level strength-3 arrays

Bagus Sartono a, Peter Goos a,b, Eric D. Schoen a,c,n

a University of Antwerp, Belgiumb Erasmus University Rotterdam, Netherlandsc TNO Research Group Quality and Safety, Zeist, Netherlands

a r t i c l e i n f o

Article history:

Received 25 May 2010

Received in revised form

25 June 2011

Accepted 26 September 2011Available online 17 October 2011

Keywords:

Experimental design

Resolution IV

Non-regular design

Orthogonal array

Regular design

58/$ - see front matter & 2011 Elsevier B.V. A

016/j.jspi.2011.09.014

esponding author at: University of Antwerp,

ail address: [email protected] (E.D. Schoen

a b s t r a c t

Generalized aberration (GA) is one of the most frequently used criteria to quantify the

suitability of an orthogonal array (OA) to be used as an experimental design. The two

main motivations for GA are that it quantifies bias in a main-effects only model and that

it is a good surrogate for estimation efficiencies of models with all the main effects and

some two-factor interaction components. We demonstrate that these motivations are

not appropriate for three-level OAs of strength 3 and we propose a direct classification

with other criteria instead. To illustrate, we classified complete series of three-level

strength-3 OAs with 27, 54 and 81 runs using the GA criterion, the rank of the matrix

with two-factor interaction contrasts, the estimation efficiency of two-factor interac-

tions, the projection estimation capacity, and a new model robustness criterion. For all

of the series, we provide a list of admissible designs according to these criteria.

& 2011 Elsevier B.V. All rights reserved.

1. Introduction

Scientific experiments are frequently conducted according to an orthogonal array (OA). An OA is an N � n matrix ofsymbols, whose rows correspond to the N runs of the experiment and whose columns correspond to the n factors studiedin the experiment. The number of different symbols in a given column of an OA corresponds to the number of levels of thecorresponding factor. Every OA has a certain strength t, which means that, for any t columns, every t-tuple of levelsappears equally often in the array (Rao, 1947). Arrays that have different numbers of levels in their n columns, say,s1, . . . ,sn, are referred to as mixed-level arrays. If all columns of an OA have the same number of levels, say s, then the arrayis called a symmetrical or pure-level array and denoted by OA(N; sn; t). In this article, we study pure-level arrays, and weassume without loss of generality that the levels of an s-level factor are represented by 0, . . . ,s�1. We refer to Hedayatet al. (1999) for a comprehensive account on properties and explicit constructions of orthogonal arrays.

The numbers t, N, n and s are the parameters of an array. For any specific set of parameters, there may be many arrayswhich can be partitioned in so-called isomorphism classes. All arrays within one isomorphism class can be obtained fromeach other by a sequence of row permutations, column permutations and/or level permutations. These arrays aremathematically and statistically equivalent. Therefore, for statistical purposes, it suffices to study only one instance ofevery isomorphism class. This explains why much research has been done to determine a minimum complete set of OAsfor given sets of the parameters t, N, n and s. Such a set has a single representative from each isomorphism class.

Minimum complete sets of regular two-level designs of up to 64 runs are given by Chen et al. (1993), whereasminimum complete sets of regular three-level designs of up to 729 runs are given in Xu (2005). Regular designs form

ll rights reserved.

Belgium. Tel.: þ32 3 2204364.

).

www.elsevier.com/locate/jspi

www.elsevier.com/locate/jspi

dx.doi.org/10.1016/j.jspi.2011.09.014

mailto:[email protected]

dx.doi.org/10.1016/j.jspi.2011.09.014

B. Sartono et al. / Journal of Statistical Planning and Inference 142 (2012) 794–809 795

a subset of the complete set of OAs. Designs from this subset can be constructed from a set of basic s-level factors, with s aprime number. The settings of additional factors are calculated by addition modulo s of two or more basic factors specifiedin so-called generators (see, e.g. Wu and Hamada, 2000). However, only a small fraction of all OAs belong to the class ofregular designs. Constructing minimum complete sets of OAs is therefore substantially more challenging than constructingminimum complete sets of regular designs. The generation of minimum complete sets is discussed by Sun et al. (2002) forpure two-level arrays of strength 2, Bulutoglu and Margot (2008) for pure-level OAs in general, and Schoen et al. (2010) forgeneral (pure-level and mixed-level) OAs.

Once a minimum complete set of OAs has been obtained, it is important to rank its non-isomorphic OAs from best toworst. By far the most widely applied criterion to rank non-isomorphic OAs is the generalized aberration (GA) criterionproposed by Xu and Wu (2001) and Ma and Fang (2001). The GA criterion reduces to the G2 aberration criterion proposedby Tang and Deng (1999) for two-level OAs, which, in turn, reduces to the aberration criterion for regular two-level designsdeveloped by Fries and Hunter (1980).

The GA criterion is based on the entries of the generalized word length vector W ¼ ðA3,A4,A5, . . . ,AnÞ. The elements Ai ofthis vector are called generalized word counts of length i. Roughly speaking, an entry Ap of the generalized word lengthpattern measures the extent to which effects involving q of the factors (with qop) are confounded with effects involvingp�q of the factors. Small Ap values are generally desirable, especially for p as low as 3 and 4, as this implies that there islittle aliasing between main effects and two-factor interaction effects as well as among the two-factor interactions.An array R1 is said to have less aberration than an array R2 if there is a p such that ApðR1ÞoApðR2Þ and AiðR1Þ ¼ AiðR2Þ fori¼ 3, . . . ,p�1. If there is no design with less aberration than R1, then R1 is a minimum generalized aberration (MGA) design.

In this paper, we address the issue of classifying three-level OAs of strength 3. These designs have four components foreach two-factor interaction. While we believe that the GA criterion is very useful for ranking strength-2 OAs, it might beless suitable for classifying three-level strength-3 arrays. As an early example, the seven MGA arrays of the type OA(81; 37;3) have A4 ¼ 10. One of these permits estimation of models with four interactions (16 components) with an averageD-efficiency of 86.1%. For 14 interactions (56 components), this efficiency drops to 0.7%. By contrast, there is a design withA4¼11.36 that has average efficiencies of 92.9% and 56.5% for models with four and 14 interactions, respectively.

The goal of this paper is, first, to show that the GA criterion is indeed less suitable for the classification of three-levelstrength-3 arrays for statistical purposes, and, second, to classify these arrays with more appropriate criteria. We study inparticular the rank of the matrix with two-factor interaction contrasts (Cheng et al., 2008), the D-efficiency of models withk two-factor interactions (Ai et al., 2005), the estimation capacity (Cheng et al., 1999), the projection estimation capacity(Loeppky et al., 2007), and a new model robustness criterion. Classification using these criteria is illustrated usingthe complete series of OAs of the type OA(N; 3n; 3), with Nr81. The single array of the type OA(27; 34; 3) is a well-knownregular array. The arrays with N¼54 and n¼5 were first enumerated by Hedayat et al. (1997), whereas those with N¼54and n¼4 were catalogued by Schoen et al. (2010). To the best of our knowledge, the OA(81; 3n; 3) family of arrays that westudy here is new.

The rest of the paper is structured as follows. In Section 2, we briefly discuss the enumeration of minimum completesets of orthogonal arrays. In Section 3, we define the classification criteria that we use, and we review the motivation forusing the GA criterion. Section 4 presents an empirical study of the relation between the various criteria in theclassification of the three-level OAs using the series of 486 arrays of the type OA(81; 37; 3) as an example. Next, we providedetails of the best designs in all the series of OAs that we studied. Finally, there is a brief general discussion in Section 6.

2. Enumeration algorithm

For the generation of the designs studied here, we used the minimum complete set (MCS) algorithm of Schoen et al.(2010). In this section, we illustrate the algorithm’s various stages with the construction of a minimum complete set ofdesigns of the type OA(9; 33; 2). The MCS algorithm generates a single representative for each isomorphism class of thedesigns with a given set of parameters. This representative is the lexicographically smallest member of its class.

To determine whether a design D1 is lexicographically smaller than a design D2, we compare the designs in a column-wise fashion. First, we identify the first column in which the two designs differ. Next, within that column, we identify thefirst entry in which the designs differ. If D1 has a smaller value for this entry than D2, then D1 is lexicographically smallerthan D2. If D1 has a larger value for this entry, then it is lexicographically larger than D2. A design D1 is lexicographicallyminimal within its isomorphism class if there is no design in that class that is lexicographically smaller than D1.An example of a lexicographically minimal design is the 32 factorial design shown in the left panel of Table 1. It can easilybe seen that no other sorting of the rows of this design results in a lexicographically smaller array, so that it islexicographically minimal in the class of OA(9;32;2) designs.

The MCS algorithm involves an element-wise extension including a check of the compatibility with the desiredstrength, and a test for lexicographic minimality. The extension starts from a so-called root node R. The array of the typeOAð9;32;2Þ shown in the table is such a root node.

When extending the root note with an extra factor, the first element of the third factor is taken to be 0. Otherwise, somepermutation of the third factor’s levels results in an array that is lexicographically smaller than the original extended array.For the third factor’s second level, the options are a 0 and a 1. The level 2 is not considered, because, again, a permutationof the third factor’s levels exists that results in a lexicographically smaller array. The second panel of Table 1 shows the

Table 1

Construction of arrays of the type OAð9;33;3Þ.

Root False step Array 1 Array 2

0 0 0 0 0 0 0 0 0 0 0

0 1 0 1 0 0 1 1 0 1 1

0 2 0 2 0 2 2 0 2 2

1 0 1 0 1 0 1 1 0 2

1 1 1 1 1 1 2 1 1 0

1 2 1 2 1 2 0 1 2 1

2 0 2 0 2 0 2 2 0 1

2 1 2 1 2 1 0 2 1 2

2 2 2 2 2 2 1 2 2 0

B. Sartono et al. / Journal of Statistical Planning and Inference 142 (2012) 794–809796

extension that results from choosing the first option, i.e. level 0 for the second element in the third column. This choice isnot compatible with the requirement that we want to obtain an orthogonal array of strength 2. As a matter of fact, thischoice results in a design where the (0, 0) pair of factor levels appears twice for the first and third factor. Since there arenine rows and nine different level combinations of two factors, each pair can only occur once to have an orthogonal arrayof strength 2. As a result, a lexicographically minimal representative can only have a 0 as the first level of the third factorand a 1 as the second level of the third factor.

The first element in the third column of the extended design that can have more than one value is the fourth. Thatelement can either take the value 1 or the value 2. Each of these choices is compatible with the requirement that the arrayhas to be of strength 2. For each of the two possibilities for the fourth element, there is only one way in which the thirdcolumn can be completed so that the resulting design is a strength-2 orthogonal array. The resulting arrays are given asArray 1 and Array 2 in Table 1.

Upon completion of a column, the lexicographic test is applied. This test shows that Array 1 cannot be turned into alexicographically smaller array. Array 2 can be turned into Array 1 by switching the levels 1 and 2 of the first factor andsorting of the rows in ascending order of the first factor’s level. Therefore, Array 2 is discarded, and we conclude that thereis a single isomorphism class for arrays of the type OAð9;33;2Þ.

Now, suppose that we want to construct a minimum complete set of designs of the type OAð9;34;2Þ. The search usesthe single lexicographically minimum design of the type OAð9;33;2Þ, and attempts to extend this design with a fourthcolumn. Starting with a 0 and a 1 as the first and second element, respectively, there is only one possible choice for each ofthe remaining elements of the fourth column. We conclude that there is just one isomorphism class for designs of the typeOAð9;34;2Þ.

An attractive feature of the MCS algorithm is that it circumvents the use of pairwise comparisons between constructeddesigns because of its lexicographic test. This test checks whether any given design can be changed into a design that islexicographically smaller by permuting its columns and the factor levels. As soon as a permutation is found that results in alexicographically smaller design, then the design is discarded. The extension algorithm ensures that none of the lexicogra-phically minimal designs is missed, so that discarding any array that is not lexicographically minimal saves computation time.

3. Classification criteria

In this section, we give more details on the criteria we use to classify designs: the generalized aberration (GA) criterion(Xu and Wu, 2001; Ma and Fang, 2001), the rank of the matrix of two-factor interaction contrasts (Cheng et al., 2008), theestimation efficiency (Cheng et al., 2002; Ai et al., 2005), the estimation capacity (Cheng et al., 1999), the projectionestimation capacity (Loeppky et al., 2007) and a newly proposed model robustness criterion.

3.1. Generalized aberration

The GA criterion is based on the entries of the generalized word length vector W¼(A3,A4,A5, . . . ,An). For regular s-leveldesigns, the Ap values in W are s�1 times as large as the entries of the ordinary word length pattern (Xu and Wu, 2001).

For the strength-3 designs studied in this paper, there is no confounding of two-factor interactions and main effects.For this reason, A3 ¼ 0. Ranking of a series of strength-3 designs based on W means ranking the arrays in the series inascending order of A4,A5, . . . ,An. The design ranked first is an MGA design; there may be more than one such design. Weused the complete vector W for the ranking. However, A4 is the sole practically relevant component of W if, as is customaryin the literature, interactions involving three or more factors are considered negligible. For this reason, we only show theA4 value in the summaries of our classification results.

The statistical justification for using the GA criterion to classify series of general orthogonal arrays is based on twodifferent arguments. One of these involves the expected bias resulting from the omission of higher-order terms in a statisticalmodel containing the main effects. It turns out that, for pure-level designs, the MGA design sequentially minimizes the bias of


the main effects’ estimates due to interactions, starting with the bias due to two-factor interactions and ending with the biasdue to (n�1)-factor interactions (Xu and Wu, 2001).

The second argument to justify the GA criterion for classification purposes is based on the average D-efficiency acrossall possible models including all main effects and a certain number of two-factor interaction components. By an argumentdue to Cheng et al. (2002), one approach to obtain a large average D-efficiency criterion value for two-level orthogonalarrays and small numbers of interactions is to minimize a linear combination of the A3 and A4 values, where the weight forthe A3 value is much larger than that for the A4 value. Therefore, to optimize the average D-efficiency, one should firstminimize the A3 value, and then the A4 value. This argument was generalized to any type of pure-level design by Ai et al.(2005) for models involving small numbers of interaction components.

We question whether the above arguments are appropriate for three-level strength-3 designs, for the following reasons.First, interactions involving three or more factors are usually negligible. In addition, main-effect estimates in strength-3designs are not biased by the presence of two-factor interactions. Therefore, the expected-bias argument to justify the useof the GA criterion is not valid for strength-3 designs.

We also believe that the efficiency-based justification of the GA criterion is not appropriate for ranking three-levelstrength-3 arrays. This is because strength-3 designs are used in situations where a substantial number of interactioneffects are expected. For three-level factors, estimation of a two-factor interaction effect requires estimating fourinteraction components. Hence, the use of three-level strength-3 arrays implies that the interest is in many interactioncomponents. The efficiency-based argument of Cheng et al. (2002) and Ai et al. (2005) was developed assuming smallnumbers of interactions only, so that the argument is less credible in situations involving three-level strength-3 designs.

3.2. Rank of the matrix of two-factor interaction contrasts

Cheng et al. (2008) classified two-level OAs of strength 3 using the rank r of the N � nðn�1Þ=2 two-factor interactioncontrast matrix X2. For three-level OAs, this matrix has 2nðn�1Þ columns. Generally, a higher rank r makes an array moreattractive to use as an experimental design. Note, however, that a higher rank does not necessarily imply that a largernumber of two-factor interaction effects can be estimated. This is because the estimability of a two-factor interactioninvolving three-level factors requires four specific two-factor interaction components to be estimable and the rankcriterion ignores the fact that the interaction components are partitioned in clusters of four, where every clustercorresponds to a specific two-factor interaction effect. A major advantage of the rank criterion is that it has a lowcomputational cost.

3.3. Estimation efficiency

The use of design efficiencies for classifying OAs was originally proposed by Cheng et al. (2002) and Ai et al. (2005).More specifically, these authors proposed calculating average D-efficiencies across all models including all the main effectsand a certain number of two-factor interaction components. In this paper, we also consider the average D-efficienciesacross all models including all the main effects and a certain number of two-factor interaction effects. As a result, in thispaper, we use two types of efficiency vectors:

1.
The vector ðD2,D3, . . . ,Dnðn�1Þ=2Þ, where nðn�1Þ=2 is the total number of two-factor interaction effects, and Di is theaverage D-efficiency for models involving i complete interaction effects.
2.
The vector ðDn
2,Dn

3, . . . ,Dn

2nðn�1ÞÞ, where 2nðn�1Þ is the total number of two-factor interaction components, and Dn

i is theaverage D-efficiency for models involving i interaction components. Because it is computationally infeasible to calculatethe efficiencies Dn

6, . . . ,Dn

2nðn�1Þ, we restrict ourselves to ðDn

2,Dn

3, . . . ,Dn

5Þ in this paper.

Note that, for any OA of strength three, D1 ¼Dn

1 ¼ 100% because such an array has all its main effects clear from two-factorinteraction effects. We included the Dn

i efficiencies in our study to investigate their relation with the A4 value in modelswith only a few interaction components, but we do not use them for classifying OAs in this paper, because we preferfocussing on designs that permit estimation of several complete two-factor interaction effects.

We use two ways of ranking OAs in terms of the Di efficiency vector. One way involves sequentially maximizing theD2,D3, . . . ,Dnðn�1Þ=2 values, while the second way involves sequentially maximizing the Dnðn�1Þ=2,Dnðn�1Þ=2�1, . . . ,D2 values.The former approach guarantees maximum estimation efficiency of the simplest possible models, namely those involvingall main effects and few interaction effects. The latter yields a maximum estimation efficiency of the most complexmodels. We find the latter approach useful for strength-3 arrays, because such arrays are used for estimating main effectsin the presence of two-factor interactions. So, strength-3 arrays are used in case two-factor interaction effects are indeedanticipated.


3.4. Estimation capacity

Cheng et al. (1999) define the estimation capacity ECk of an array as the percentage of estimable models among allmodels including main effects and k two-factor interaction effects. We calculated the complete vector ðEC2,EC3, . . . ,ECnðn�1Þ=2Þ for all the designs. As with the estimation efficiency, we use two ways of ranking OAs in terms of estimationcapacity. The first way involves sequentially maximizing the EC2,EC3, . . . ,ECnðn�1Þ=2 values, while the second way involvessequentially maximizing the ECnðn�1Þ=2,ECnðn�1Þ=2�1, . . . ,EC2 values.

3.5. Projection estimation capacity

Loeppky et al. (2007) proposed a criterion based on the projections of an s-level array with n factors into i¼ 2;3, . . . ,qfactors, where q is the largest integer such that 1þqðs�1Þþqðq�1Þðs�1Þ2=2rN. They define the projection estimationcapacity PECi of an array as the percentage of estimable models among all possible models with all main effects and alltwo-factor interaction effects involving i of the factors. They classify designs based on the vector PEC¼ ðPEC1,PEC2, . . . ,PECqÞ, and suggest sequentially maximizing this vector either from left to right, or from right to left. When computing theprojection estimation capacity, we consider a model containing two-factor interactions estimable only if all components ofall of these interactions are estimable.

Since a strength-3 OA contains all level combinations of each set of three factors equally often, any projection onto twoor three factors involves a full factorial design and allows independent estimation of all main effects and two-factorinteraction effects. Hence, for strength-3 OAs, PEC2 ¼ PEC3 ¼ 1. Moreover, for all three-level designs with up to 81 runs,PEC7 ¼ 0. As a result, the PEC vector of interest is ðPEC4,PEC5,PEC6Þ.

3.6. Robustness

One reason to prefer an OA over another might be its robustness to missing data. The robustness criterion we use isbased on the assumption that one level of one of the n factors leads to useless results, in which case all the rows involvingthat factor level are removed from the array. The remaining 2=3 fraction of the original three-level array is an OA ofstrength 2 (Hedayat et al., 1999). With n factors at three levels, the total number of 2=3 fractions is 3n. Each of the possiblefractions has its own word length vector W, and its own A3 value.

As a measure M for the robustness of an OA, we use the maximum A3 value over all 3n fractions that can be obtained bydropping one level of one of the n factors. Arrays with a small M value are desirable, as these guarantee that the designresulting from dropping the runs corresponding to one factor’s level involves little confounding. As the M criterion is basedon the word length vector W, we expect that it will be related to the A4 value of the complete design.

4. Relations among the classification criteria

Using the criteria discussed in the previous section, we classified all arrays of the type OAðN;3n;3Þwith Nr81, i.e. of allstrength-3 three-level OAs with up to 81 runs. Table 2 shows the numbers of isomorphism classes for N¼27, 54, and 81.To the best of our knowledge, the numbers for the 81-run arrays are new to the literature. The numbers in brackets are thenumbers of non-isomorphic regular arrays among the total numbers of non-isomorphic OAs. The numbers of regulararrays were given earlier in Xu (2005).

Before reporting the best designs in each class of OAs, we compare the rankings obtained by the different criteriadescribed in the previous section. The reason we do so is that, if the criteria used for the classification are strongly related,we could simplify the classification by omitting some of the criteria. In particular, if these criteria would indeed be stronglyrelated, we could use the computationally less expensive A4 value rather than the Di or Dn

i efficiencies for classificationpurposes.

Table 2Numbers of isomorphism classes for OAs of the

type OAðN;3n;3Þ. Numbers in brackets are the

numbers of non-isomorphic regular arrays.

n N¼27 N¼54 N¼81

4 1(1) 7 (1) 32 (2)

5 – 4 (0) 17056 (2)

6 – – 1099 (2)

7 – – 486 (2)

8 – – 235 (3)

9 – – 1 (1)

10 – – 1 (1)


We have studied the relations among the criteria in the 81-run arrays with 5–8 factors, because there are manyisomorphism classes for these cases. Here, we use the results of the seven-factor OA series as an illustration, but theconclusions are valid for the other series as well.

4.1. Relation between A4 and other criteria

First, we studied the connection between, on the one hand, the Dn

i values for 2r ir5, the Di values for 2r ir7, thePECi values for 4r ir6, and the M value, and, on the other hand, the A4 value. Fig. 1 shows plots for the Dn

5, D2, PEC4, and M

values. The circles in the plots correspond to the 486 non-isomorphic seven-factor designs. The seven best designs in termsof GA have A4 ¼ 10.

In accordance with the theory developed by Ai et al. (2005), Fig. 1(a) shows a strong relation between the Dn

5 efficiencyand the A4 value. Note, however, that the range of the Dn

5 efficiency values is only 0.5%. In addition, achieving the minimumA4 value does not guarantee having the highest efficiency, because the seven MGA designs all have different efficiencies.

Fig. 1(b) shows that the best designs in terms of D2 efficiency do not have the smallest A4 value, but rather average A4

values. The difference between the overall maximum D2 efficiency and that of the worst MGA designs is about 5%. The plotfor D7 (not shown here) reveals that this difference can become as large as 70%. We conclude that it makes no sense to usethe A4 value as a surrogate for Di efficiencies for models with two or more complete interactions.

The pattern shown in Fig. 1(c) for the PEC4 value is roughly the same as that for the D2 value: the best designs in terms ofthe PEC4 value are not those with minimum A4 value. The plot for the PEC6 value (not shown) reveals that the PEC6 value iszero for the MGA designs, while it amounts to 57% for the designs that sequentially maximize the PEC vector from right to left.

The plot of the M value against the A4 value in Fig. 1(d) shows that the best design in terms of the M value is also best interms of the A4 value. This does not come as a surprise, because the M value is based on the A3 values of fractions of theOAs. However, for larger M values, designs with a given value of M can have a broad range of A4 values.

Finally, there is no clear relation between the rank r of the two-factor interaction contrast matrix and the A4 value.For instance, the overall best r value is 66, while the best MGA design in terms of the rank criterion only has an r value of 60.

4.2. Relation among other criteria than A4

We calculated the complete ðEC2, . . . ,ECnðn�1Þ=2Þ and ðD2, . . . ,Dnðn�1Þ=2Þ vectors for all the designs, and we found that thereis a correlation of 98% or higher between the ECi and Di values for all i. Therefore, all the results obtained for estimationefficiencies carry over to estimation capacities. For this reason, we do not consider estimation capacity separately.

Next, we considered the relations among the Di values, the PECi values and the rank r. There was no clear relationbetween r, on the one hand, and the Di or PECi values, on the other hand. Illustrative of the relation between the Di and PECi

values are the plots of the PEC4 value against the D6 value, and of the PEC6 value against the D15 value in Fig. 2. We matchthe D6 and D15 values with the PEC4 and PEC6 values, respectively, because their calculation is based on the same numberof interaction effects.

Fig. 2(a) shows that the set of designs with the best PEC4 value includes the design with the best D6 value, but, for givenPEC4 values, there is a wide range of D6 efficiencies. From Fig. 2(b), we conclude that designs with the best PEC6 value aredifferent from those with the best D15 value.

We conclude that, for the classification of the three-level strength-3 arrays, the Di efficiencies, the PECi values, the rankr, and the M value are useful criteria to consider, and that there is no added value in using the ECi values. The A4 value doesnot seem to be a good indicator of a design’s ability to fit various models, but we include this criterion in the classificationto verify this claim.

5. Classification of arrays of the type OA(N; 3n; 3) with Nr81

When using more than one classification criterion, it is likely that some OAs are better than others according to onecriterion but worse according to another criterion. To deal with this multiple criteria issue when characterizing blockedtwo-level arrays, Sun et al. (1997) proposed to list only admissible arrays. If, for a particular array OA1, there exists an arrayOA2 that is strictly better in terms of at least one of the criteria and that is equally good in terms of the remaining criteria,then OA1 is inadmissible. Otherwise, OA1 is admissible.

It should be pointed out that the list of admissible designs may change when a new criterion is added to the list ofclassification criteria. When two designs that are admissible under the original classification criteria are tied, theadditional criterion may break the tie in which case one of the designs becomes inadmissible. In all other circumstances,any design that is admissible under the original criteria remains admissible under the new set of criteria. Moreover, it ispossible that there is a set of originally inadmissible designs that becomes admissible under the new set of criteria.

Our list of admissible designs is based on the A4 value, the Di efficiencies, the PECi values, the rank r, and the M value. So, it isbased on a broad range of estimation-oriented and efficiency-oriented criteria. As pointed out earlier, when determining the Di

efficiencies and the PECi values, we consider only models that involve all the components of given two-factor interaction effects.There is just one isomorphism class for arrays of the type OAð27;34;3Þ. As a result, all 27-run strength-3 OAs with four

factors are isomorphic to the well-known regular 34�1IV design with generator D¼ABC. For this array, the rank r of the

Fig. 1. Relation between the A4 value and other classification criteria for arrays of the type OAð81;37;3Þ. (a) Average D-efficiency of matrices with five

two-factor interaction components plotted against the A4 value. (b) Average D-efficiency of matrices with all eight components of two two-factor

interactions plotted against the A4 value. (c) Percentage of estimable models with all main effects and all two-factor interactions involving four of the

factors plotted against the A4 value. (d) Model robustness M plotted against the A4 value.


Fig. 2. Projection estimation capacity into four or six factors plotted against estimation efficiency of six or 15 two-factor interactions for arrays of the

type OAð81;37;3Þ: (a) PEC4 against D6 and (b) PEC6 against D15.


matrix of two-factor interaction contrasts equals 18, the A4 value equals 2, and the D2 and D3 values are 80% and 40%,respectively. For iZ4,Di ¼ 0. The array’s M value is 0.50. Every subset of three factors is a full factorial design. Therefore,PEC3¼100 %.

The fact that there is only a single isomorphism class for arrays of type OAð27;34;3Þ and that all of its properties arewell known implies that, for the purpose of this paper, we can concentrate on the 54-run and 81-run series. For theseseries, we denote the arrays by N:n:q, where N is the number of runs, n is the number of factors, and q is the lexicographicranking of the lexicographically minimal representatives of each OA isomorphism class, for given N and n.

5.1. 54 runs

5.1.1. Four factors

There are seven non-isomorphic arrays of the type OAð54;34;3Þ. Their classification is presented in Table 3. For each ofthe seven isomorphism classes, the table shows the A4 value, the rank r of the two-factor interaction contrast matrix, theDi-efficiency values for i ranging from 2 to 6, the PEC4 value and the M criterion value measuring the robustness.

One of the designs of the type OAð54;34;3Þ, namely the array labeled 54.4.1, can be viewed as regular, because it is aduplicated regular 34�1

IV fractional factorial design. For this reason, the values of all the classification criteria for that designare the same as for the 27-run array.

Five of the arrays of the type OAð54;34;3Þ permit estimation of a model with all the main effects and all the two-factorinteraction effects. Therefore, these arrays have a PEC4 value of 100%. Also, the rank of the two-factor interaction contrastmatrix is 2nðn�1Þ ¼ 24 for these arrays, and their Di values are strictly positive for all i.

Array 54:4:7 is the only admissible array in the OAð54;34;3Þ series. It has the smallest A4 and M values among all arraysthat permit estimation of the full two-factor interactions model, and the largest Di efficiencies. Array 54.4.7 is given inTable A.1 of the Appendix.

5.1.2. Five factors

There are four non-isomorphic arrays of the type OAð54;35;3Þ. Their classification is presented in Table 4. For each ofthe four isomorphism classes, the table shows the A4 value, the rank r of the two-factor interaction contrast matrix, the Di

values for i ranging from 2 to 9, the PEC4 value, and the M criterion value.

Table 3

Classification of all arrays of the type OAð54;34;3Þ.

ID A4 r D2 D3 D4 D5 D6 PEC4 M

54.4.1 2.00 18 80 40 0 0 0 0 0.500

54.4.2 1.00 24 96.8 93.5 90.2 87.1 84.1 100 0.500

54.4.3 1.00 23 91.5 76.3 55.4 29.7 0 0 0.250

54.4.4 0.78 24 97.7 95.4 93.1 90.8 88.7 100 0.250

54.4.5 0.67 24 98.1 96.2 94.3 92.4 90.6 100 0.250

54.4.6 0.61 24 98.3 96.6 94.9 93.3 91.7 100 0.167

54.4.7 0.50 24 98.6 97.2 95.8 94.4 93.1 100 0.125

Table 4

Classification of all arrays of the type OAð54;35;3Þ.

ID A4 r D2 D3 D4 D5 D6 D7 D8 D9 PEC4 M

54.5.1 3.00 35 95.3 88.3 79.1 67.9 53.4 31.1 0 0 80 0.250

54.5.2 3.00 39 97.2 94.1 90.4 85.2 77.5 65.9 49.4 27.4 100 0.250

54.5.3 3.06 31 97.2 94.0 90.1 85.0 69.4 0 0 0 100 0.167

54.5.4 3.06 36 97.2 94.1 90.6 86.3 71.7 0 0 0 100 0.167


To estimate all 5ð5�1Þ=2¼ 10 two-factor interaction effects in a model involving five three-level factors, 40 degrees offreedom are required. However, Table 4 shows that the highest rank r of the two-factor interactions contrast matrix amongthe four non-isomorphic arrays of the type OAð54;35;3Þ is 39. As a result, none of the four arrays allows a model involvingall two-factor interactions to be estimated. Hence, PEC5 ¼D10 ¼ 0 for each of the arrays.

Table 4 also shows that arrays 54:5:1 and 54:5:2 perform equally well in terms of the A4 and M values. However, array54:5:1 is outperformed by array 54:5:2 in terms of the rank r, the PEC4 value and all Di values. Hence, array 54:5:1 is notadmissible. This demonstrates the usefulness of classification criteria other than A4. Comparing arrays 54:5:3 and 54:5:4learns that the two designs perform equally well on all criteria except two: array 54:5:4 has a higher value of r, and higherDi values. For this reason, array 54:5:3, also, is inadmissible.

Interestingly, sequentially maximizing the Di vector from left to right and from right to left in Table 4 results in differentdesigns being ranked first. Sequentially maximizing from left to right leads to array 54:5:4 being ranked first, while array54:5:2 is ranked first if the Di vector is maximized from right to left. Design 54:5:2 is therefore preferable when manyinteractions are expected, while design 54:5:4 performs better when only a few interactions are expected.

As a conclusion, the arrays 54:5:2 and 54:5:4 are the two admissible designs in the OAð54;35;3Þ series. The former isbetter in terms of the A4 value, the rank r and the Di values for large i. Array 54.5.4 is better in terms of the Di values forsmall i and in terms of the robustness criterion M. Both arrays have PEC4 ¼ 1. We have displayed the two admissibledesigns in Table A.1 of the Appendix.

5.2. 81 runs

5.2.1. Four factors

As indicated in Table 2, there are 32 non-isomorphic arrays of the type OAð81;34;3Þ. Two arrays of this type do not havefull rank r for the two-factor interaction contrast matrix. For one of these, this is due to the fact that it is a triplicatedregular 34�1

IV design. As a result, the classification criterion values for this array are identical to those of the 27-run arraydiscussed at the start of this section. For the second array lacking a full rank, the low rank is due to the fact that the arrayconsists of one replicate of array 54.4.3 and one replicate of the regular 34�1

IV design. Each of these building blocks has a lowrank for the two-factor interaction contrast matrix and low estimation efficiencies.

The full factorial 34 design is the only admissible design of the series. This design allows estimation of a fourth-ordersaturated model, and, as a result, has a zero A4 value, and Di values of 100%. Note, however, that array 54.4.7 permitsestimation of all the two-factor interactions with a high D-efficiency at lower cost than the full factorial 34 design.

5.2.2. Five factors

In the five-factor series labeled OAð81;35;3Þ, 40 out of the 17,056 non-isomorphic arrays do not permit all two-factorinteraction effects to be estimated. The only admissible design is the regular 35�1

V fractional factorial design with generatorE¼ABCD. Using this array, all two-factor interaction effects can be estimated with 100% D-efficiency. Hence, the array has a

Table 5

The 12 admissible arrays and the two regular arrays of the type OAð81;36;3Þ.

ID A4 r ðD2 ,D3 ,D4Þ ðD13 ,D14 ,D15Þ ðPEC4 ,PEC5 ,PEC6Þ M

81.6.440 4 60 ð98:4,96:7,94:9Þ ð77:3,75:2,73:0Þ ð100;100,100Þ 0.778

81.6.452 4 60 ð98:4,96:7,95:0Þ ð75:8,73:0,70:0Þ ð100;100,100Þ 0.759

81.6.649 4 60 ð98:4,96:7,94:9Þ ð76:0,73:6,71:2Þ ð100;100,100Þ 0.778

81.6.778 4 60 ð98:3,96:5,94:7Þ ð77:2,75:2,73:2Þ ð100;100,100Þ 0.759

81.6.831 4 60 ð98:3,96:4,94:5Þ ð78:0,76:3,74:5Þ ð100;100,100Þ 0.778

81.6.994 4 60 ð98:4,96:7,94:9Þ ð77:4,75:4,73:3Þ ð100;100,100Þ 0.778

81.6.1009 4 60 ð98:4,96:7,95:0Þ ð76:2,73:8,71:2Þ ð100;100,100Þ 0.796

81.6.1051 4.2 59 ð98:3,96:6,94:8Þ ð48:1,27:7,0Þ ð100;100,0Þ 0.741

81.6.1081 4 60 ð98:4,96:7,94:9Þ ð75:1,72:5,69:9Þ ð100;100,100Þ 0.852

81.6.1085 4 60 ð98:4,96:7,95:0Þ ð74:7,71:8,68:8Þ ð100;100,100Þ 0.796

81.6.1086 4 60 ð98:4,96:7,95:0Þ ð74:7,71:8,68:8Þ ð100;100,100Þ 0.796

81.6.1099 4 56 ð98:4,96:7,94:5Þ ð4:4,0;0Þ ð100;100,0Þ 0.667

81.6.1 6 44 ð91:4,74:7,52:7Þ ð0;0,0Þ ð80;0,0Þ 1

81.6.4 4 48 ð94:3,83:1,67:6Þ ð0;0,0Þ ð86:7,33:3,0Þ 1


zero A4 value and Di values of 100%. Note, however, that the 54-run array 54.5.2 only lacks one degree of freedom toestimate all the two-factor interaction effects. That array is therefore a good alternative if budgetary constraints render theuse of the 81-run 35�1

V fractional factorial design impossible.

5.2.3. Six factors

Table 5 presents the classification of all admissible and all regular arrays of the type OAð81;36;3Þ. The admissibledesigns are listed in Table A.2 in the Appendix.

The arrays 81.6.1 and 81.6.4 are regular designs. The former array has generators E¼ABC and F¼ABD, with generalizedword length pattern W ¼ ð6;0,2Þ, whereas the latter array has generators E¼ ABC2 and F¼ABCD and generalized wordlength pattern W ¼ ð4, 4, 0Þ. Array 81.6.4 is one of 94 MGA designs. However, like 90 other MGA designs, array 81.6.4 isinadmissible because it performs worse on all of the other classification criteria than the admissible MGA arrays 81.6.778,81.6.831 and 81.6.1099 (the eight other admissible arrays with A4¼4 have larger values of A5). The most striking weaknessof the regular array 81.6.4 is that the rank r of its two-factor interaction contrast matrix is only 48, while 10 out of the 12admissible arrays have a full rank two-factor interaction contrast matrix (r¼ 60). The array 81.6.1099 does not have a full-rank two-factor interaction contrast matrix, but it is admissible because it has the best value for the robustness criterionM. The admissible design 81.6.1051 has a slightly better M value than the full-rank arrays and a better rank r than array81.6.1099.

The 10 admissible arrays with r¼60 have slightly different estimation efficiency vectors and slightly different values ofthe robustness criterion. The arrays ranked best when sequentially maximizing the Di vector from left to right are81.6.1085 and 81.6.1086. Both have exactly the same Di vector. Their superiority over the other designs in the list appearsonly in the third decimal of the first few Di values, however. The best array when sequentially maximizing the Di vectorfrom right to left is 81.6.831.

5.2.4. Seven factors

Table 6 contains the classification of all admissible arrays and the two regular arrays of the type OAð81;37;3Þ. Theadmissible designs are listed in Table A.3 in the Appendix.

The two regular OAs are array 81.7.1, which has generators E¼ABCD, F ¼ AB2C and G¼ AB2D, and array 81.7.4, whichhas generators E¼ABCD, F ¼ AB2C and G¼ AC2D. Both of the regular arrays are inadmissible, even though the latter is anMGA design. Both regular designs perform poorly with respect to estimation efficiency, projection estimation capacity, andrank of the two-factor interaction contrast matrix.

There are seven MGA designs among the OAs of the type OAð81;37;3Þ. Their generalized word length pattern isW ¼ ð10,12,2,2Þ. MGA designs 81.7.247 and 81.474 are the only admissible MGA designs because, among all MGA designs,they perform best in terms of the classification criteria other than the A4 value. They also have the best value for therobustness criterion M, and they allow 60 two-factor interaction components to be estimated, because the rank r of thetwo-factor interaction contrast matrix is 60. There are, however, many non-MGA designs which allow substantially moretwo-factor interaction components to be estimated. The best array in this respect is array 81.7.461, whose 80 availabledegrees of freedom can be used to estimate all seven main effects (this requires 14 degrees of freedom) and 66 two-factorinteraction components.

Of particular interest are seven arrays with ðPEC4,PEC5,PEC6Þ vector equal to ð100,100,42:9Þ. All projections onto fivefactors of each of these arrays permit the estimation of all main effects and all two-factor interactions involving the five

Table 6

The 28 admissible arrays and the two regular arrays of the type OAð81;37;3Þ.


81.7.31 10.44 60 ð96:8,92:2,86:1Þ ð2:1,0;0Þ ð97:1,85:7,0Þ 1.611

81.7.68 10.69 62 ð96:8,92:2,86:3Þ ð5:3,1:4,0Þ ð97:1,85:7,57:1Þ 1.611

81.7.76 10.69 62 ð96:8,92:2,86:3Þ ð4:8,1:2,0Þ ð97:1,85:7,57:1Þ 1.611

81.7.80 10.44 60 ð96:9,92:2,86:2Þ ð1:8,0;0Þ ð97:1,85:7,0Þ 1.611

81.7.87 10.44 64 ð96:4,91:0,84:0Þ ð3:4,1:1,0:2Þ ð94:3,71:4,28:6Þ 1.611

81.7.179 10.44 64 ð96:4,91:0,84:0Þ ð3:3,1:0,0:2Þ ð94:3,61:9,28:6Þ 1.611

81.7.180 10.67 64 ð96:5,91:0,84:1Þ ð4:7,1:8,0:4Þ ð94:3,71:4,28:6Þ 1.667

81.7.245 10.54 62 ð96:9,92:2,86:3Þ ð4,0:7,0Þ ð97:1,85:7,0Þ 1.611

81.7.246 10.54 62 ð96:9,92:2,86:3Þ ð4:5,0:8,0Þ ð97:1,85:7,28:6Þ 1.611

81.7.247 10 60 ð96:9,92:3,86:1Þ ð0:7,0;0Þ ð97:1,85:7,0Þ 1.500

81.7.250 10.69 62 ð96:8,92:1,86:1Þ ð5:4,1:5,0Þ ð97:1,85:7,57:1Þ 1.611

81.7.294 10.84 62 ð96:8,92:1,86:2Þ ð5:5,1:5,0Þ ð97:1,85:7,57:1Þ 1.630

81.7.297 10.74 62 ð96:8,92:2,86:2Þ ð5:5,1:5,0Þ ð97:1,85:7,28:6Þ 1.630

81.7.438 10.59 62 ð96:9,92:3,86:2Þ ð1:9,0;0Þ ð97:1,66:7,0Þ 1.574

81.7.461 10.44 66 ð96:8,92:2,86:2Þ ð3:2,0:8,0Þ ð97:1,76:2,28:6Þ 1.611

81.7.472 10.59 62 ð96:9,92:2,86:1Þ ð0:9,0;0Þ ð97:1,66:7,0Þ 1.574

81.7.473 10.59 62 ð96:9,92:2,86:1Þ ð0:9,0;0Þ ð97:1,66:7,0Þ 1.574

81.7.474 10 60 ð96:9,92:3,86:0Þ ð0;0,0Þ ð97:1,66:7,0Þ 1.500

81.7.476 10.91 64 ð97:8,95:4,92:9Þ ð0;0,0Þ ð100,85:7,0Þ 1.630

81.7.477 10.67 64 ð96:6,91:8,85:7Þ ð0;0,0Þ ð91:4,42:9,0Þ 1.556

81.7.478 11.11 63 ð97:7,95:2,92:6Þ ð26:0,9:9,0Þ ð100;100,42:9Þ 1.685

81.7.479 11.11 63 ð97:7,95:2,92:6Þ ð28:1,12:7,0Þ ð100;100,42:9Þ 1.685

81.7.480 11.06 63 ð97:7,95:2,92:7Þ ð22:4,7;0Þ ð100;100,42:9Þ 1.667

81.7.481 11.06 63 ð97:7,95:2,92:6Þ ð19:2,4:8,0Þ ð100;100,42:9Þ 1.667

81.7.482 10.89 60 ð97:7,95:1,91:8Þ ð0:9,0:1,0Þ ð100,71:4,0Þ 1.611

81.7.484 11.36 63 ð97:7,95:4,92:9Þ ð56:5,40:7,0Þ ð100;100,42:9Þ 1.667

81.7.485 11.36 63 ð97:7,95:4,92:9Þ ð56:5,41:4,0Þ ð100;100,42:9Þ 1.667

81.7.486 11.21 63 ð97:8,95:4,93:0Þ ð47:5,21:7,0Þ ð100;100,42:9Þ 1.648

81.7.1 12 56 ð91:4,75:1,54:2Þ ð0;0,0Þ ð82:9,14:3,0Þ 2.000

81.7.4 10 56 ð92:9,79:2,61:1Þ ð0;0,0Þ ð85:7,28:6,0Þ 1.500


factors. The arrays 81.7.484 and 81.7.485 are the best arrays for estimating models with large numbers of interactions,because they have the largest D14 and D15 values. The arrays 81.7.478, 81.7.479, 81.7.480, 81.7.481 and 81.7.486 have asubstantially poorer estimation efficiency when a lot of interactions are present in the model (as reflected by the D14 andD15 values, for example). These arrays are admissible only because they have a slightly better generalized word lengthpattern than arrays 81.7.484 and 81.7.485.

5.2.5. Eight factors

Table 7 presents the classification of all admissible and all regular arrays of the type OAð81;38;3Þ. The admissible designs are

given in the Appendix in Table A.4. The OAð81;38;3Þ series consists of 235 arrays, including three regular 38�4IV designs. One of the

three regular designs, array 81.8.7, has generators E¼ABCD, F ¼ AB2C, G¼ AC2D, and H¼ AB2D2. It is admissible because it is theonly MGA design and it has the best value of the robustness criterion M. Its generalized word length pattern isW ¼ ð20,32,8,16,4Þ. While this array is desirable in terms of the GA criterion and the robustness criterion, it has the smallestnumber of estimable two-factor interaction components. As a matter of fact, the rank of its two-factor interaction contrast matrixis only 60, whereas all other admissible designs have r¼64, which is the maximum possible rank for eight factors and strength 3.

Perhaps the most interesting arrays in the OAð81;38;3Þ series are the arrays 81.8.234 and 81.8.235. These arrays havePEC4 ¼ PEC5 ¼ 100 and PEC6 ¼ 42:9 as well as excellent estimation efficiencies. Thus, all projections onto five factors ofthese two arrays permit the estimation of all main effects and all two-factor interactions involving the five factors. The twoarrays have a slightly different generalized word length pattern.

5.2.6. Nine and 10 factors

To complete the classification of all arrays of the type OAð81;3n;3Þ, there is a single array with nine factors and a single

array with 10 factors. Both of these arrays are regular. The array of type OAð81;39;3Þ has generators E¼ABCD, F ¼ AB2C,

G¼ AC2D, H¼ AB2D2, and P¼ BCD2, and an A4 value of 36. The array of type OAð81;310;3Þ has generators E¼ABCD, F ¼ AB2C,

G¼ AC2D, H¼ AB2D2, P¼ BCD2, and Q ¼ ABC2D2, and an A4 value of 60. The rank r of the two-factor interaction contrast matrix

Table 7

The 19 admissible arrays and two further regular arrays of the type OAð81;38;3Þ.


81.8.6 20.89 64 ð95:3,87:4,76:9Þ ð0;0,0Þ ð94:3,71:4,0Þ 2.722

81.8.7 20 60 ð92:1,77:2,58:1Þ ð0;0,0Þ ð85:7,28:6,0Þ 2.500

81.8.34 21.60 64 ð96:4,91:1,84:4Þ ð0:8,0:1,0Þ ð97:1,85:7,14:3Þ 2.889

81.8.35 21.38 64 ð96:4,91:2,84:4Þ ð0:9,0:1,0Þ ð97:1,85:7,17:9Þ 2.722

81.8.39 21.33 64 ð96:0,90:0,82:2Þ ð0:8,0;0Þ ð94:3,71:4,0Þ 2.778

81.8.43 21.60 64 ð96:4,91:2,84:6Þ ð2:0,0:2,0Þ ð97:1,85:7,7:1Þ 2.833

81.8.56 21.68 64 ð96:5,91:3,84:7Þ ð3:2,0:6,0Þ ð97:1,85:7,39:3Þ 2.815

81.8.58 21.63 64 ð96:4,91:3,84:6Þ ð1:6,0;0Þ ð97:1,71:4,0Þ 2.778

81.8.61 21.63 64 ð96:4,91:3,84:6Þ ð1:8,0;0Þ ð97:1,71:4,0Þ 2.833

81.8.127 21.33 64 ð95:6,88:7,79:7Þ ð0;0,0Þ ð91:4,28:6,0Þ 2.722

81.8.155 20.89 64 ð94:7,86:2,75:4Þ ð0;0,0Þ ð85:7,28:6,0Þ 2.611

81.8.186 21.63 64 ð96:4,91:2,84:6Þ ð1:3,0;0Þ ð97:1,78:6,0Þ 2.778

81.8.217 21.68 64 ð96:5,91:3,84:7Þ ð1:7,0;0Þ ð97:1,85:7,0Þ 2.722

81.8.230 21.83 64 ð97:5,94:9,92:2Þ ð0;0,0Þ ð100,85:7,0Þ 2.759

81.8.231 21.33 64 ð96:3,91:0,84:2Þ ð0;0,0Þ ð91:4,42:9,0Þ 2.667

81.8.232 22.22 64 ð97:4,94:7,91:8Þ ð17:2,6:1,0:2Þ ð100;100,42:9Þ 2.815

81.8.233 22.12 64 ð97:4,94:7,91:9Þ ð13:4,3:5,0:1Þ ð100;100,42:9Þ 2.815

81.8.234 22.72 64 ð97:5,94:9,92:2Þ ð44:7,24:3,1:4Þ ð100;100,42:9Þ 2.870

81.8.235 22.42 64 ð97:5,94:9,92:3Þ ð30:7,10:9,0:7Þ ð100;100,42:9Þ 2.870

81.8.1 22 64 ð91:3,75:0,54:4Þ ð0;0,0Þ ð84:3,21:4,0Þ 3

81.8.2 24 64 ð90:5,72:8,50:9Þ ð0;0,0Þ ð82:9,14:3,0Þ 3


is 60 in both cases. For the 10-factor array, 60 is the maximum rank possible for the two-factor interaction contrast matrixgiven that 80 degrees of freedom are available and 20 parameters are required to model the 10 main effects.

6. Discussion

This article features the classification of three-level strength-3 arrays with run size Nr81. Our classification accordingto various criteria resulted in a list of admissible designs that is useful for practitioners. There is a single admissible designfor the 27-run series, the 54-run series with four factors, and the 81-run series with four, five, nine, and 10 factors. For the54-run series with five factors, there were two admissible designs. Finally, there were 12, 28, and 19 admissible designs for81 runs and six, seven, and eight factors, respectively.

In Section 3.1, we reproduced an expected-bias argument and an efficiency-based argument to justify the generalizedaberration criterion to classify OAs. We claimed that these arguments might be less suitable for the strength-3 arraysstudied here. This claim is strongly supported by the plots of various criterion values against the A4 value in Section 4, andthe lists of admissible designs in Section 5. We observed that the designs with the smallest A4 value generally had inferiorestimation efficiencies Di, projection estimation capacities PECi and ranks r of the matrix of two-factor interactioncontrasts when compared to the best designs according to these other criteria.

For models involving at most five interaction components, we verified the existence of an inverse relationship betweenthe A4 value and the Dn

i values. These efficiency values may, however, not be the most sensible measure of estimationefficiency in three-level strength-3 designs, because we are particularly interested in interaction effects that cannot becaptured by a single interaction component. Therefore, we have used an alternative definition of efficiency, that focuses onthe estimation of all components of given two-factor interactions simultaneously.

Our study revealed that there is a connection between the A4 value and the robustness criterion M, because designs thatminimize the A4 value tend to have low values of M. However, when robustness against missing observations is not anissue, then even this motivation to minimize the A4 value disappears. One could then argue that the A4 and M valuesshould not be considered when determining the lists of admissible designs. This would result in a reduction of thenumbers of admissible 81-run designs to eight, nine and two, for six, seven, and eight factors, respectively.

Finally, we like to stress that this paper is not intended as a general criticism on the use of generalized aberration.In view of the expected-bias argument mentioned in Section 3.1, we are convinced that generalized aberration is a veryuseful criterion to classify strength-2 designs. In addition, two-level designs have only one component for each two-factorinteraction. We therefore believe that the efficiency-based argument to use the generalized aberration criterion is moreappropriate for two-level strength-3 designs than for three-level strength-3 designs.

Acknowledgments

The third author was supported by a grant from the Flemish foundation of scientific research FWO. The comments ofthe associate editor and an anonymous referee led to a greatly improved argumentation in the paper.


Appendix A. Admissible non-regular three-level arrays of strength 3

The Tables A.1–A.4 list all admissible non-regular arrays with 54 and 81 runs discussed in the text. The arrays are givenin a condensed form. First, we omitted the first three columns of each array, because these columns simply forma duplicated 33 full factorial design for all arrays with 54 runs and a triplicated 33 full factorial design for all arrays with81 runs. The rows of the two or three replicates of the full factorial design are sorted lexicographically, so that, for a givenrun of the full factorial design, the two or three replicates appear in pairs or triples. Also, we omitted the first row of the81-run arrays, because that row is a zero row for each of the 81-run arrays. The next step was to transpose the arrays, sothat we obtained an ðn�3Þ � 54 matrix for the 54-run arrays and an ðn�3Þ � 80 matrix for the 81-run arrays. The resultingmatrices thus have either 27 or 40 pairs of digits. The final step was to replace each pair of successive digits, which can beviewed as a base-3 number, by its base-10 representation. The end result is an ðn�3Þ � 27 matrix for the 54-run arrays andan ðn�3Þ � 40 matrix for the 81-run arrays.

We illustrate this procedure by reconstructing the fourth column of array 81.6.778. In Table A.2, we find0458458028014468038045045278064064564562 as the condensed form of this fourth column (note that the first threecolumns were omitted because they correspond to the 33 factorial design). We then convert every digit into the two digitsof the corresponding base-three number. So we replace 0 with 00, 4 with 11, etc. Subsequently, we add a 0 as the firstsymbol of the column (because we dropped the first row of every 81-run array). The resulting column, in transposed form,is 000111222111222000222000111112022001022001112001112022122002011002011122011122002.

Table A.1

Admissible arrays of the type OAð54,3n ,3Þ for n¼4 and n¼5.

54.4.7

125251512251512125512125251

54.5.2

048521521521152215521215152

125536671536084716671716325

54.5.4

055512521512125251521251118

057563736736615381563372615

Table A.2

Admissible arrays of the type OAð81,36 ,3Þ.

440 452

0458468048064468032148165178064165161468 0458468048064468032148165178064165161468

5165115285211562146381054661552762135512 5165115285211562146381054661552762135512

5646181365705642637512505646642505637246 5646242637507642635721707376642707365183

649 778

1178178048035178048035035578035035565561 0458458028014468038045045278064064564562

3725725125123725125123123785123123783763 5165165165165112585107248055231125372048

5172523507651637642165516723516651235365 5521635516635246242251637372642242516367

831 994

0458458028014468038045045278064064564562 0458468048064468035165064578064064564562

5165165165165115282117538055211155062348 5165115285211211585064155075261512632348

5646505651505242646237251372642242516367 5646181372642381176181242521381246376367

1009 1051

0458468048064468035165064578064064564562 0478462178161462165165075178161075164078

5165115285211211585064155075261512632348 5125238412536548020148166732247535205505

5646383165705381167383505251381507167176 5235523707172172712237521723705642365507

1081 1085

1178165178161165178161035578161035565165 1178165178161165178161035578161035565165

3725767321367767321367123781367123785240 3752552122047663531468538045803270463552

5516181183646646642505635167181521646176 5176721235516721237523165367516365383637

1086 1099

1178165178161165178161035578161035565165 1178178048035265135138138175162162462465

3752552122047663531468538045803270463552 3781781121125664364382382147076076546532

5237712505707712507721167167707367183376 5183642637516716635512167635642507653505

Table A.3


31 68

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516521651521635181505521516651165505 5165516646651521521165181521521646165176

5521176651505635646646505176176181181646 5521242523635365721635516246251183521172

76 80

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516646651521521165181521521646165176 5165516646651521521165181521521646165176

5521253512635705381635516246251172521183 5521505521635165651635516516521181521165

87 179

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516646651521521165646181521181505176 5165516646651651505165516651516651505165

5521505181646651505521505176165651505521 5521181176521646521176635165635521516181

180 245

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516646651651505165516651516651505165 5165516651521516651165521181646181505176

5521635176521521521176635165635521505521 5521176242653635383635507507716512651165

246 247

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516651521516651165521181646181505176 5165516651521516651165521181646181505176

5521176253642635712635507507376523651165 5521176505651635181635516516646505651165

250 294

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516651521516651165521181646181505176 5165516651521521635635505521516181521165

5521242176721707521635365505642516721167 5521242253712707716365381242242653707176

297 438

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516651521521635635505521516181521165 5167376721646646635235523383376521637165

5521253176381365516635721251253646365176 5521646646165165516635521651646176181176

461 472

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5176176651521521521176635165651521176505 5176242651716637376653505183521176172637

5505646181646181635646505516521505651165 5516651505635521635181505521165516181521

473 474

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5176242651716637376653505183521176172637 5176516651635521521181176165651505181516

5635383521246512716637165637521635183512 5516635176651165651646516181521505505635

476 477

0458468048064468035075074278064074264265 0458468048064468048064014878064014875261

5165115285211115278204235385211505161486 5165115285211115285211165185211165163468

5365653716165251705376635376367505246651 5167651516367653716165172172365705167653

5521646251381505723251381365705165521637 5521505716246181365705521181381635521507

478 479

0458458028014468038045045278064064564562 0458458028014468038045045278064064564562

5165165165165105572235567055501242631378 5165165165165105572235567055501242631378

5176516181651521637352508335176722335422 5176516181651521637352508335176722335422

5637712512253383381172172646242642512507 5642253507712732706537172636737532523642

480 481

0458458028014468038045045278064064564562 0458458028014468038045045278064064564562

5165165165165105572237565055501242361648 5165165165165105572237565055501242361648

5183642637516381183172365646642246376507 5183642637516381183172365646642246376507

5637653512505736162571704806553731665811 5646176646181521507367635505165723376651

482 484

0458458028014468038045045278064064564562 0458468048064468032175164278064164261565

5165165165165115275263263485211211771766 5165105585501505246167642365512581031655

5521635516635176721246381181235376181523 5183381176642112781172846057633512633525

5646505651505505721365251505705381646507 5637736166553183366571531672642641812804


Table A.3 (continued )

485 486

0458468048064468032175164278064164261565 0458468048064468032175164278064164261565

5165105585501505246167642365512581031655 5165115285211181177065215635505805334182

5183381176642112781172846057633512633525 5367642646642317684076283427613617241652

5646521505165536702467523636537732523635 5521181365705635183563703776521231842770

Table A.4


6 7

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516521651516521651165181651165176516 5165516521651516521651165181651165176516

5521176635521521181176635516635521181176 5521176635521635521176635516635521176635

5646505651505181505646505646181646181505 5521635516635176651176521165521635516635

34 35

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516521651521635181505521516651165505 5165516521651521635181505521516651165505

5521176637383653246642505172176707181646 5521176637383653246642505172176707181646

5521521183242172716637635507521383181176 5521635512707507376653165653635242181516

39 43

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516521651521635181505521516651165505 5165516521651521635181505521516651165505

5521176651505635646646505176176181181646 5521181507383523246642635172181707181516

5521635505646516176651165651635505181516 5521505653242642716637165507505383181646

56 58

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516646651521521165181521521646165176 5165516646651521521165181521521646165176

5521176251383523642635507167172721521176 5521176521181521635635516176165651521176

5521635376707167523635512642653246521165 5521635646505165516635521651646176521165

61 127

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516646651521521165181521521646165176 5165516646651521521646165181521165181516

5521176521635505651635516176181181521176 5521176181646505181521521176651635505176

5521635646176176505635521651635516521165 5646505651176181505521646176635505651505

155 186

0458458028014458028014014888014014874860 0458458028014458028014014888014014874860

5165165165165165165165165165165165165165 5165165165165165165165165165165165165165

5165516646651521521646165181521165181516 5165516651521516651165521181646181505176

5521505181646165521521521176651635505505 5521176181646635646635505505516521651165

5646176651176521165521646176635505651176 5521521646165176505635651521181516181176

217 230

0458458028014458028014014888014014874860 0458468048064468035075074278064074264265

5165165165165165165165165165165165165165 5165115285211115278204235385211505161486

5167376651523523376251181181646705167376 5365653716165251705376635376367505246651

5521635505521181646635516176176635521635 5521646251381505723251381365705165521637

5642716521167507705381646176635235642716 5721507705505381386364151676242135686504

231 232

0458468048064468048064014878064014875261 0458458028014468038045045278064064564562

5165115285211115285211165185211165163468 5165165165165105572235567055501242631378

5167651516367653716165172172365705167653 5176516181651521637352508335176722335422

5521505716246181365705521181381635521507 5637712512253383381172172646242642512507

5523181181383642246646507167705365523183 5642253507712732706537172636737532523642

233 234

0458458028014468038045045278064064564562 0458468048064468032175164278064164261565

5165165165165105572237565055501242361648 5165105585501505246167642365512581031655

5183642637516381183172365646642246376507 5183381176642112781172846057633512633525

5637653512505736162571704806553731665811 5637736166553183366571531672642641812804

5646176646181521507367635505165723376651 5646521505165536702467523636537732523635

235

0458468048064468032175164278064164261565

5165115285211181177065215635505805334182

5367642646642317684076283427613617241652

5521181365705635183563703776521231842770

5723637176523242646376635242642183635642



References

Ai, M.Y., Li, P.F., Zhang, R.C., 2005. Optimal criteria and equivalence for nonregular fractional factorial designs. Metrika 62, 73–83.Bulutoglu, D.A., Margot, F., 2008. Classification of orthogonal arrays by integer programming. Journal of Statistical Planning and Inference 138, 654–666.Chen, J., Sun, D.X., Wu, C.F.J., 1993. A catalogue of two-level and three-level fractional factorial designs with small runs. International Statistical Review

61, 131–145.Cheng, C.S., Deng, L.Y., Tang, B., 2002. Generalized minimum aberration and design efficiency for nonregular fractional factorial designs. Statistica Sinica

12, 991–1000.Cheng, C.S., Mee, R.W., Yee, O., 2008. Second order saturated orthogonal arrays of strength three. Statistica Sinica 18, 105–119.Cheng, C.S., Steinberg, D.M., Sun, D.X., 1999. Minimum aberration and model robustness for two-level factorial designs. Journal of the Royal Statistical

Society Series B 61, 85–94.Fries, A., Hunter, W.G., 1980. Minimum aberration 2k�p designs. Technometrics 22, 601–608.Hedayat, A.S., Seiden, E., Stufken, J., 1997. On the maximal number of factors and the enumeration of 3-symbol orthogonal arrays of strength 3 and index 2.

Journal of Statistical Planning and Inference 58, 43–63.Hedayat, A.S., Sloane, N.J.A., Stufken, J., 1999. Orthogonal Arrays: Theory and Applications. Springer, New York, NY, USA.Loeppky, J.L., Sitter, R.R., Tang, B., 2007. Nonregular designs with desirable projection properties. Technometrics 49, 454–467.Ma, C.X., Fang, K.T., 2001. A note on generalized aberration in fractional designs. Metrika 53, 85–93.Rao, C.R., 1947. Factorial experiments derivable from combinatorial arrangements of arrays. Journal of the Royal Statistical Society Supplement 9,

128–139.Schoen, E.D., Eendebak, P.T., Nguyen, M.V.M., 2010. Complete enumeration of pure-level and mixed-level orthogonal arrays. Journal of Combinatorial

Designs 18, 123–140.Sun, D.X., Li, W., Ye, K.Q., 2002. An algorithm for sequentially constructing nonisomorphic orthogonal designs and its applications. Technical Report,

Department of Applied Mathematics and Statistics, SUNY at Stony Brook.Sun, D.X., Wu, C.F.J., Chen, Y., 1997. Optimal blocking schemes for 2n and 2n�p designs. Technometrics 39, 298–307.Tang, B., Deng, L.Y., 1999. Minimum G2-aberration for nonregular fractional factorial designs. Annals of Statistics 27, 1914–1926.Wu, C.J.F., Hamada, M., 2000. Experiments: Planning, Analysis and Parameter Design Optimization. Wiley, New York, NY, USA.Xu, H., 2005. A catalogue of three-level regular fractional factorial designs. Metrika 62, 259–281.Xu, H., Wu, C.F.J., 2001. Generalized minimum aberration for asymmetrical fractional factorial designs. Annals of Statistics 29, 1066–1077.

classification of three-level strength-3 arrays

Documents