designs in constrained experimental regions...lower bounds whole simplex design in pseudocomponents...

Designs in constrained experimental regions Mixture Experiments Prof Eddie Schrevens Biosystems Department Faculty of Bioscience Engineering K. Universiteit Leuven Belgium [email protected]

Outline

• Paul Darius and mixtures

• Part I

o Definitions

o Design and analysis of mixture experiments

o Whole simplex designs

• Multiple grain cracker: simple mixture

• Composed fertilizer: constrained, double mixture amount experiment

• Part II

o Constrained mixture experiments with non-homomorphic experimental regions

• Investigating the ill conditioning

o Optimal design revisited

Part I

Mixtures?

Paul Darius and design and analysis of mixtures

Why mixtures?

• Investigate the effects of composition and process variables

on responses

o Investigate the importance of the compositional components

o Investigate the importance of the process variables

o Investigate respective interactions

o Predict the optimal response

• Pancake, concrete, …

Mixture constraints

• Mixture of q components xi for i in 1 tot q

1

1q

i

i

x

0 1ix

The factor spaces

Simplex factorspace for a two component mixture

x1

x2

x1 = 1 x2 = 1

(1,0) (0,1)

(1,0)

(0,1) (0,0)

(0,0,1)

(0,1,0)

(1,0,0)

x2

x1

x1 = 1

(1,0,0)

(0,1,0)

x2 = 1

(0,0,1)

x3 = 1

Simplex factorspace for a 3 component mixture

x3

q=2 q=3

Consequences

• Experimental factors not independent

o One exact collinearity/dependence relation

• Classical, orthogonal experimental designs not applicable

• Specific design and analysis methodology

• Multivariate approach is necessary

Additional constraints

• Single component constraints

• Mulitiple component constraints in p of the

q components

1

p

j ij i j

i

a c x b

0 1i i il x u

Complex constrained mixture experiment

0.10 x1 0.40

0.25 x2 0.50

0.20 x3 0.60

2.3 x1 + 3 x2 + 4 x3 3 x1 = 1

x2 = 1 x3 = 1

Experimental regions

in high dimensionality

Irregular convex

hyper-polyhedrons

Process variables

Combined region of a 22 factorial design and

a {3,2} simplex lattice design

x2 = 1

-1

+1

-1 +1 z1

z1

z2

z2

x1 = 1

x3 = 1

a b

x3 = 1

x3 = 1

x2 = 1

x2 = 1

x1 = 1

Combined region for a 3 component mixture

(x1,x2,x3) and 1 process variable (z) in 2 levels

x1 = 1

z = 1

z = 2

Experimental factor not bound by the mixture constraints

Multiple mixtures

Y2 = 1 Y3 = 1

Y1 = 1

X3 = 1

X1 = 1

{3,2;3,2} double lattice design

X2 = 1

3

1

1i

i

Y p

3

1

i

i

X p

Mixture of mixtures of mixtures…

Double mixture: each mixture is

itself a mixture or a mixture of

mixtures, mixed with proportions p

and 1-p

Defined by multiple component

equalities

Analysis of mixture experiments • Numeric responses

o Specific mixture models: Sheffé canonical polynomials for simple mixtures

o Incorporation of the mixture constraint in the model

o Combination of submodels for process variable and multiple mixture models by multiplying the models

o Relies on OLS and RSM in classical approaches

o Ridge regression, PCA and PLS regression multicolinearity

• Categorical responses o Generalized linear modelsmodels

o PLSD

o TBM

1

q

i i

i

y x

1

q q q

i i ij i j

i i j

y x x x

Mixture experimental design

• Transform to independent factors

• Whole simplex experiments

• Homomorphic experimental regions

o Same shape as the overall simplex

• Complex constrained experimental regions

o Irregular convex hyper-polyhedron

Transform to independent factors

Advantages

Known methodology for design

and analysis

Disadvantages

Difficult interpretation

Exp factors are linear

combinations of the original

mixture components

Interactions, multiple mixtures

For mixtures with additional

constraints near-

multicollinearity problems

persist

(1/3,

1/3,

1/3

)

[0,0,0]

(0,0)

w2

w1

x3

x2

x1

x3

x2

x1

Transformation to independent variables

~

~

~

Get rid of the mixture constraint

Whole simplex experiments

{3,3} simplex lattice design

(0,1/2,0,

1/2)

(1/2,0,

1/2,0)

(0,0,1/2,

1/2)

(1/2,

1/2,0,0) (

1/2,0,0,

1/2)

(0,1/2,

1/2,0)

x4 = 1

x3 = 1

x2 = 1

x1 = 1

x3 = 1 x2 = 1

x1 = 1

{4,2} simplex lattice design

{q,m} simplex lattice design

3 component simplex centroid design

(1/2,0,

1/2,0)

(1/3,0,

1/3,

1/3)

(0,0,1/2,

1/2)

(1/2,0,0,

1/2) (

1/2,

1/2,0,0)

(0,1/2,

1/2,0)

x2 = 1 x4 = 1

x3 = 1

x1 = 1

x3 = 1 x2 = 1

x1 = 1

Simplex centroid designs

4 component simplex

centroid design

(0,1/3,

1/3,

1/3) (0,

1/2,0,

1/2)

(1/4,

1/4,

1/4,

1/4)

(1/3,

1/3,

1/3,0)

(1/3,

1/3,0,

1/3)

Many appropriate optimal design plans exist

Homomorphic experimental regions

x1 = 0.35

x3 = 0.20x2 = 0.15

x3’ = 1 x2

’ = 1

x1 = 1

x1’ = 1

x3 = 1 x2 = 1

x1 0.35

x2 0.15

x3 0.20

' i ii q

i

i 1

x lx

1 l

i i0 l x 1

q

i

i 1

l 1

L-pseudocomponents xi’

Lower bounds Whole simplex design in pseudocomponents

xi’

Whole simplex experiments in raw mixture

components or pseudocomponents: examples

• Most ‘optimal’ approaches for design and analysis of

mixture experiments

• Straightforward design and analysis, even for complex

problems

• Examples

o Multigrain crackers

o Composed liquid fertilizer as a constrained, double mixture amount

experiment

Multigrain crackers

Objectives

o Develop a multiple grain cracker where a proportion p of the

wheat flower is replaced with buckwheat, oats, barley and rye

o Optimising the flour composition of a multiple grain cracker in

relation to consumer preference, scored by magnitude estimation

Multigrain crackers

Defining constraints of the experimental region

o All components of the dough mixture are constant except flour

o Proportion p of the wheat flour can be replaced by buckwheat, oats, barley and/or rye

• 1-p Pwheat 1

• 0 Pbuckwheat p

• 0 Poats p

• 0 Pbarley p

• 0 Prye p

o Homomorphic experimental region with 5 components

o Pseudocomponents transformation to whole simplex

o {5,2} simplex lattice is proposed

Multigrain crackers

Mixture Wheat Buckwheat Oats Barley Rye

1 1 0 0 0 0

2 0 1 0 0 0

3 0 0 1 0 0

4 0 0 0 1 0

5 0 0 0 0 1

6 0.5 0.5 0 0 0

7 0.5 0 0.5 0 0

8 0.5 0 0 0.5 0

9 0.5 0 0 0 0.5

10 0 0.5 0.5 0 0

11 0 0.5 0 0.5 0

12 0 0.5 0 0 0.5

13 0 0 0.5 0.5 0

14 0 0 0.5 0 0.5

15 0 0 0 0.5 0.5

{5,2} simplex lattice design in pseudocomponents

Multigrain crackers

Full model: quadratic Scheffé canonical polynomial

o 5 linear terms

o 10 cross product terms

Pref. = 10000wheat + 01000buckwheat + 00100oats + 00010barley

+ 00001rye + 11000wheat*buckwheat + 10100wheat*oats +

10010wheat*barley + 10001wheat*rye + 01100buckwheat*oats +

01010buckwheat*barley + 01001buckwheat*rye + 00110oats*barley

+ 00101oats*rye + 00011barley*rye

Multigrain crackers

Model reduction based on all possible models and

maximum R2

pref. = 66.62 wheat + 76.51 buckwheat + 62.41

oats + 40.99 barley + 32.81 rye – 129.39

wheat*buckwheat

R2 = 0.98 CV=6 %

0.2

0.1

0

0 0.1 0.2

Buckwheat

Wheat

Oats

Barley

Adj. pref. (%)

1 0

1

62.4

Adj. pref. (%)

62.7

0.9 0

0.9

Barley

Oats

63.2

0.8 Oats 0

0.8

Barley

Adj. pref. (%)

63.8

0.9

0

0.9

Oats

Barley

64.2

0.8

0

0.8

Oats

Barley

Adj. pref. (%)

65.2

0.8

0

0.8

Oats Barley

41.5

37.1 39.3

32.8 36.2 39.6

Adj. pref. (%)

Adj. pref. (%)

Figure 5: Adjusted preference of the Belgian experts for 3 wheat and buckwheat levels

Multigrain crackers

Conclusions

o Replacing the wheat flour partially with buckwheat and oats had a

positive effect on the consumers preference

o A antagonistic interaction between wheat and buckwheat persists

o Rye and barley addition resulted in low preference

Composed liquid fertilizer as a constrained, double

mixture amount experiment

Objectives

Optimize the composition of a composed liquid fertilizer in

relation to plant response (Ca content of Rye grass)



Defining constraints Aqueous solutions of inorganic ions

K+ Ca2+ Mg2+ NO3- H2PO4

- SO42-

with C = total concentration in units of charge (meq/l)

Prepared by dissolving salts

fi KNO3, Ca(NO3)2, MgSO4, …

Ionic balance constraint: balance of charge [K+] + [Ca2+] + [Mg2+] = [NO3

-] + [H2PO4-] + [SO42-] =C/2

Double mixture constraints: multicomponent equality

constraints

Composed liquid fertilizer as a constrained

double mixture amount experiment

4

5 3

1

2

Ca2+

= 1

K+ = 1

Mg2+

= 1 Cation factorspace

6

5

6 4

2 3

1

NO3- = 1

H2PO4- = 1 SO4

2- = 1

Anion factorspace

0.44 K+ 0.79

0.09 Ca2+ 0.44

0.12 Mg2+ 0.47

0.56 NO3- 0.82

0.09 H2PO4- 0.35

0.09 SO42- 0.35

{3,2} Simlex Lattice



• Mixtures ({3,2} SimLat) in the separate mixtures are transformed to pseudocomponents

• Total concentration (C) is defined as a process variable

• The mixture design is the combination of the two constituent mixtures (Cations, Anions), mixed with proportion 0.5

• This design is repeated at the levels of the process variable (C = total concentration or amount)



• Pseudocomponents transformations

+K 0.44k

0.35

2Ca 0.09ca

0.35

2Mg 0.12mg

0.35

-

3NO 0.56n

0.26

-

2 4H PO 0.09p

0.26

2-

4SO 0.09s

0.26

Total milli-eq concentration (C)

n = 0

p = 0

s = 0.5

n = 0

p = 0.5

s = 0

n = 0.5

p = 0

s = 0

k

mg ca

12.5 meq/l 50 meq/l

Double mixture experiment with one process variable in two levels

n = 0.5

p = 0

s = 0

k

mg ca

n = 0

p = 0

s = 0.5

n = 0

p = 0.5

s = 0

Design representation



Number of treatment combinations

o Cations x anions: 6 x 6 = 36

o 2 levels in the process variable: 36 * 2 = 72

o 5 replications per treatment: 360 exp units

o Checkpoints?



Model formulation

o The second degree double mixture model with process variable is

obtained by combining the process variable model with a second

degree double mixture model, resulting in a total of 72 terms

f(x,z) = b0100100 kn + b0

100010 kp + b0100001 ks + b0

100110 knp + b0100101 kns + b0

100011 kps +

b0010100 can + b0

010010 cap + b0010001 cas + b0

010110 canp + b0010101 cans + b0

010011 caps +

b0001100 mgn + b0

001010 mgp + b0001001 mgs + b0

001110 mgnp + b0001101 mgns + b0

001011 mgps +

b0110100 kcan + b0

11001 kcap + b0110001 kcas + b0110110 kcanp + b0

110101 kcans + b0110011

kcaps + b0101100 kmgn + b0

101010 kmgp + b0101001 kmgs + b0

101110 kmgnp + b0101101 kmgns +

b0101011 kmgps + b0

011100 camgn + b0011010 camgp + b0

011001 camgs + b0011110 camgnp +

b0011101 camgns + b0

011011 camgps + b1100100 knz + b1

100010 kpz + b1100001 ksz + b1

100110 knpz +

b1100101 knsz + b1

100011 kpsz + b1010100 canz + b1

010010 capz + b1010001 casz + b1

010110 canpz +

b1010101 cansz + b1

010011 capsz + b1001100 mgnz + b1

001010 mgpz + b1001001 mgsz + b1

001110

mgnpz + b1001101 mgnsz + b1

001011 mgpsz + b1110100 kcanz + b1

110010 kcapz + b1110001 kcasz +

b1110110 kcanpz + b1

110101 kcansz + b1110011 kcapsz + b1

101100 kmgnz + b1101010 kmgpz +

b1101001 kmgsz + b1

101110 kmgnpz + b1101101 kmgnsz + b1

101011 kmgpsz + b1011100 camgnz +

b1011010 camgpz + b1

011001 camgsz + b1011110 camgnpz + b1

011101 camgnsz + b1011011 camgpsz

With x: mixture variable

z: process variable

k, ca, mg, n, p and s: pseudocomponents in proportions

Composed liquid fertilizer as a constrained, double mixture

amount: model formulation

bz k ca mg n p s: parameters


mixture amount

Reduced Model 25 terms:

calcium = 211.65 kn + 217.64 kp + 237.35 ks + 672.47 can

+ 657.87 cap + 700.02 cas + 245.40 cap

+ 176.66 mgn + 125.13 mgp + 123.59 mgs

+ 928.30 kcan + 542.34 kcap + 2437.85 kcans

- 282.50 kmgs + 661.36 camgn + 716.75 camgp

+ 318.91 camgs +2418.63 camgns + 37.37 knz

+ 88.21 canz + 78.72 mgnz + 36.88 mgpz

+ 60.66 mgsz + 795.67 kcapz + 573.17 kcasz

R2 = 0.99, CV = 4.9

n = 0.25, p = 0, s = 0.25 n = 0.25, p = 0.25, s = 0

n = 0, p = 0.25, s = 0.25 n = 0, p = 0, s = 0.5 n = 0, p = 0.5, s = 0

n = 0.5, p = 0, s = 0

Calcium (mmol/kg DW)

200

110

200.5

k 0.250

0.25

0.5

ca0.5

0


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca

12.5 mval/l

k = 0.25, ca = 0, mg = 0.25 k = 0.25, ca = 0.25, mg = 0

k = 0, ca = 0.25, mg = 0.25 k = 0, ca = 0, mg = 0.5 k = 0, ca = 0.5, mg = 0

k = 0.5, ca = 0, mg = 0


200

110

200.5

n 0.250

0.25

0.5

p


200

110

2020

n 0.250

0.25

0.5

p


200

110

2020

n 0.250

0.25

0.5

p


200

110

2020

n 0.250

0.25

0.5

p


200

110

2020

n 0.250

0.25

0.5

p


200

110

2020

n 0.250

0.25

0.5

p

12.5 mval/l

n = 0.25, p = 0, s = 0.25 n = 0.25, p = 0.25, s = 0

n = 0, p = 0.25, s = 0.25 n = 0, p = 0, s = 0.5 n = 0, p = 0.5, s = 0

n = 0.5, p = 0, s = 0


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca


200

110

200.5

k 0.250

0.25

0.5

ca

50 mval/l

k = 0.25, ca = 0, mg = 0.25 k = 0.25, ca = 0.25, mg = 0

k = 0, ca = 0.25, mg = 0.25 k = 0, ca = 0, mg = 0.5 k = 0, ca = 0.5, mg = 0

k = 0.5, ca = 0, mg = 0


200

110

200.5

n 0.250

0.25

0.5

p


200

110

200.5

n 0.250

0.25

0.5

p


200

110

200.5

n 0.250

0.25

0.5

p


200

110

200.5

n 0.250

0.25

0.5

p


200

110

200.5

n 0.250

0.25

0.5

p


200

110

200.5

n 0.250

0.25

0.5

p

50 mval/l



Conclusions

o Ca in the fertilizer increases the Ca content in the Rye grass

o Marked Ca*K antagonistic effect on the Ca content in the Rye

grass

o Marked Mg*Ca antagonistic effect on Ca content in the Rye grass

o No cation – anion interactions

o Total concentration has an additive effect, not interacting with

composition

Part II

Complex constrained mixtures

Irregular, convex hyper-polyhedrons

Optimal design strategy revisited

Mixture experimental design

• Transform to independent factors o Classical design and analysis

• Whole simplex experiments o {q,m} simplex lattice, simplex centroid (D-optimal with respect to

corresponding mixture models)

• Homomorphic experimental regions o Same shape as the overall simplex

o Pseudo-components transformation expands the experimental

region to the whole simplex

o Whole simplex experiments in pseudocomponents

• Complex constrained experimental regions o Irregular, convex hyper-polyhedrons

Irregular, convex (hyper)-polyhedron

Experimental region for the optimization of

a 4 component explosive

Complex constrained experimental regions

Classical approach to design

• Each specific problem has its own optimal design related

to size, shape and location of the experimental region

defined by the set of additional constraints

• Optimal design theory is indispensable

• Implies computer aided design of experiments, especially

in high dimensionality

• Extremely computational intensive exchange

algorithms

• Ill conditioning is more rule than exception


Classical procedure

• Constraints expert or screening

• Consistency check of constraints!

• Vertices of the hyper-polyhedron are computed

• ‘Mixture’ model is chosen

• Centroids of different dimensionality are

calculated

List of candidate points: vertices extended

with centroids


Classical procedure

• Optimality criteria is chosen: D, G, V, A, …

• Optimal design is selected by an exchange algorithm taking into account the selected model o Branch and bound, excursion algorithms or all possible subsets

• This algorithm is runned for different numbers of design points

Optimal design for the pre-specified model

and optimality criterion

Classical design procedures: example

Constraints

20.22 0.47x

30.06 0.56x

10.22 0.72x

X1

X2

X3

Vertex

Edge centroid

Plane centroid

List of 9 candidate points

4

2

3

7

8

5

6

9 1

List of candidate points for the linear mixture

model

Id X1 X2 X3

1 0.720 0.220 0.060

2 0.470 0.470 0.060

3 0.220 0.470 0.310

4 0.220 0.220 0.560

5 0.470 0.220 0.310

6 0.595 0.345 0.060

7 0.345 0.470 0.185

8 0.220 0.345 0.435

9 0.408 0.345 0.248

Design matrix X

List of candidate points for the quadratic

mixture model

Id X1 X2 X3 X1X2 X1X3 X2X3

1 0.720 0.220 0.060 0.158400 0.043200 0.013200

2 0.470 0.470 0.060 0.220900 0.028200 0.028200

3 0.220 0.470 0.310 0.103400 0.068200 0.145700

4 0.220 0.220 0.560 0.048400 0.123200 0.123200

5 0.470 0.220 0.310 0.103400 0.145700 0.068200

6 0.595 0.345 0.060 0.205275 0.035700 0.020700

7 0.345 0.470 0.185 0.162150 0.063825 0.086950

8 0.220 0.345 0.435 0.075900 0.095700 0.150075

9 0.408 0.345 0.248 0.140760 0.101184 0.085560

Extended design matrix X

Design procedures: example

• Given the expanded design matrix X for OLS estimation, representing model specifications

• For optimal parameter estimation: o A-optimality minimises the trace of (X’X/n)-1, resulting in a minimal

variance of the parameters

o D-optimality minimizes the determinant of (X’X/n) -1

o E-optimality minimizes the maximum eigenvalue of (X’X/n) -1

• For optimal response estimation: o G-optimality minimises the maximum prediction variance over a

specified set of design points (normalised for the number of design points)

o V-optimality minimises the average prediction variance over a specified set of design points (normalised for the number of design points)

Design procedures: example, 1st order

• Calculation of optimality criteria for all subsets of 4, 5, 6, 7, 8, 9 points out of the 9 candidates

• Set of points 1,2,3,4 has the lowest D-criterion for a first order linear mixture model

Number of

design points

D-optimality Candidate point

number

4 1638 1 2 3 4

5 2133 1 2 3 4 5

1 2 3 4 7

6 2457 1 2 3 4 5 7

1 2 3 4 5 8

7 2804 1 2 3 4 5 6 7

1 2 3 4 5 7 8

8 3013 1 2 3 4 5 6 7 8

9 3812 1 2 3 4 5 6 7 8 9

Optimal design

Design procedures: example 2d order

• Calculation of optimality criteria for all subsets of 6, 7, 8, 9 points out of the 9 candidates of the expanded matrix

• Quadratic mixture model

• Set of points 1,2,3,4,6,8 has the lowest D-criterion for a second order linear mixture model

Number

design

points

D-optimality Candidate point

number

6 3.5 E14 1 2 3 4 6 8

7 5.9 E14 1 2 3 4 5 6 8

8 6.6 E14 1 2 3 4 5 6 8 9

9 8.5 E14 1 2 3 4 5 6 7 8 9

Optimal design

How to deal with ill conditioning

(multicollinearity) in experiments with

mixtures?

• In many situations the additional constraints induce ill-conditioning or near-collinearities in the design matrix on top of the mixture constraint (exact collinearity)

• On top of this, the ill conditioning is increased by model formulation

• Implications for OLS o Large estimates

o Large variances and covariances of the estimates

o Incorrect signs of the estimates

o Unreliable test statistics

o Unreliable variable selection

How to deal with ill conditioning

(multicollinearity) in experiments with

mixtures?

• Duality o Parameter predicted response estimation

o Alternative optimality criteria ill conditioning

• Investigate ill conditioning of a selected optimal

design

• Decrease ill conditioning by re-design

• (Decrease ill conditioning by re-modelling)

Investigate collinearity structures

Classical approaches: VIF

OLS of y=X and svd of X=ULV’

var() = σ2 (X’X)-1 = σ2 (VL-2V’)

For the kth element of

2qkj2 2

k 2j 1 j

vvar( )

lkVIF

VIF=diag (X’X)-1 = diag(V’L-2V)

Investigate collinearity structures of constrained

mixture design matrices


W=ULV’

VIF=diag (W’W)-1 = diag(V’L-2V)

ij

ijn

2

ij

i 1

xw

x

Standardizing the terms of the model

generates the most stable parameter

estimation




VIFi can be decomposed in components associated with the respective singular values

VIF decomposition

2qij

i 2j 1 j

vVIF

l

2

ij

ij 2

j

vVIF

l The proportion of VIFi

Corresponding to lj ij

i

i

VIFVIF

VIF

High variance inflation can be allocated to eigenvectors (PC)

with small corresponding singular values lj

With X or W = ULV’



• An exploratory multivariate analysis approach

• Different factorization of the standardized design

matrix o Factorisation 1: correlational structure

o Factorisation 2: collinearity constraints

• Graphical representation by Biplots o Rank 2 approximation

o Superimposing standardized scores and loading plot

Factorisation 1: correlational structure

Starting from the standardized design matrix Z , the correlation matrix R=Z’Z

Singular value decomposition: Z=ULV’

Factorisation 1 Z=GH’

With G=U and H=VL

Then R=H’H Correlation

Z=GH’ Values of design points are orthogonal projections on the exp factors

G Standardized principal components scores

Superimposing the first and second columns of G and H in a biplot results in a rank 2 graphical approximation of the correlational structure of the design

Factorisation 2: collinearity constraints

Starting from the standardized design matrix Z

Singular value decomposition: Z=ULV’

Factorisation 2 Z=PQ’

With P=UL and Q=V

Then Q: principal component loadings

P: principal components scores

Superimposing the columns of P and Q, corresponding to the smallest singular values results in a rank 2 graphical approximation or biplot, that gives information about collinearity equations


Example: {3,1} simplex lattice

X1

X2

X3

First component

Second

com

ponent

-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1

2 3

X1

X2 X3

Id X1 X2 X3

1 1 0 0

2 0 1 0

3 0 0 1



X1

X2

X3

Correlation matrix R

1 -0.5 -0.5

-0.5 1 -0.5

-0.5 -0.5 1

Id X1 X2 X3

1 1 0 0

2 0 1 0

3 0 0 1 Singular values of X

1

1

1

VIF decomposition

VIF VIF1 VIF2 VIF3

1 1 0 0

1 0 1 0

1 0 0 1

X1

X2

X3

Id X1 X2 X3

1 1 0 0

2 0 1 0

3 0 0 1



X1

X2

X3

First component

Second

com

ponent

-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1

2

3

4

5

6

X1

X2

X3

X1X2

X1X3

X2X3

Id X1 X2 X3 X1X2 X1X3 X2X3

1 1.0 0.0 0.0 0.00 0.00 0.00

2 0.0 1.0 0.0 0.00 0.00 0.00

3 0.0 0.0 1.0 0.00 0.00 0.00

4 0.5 0.5 0.0 0.25 0.00 0.00

5 0.0 0.5 0.5 0.00 0.00 0.25

6 0.5 0.0 0.5 0.00 0.25 0.00



X1

X2

X3

Singular values of W

1.41

1.15

1.15

0.7

0.7

0.5

VIF decomposition

VIF VIF1 VIF2 VIF3 VIF4 VIF5 VIF6

1 1.5 0.1 0.2 0 0 0.8 0.4

2 1.5 0.1 0.05 0.15 0.6 0.2 0.4

3 1.5 0.1 0.05 0.15 0.6 0.2 0.4

12 1.5 0.066 0.07 0.22 0.4 0.13 0.6

13 1.5 0.066 0.07 0.22 0.4 0.13 0.6

23 1.5 0.066 0.3 0 0 0.53 0.6

Correlation matrix

1 -0.5 -0.5 0.2 0.2 -0.4

-0.5 1 -0.5 0.2 -0.4 0.2

-0.5 -0.5 1 -0.4 0.2 0.2

0.2 0.2 -0.4 1 -0.2 -0.2

0.2 -0.4 0.2 -0.2 1 -0.2

-0.4 0.2 0.2 -0.2 -0.2 1

X1

X2

X3


Example: constrained mixture Id X1 X2 X3

1 0.7 0.1 0.2

2 0.7 0.2 0.1

3 0.3 0.6 0.1

4 0.2 0.6 0.2

First component

Second

com

ponent

-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1

2 3

4

X1 X2

X3

1

3

2 4

1

2

3

0.2 0.7

0.1 0.6

0.1 0.2

x

x

x

X1

X2

X3


Example: constrained mixture

Singular values of W

1.57

0.67

0.29

Id X1 X2 X3

1 0.7 0.1 0.2

2 0.7 0.2 0.1

3 0.3 0.6 0.1

4 0.2 0.6 0.2

Correlation matrix

1 -0.97 -0.11

-0.97 1 -0.11

-0.11 -0.11 1

VIF decomposition

VIF VIF1 VIF2 VIF3

1 3.87 0.1283436 0.9900934 2.7530263

2 3.06 0.1217495 1.2260788 1.7133913

3 7.25 0.1558498 0.005732 7.0896378

1

3

2 4


Example: optimization of bread dough composition

Component Lower

bounds

Upper

bounds

Water 0.2 0.4

Flour 0.5 0.8

Salt 0.03 0.044

Additive 0.0091 0.0095

Yeast 0.0045 0.0048

Constraints

16 vertices

Extended with 65 2,3,4,5

dimensional centroids

81 candidate points 16 points D-optimal design


Example: optimization of bread dough composition with a

linear mixture model

First component

Second c

om

ponent

-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

12

34

56

78

910

1112

1314

1516

WaterFlour

Ad

Salt

Yeast


Example: optimization bread dough composition with

quadratic mixture model

1234

5678

9101112

13141516

Water1 X1X4 X1X5 Flour2

X1X2

X1X3

X2X3 Salt3

X2X4

X2X5

Yeast5

Ad4

X4X5




Fifth component

Fourth c

om

ponent

-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

WaterFlour

Ad

Salt

Yeast

Singular values of Z

1.41

1.00

1.00

1.00

2.120 E-16

PC5 = -0.7 Water -0.7 Flour -

0.005 Salt -0.001Ad -

0.001Yeast 0

‘Near’ collinearity constraint




VIF decomposition

VIF VIF1 VIF2 VIF3 VIF4 VIF5

1 320.11888 0.0389517 5.8668324 0.5888333 18.047538 295.57672

2 1363.468 0.0409593 2.334777 3.4602724 78.632188 1278.9998

3 31.279872 0.0414719 0.0990308 27.645318 0.1196886 3.3743626

4 2123.2871 0.0423696 0.0644128 1.7034599 254.33501 1867.1418

5 953.12182 0.0423552 0.0650038 1.774838 846.05622 105.18341

Singular values

of W

2.20 0.34 0.17 0.029 0.017



quadratic mixture model

Fifteenth component

Fourteen

th c

om

ponent

-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

12345678910111213141516

Water1Flour2

Ad3

Salt4

Yeast5X1X2X1X3

X1X4

X1X5

X2X3

X2X4

X2X5

X3X4X3X5X4X5

Component Singular values

1 2.84

2 1.95

3 1.43

4 1.02

5 0.19

6 0.04

7 0.03

8 0.02

9 0.018

10 0.01

11 2.8 E-16 ~ 0

12 2.6 E-16 ~ 0

13 2.2 E-16 ~ 0

14 1.5 E-16 ~ 0

15 4.8 E-17 ~ 0


Example: optimization of bread

dough composition with quadratic

mixture model

Thirteenth component

Tw

elfth

com

ponent

-1.0 -0.6 -0.2 0.2 0.4 0.6 0.8 1.0

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

12345678910111213141516Water1

Flour2

Ad3

Salt4

Yeast5

X1X2

X1X3

X1X4

X1X5

X2X3

X2X4

X2X5

X3X4X3X5X4X5


1 2.84

2 1.95

3 1.43

4 1.02

5 0.19

6 0.04

7 0.03

8 0.02

9 0.018

10 0.01

11 2.8 E-16 ~ 0

12 2.6 E-16 ~ 0

13 2.2 E-16 ~ 0

14 1.5 E-16 ~ 0

15 4.8 E-17 ~ 0


Example: optimization of bread dough

composition with quadratic mixture model

11 2.8 E-16 ~ 0

12 2.6 E-16 ~ 0

13 2.2 E-16 ~ 0

14 1.5 E-16 ~ 0

15 4.8 E-17 ~ 0


Corresponding PC are defining

the collinearity constraints

PC11 PC12 PC13 PC14 PC15

Water1 -0.29 -0.06 0.44 0.33 0.54

Flour2 0.09 -0.47 -0.22 0.39 0.49

Ad3 0.55 0.27 0.06 0.01 0.25

Salt4 -0.03 0.01 -0.04 -0.10 0.08

Yeast5 -0.06 0.13 -0.11 0.10 0.01

X1X2 0.33 -0.36 -0.58 0.05 -0.04

X1X3 -0.32 -0.25 -0.12 0.01 -0.15

X1X4 0.18 -0.07 0.21 0.49 -0.39

X1X5 0.19 -0.43 0.35 -0.32 -0.03

X2X3 -0.48 -0.27 -0.07 0.01 -0.19

X2X4 0.17 -0.06 0.23 0.49 -0.39

X2X5 0.19 -0.44 0.37 -0.33 -0.03

X3X4 0.006 -0.007 0.01 0.03 -0.02

X3X5 0.01 -0.03 0.02 -0.02 -0.003

X4X5 0.001 -0.001 0.001 -0.0002 -0.0008


PC loadings corresponding with singular values ~ 0

How to use this information for more ‘optimal’

design?

• Instead of optimizing variance properties of parameters and predicted response under OLS estimation

• Minimize correlational structure

o Scree plot of singular values

o Optimize factorisation 1

• Minimize collinearity structure

o Optimize factorisation 2

• Redesign to uniform singular value profile

Guarantee design

in the ‘full’ region

in the exchange

algorithm

• Protected design points, fi vertices

• Adapt constraints

Solutions for the bread example

• Solutions for ill conditioning o Redesign

• Categorisation of the components

• Define small ranged components as process variables

o (Remodel)

• Model reduction

• Adapted models

Conclusion

• Plenty of interesting ideas for better mixture design

optimality criteria for constrained mixture experiments by

detailled study of ill conditioning

• Correlational and collinearity structures

• Computational problems to calculate for realistic problems

in industry exchange algorithms in high dimensionality

Thank you for your attention

Questions?

designs in constrained experimental regions...lower bounds whole simplex design in pseudocomponents...

Documents