everything you wanted to know about definitive screening designs
DESCRIPTION
An introduction to definitive screening designs (DSDs). These slides describe issues with standard screening designs and how to overcome these issues by using DSDs and orthogonally blocked DSD, first introduced by Bradley Jones of SAS and Christopher Nachtsheim of the Carlson School of Management, University of Minnesota. For information about using JMP software for design of experiments and DSDs, see http://www.jmp.com/applications/doe/TRANSCRIPT
![Page 1: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/1.jpg)
Copyright © 2013, SAS Insti tute Inc. Al l r ights reserved.
EVERYTHING YOU WANTED TO KNOW
ABOUT DEFINITIVE SCREENING DESIGNS
(but were afraid to ask)
Bradley Jones
April 2014
![Page 2: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/2.jpg)
Joint work with Chris Nachtsheim
![Page 3: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/3.jpg)
DSDs Introduced in 2011
![Page 4: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/4.jpg)
Paper won both the Brumbaugh and Lloyd S. Nelson Awards
![Page 5: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/5.jpg)
Screening Design – Wish List
1. Orthogonal main effects.
2. Main effects uncorrelated with two-factor interactions and quadratic effects.
3. Estimable quadratic effects – three-level design.
4. Small number of runs – order of the number of factors.
5. Two-factor interactions not confounded with each other.
6. Good projective properties.
![Page 6: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/6.jpg)
Motivation: Problems with Standard Screening Designs
Resolution III designs confound main effects and two-factor interactions.
Plackett-Burman designs have “complex aliasing of the main effects by two-factor interactions.
Resolution IV designs confound two-factor interactions with each other, so if one is active, you usually need further runs to resolve the active effects.
Center runs give an overall measure of curvature but you do not know which factor(s) are causing the curvature.
Example: JMP Demo
![Page 7: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/7.jpg)
Solution: Definitive Screening Designs
1. Orthogonal for the main effects.
2. The number of required runs is only one more than twice the number of factors. ***
3. Unlike resolution III designs, main effects are independent of two-factor interactions.
4. Unlike resolution IV designs, two-factor interactions are not completely confounded with other two-factor interactions, although they may be correlated
5. Unlike resolution III, IV and V designs with added center points, all quadratic effects are estimable in models comprised of any number of linear and quadratic main effects terms.
6. Quadratic effects are orthogonal to main effects and not completely confounded (though correlated) with interaction effects.
7. If there are more than six factors, the designs are capable of efficiently estimating all possible full quadratic models involving three or fewer factors
![Page 8: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/8.jpg)
Six factor DSD table.
6 factors and 13 runs.
Run A B C D E F
1 0 1 -1 -1 -1 -1
2 0 -1 1 1 1 1
3 1 0 -1 1 1 -1
4 -1 0 1 -1 -1 1
5 -1 -1 0 1 -1 -1
6 1 1 0 -1 1 1
7 -1 1 1 0 1 -1
8 1 -1 -1 0 -1 1
9 1 -1 1 -1 0 -1
10 -1 1 -1 1 0 1
11 1 1 1 1 -1 0
12 -1 -1 -1 -1 1 0
13 0 0 0 0 0 0
![Page 9: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/9.jpg)
DSD structure – 1
Run A B C D E F
1 0 1 -1 -1 -1 -1
2 0 -1 1 1 1 1
3 1 0 -1 1 1 -1
4 -1 0 1 -1 -1 1
5 -1 -1 0 1 -1 -1
6 1 1 0 -1 1 1
7 -1 1 1 0 1 -1
8 1 -1 -1 0 -1 1
9 1 -1 1 -1 0 -1
10 -1 1 -1 1 0 1
11 1 1 1 1 -1 0
12 -1 -1 -1 -1 1 0
13 0 0 0 0 0 0
Six fold-over
pairs
![Page 10: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/10.jpg)
DSD structure – 2
Run A B C D E F
1 0 1 -1 -1 -1 -1
2 0 -1 1 1 1 1
3 1 0 -1 1 1 -1
4 -1 0 1 -1 -1 1
5 -1 -1 0 1 -1 -1
6 1 1 0 -1 1 1
7 -1 1 1 0 1 -1
8 1 -1 -1 0 -1 1
9 1 -1 1 -1 0 -1
10 -1 1 -1 1 0 1
11 1 1 1 1 -1 0
12 -1 -1 -1 -1 1 0
13 0 0 0 0 0 0
Center value in
each row
![Page 11: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/11.jpg)
DSD structure – 3
Run A B C D E F
1 0 1 -1 -1 -1 -1
2 0 -1 1 1 1 1
3 1 0 -1 1 1 -1
4 -1 0 1 -1 -1 1
5 -1 -1 0 1 -1 -1
6 1 1 0 -1 1 1
7 -1 1 1 0 1 -1
8 1 -1 -1 0 -1 1
9 1 -1 1 -1 0 -1
10 -1 1 -1 1 0 1
11 1 1 1 1 -1 0
12 -1 -1 -1 -1 1 0
13 0 0 0 0 0 0One overall
center run.
![Page 12: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/12.jpg)
How do you make a DSD?
DSDs are constructed using conference matrices
What is a conference matrix?
An mxm square matrix C with 0 diagonal and +1 or -1 off diagonal elements so that:
CTC (m 1)Imm
![Page 13: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/13.jpg)
Sample Conference Matrix
![Page 14: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/14.jpg)
For six factors the model of interest has 6 main effects and 15 interactions.
But n = 12, so we can only fit the intercept and the main effects:
Standard result: some main effects estimates are biased:
where the “alias” matrix is:
Screening Conundrum 1 – Two Models
14
![Page 15: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/15.jpg)
Alias Matrices
For the DSD there is no aliasing between main effects and two-factor interactions.
The D-optimal design (AKA Plackett-Burman) with one added center point has
substantial aliasing of each main effect with a number of two-factor interactions.
![Page 16: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/16.jpg)
Column Correlations for 6 factor DSD
0.25
0.5
0.4655
0.1333
0.0
![Page 17: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/17.jpg)
Are there any trade-offs versus the D-optimal design with an added center run?
Confidence intervals for the main effects are a little less than 10% longer.
![Page 18: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/18.jpg)
Screening Conundrum 2 – Confounding
Resolution IV designs confound two-factor interactions with each other.
So, if some two-factor interaction is large, you are left with ambiguity about which model is correct.
![Page 19: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/19.jpg)
Correlation Cell Plot for 8 Factor Screening Design
AB is
confounded with
CE, DH & FG
AB FG r = 1
AB DH r = 1
AB CE r = 1
![Page 20: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/20.jpg)
Correlation Cell Plot for 8 Factor DSD
|r| = 2/3
|r| = 1/6
r = 0
![Page 21: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/21.jpg)
What if some factors are categorical?
![Page 22: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/22.jpg)
DSD with categorical factors - construction
Change pairs of
zeros to different
levels
![Page 23: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/23.jpg)
DSD with a 2-level categorical factor
|r| = 0.169
|r| = 0.68
|r| = 0.5
|r| = 0.25
![Page 24: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/24.jpg)
What if you want to block a DSD?
In revision with Technometrics
![Page 25: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/25.jpg)
Constructing Orthogonally Blocked DSDs
1. Create DSD in standard order
2. Create blocks from groups of fold-over pairs
3. If you have all continuous factors, add one center run per block to estimate quadratic effects
4. If you have categorical factors, add pairs of center runs to estimate quadratic effects*
* How many pairs do I need? Answer on next slide.
![Page 26: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/26.jpg)
How many pairs?
1. Let n be the number of runs in the conference matrix, m be the number of continuous factors and b be the desired number of blocks.
2. Add b – (n – m) pairs of center runs
![Page 27: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/27.jpg)
Table of Orthogonal Blocking Alternatives
![Page 28: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/28.jpg)
Ideas for analysis of DSDs
Simplest idea – fit the main effects model.
The main effects are not biased so you can believe their magnitude. Unfortunately, if there are strong interaction or quadratic effects, the RMSE, which estimates s, will be inflated.
![Page 29: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/29.jpg)
Idea #2 for Analysis
Use a version of stepwise regression
Model:
all main effects
all quadratic effects
all two-factor interactions
![Page 30: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/30.jpg)
Idea #2 for Analysis
Procedure:
1. Enter all main effects
2. Add any large quadratic effect
3. Add any large two-factor interaction
4. Remove any main effect that is small and not featured in any 2nd order effect
5. Beware of models with more than n/2 terms because of possible over-fitting
![Page 31: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/31.jpg)
Idea #3 for Analysis
Use a version of all subsets regression
Model:
all main effects
all quadratic effects
all two-factor interactions
n/2 - 1
![Page 32: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/32.jpg)
Recapitulation – Definitive Screening Design
1. Orthogonal main effects plans.
2. Two-factor interactions are uncorrelated with main effects.
3. Quadratic effects are uncorrelated with main effects.
4. All quadratic effects are estimable.
5. The number of runs is only one more than twice the number of factors.
6. For six factors or more, the designs can estimate all possible full quadratic models involving three or fewer factors
![Page 33: Everything You Wanted to Know About Definitive Screening Designs](https://reader031.vdocuments.mx/reader031/viewer/2022012309/5478be2eb47959a4098b458f/html5/thumbnails/33.jpg)
References
1. Box, G. E. P. and J. S. Hunter (2008). The 2k−p fractional factorial designs. Technometrics 3, 449–458.
2. Goethals, J. and Seidel, J. (1967). ”Orthogonal matrices with zero diagonal”. Canadian Journal of Mathematics, 19, pp. 1001–1010.
3. Tsai, P. W., Gilmour, S. G., and R. Mead (2000). Projective three-level main effects designs robust to model uncertainty. Biometrika 87, 467–475.
4. Jones, B. and Nachtsheim, C. J. (2011) “Efficient Designs with Minimal Aliasing” Technometrics, 53. 62-71.
5. Jones, B and Nachtsheim, C. (2011) “A Class of Three-Level Designs for Definitive Screening in the Presence of Second-Order Effects” Journal of Quality Technology, 43. 1-15.