analyzing supersaturated designs using biased estimation qprc 2003 by adnan bashir and james simpson...
TRANSCRIPT
Analyzing Supersaturated Designs Using Biased Estimation
QPRC 2003
By
Adnan Bashir and
James Simpson
May 23,2003
FAMU-FSU College of Engineering, Department of Industrial Engineering
Outline
• Introduction
• Motivation example
• Research objectives
• Proposed analysis method– Multicollinearity & ridge– Best subset model– Simulated case studies– Example– Results
• Conclusion & recommendations• Future research
FAMU-FSU College of Engineering, Department of Industrial Engineering
Introduction
• Many studies and experiments contain a large number of variables
• Fewer variables are significant
• Which are those few factors? How do we find those factors?
• Screening experiments (Design & Analysis) are used to find those important factors
• Several methods & techniques (Design & Analysis) are available to screen
FAMU-FSU College of Engineering, Department of Industrial Engineering
Motivation exampleComposites Production
INPUTS (Factors)
Resin Flow Rate (x1)
Type of Resin (x2)
Gate Location (x3)
Fiber Weave (x4)Mold Complexity
(x5)
Fiber Weight (x6)
Curing Type (x7)
Pressure (x8)
OUTPUTS(Responses)
Fiber Permeability
Product Quality
Tensile Strength
Noise
Process
Raw Materials
FAMU-FSU College of Engineering, Department of Industrial Engineering
Motivation example (continued)
Response y = Tensile strength
1 2 8( , ,..., )y f x x xEach experiment costs $500, requires 8 hours, budget $3,000 (6 experiments)
FAMU-FSU College of Engineering, Department of Industrial Engineering
1 2 3 4 5 6 7 8 Y
1 1 1 1 1 1 1 1 1
2 -1 -1 -1 -1 -1 -1 1 1
3 -1 -1 -1 1 1 1 -1 -1
4 -1 1 1 -1 -1 1 -1 -1
5 1 -1 1 -1 1 -1 -1 1
6 1 1 -1 1 -1 -1 1 -1
1: High level-1: Low level
• Supersaturated Designs: number of factors m ≥ number of runs n
• Columns are not Orthogonal
Research Objectives
• Propose an efficient technique to screen the important factors in an experiment with fewer number of runs
– Construct improved supersaturated designs
– Develop an accurate, reliable and efficient technique to analyze supersaturated designs
FAMU-FSU College of Engineering, Department of Industrial Engineering
Analysis of SSDs – Current Methods
• Stepwise regression, most commonly used
– Lin (1993, 1995), Wang (1995), Nguyen (1996)
• All possible regressions
– Abraham, Chipman, and Vijayan (1999)
• Bayesian method
– Box and Meyer (1993)
Investigated techniques
• Principle components, partial least squares and flexible regression methods (MARS & CART)
FAMU-FSU College of Engineering, Department of Industrial Engineering
Analysis of SSDs – Proposed Method
• Modified best subset via ridge regression (MBS-RR)
– Ridge regression for multicollinearity
– Best subset for variable selection in each model
– Criterion based selection to identify best model
FAMU-FSU College of Engineering, Department of Industrial Engineering
Ridge Regression Motivation
Consider a centered, scaled matrix, X*
1 2
max
min
1.0 0.999*' *
0.999 1.0
:
1.999 0.001
1.9991999.0
0.001
X X
with eigenvalues
Consider adding k > 0 to each diagonal of X*'X* , say k = 0.1
1 2
max
min
1.10 0.999( *' * )
0.999 1.10
:
2.099 0.101
2.09920.8
0.101
X X kI
with eigenvalues
FAMU-FSU College of Engineering, Department of Industrial Engineering
Ordinary Least Squares
Ridge Regression
Ridge Regression
• Ridge regression estimates
where k is referred to as a shrinkage parameter
• Thus,
ˆ( ' ) 'RX X kI X Y
1ˆ ( ' ) 'R X X kI X Y
FAMU-FSU College of Engineering, Department of Industrial Engineering
Geometric interpretation of ridge regression
Ridge Regression, (continued)Shrinkage parameter
2ˆˆ ˆ'
pk
FAMU-FSU College of Engineering, Department of Industrial Engineering
•Hoerl and Kennard (1975) suggest
where p is number of parameter
• are found from the least squares solution
2ˆ ˆand
Shrinkage Parameter Ridge Trace
Ridge trace for nine regressors (Adapted from Montgomery, Peck, & Vining; 2001)
FAMU-FSU College of Engineering, Department of Industrial Engineering
Read X, Y
Select the best 1-factor modelBy OLS (k=0)
Calculate k, and find the best 2-factor model by all possible subsets
Adding 1 factor at a time to the best 2-factor model, from the remaining
variables to get the best 3-factor model
Proposed Analysis Method
FAMU-FSU College of Engineering, Department of Industrial Engineering
Cont’d.
Proposed Analysis Method
Is the stopping rule satisfied?
Adding 1 factor at a time to the best 3-factor model, from the remaining
variables to get the best 4-factor model
Is the stopping rule satisfied?
Adding 1 factor at a time to the best 7-factor model, from the remaining
variables to get the best 8-factor model
Final Model with Min. Cp
FAMU-FSU College of Engineering, Department of Industrial Engineering
Yes
Yes
No
No
Selecting the Best Model
1
1
( )100if then continue, else stop.pi pi
p
C Cdiff
C
FAMU-FSU College of Engineering, Department of Industrial Engineering
Where diff: user defined tolerance
Cp
Method Comparison-Monte Carlo
Simulation & Design of Experiments
Factors considered in the simulation study
III Fractional Factorial Design Matrix 5 22
FAMU-FSU College of Engineering, Department of Industrial Engineering
Analysis Method Comparison
• The performance measures, Type I and Type II errors
.
( . . )
No of Insignificant Factors SelectedType I error
Total No of Factors No of Significant Factors
.
.
No of Significant Factors Not SelectedType II error
No of Significant Factors
FAMU-FSU College of Engineering, Department of Industrial Engineering
Case Studies with Corresponding Models
FAMU-FSU College of Engineering, Department of Industrial Engineering
Method Comparison Results, Type I errors
FAMU-FSU College of Engineering, Department of Industrial Engineering
Factors Type I errors (%)
No. of No. of No. of Sig. Collin- Error Average Average
Runs Factors Factors earity Variance Proposed Stepwise
12 20 3 H 3.00 14.10 12.92
18 40 3 L 3.00 8.93 16.43
18 40 7 H 3.00 8.70 14.56
12 20 7 L 3.00 9.06 12.52
12 40 7 L 0.50 2.88 10.42
18 20 3 L 0.50 0.00 13.26
18 20 7 H 0.50 0.00 13.27
12 40 3 H 0.50 0.00 17.28
15 30 5 M 1.75 6.56 10.67
15 30 5 M 1.75 6.80 11.76
15 30 5 M 1.75 5.20 11.28
Method Comparison Results, Type II errors
FAMU-FSU College of Engineering, Department of Industrial Engineering
Factors Type II errors (%)
No. of No. of No. of Sig. Collin- Error Average Average
Runs Factors Factors earity Variance Proposed Stepwise
12 20 3 H 3.00 8.67 63.40
18 40 3 L 3.00 0.00 0.00
18 40 7 H 3.00 37.20 49.86
12 20 7 L 3.00 26.19 30.53
12 40 7 L 0.50 17.50 14.86
18 20 3 L 0.50 0.00 0.00
18 20 7 H 0.50 2.98 3.36
12 40 3 H 0.50 0.00 0.00
15 30 5 M 1.75 3.20 3.53
15 30 5 M 1.75 4.00 4.80
15 30 5 M 1.75 2.00 3.60
Factors Contributing to Method PerformanceType II ErrorsStepwise Method
FAMU-FSU College of Engineering, Department of Industrial Engineering
DESIGN-EXPERT PlotType II err.(SW)
A: # of runsB: # of factorsC: collinearityD: err. dist.E: # of sig. fact.
Half Normal plot
Half Norm
al %
pro
bability
|Effect|
0.00 7.88 15.77 23.65 31.54
0
20
40
60
70
80
85
90
95
97
99
A
B
C
D
E
CD
var
Factors Contributing to Method PerformanceType II Errors
DESIGN-EXPERT PlotType II err.(PM)
A: # of runsB: # of factorsC: collinearityD: err. dist.E: # of sig. fact.
Half Normal plot
Half Norm
al %
pro
bability
|Effect|
0.00 4.70 9.40 14.10 18.80
0
20
40
60
70
80
85
90
95
97
99
A
B
C
D
E
Proposed Method
FAMU-FSU College of Engineering, Department of Industrial Engineering
var
Summary Results
FAMU-FSU College of Engineering, Department of Industrial Engineering
A: No. of runsB: No. of factorsC: MulticollinearityD: Error varianceE: No. of Sig. factors
Conclusions & Recommendations
SSDs Analysis: Best Subset Ridge Regression
• Use ridge regression estimation
• Best subset variable selection method outperforms stepwise regression
FAMU-FSU College of Engineering, Department of Industrial Engineering
Future Research
Analyzing SSDs• Multiple criteria in selecting the best model• All possible subset, 3 factor model • Streamline program code • Real-life case studies• Genetic algorithm for variable selection
FAMU-FSU College of Engineering, Department of Industrial Engineering
Acknowledgement
• Dr. Carroll Croarkin, chair of selection committee for Mary G. Natrella
• Selection Committee for Mary G. Natrella Scholarship
• Dr. Simpson, Supervisor
FAMU-FSU College of Engineering, Department of Industrial Engineering