jmp demo slides
TRANSCRIPT
Laboratory for Interdisciplinary Statistical Analysis
LISA helps VT researchers benefit from the use of Statistics
Collaboration Walk In Consulting
Experimental Design • Data Analysis • Interpreting ResultsGrant Proposals • Software (R, SAS, JMP, SPSS...)
CollaborationFrom our website request a meeting for personalized statistical adviceGreat advice right now:
Walk-In ConsultingMonday—Friday* 12-2PM for questions
requiring <30 mins*Mon—Thurs during the summerGreat advice right now:
Meet with LISA before collecting your data
Short Courses
g
Designed to help graduate students apply statistics in their research
All services are FREE for VT researchers. We assist with research—not class projects or homework.All services are FREE for VT researchers. We assist with research not class projects or homework.
www.lisa.stat.vt.edu1
Using JMP® for Statistical Analysis Part IIPart II
– Design and Analysis of ExperimentsWandi Huang
Laboratory for Interdisciplinary Statistical AnalysisDepartment of Statistics, Virginia Tech
http://www.lisa.stat.vt.edu/
02/08/2011
2
Course OutlineCourse Outline
• Introduction and Basic Principles
• Introduction to Factorial Designs
• Screening Designs
• Response Surface Designs• Response Surface Designs
• Resources
3
Section Outline ‐ IntroductionSection Outline Introduction
Ob ti l St d D i f E i t (DOE)• Observational Study vs. Design of Experiments (DOE)• Terminology• EffectsEffects• Principles of Good Design • Sequential Nature of DOE• Advantages of DOE
4
DOE DefinitionDOE Definition
A designed experiment is a test or series of tests in g pwhich purposeful changes are made to the input variables of a process or system so that we may observe and identify the reasons for the changes inobserve and identify the reasons for the changes in the output response.
– From Design and Analysis of Experiments by Douglas Montgomery
5
IntroductionIntroduction
• Drawing Conclusions from a Designed Experiment vs• Drawing Conclusions from a Designed Experiment vs. from Observational Study
– A plot that shows a relationship between two variables does not necessarily prove a true cause‐and‐effect relationship
– Correlation between two variables often occurs because they yare both associated with a third factor – lurking variable
– A scatter plot is useful for identifying potential relationships, but designed experiments must be used to verify causalityg p y y
6
IntroductionIntroduction
• Key pointKey point– To determine cause and effect in a system you have to interfere with it (not just passively observe it)to interfere with it (not just passively observe it)
From George Box (Famous Statistician) ‐ 1966From George Box (Famous Statistician) 1966
7
TerminologyTerminology
• Factor• Factor– Also known as input, “X” variable, explanatory variable, or independent variable
– A variable that is controlled for purposes of the experiment. It is believed that the factor will have some relationship with the response.p p
• Level– A particular value of a factor– For example, if temperature is a factor, levels could be 250˚, 300˚ or 350˚ F
8
TerminologyTerminology
R• Response– Usually a measurement on a product or process that you are interested inare interested in
– Also known as the output, “Y” variable, response variable, or dependent variable
– An experiment may have multiple responses but often it is best to focus on the most important response(s)
The methodology discussed in this course does not– The methodology discussed in this course does not typically handle non‐continuous response variables
9
TerminologyTerminology
Noise Factors/Variables - a factor that potentially impacts the p y presponse but you are not necessarily interested in controlling
Or it is expensive to controlYou want to make the response consistent across settings of p gthe other factors (control factors)This “robustness” minimizes the impact of the noise factorExamples of Noise Factors/Variables
Raw MaterialEnvironmental ConditionsDifferent Suppliers
keep noise variables in mind when designing andanalyzing an experiment!
10
TerminologyGeneral ModelTerminology
x2 xpx1
. . .Factors
Inputs Outputsy
zqz2z1
. . .Noise
11
TerminologyTerminology
• Coded vs Uncoded Variables• Coded vs. Uncoded Variables– Using coded variables consists of coding factor levels to a scale where ‐1 is equal to the low level and 1 is equal to q qthe high level
– Provides the factors on a common or dimensionless scale
– Determines the relative impact of the factors within the design space
– Uncoded variables are the original factor levels– Uncoded variables are the original factor levels
– Statistical analysis should be done using coded variables –which JMP does
– Output will be in terms of coded variables12
TerminologyTerminology
• Coded vs. Uncoded Variables ‐ examplesCoded vs. Uncoded Variables examples
Factor Uncoded CodedMachine A -1
B 1
Temperature 300˚ -1
350˚ 0
400˚ 1
Time 15 minutes -1
60 minutes 1
EffectsEffects
• Main effect – The change in the response due to the change in a factor levelg– This effect can be linear or curved
• Interaction effect – The change in the response due to simultaneous change inresponse due to simultaneous change in factor levels
14
EffectsEffects95
90A linear main effect for a
85
factor with two levels is simply represented byTo
rque
80
represented by a straight line
75
Low High
Line Speed
15
EffectsEffects95
85
90A main effect for a factor with more than
80
85 with more than 2 levels is similar
Torq
ue
70
75 Note – Only consider the main effects if
T
65
Low Middle High
no interaction is present
Low Middle High
Line Speed16
Interaction EffectInteraction Effect
• If an interaction effect is present, lines on an interaction plot will not be parallel
• If the interaction is rque
present, then using the main effects will be misleading
To
Low HighClutch Setting (High)
Clutch Setting (Low)Line Speed
Low High Clutch Setting (Low)
Principles of Experimental DesignPrinciples of Experimental Design
Good experimental designs will employ the following principles where appropriate:
– Randomization– Replication (instead of just repetition)– Blocking
18
Principles of Experimental DesignPrinciples of Experimental Design
• Randomization both the assignment of the• Randomization – both the assignment of the experimental units to factor combinations and the order in which the individual runs of the experiment are
f d d l d t i dperformed are randomly determined– Randomization is a valuable device for dealing with unavoidable sources of variability and reduce the risk of an unknown or unexpected occurrence jeopardizing accurate conclusions
– It helps to avoid time effectsp– By properly randomizing the experiment, one “averages out” the effects of noise variables that may be presentbe present
19
Principles of Experimental DesignPrinciples of Experimental Design
• Factor combination – a specific combination of factor levelsFactor combination a specific combination of factor levels that is used in an experiment. Also known as a treatment combination
• Experimental Unit ‐ the smallest entity to which a factor combination is applied
• Observational Unit – the entity that is measured– Not necessarily the same as the experimental unit
– If an experimental unit is measured multiple times, each measurement isIf an experimental unit is measured multiple times, each measurement is an observational unit
• A run occurs each time we impose and carry out a particular f bi i i l i l ifactor combination on a particular experimental unit– We can have multiple measurements on each run
20
Principles of Experimental DesignPrinciples of Experimental Design
R li ti Th f f th f t• Replication – The performance of the same factor combination multiple times on different experimental units. Each different experimental p punit is a replicate
• Repetition – is when the same experimental unit is measured more than once or when several units are sampled during the same run.– When repetition occurs the experimental unit and– When repetition occurs, the experimental unit and observational unit are not the same
• An experiment can have replication and/or repetition
21
Replication ExampleReplication Example
• The goal of the Baby Care experiment was to identifyThe goal of the Baby Care experiment was to identify process variables that have a significant impact on product quality– Several factors considered:
– For each factor combination, the levels of each factor were set, then 15 pads were tested
– One factor combination was done at two different times so a total of 30 pads were tested
– The 15 are repetitions, and the factor combination performed at two different times are replicates
Principles of Experimental DesignPrinciples of Experimental Design
• Blocking – is a technique that is used to account for theBlocking is a technique that is used to account for the unwanted variation that could be caused by non‐homogeneous conditions, so they do not distort the analysis of the factors that are of interestthe factors that are of interest– In a designed experiment, a block is a portion of the experimental
material that should be more homogeneous than the entire set of material
– Blocking involves making comparisons among the experimental conditions within each block
– Multiple randomization schemes are needed ‐ both the blocks and within blocks
– One is usually not interested in estimating the effects of the blocking variable
– Examples: machines days operators batches of raw materialExamples: machines, days, operators, batches of raw material
23
Blocking in a Designed ExperimentBlocking in a Designed Experiment
E lExample
I h i l i t l fIn a chemical process experiment only four experimental trials can be made from a single batch. In this experiment we are studying the effects of fourIn this experiment we are studying the effects of four different binder types on properties of the final product. Four batches of materials were made.
24
Blocking in a Designed ExperimentBlocking in a Designed Experiment
Batch 1 Batch 2 Batch 3 Batch 4
B1 = 89.5B3 = 93.0
B1 = 88.9B2 = 93.7
B4 = 97.3B3 = 96.9
B1 = 96.7B3 = 101.1B3 93.0
B2 = 89.1B4 = 96.4
B2 93.7B3 = 93.7B4 = 97.4
B3 96.9B1 = 95.1B2 = 95.3
B3 101.1B2 = 97.2B4 = 100.9
Mean = 92.0 Mean = 93.4 Mean = 96.2 Mean = 99.0
25
Blocking in a Designed Experimentg g p
100
105
Binder 1
90
95
Yie
ld
Binder 1Binder 2Binder 3
80
85
90Y Binder 3Binder 4
80Batch 1 Batch 2 Batch 3 Batch 4
Excel output26
Blocking in a Designed Experimentg g p
File → Open 1-1Binder.JMPAnalyze → Fit Y by XAnalyze → Fit Y by X▼Red Triangle → Quantiles▼Red Triangle → Means/ANOVA
27Not SignificantMSE = 11.3
Blocking in a Designed ExperimentBlocking in a Designed Experiment
Accounting for the variability in the batches in the statisticalAccounting for the variability in the batches in the statistical analysis
Analyze → Fit Model
Significant
MSE = 2.3
Notice the great reduction in the estimated experimental (MSE) h b t h i d bl ki ff terror (MSE) when batch is used as a blocking effect
Sequential ExperimentationSequential Experimentation
• Experimentation is an iterative processExperimentation is an iterative process– It is often best to do a smaller experiment
initially, and then do additional experiments based on the initial results
• Sir Ronald Fisher once said “the best time to design an experiment is after you’ve done it.”you e do e t
• “It is best not to plan a large ‘all‐encompassing’ experiment at the outset because this is the time when you knowbecause this is the time when you know least about the system.” G.E.P. Box
Sequential ExperimentationSequential Experimentation
Purpose
Screening DesignNarrow down a list of many factors to the
most important onesmost important ones
Understand relationshipsFactorial Design
Understand relationships and interactions between
factors and response
Response Optimize response by interpolating factor Surface Designinterpolating factor
settings30
Advantages of DOEAdvantages of DOE
• More efficient than one‐factor‐at‐a‐time experimento e e c e a o e ac o a a e e pe e
• Avoids misleading conclusions that can occur when interactions are present
• Factors are not confounded with one another– Confounding occurs when you cannot distinguish which factor is
ll i ti threally impacting the response
• Can create a mathematical model useful for prediction
• Yields conclusions that are valid over a range of• Yields conclusions that are valid over a range of experimental conditions– Interpolation is valid (with continuous factors)
31
Course OutlineCourse Outline
• Introduction and Basic Principles
• Introduction to Factorial Designs
• Screening Designs
• Response Surface Designs• Response Surface Designs
• Resources
32
Introduction to Factorial DesignsIntroduction to Factorial Designs
• In a standard factorial design, at least one run is made at each of the possible factor combinations
The number of runs required can be quite large!– The number of runs required can be quite large!• 2 factors with 3 levels each: 32=9 factor combinations• 3 factors with 3 levels each: 33=27 factor combinations• 4 factors with 3 levels each: 34=81 factor combinations
– Most common type are those using only 2 levels for each factor – 2k designsfor each factor 2 designs
• k is the number of factors
33
Introduction to Factorial DesignsIntroduction to Factorial Designs
• 2k factorial design is a very important special case of2 factorial design is a very important special case of a factorial design where each of the k factors of interest has only two levels – Useful at early stages of development work as a screening experiment (with not too many factors)
U f l f d t di i t ti b t f t– Useful for understanding interaction between factors
– Can not adequately evaluate curvature in the response function (will discuss this more later)u ct o ( d scuss t s o e ate )
34
ExampleHighTablet Dissolution
Example
Relative Humidity
Example -23 Factorial Design
High
Low
Pan Speed
Inlet Air Humidity
Low High Low
Pan Speed
Study how Pan Speed, Inlet Air Humidity and Relative H idit ff t T bl t Di l tiHumidity affect Tablet Dissolution
35
Example23 Factorial Design - The data table
Example
Run Pan Speed
Relative Humidity
Inlet Air Humidity
Tablet Dissolution
1 9 35 7 89.8
2 9 35 14 89.7
3 9 65 7 84.6
4 9 65 14 87.9
5 11 35 7 87.2
6 11 35 14 88.8
7 11 65 7 70 17 11 65 7 70.1
8 11 65 14 73.1
Create a 23 Factorial Design in JMPCreate a 2 Factorial Design in JMP
• DOE → Full Factorial Designg
Create a 23 Factorial Design in JMPCreate a 2 Factorial Design in JMP
• In this table, enter the values for Tablet ,Dissolution manually
Studying Main Effects in a 2k Factorial DesignStudying Main Effects in a 2 Factorial Design
Recall that a Main Effect is the change in the response due to the change in a factor level.
This is computed by averaging responses across all of the levels of all of the other factors.
There will be a main effect for each factor. Because they are averages, they are only useful when there is no interaction effect present.
39
Studying Main Effects in a 2k
73.187.9
Factorial Design
70.184.6
88 0Y 79 8Y =
88.889.7
88.0LowY = 79.8HighY =
Pan Speed
Low High
89.8 87.2
Pan Speed
So to calculate the main effect of Pan Speed we take the average response at the high level and g p gsubtract the average response at the low level.
The main effect of Pan Speed is –8.2 40
Studying Main Effects in a 2k
78 9H hY =
Factorial Design
High73.187.9
78.9HighY
70.184.6
Relative Humidity
Likewise, the main effect of Relative
Low88.889.7
of Relative Humidityis –10.0
89.8 87.2
88.9LowY =41
Studying Main Effects in a 2k
84 9Y =
Factorial Design
73.187.9
84.9HighY
70.184.6Likewise, the
main effect of Inlet Air
88.889.7
of Inlet Air Humidity
is 2High
89.8 87.2 Inlet Air HumidityLow
g
82.9LowY =42
Studying Interaction Effects in a 2k Factorial DesignStudying Interaction Effects in a 2 Factorial Design
Interaction between factors – When the difference inInteraction between factors When the difference in the response between the low level and high level of a factor is not the same at all levels of the other factors
In other words, the main effect for one factor depends pon the level of the other factor.
43
Studying Interaction Effects in a 2k
73.187.986 3Y = 716H h PSY =High
Factorial Design
70.184.6
86.3Low PSY − = 71.6High PSY −
Relative Humidity
88.889.7 Low
Pan Speed
Low High
89.8 87.288.0High PSY − =89.8Low PSY − =
Pan Speed
The average effect (or main effect) of Pan Speed At High Relative Humidity is –14.7 (71.6-86.3)
There is likely an interaction between Pan
Speed and Relative Humidity
The average effect (or main effect) of Pan Speed At Low Relative Humidity is –1.8 (88.0-89.8)
44
JMP: Analysis of DesignJMP: Analysis of Design
• We’re going to examine this interaction in JMP. To do this we need to build a model, which we will be using for the next several slidesfor the next several slides . . .
• The JMP file for the dissolution datashould have a model “built in”should have a model built in
• ▼Red Triangle (next to “Model”in the upper left panel of the data table)→ S i→ Run Script
• If this red triangle is not present,Analyze→ Fit ModelAnalyze → Fit Model
JMP: Analysis of Design
The main effects and two way Make sure the
JMP: Analysis of Design
interactions should already be listed in the Construct Model Effects section
response variable is listed
If not, make sure that
Do the Minimal Reportthe degree is “2”
Click on the 3 factors (using CTRL-click)
Reportemphasis
Click on( g )
Click Macros, then click Factorial to degree
Click on Run
JMP: Interaction PlotJMP: Interaction Plot
Now we can get the plot!Now we can get the plot!▼Red Triangle (next to “Response Tablet
Dissolution”) → Factor Profiling → Interaction PlotsIf the lines are (nearly) parallel,
then there is no interaction.
) g
ConclusionsThere appears to be an interaction between Pan Speed and Relative Humidity. Other interactions are not very strong or non-existent.
Al th i t bi i ff tAlso: there is not a big main effect for Inlet Air Humidity.
47
Interaction Plot ExamplesInteraction Plot ‐ Examples
90No interaction
80
85
90
1
75
80 -1
1 0 1
Strong interaction
Weak interaction
-1 0 1
8590 -1
85
90
707580
175
80
85
-111
-1
-1 0 175
-1 0 1
1
Analysis of Factorial DesignsAnalysis of Factorial Designs
• A first order regression model with interactions is often• A first‐order regression model with interactions is often used in the case of a 2k factorial design
• Model is used with “coded variables” (‐1 to 1)• β0 is defined as the intercept and is the grand average of all
the data• β is defined as the regression coefficient for the ith factor• βi is defined as the regression coefficient for the i factor
0
k k
i i ij i jy x x xβ β β= + +∑ ∑ ∑01 2
i i ij i ji i j
y β β β= < =∑ ∑ ∑
M i TMain Effects
Two-wayInteractions
Analysis of Factorial DesignsAnalysis of Factorial Designs
• Analysis of Variance is used in order to determine if ycollectively the regression coefficients are statistically significant
t t ti ti d t d t i if th i di id l ffi i t• t‐statistics are used to determine if the individual coefficient has a statistically significant effect on the response variable
Hypothesis for testing individualregression coefficients
The test statistic for the hypothesis
0
1
: 0: 0
i
i
HH
ββ
=≠
0
ˆˆ( )i
i
tseββ
=1 iβ
50
JMP: Analysis of DesignJMP: Analysis of Design
Act al b Predicted PlotResponse Tablet Dissolution
Scroll back up in the Fit Model
80
85
90
Tabl
etso
lutio
n A
ctua
l
Actual by Predicted Plot Scroll back up in the Fit Model output window (where we made
the interaction plots)
70
75
Dis
s
70 75 80 85 90Tablet Dissolution Predicted
P=0.0641 RSq=1.00 RMSE=0.7071
Analysis of Variance
If this is not significant, it is not RSquareRSquare AdjRoot Mean Square ErrorMean of ResponseObservations (or Sum Wgts)
0.9988290.9918020.707107
83.98
Summary of Fit likely that the individualt-tests will be significant
Observations (or Sum Wgts) 8
ModelError
Source61
DF426.42000
0 50000
Sum ofSquares
71.07000 5000
Mean Square142.1400
F Ratio
Prob > F
Analysis of Variance Sometimes it is reasonable to use a higher alpha level (e.g., 0.10) while in early stages of Error
C. Total17
0.50000426.92000
0.50000.0641
Prob > F ) y gexperimentation
51
JMP: Analysis of DesignJMP: Analysis of Design
InterceptPan Speed(9 11)
Term83.9
4 1
Estimate0.250 25
Std Error335.60
16 40
t Ratio0.0019*0 0388*
Prob>|t|
Parameter Estimates
Pan Speed(9,11)Relative Humidity(35,65)Inlet Air Humidity(7,14)Pan Speed*Relative Humidity
-4.1-4.9750.975
-3.225
0.250.250.250.25
-16.40-19.90
3.90-12.90
0.0388*0.0320*0.15980.0493*
A statistical test on whether or not βi is = 0 is the same
Pan Speed*Inlet Air HumidityRelative Humidity*Inlet Air Humidity
0.1750.6
0.250.25
0.702.40
0.61120.2513
A statistical test on whether or not βi is 0 is the same thing as testing whether or not the ith factor is significant.
These parameter estimates will only be in coded form if you’ve
Caution:p y y
created the design in JMP and entered in the response data.If you bring in the data from Excel and analyze it, be sure to
enter the factors in coded variables. 52
JMP: Removing Insignificant Model TermsJMP: Removing Insignificant Model Terms
Go back to the “Fit Model” dialog.
Select the two non-Select the two nonsignificant interactions and click on Remove.
Then click on Run again.
53
JMP: Removing Insignificant Model TermsJMP: Removing Insignificant Model Terms
Here is our final model:
RSquareRSquare Adj
0.9915090 980188
Summary of Fit
Response Tablet Dissolution
Measures the model’s capability to fit the present data. R t th ti f i ti iRSquare Adj
Root Mean Square ErrorMean of ResponseObservations (or Sum Wgts)
0.9801881.099242
83.9 8
Analysis of Variance
Represents the proportion of variation in the response data that is explained by the model
ModelErrorC. Total
Source 4 3 7
DF 423.29500 3.62500
426.92000
Sum of Squares 105.824 1.208
Mean Square 87.5783
F Ratio
0.0019Prob > F
y
All the effects are very significant except for the
main effect of Inlet Air
InterceptPand Speed(9,11)
Term 83.9 -4.1
Estimate0.3886410.388641
Std Error215.88-10.55
t Ratio<.00010.0018
Prob>|t|
Parameter Estimatesmain effect of Inlet Air
Humidity, which is marginally significant
(could remove this effect)Relative Humidity(35,65)Inlet Air Humidity(7,14)Pand Speed*Relative Humidity
-4.975 0.975 -3.225
0.3886410.3886410.388641
-12.80 2.51 -8.30
0.00100.08700.0037
( )
54
Model BuildingModel Building
• Model building is an iterative procedureg p– Goal is to get an accurate, parsimonious representation of reality
• Start the analysis by including all the factors and the interactions between them
l h ff h h f• Eliminate those effects which are not significant– Eliminate non‐significant interactions first
Th li i t i ifi t i ff t if th– Then eliminate non‐significant main effects if they are not included in any significant interactions
“All models are wrong
55
All models are wrong, some are useful.”
– George Box
Model Checking in Factorial DesignsModel Checking in Factorial Designs
• Evaluating Model Adequacy is an important part of the g q y p pregression analysis– It is always important to examine the fitted model to ensure
it provides an adequate approximation to the true systemit provides an adequate approximation to the true system
– We are essentially verifying the assumptions that allow us to do regression
• Normality of the response
• Equal variance of the response
Proceeding with exploration and optimization of the fitted– Proceeding with exploration and optimization of the fitted response surface will likely give poor or misleading results unless the model is an adequate fit
56
Model Checking in Factorial DesignsModel Checking in Factorial Designs
• The residuals play an important role inˆ( )y y−The residuals, play an important role in judging model adequacy– Normal quantile plot (also known as the normal probability
( )i iy y
plot) of the residuals checks the normality assumptions• If the residuals plot approximately along a straight line, then the normality assumption is satisfied
– Plot of residuals vs. the predicted response, • If the residuals are scattered randomly on the plot, then it can be assumed that the variance is constant for all values of y. y
57
JMP: Checking for Equal Variance with a Plot f h d l
▼Red Triangle (upper left of Fit Model results window)
of the Residuals
g→ Row Diagnostics → Plot Residual by Predicted
Scroll down to see plot
The residuals havei il d th 0 5
1.0
1.5
Res
idua
l
Residual by Predicted Plot
Response Tablet dissolution
similar spread across thex-axis (but it is hard to besure for the small x values
1 5
-1.0
-0.5
0.0
0.5
Tabl
et d
isso
lutio
n R
since there isn’t a lot of data)-1.5
70 75 80 85 90 95Tablet dissolution Predicted
58
Model Checking in Factorial DesignsModel Checking in Factorial Designs
Any pattern in a residuals plot (against run order) suggests
1
2
3
interdependence among the runs
-3
-2
-1
0
1
Res
idua
ls
Individual Measurement of Column 7
Control Chart
Megaphone shape –increasing variability
↓
Stable – no problems →
-45 10 15 20 25 30 35 40 45 50
Sample
-10
-5
0
5
10
15
Res
idua
l
1 0
1.5
2.0
5 10 15 20 25 30 35 40 45 50Sample
-1.0
-0.5
0.0
0.5
1.0
Res
idua
ls ← Curvature oftenindicates “modelmisspecification”
-1.55 10 15 20 25 30 35 40 45 50
Sample
These plots can also be good for detecting outliers
JMP: Checking for Normality of the Residuals
▼Red Triangle (in Fit Model results window)
JMP: Checking for Normality of the Residuals
g→ Save Columns → Residuals
Analyze → Distribution 1.5.01 .05.10 .25 .50 .75 .90.95 .99
Residual Tablet dissolution
Distributions
y* Select “Residual Tablet
Dissolution” for “Y, Columns”* ▼Red Triangle (next to “Residual 0
0.5
1
.01 .05.10 .25 .50 .75 .90.95 .99
▼Red Triangle (next to ResidualTablet Dissolution”)→ Normal Quantile Plot -1
-0.5
0
The normality assumption is satisfiedsince the black points are near thediagonal line
-1.5
-3 -2 -1 0 1 2 3
Normal Quantile Plot
diagonal line.
60
Predictions in JMP
JMP lets you save something called a Prediction Formula
Predictions in JMP
▼Red Triangle (in Fit Model results window)→ Save Columns → Prediction Formula
Return to the data table (leave results window open!) and right click on the column header for “Pred Formula Tablet Dissolution”; choose “Formula” from the menuDissolution ; choose Formula from the menu
61
Predictions in JMPPredictions in JMP
The mathematicalThe mathematical model in terms of
the original variables is shown in the bottom of the
dialog box.dialog box.
If desired, it can be copied/pasted intocopied/pasted into other programs.
62
Optimization in JMP
Go back again to the Fit Model output window
Optimization in JMP
▼Red Triangle → Factor Profiling → Profiler
Scroll down to see the profilerScroll down to see the profiler
▼Red Triangle(next to theProfiler)→ Desirability
Functions
▼Red Triangle(next to theProfiler)→ Maximize Desirability
63
Optimization in JMPOptimization in JMP
80
85
90
Tabl
etso
luti
on0.
725
6563
3
Prediction Profiler
To maximize Tablet
Di l ti P70
75T
Dis
s 90?.
760.
751
abili
ty38
23Dissolution, Pan
Speed and Relative
Humidity should0
0.25
Des
ira
0.99
9
9.5 10
10.5 11 35 40 45 50 55 60 65
35
7 9 11 13
14
0
0.25 0.5
0.75 1
Humidity should be at their low levels and Inlet Air Humidity at9
Pan Speed
Relative
Humidity
Inlet Air
Humidity Desirability
Air Humidity at its high level
The Profiler will tell you theThe predicted response and The Profiler will tell you the factor settings that meet
your objective
The predicted response and its variability is also shown
64
DOE from Beginning to EndDOE from Beginning to End
1. Set up the designp g2. Collect data and input into software (JMP)3. Fit a model with all main effects and
interactions 4. Analysis
1. Model building2. Check for model adequacy on final model3 ANOVA/Parameter estimates/Summary of Fit3. ANOVA/Parameter estimates/Summary of Fit4. Interaction plots5. Prediction Formula6. Optimization
65
Course OutlineCourse Outline
• Introduction and Basic Principles
• Introduction to Factorial Designs
• Screening Designs
• Response Surface Designs• Response Surface Designs
• Resources
66
Section Outline ‐ Screening DesignsSection Outline Screening Designs
• Basic principles• Basic principles
• Fractional factorial designs
• Plackett – Burman designs
• Setup in JMPp
• Analysis in JMP
67
Screening DesignsScreening Designs
• A screening design is used when you are interested inA screening design is used when you are interested in determining which factors are most influential on a response with a small number of runs
• More interested in reducing number of factors than their• More interested in reducing number of factors than their interactions
• 2 major types are– Fractional factorial design– Plackett‐Burman design
68
TerminologyTerminology
• Confounding (also known as aliasing)– A confounding design is one where 2 or more treatment (main and/or interaction) effects are estimated by the same linear combination of the experimental observations
– Exists when a change in the response is due to multiple effects
– It is not possible to determine which effect is really causing h hthe change
– Can occur accidentally in poorly designed experiments– Done deliberately and systematically in screening designs– In general, we confound main effects and 2 way interactions with higher order interactions
– Most software generated designs will give you the g g g yminimum amount of confounding possible
69
Fractional Factorial DesignsFractional Factorial Designs
Oft b t f ll b f f t• Often best for a smaller number of factors– Requires 4, 8, 16, 32, 64, etc. runs
– Uses a fraction of the runs from a full factorial design– Uses a fraction of the runs from a full factorial design
• Notation ‐ 2k‐p
– k is the number of factors consideredk is the number of factors considered
– p is the level of fractionation
• 23‐1, 24‐1, 25‐1, etc. are half fraction designs
• 24‐2, 25‐2, 26‐2, etc. are quarter fraction designs, , , q g
70
Fractional Factorial DesignsFractional Factorial Designs
Resolution 3Resolution 4Resolution 5
Design Resolution Table for Fractional Factorial Designs
2 3 4 5 6 7 8 9 10 11 12 13 14 154 Full 1/2
Number of Factors
8 Full 1/2 1/4 1/8 1/1616 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/512 1/1024 1/204832 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/512 1/102464 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/51264 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256 1/512128 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128 1/256256 Full 1/2 1/4 1/8 1/16 1/32 1/64 1/128512 Full 1/2 1/4 1/8 1/16 1/32 1/64
71
Fractional Factorial DesignsFractional Factorial Designs
-,+,+ +,+,+Example – 23-1
Runs not performed -,+,- +,+,-
p
ctor
CFa
+,-,+
Factor B+,-,--,-,-Runs performed
Factor A
Fractional Factorial Design ExampleFractional Factorial Design Example
• Resistivity of wafery– Five factors in a manufacturing process for an integrated circuit were investigated in a 25‐1 design with the objective f l i h th f t ff t th i ti it f thof learning how these factors affect the resistivity of the
wafer
– 5 factors – each with 2 levels• implant dose, temperature, time, oxide thickness, furnace position
– 25‐1 fractional factorial design – Resolution 5 design2 fractional factorial design Resolution 5 design• 16 runs were done
– Every main effect is confounded with a four‐factor interaction
Plackett‐Burman DesignsPlackett Burman Designs
• Most useful when you havemany factors to study• Most useful when you have many factors to study– Only exist for set number of runs – such as 12, 20, 24, 28, 36, 40, 44, 48, etc.
– Any multiple of 4 that is not a power of 2• Don’t exist for 4, 8, 16, 32, 64, etc.
– Only for 2 levels per factorOnly for 2 levels per factor
• Can only study main effects– Resolution 3 designsg– Some projection properties to a small number of significant factors
74
Plackett‐Burman DesignsPlackett‐Burman Designs
• Example – 10 factors could be studied withExample 10 factors could be studied with a 12 run Plackett‐Burman design
Setup of Screening Designs in JMPSetup of Screening Designs in JMP
Using JMP to setup a screening design will g p g ggreatly simplify the analysis
DOE → Screening DesignThe default is to include a response called “Y”, if you want to change it
dd l i l li k hor add multiple responses click here
Add the appropriate number of Continuous or Categorical factorsContinuous or Categorical factors
You can fix mistakes by clicking on Remove Selected
76
Setup of Screening Designs in JMPSetup of Screening Designs in JMP
Double click on the Name or Values to change the factorchange the factor name or level –These are the values that will appear in your
output and on all graphs
When you are ready, click Continue
77
Setup of Screening Designs in JMPSetup of Screening Designs in JMP
Possible designs and their effects
that can bethat can be estimated are
shown
Pick the appropriate design
Scroll down to the bottom and click
Continue
78
Setup of Screening Designs in JMPSetup of Screening Designs in JMP
Make sure that youTo see what effects are confounded with each
other check here but don’t change the Make sure that you randomize the runs
other check here – but don t change the generating rules or aliasing
Don’t add replicates
When you areWhen you are ready to generate the design click
Make Table
79
Analysis of Screening DesignsAnalysis of Screening Designs
• There are two methods for analyzing screening designs
h l h d ( h (h lf ) l l )• Graphical methods (such as (half‐) normal plot)– This is the only option whennumber of runs ≤ (number of effects + 1)u be o u s ( u be o e ects )
• Standard analysis using the parameter estimates, significance testing and model building– This is possible only whennumber of runs > (number of effects + 1)
80
Analysis of Screening Designs in JMPAnalysis of Screening Designs in JMP
• Standard procedure for analyzing a screening design is:1. Setup the designp g
2. Enter the response data
3. Run scriptp
4. Analysis1. Model buildingg
– Half normal plot
– ANOVA
2 F ll i t2. Follow up experiments
81
Analysis of Screening Designs in JMPAnalysis of Screening Designs in JMP
• Resistivity example
• DOE →
Screening DesignScreening Design
82
Analysis of Screening Designs in JMPAnalysis of Screening Designs in JMP
• Resistivity example
• Enter data for Resistivity
83
Analysis of Screening Designs in JMPAnalysis of Screening Designs in JMP
• Resistivity example
JMP file should have a model “built in”
• ▼Red Triangle (next to “Model”• ▼Red Triangle (next to Modelin the upper left panel of the data table)→ Run Script→ Run Script
• If this red triangle is not present,Analyze→ Fit Model andAnalyze → Fit Model and
add all main and 2‐way effects
R th d l• Run the model84
Analysis of Screening Designs in JMPAnalysis of Screening Designs in JMP
Step 1 – Model Building
(optional) Click to change the default normal plot to a half normal plot
▼Red Triangle → Effect Screening → Normal Plot
(optional) Click to change the default normal plot to a half normal plot
Signif. effects: dose, temperature, time, dose*temperature 85
Analysis of Screening Designs in JMPAnalysis of Screening Designs in JMP
Final Model
86
Analysis of Screening DesignsAnalysis of Screening Designs
A l i th d i th f t i l• Analysis can then proceed as in the factorial designs case on the most significant terms
• Follow up experiments– Now that the most important factors have been identified, additional experimentation can be done to better understand interactions and possible curvatureand possible curvature
87
Course OutlineCourse Outline
• Introduction and Basic Principles
• Introduction to Factorial Designs
• Screening Designs
• Response Surface Designs• Response Surface Designs
• Resources
88
Section Outline –Response Surface Designs
• TerminologyTerminology• Types
– Central Composite Designp g– Box‐Behnken Design
• Setup• Graphical Methods
– Contour PlotR S f Pl t– Response Surface Plot
• Analysis
89
Response Surface DesignsResponse Surface Designs
• Appropriate when you have continuous factors– And you want to be able to interpolate in order to optimize the response
• Best for a smaller number of factors• Best for a smaller number of factors– Most economical for fewer than 6 factors
• Useful when there is curvatureUseful when there is curvature– A non‐linear relationship between the factors and response
l f l d h d• Also RSD are often implemented when prediction is of utmost importance
90
Response Surface DesignsResponse Surface Designs
• Center points• Center points– Points at the center of the design region (all the factors at the middle of the high and low level settings)
– Often replicated to get estimates of experimental error
– Used in factorial designs to check for curvature (will ll f d ff )not allow estimation of quadratic effects)
– More often used in response surface designs in conjunction with axial points tomodel curvatureconjunction with axial points to model curvature
91
Response Surface DesignsResponse Surface Designs
• Axial points• Axial points– Points along the edges or outside of the design region formed by factorial pointsregion formed by factorial points
– Required if curvature is to be modeled
92
Example ‐ Response Surface iDesigns – 3 Factors
Red points are theBl i t Red points are the factorial points
Blue points are the axial points
G i tGreen point is the center point(s)
93
Response Surface DesignsResponse Surface Designs
• 2 major types of Response Surface Designs• 2 major types of Response Surface Designs (RSD)
C t l C it D i (CCD)– Central Composite Designs (CCD)
– Box Behnken Designs (BBD)
94
Central Composite DesignsCentral Composite Designs• A combination of factorial, axial, & center points
• Axial points can be “face centered” – using only combinations of high, low, and middle levels
• Or they can be “outside” – using more extreme factor levels
95
Box‐Behnken DesignBox Behnken Design• For three factors, the points are on the edges of the cube
• More extreme factor combinations are not performed (the corner points)
• Cannot be done sequentiallyCannot be done sequentially
• Appealing when extreme conditions (corners) are not runnable or of interest
96
Setup of RSDSetup of RSD
The set up of a RSD is similar to that of Factorial pand Screening Designs
DOE → Response Surface DesignThe default includes a response called “Y” – go here to change it
dd l i lor add multiple responses
Add the appropriate number of Continuous factors (minimum 2)Continuous factors (minimum 2)
You can change factor namesand factor levels as needed
Click “Continue” to proceed 97
Setup of RSDSetup of RSD
Select the appropriate
design
Then click on Continue
98
Setup of RSDSetup of RSDYou can change the location of
the axial pointsMake sure
that you randomize
the axial points
the runs
Specify the # of additionaladditional
replicates and center points
When you are ready to make the design table
click Make Table
99
Analysis of RSDAnalysis of RSD
• When the curvature in the true response surface is strong enough the first‐order model with interactions is inadequateinteractions is inadequate
• A second‐order model (shown below) will likely be required in these situationsbe required in these situations
20
k k k
i i ij i j ii iy x x x xβ β β β= + + +∑ ∑ ∑ ∑01 2 1
i i ij i j ii ii i j i= < = =∑ ∑ ∑ ∑
LinearEffects
Two-way Interactions
QuadraticEffectsEffects Interactions Effects
Analysis of RSDAnalysis of RSD
M d l b ildi i RSD i th it i• Model building in RSD is the same as it was in factorial designs
• First eliminate non significant quadratic effects• First eliminate non‐significant quadratic effects and interactions before eliminating main effects
• Preserve model hierarchy• Preserve model hierarchy– If a factor is involved in an interaction or quadratic effect, the main effect should be included
101
Analysis of RSDAnalysis of RSD
• Standard procedure for analyzing a RSD is:p y g1. Setup the design2. Enter the response data3. Include all main effects, 2 way interactions and
quadratic terms 4. Analysisy
1. Model building2. Check for model adequacy on final model3. ANOVA/Parameter estimates/Summary of Fit3. ANOVA/Parameter estimates/Summary of Fit4. Graphical Methods
» Interaction plots» Optimizationp» Contour Plots & Response Surface Plots
5. Conclusions102
Example Breadwrapper ExperimentExample – Breadwrapper Experiment
• An experiment was conducted to study theAn experiment was conducted to study the strengh of breadwrapper stock in grams per square
– 3 factors: sealing temperature, cooling temperature, percent additive
A CCD d ith t i t– A CCD was used with one center point
– Objective is to maximize the breadwrapper strengh
Example – Breadwrapper Experiment
• We setup the design and run the experiment just as
Example – Breadwrapper Experiment
we did for factorial and screening designs
Example – Breadwrapper Experiment
• Manually enter the values for Strength
Example – Breadwrapper Experiment Example – Breadwrapper ExperimentExample – Breadwrapper Experiment
• This JMP file should have a model “built in”
• ▼Red Triangle (next to▼Red Triangle (next to“Model” in the upper left panelof the data table) → Run Script)→ p
• If this red triangle isn’t there,use Analyze→ Fit Modeluse Analyze → Fit Model
106
JMP: Analysis of DesignJMP: Analysis of Design1. The main effects, two way interactions, and quadratic terms
3. Make sure the response
should already be listed in the construct model effects section
response variable is listed
4. Do the Minimal Report2. If not, click on Report
emphasis
5 Click on
the 3 factors (using CTRL-click)
Click Macros, 5. Click on Run
,then click
Response Surface
Analysis of RSD
Step 1 – Model Building
Analysis of RSD
Use output and pprocess knowledge to help determine
which non-significant c o s g caeffects to eliminate
from the model
Fi tFirst, removenon-significant
interactions and quadratic terms
108
Analysis of RSD
Step 1 – Model Building
Analysis of RSD
Here is the analysis after eliminating the insignificant terms
109
Analysis of RSD
Step 2 – Model Checking (Equal Variance)
Analysis of RSD
▼Red Triangle (upper left of Fit Model results window)▼Red Triangle (upper left of Fit Model results window)→ Row Diagnostics → Plot Residual by Predicted
Scroll down to see plotp
The residuals have similarspread across x-axis
110
Analysis of RSDAnalysis of RSD
Step 2 – Model Checking(Normality)
▼Red Triangle (in Fit Modellt i d )results window)
→ Save Columns→ Residuals
Analyze → Distribution* Select “Residual Strength” for
“Y C l ”“Y, Columns”* ▼Red Triangle (“Residual Strength”) → Normal Quantile Plot
The normality assumption is satisfied since the black points areThe normality assumption is satisfied since the black points are near the diagonal line.
111
Analysis of RSD
Step 3 – ANOVA/Parameter Estimates/Summary of Fit
Analysis of RSD
RSquare Summary of Fit
Final Parameter Estimates
112
Analysis of RSD
Step 4 – Graphical Methods
Analysis of RSD
Graphical methods become even more crucial in RSD because we want to understand better where the optimum is as well as how the response changes.
There are many that we can use: interaction plots, t l t d f l tcontour plots, and response surface plots
Step 4 Graphical Methods (Interaction Plot)Step 4 – Graphical Methods (Interaction Plot)
▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Interaction Plot
113
→ Factor Profiling → Interaction Plot
Analysis of RSDAnalysis of RSD
Step 4 – Graphical Methods (Optimization/Profiler)
▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Profiler
If necessary, scroll down to the Prediction Profiler
▼Red Triangle (next to Prediction Profiler)g ( )→ Desirability Functions
▼Red Triangle (next to Prediction Profiler)→ Maximize Desirability
114
Analysis of RSDAnalysis of RSD
Step 4 – Graphical Methods (Optimization/Profiler)
To maximize the t th listrength, sealing temp should be
set at its low l l lilevel, cooling temp at mid level, and
percent additive at high level
The Profiler will tell you theThe predicted response and The Profiler will tell you the factor settings that meet
your objective
The predicted response and its variability is also shown
115
Analysis of RSD
Step 4 – Graphical Methods (Contour Plot)
Analysis of RSD
▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Contour Profiler
Scroll to the Contour Profiler The predicted response for the specified factor
levels is herelevels is here
The factor levels can be changed
Or if you want to enter a value for the response and see all the factor
levels that will achieve that response, do it here
116
Analysis of RSDAnalysis of RSD
Step 4 – Graphical Methods (Contour Plot)
To add contours to the Contour Plot:
▼Red Triangle (next to Contour Profiler) → Contour Grid▼Red Triangle (next to Contour Profiler) → Contour Grid
You then have to specify the contours. Use the
defaults for now.
117
Analysis of RSD
Step 4 – Graphical Methods (Contour Plot)
Analysis of RSD
The lines on the plot represent the predicted values of the
response for different valuesresponse for different values of the three factors
118
Analysis of RSD
Step 4 – Graphical Methods (Contour Plot & Response Surface Plot)
Analysis of RSD
To find the region of operability using a given
range of the response, enterrange of the response, enter in values for the Lo Limit
and/or Hi Limit
Low = 9Low = 9
The white area represents factor settings that are within
the specified limits
119
Analysis of RSDAnalysis of RSD
Step 4 – Graphical Methods (Contour Plot)
▼Red Triangle (upper left of Fit Model results window)→ Factor Profiling → Surface Profiler
Scroll to the surface profiler, which will be labeled “Drop”
Choose which variables to plotby clicking on buttons in this area
Response Surface Plot(Can be rotated by clicking
and dragging) 120
Analysis of RSD
Step 5 – Conclusions
Analysis of RSD
p
From all the graphical methods determine the best factor levels to meet your objectivey jBe aware the variability in the predicted response can be large
121
Course OutlineCourse Outline
• Introduction and Basic Principles
• Introduction to Factorial Designs
• Screening Designs
• Response Surface Designs• Response Surface Designs
• Resources
122
ResourcesResources
• Help menue p e u– Indexes– Tutorials– Books
• JMP documentations
S l D t– Sample Data
123
ResourcesResources
• On‐line resourcesO e esou ces– http://www.jmp.com/about/events/webcasts/for webcasts and recorded demos
– http://www.jmp.com/academic/check out Learning Library
8 Q i k G id• JMP 8 Quick Guide
124
ResourcesResources
• On‐line resourcesO e esou ces– http://www.lisa.stat.vt.edu/Welcome to LISA!
– http://www.lisa.stat.vt.edu/?q=short_coursesLISA short courses
125
Thank YouThank You
126