fitting regression models
DESCRIPTION
Fitting Regression Models. It is usually interesting to build a model to relate the response to process variables for prediction, process optimization, and process control A regression model is a mathematical model which fits to a set of sample data to approximate the exact appropriate relation - PowerPoint PPT PresentationTRANSCRIPT
1
Fitting Regression Models
• It is usually interesting to build a model to relate the response to process variables for prediction, process optimization, and process control
• A regression model is a mathematical model which fits to a set of sample data to approximate the exact appropriate relation
• Low-order polynomials are widely used• There is a strong “interaction” between the design of
experiments and regression analysis• Regression analysis is often applied to unplanned
experiments
2
Linear Regression Models
• In general, the response variable y may be related to k regressor variables by a multiple linear (first order) regression model
y = o + 1x1 + 2x2 + + kxk + • Models of more complex forms can be analyzed
similarly. E.g.,
y = o + 1x1 + 2x2 + 12x1x2 + =>
y = o + 1x1 + 2x2 + 3x3 + • Any regression model that is linear in the parameters
( values) is a linear regression model, regardless of the shape of the response surface
• The variables have to be quantitative
3
Model Fitting – Estimating Parameters• The method of least squares is typically used• Assuming that the error term are uncorrelated random
variables• The data can be expressed as
• The model equation is
yi = o + 1xi1 + 2xi2 + + kxik + i
• Least square method chooses the ´s so that the sum of the squares of the errors i is minimized
4
Model Fitting – Estimating Parameters• The least squares function is
• The least squares estimators ( ) must satisfy and
or
2
1 11
2
n
i
k
jijjoi
n
ii xyL
j
0
o
L
0
j
L
n
iiik
n
ik
n
iiik
n
i
n
iiko
n
iii
n
iikik
n
iii
n
i
n
iio
n
ii
n
iikk
n
ii
n
iio
yxxxxxx
yxxxxxxx
yxxxn
ikik
i
11
2
122
1
21
1
11
11
1212
1
21
11
11122
111
ˆˆˆˆ
ˆˆˆˆ
ˆˆˆˆ
1
5
Model Fitting – Estimating Parameters• Or in the matrix notation
y = X + • The least squares estimators of are
• The fitted model and residuals are
• Example 10-1:• Response: viscosity of a polymer (y)• Variables: reaction temperature (x1), catalyst feed
rate (x2)• The model
y = o + 1x1 + 2x2 +
yXXX 1
ˆ Xy yye ˆ
6
21 58.862.708.1566ˆ xxy
7
8
9
Fitting Regression Models in Designed Experiments
• Example 10-2: regression analysis of a 23 factorial design• Response: yield of a
process• Variables: temperature,
pressure, and concentration• Single replicate with 4
center points
10
Fitting Regression Models in Designed Experiments
• A main effects only model
y = o + 1x1 + 2x2 + 3x3 +
321 125.1625.10625.5000.51ˆ xxxy
11
Fitting Regression Models in Designed Experiments
• The regression coefficients are exactly one-half of the effect estimates in a 2k design
• Because the factorial designs are orthogonal designs, the off-diagonal elements in X´X are zero, or X´X is diagonal
• Regression method is useful when the experiment (or data) is not perfect
• Regression analysis of data with missing observations. Example 10-3: assuming run 8 of the observations in Example 10-2 was missing. Fit the main effect model using the remaining observations
y = o + 1x1 + 2x2 + 3x3 +
12
Example 10-3
321 125.1625.10625.5000.51ˆ xxxy
321 25.175.1075.525.51ˆ xxxy
Original model
13
Fitting Regression Models in Designed Experiments
• Regression analysis of experiments with inaccurate factor levels• Example 10-4: assuming the process variables are not at their
exact assumed values in Example 10-2. Fit the main effect model y = o + 1x1 + 2x2 + 3x3 +
14
Example 10-4
321 125.1625.10625.5000.51ˆ xxxy
321 08.117.1042.536.50ˆ xxxy
Original model
15
Fitting Regression Models in Designed Experiments
• Regression analysis can be used to de-alias interactions in a fractional factorial using fewer than a full fold-over fraction in a resolution III design
• Example 10-5: consider Example 8-1, assume effects A, B, C, D, and AB+CD were large – de-alias AB+CD using fewer than 8 additional runs. Consider the model
y = o + 1x1 + 2x2 + 3x3 + 4x4 + 12x1x2 + 34x3x4 +
16
• The X matrix for the model is
• Adding one run from the alternate fraction to the original 8 runs, the X matrix becomes
17
Hypothesis Testing in Multiple Regression
Measuring the usefulness of the model• Test for significance of regression – determine if there is a
linear relationship between the response y and a subset of the regressor variables x1, x2, , xk.
• Testing hypothesis
Ho: 1 = 2 = = k = 0
H1: j 0 for at least one j• Analysis of variance
SST = SSR + SSE
n
y
yXSS
n
ii
R
2
1ˆ
yXyySSE
18
Hypothesis Testing in Multiple Regression
• If Fo exceeds F,k,n-k-1, the null hypothesis is rejected
E
R
E
Ro MS
MS
knSS
kSSF
)1/(
/
19
T
E
T
R
SS
SS
SS
SSR 12
)1()(
)1(1
)1/(
)/(1 22 R
pn
n
nSS
pnSSR
T
Eadj
20
Hypothesis Testing in Multiple Regression• Testing individual and group of coefficients – determine if one
or a group of regressor variables should be included in the model
• Testing hypothesis (for an individual regression coefficient)
Ho: j = 0
H1: j 0
if Ho is not rejected, then xj can be deleted from the model.
• Ho is rejected if |to| > tn
jj
jo
Ct
2ˆ
ˆ
21
• The contribution of a particular variable, or a group of variables can be quantified using sums of squares
• Confidence intervals on individual regression coefficients
• Example 10-7
jjpnjjjjpnj CtCt 2,2/
2,2/ ˆˆˆˆ
22
• Prediction of new response observations
• The future observation yo at a point (xo1, xo2, ,xok) with x’o =[1, xo1, xo2, ,xok]
• Regression model diagnostics
• Testing for lack of fit
• Sections 10-7, 8
))(1(ˆ)(ˆ
))(1(ˆ)(ˆ
12,2/
12,2/
oopno
ooopno
xXXxtxy
yxXXxtxy