lecture 22 multiple regression (sections 19.3-19.4)
Post on 20-Dec-2015
226 views
TRANSCRIPT
Lecture 22
• Multiple Regression (Sections 19.3-19.4)
19.1 Introduction
• In this chapter we extend the simple linear regression model, and allow for any number of independent variables.
• We expect to build a model that fits the data better than the simple linear regression model.
Examples of Multiple Regression
• Business decisionmaking: La Quinta Inns wants to decide where to locate new inns. It wants to predict operating margin based on variables related to competition, market awareness, demand generators, demographics and physical location.
• College admissions: The admissions officer wants to predict which students will be most successful. She wants to predict college GPA based on GPA from high school, SAT score and amount of time participating in extracurricular activities.
More Examples
• Improving operations: A parcel delivery service would like to increase the number of packages that are sorted in each of its hub locations. Three factors that the company can control and that influence sorting performance are the number of sorting lines, the number of sorting workers, and the number of truck drivers. What can the company do to improve sorting performance?
• Understanding relationships: Executive compensation. Does it matter how long the executive has been at the firm controlling for other factors? Do CEOs pay themselves less if they have a large stake in the stock of the company controlling for other factors? Does having an MBA increase executive salary controlling for other factors?
• We shall use computer printout to – Assess the model
• How well it fits the data
• Is it useful
• Are any required conditions violated?
– Employ the model• Interpreting the coefficients
• Predictions using the prediction equation
• Estimating the expected value of the dependent variable
Introduction
• Where to locate a new motor inn?– La Quinta Motor Inns is planning an expansion.
– Management wishes to predict which sites are likely to be profitable.
– Several areas where predictors of profitability can be identified are:
• Competition
• Market awareness
• Demand generators
• Demographics
• Physical quality
Example 19.1
Profitability
Competition Market awareness Customers Community Physical
Margin
Rooms Nearest Officespace
Collegeenrollment
Income Disttwn
Distance to downtown.
Medianhouseholdincome.
Distance tothe nearestLa Quinta inn.
Number of hotels/motelsrooms within 3 miles from the site.
Coefficients
Dependent variable Independent variables
Random error variable
19.2 Model and Required Conditions• We allow for k independent variables to
potentially be related to the dependent variable:
y = 0 + 1x1+ 2x2 + …+ kxk +
kkk xxxxyE 111 ),,|(
Multiple Regression for k = 2, Graphical Demonstration - I
y = 0 + 1xy = 0 + 1xy = 0 + 1xy = 0 + 1x
X
y
X2
1
The simple linear regression modelallows for one independent variable, “x”
y =0 + 1x +
The multiple linear regression modelallows for more than one independent variable.Y = 0 + 1x1 + 2x2 +
Note how the straight line becomes a plane, and...
y = 0 + 1x1 + 2x2
y = 0 + 1x1 + 2x2
y = 0 + 1x1 + 2x2
y = 0 + 1x1 + 2x2y = 0 + 1x1 + 2x2
y = 0 + 1x1 + 2x2
y = 0 + 1x1 + 2x2
Multiple Regression for k = 2, Graphical Demonstration - II
Note how a parabola becomes a parabolic Surface.
X
y
X2
1
y= b0+ b1x2
y = b0 + b1x12 + b2x2
b0
• The error is normally distributed.
• The mean of the error is equal to zero for each combination of x’s, i.e., .
• The standard deviation is constant ( for all values of x’s.
• The errors are independent.
Required conditions for the error variable
0),,|( 1 kxxE
• Data were collected from randomly selected 100 inns that belong to La Quinta, and ran for the following suggested model:
Margin = Rooms NearestOfficeCollege + 5Income + 6Disttwn
Estimating the Coefficients and Assessing the Model, Example
Margin Number Nearest Office Space Enrollment Income Distance55.5 3203 4.2 549 8 37 2.733.8 2810 2.8 496 17.5 35 14.449 2890 2.4 254 20 35 2.6
31.9 3422 3.3 434 15.5 38 12.157.4 2687 0.9 678 15.5 42 6.949 3759 2.9 635 19 33 10.8
Xm19-01
– If the model assessment indicates good fit to the data, use it to interpret the coefficients and generate predictions.
– Assess the model fit using statistics obtained from the sample.
– Diagnose violations of required conditions. Try to remedy problems when identified.
19.3 Estimating the Coefficients and Assessing the Model
• The procedure used to perform regression analysis:– Estimate the model coefficients and statistics using least
squares using JMP.
Model Assessment
• The model is assessed using three tools:– The standard error of estimate – The coefficient of determination– The F-test of the analysis of variance
• The standard error of estimates participates in building the other tools.
• The standard deviation of the error is estimated by the Standard Error of Estimate:
• The magnitude of s is judged by comparing it to
1knSSE
s
Standard Error of Estimate
.y
• From the printout, s = 5.51
• Calculating the mean value of y,
• It seems s is not particularly small relative to y.
• Question:Can we conclude the model does not fit the data well?
739.45y
Standard Error of Estimate
• The definition is
• From the printout, R2 = 0.5251• 52.51% of the variation in operating margin is
explained by the six independent variables. 47.49% remains unexplained.
• When adjusted for degrees of freedom, Adjusted R2 = 1-[SSE/(n-k-1)] / [SS(Total)/(n-1)] =
= 49.44%
2i
2
)yy(SSE
1R
Coefficient of Determination
• We pose the question:
Is there at least one independent variable linearly related to the dependent variable?
• To answer the question we test the hypothesis
H0: 0 = 1 = 2 = … = k
H1: At least one i is not equal to zero.
• If at least one i is not equal to zero, the model has some validity.
Testing the Validity of the Model
• The hypotheses are tested by an ANOVA procedure.
Testing the Validity of the La Quinta Inns Regression Model
Analysis of Variance Source DF Sum of
Squares Mean
Square F Ratio
Model 6 3123.8320 520.639 17.1358 Error 93 2825.6259 30.383 Prob > F C. Total
99 5949.4579 <.0001
[Variation in y] = SSR + SSE. If SSR is large relative to SSE, much of the variation in y is explained by the regression model; the model is useful and thus, the null hypothesis should be rejected. Thus, we reject for large F.
Rejection region
F>F,k,n-k-1
Testing the Validity of the La Quinta Inns Regression Model
1knSSE
kSSR
F
F,k,n-k-1 = F0.05,6,100-6-1=2.17F = 17.14 > 2.17
Also, the p-value (Significance F) = 0.0000Reject the null hypothesis.
Testing the Validity of the La Quinta Inns Regression Model
ANOVAdf SS MS F Significance F
Regression 6 3123.8 520.6 17.14 0.0000Residual 93 2825.6 30.4Total 99 5949.5
Conclusion: There is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis. At least one of the i is not equal to zero. Thus, at least one independent variable is linearly related to y. This linear regression model is valid
Conclusion: There is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis. At least one of the i is not equal to zero. Thus, at least one independent variable is linearly related to y. This linear regression model is valid
• b0 = 38.14. This is the intercept, the value of y when
all the variables take the value zero. Since the data
range of all the independent variables do not cover
the value zero, do not interpret the intercept.
• b1 = – 0.0076. In this model, for each additional
room within 3 mile of the La Quinta inn, the
operating margin decreases on average by .0076%
(assuming the other variables are held constant).
Interpreting the Coefficients
• b2 = 1.65. In this model, for each additional mile that the
nearest competitor is to a La Quinta inn, the operating margin increases on average by 1.65% when the other variables are held constant.
• b3 = 0.020. For each additional 1000 sq-ft of office space, the operating margin will increase on average by .02% when the other variables are held constant.
• b4 = 0.21. For each additional thousand students the operating margin increases on average by .21% when the other variables are held constant.
Interpreting the Coefficients
• b5 = 0.41. For additional $1000 increase in median household income, the operating margin increases on average by .41%, when the other variables remain constant.
• b6 = -0.23. For each additional mile to the
downtown center, the operating margin decreases on
average by .23% when the other variables are held
constant.
Interpreting the Coefficients
• The hypothesis for each i is
• JMP printout
H0: i 0H1: i 0 d.f. = n - k -1
Test statistic
ib
iis
bt
Testing the Coefficients
Parameter Estimates Term Estimate Std Error t Ratio Prob>|t|
Intercept 38.138575 6.992948 5.45 <.0001 Number -0.007618 0.001255 -6.07 <.0001 Nearest 1.6462371 0.632837 2.60 0.0108 Office Space 0.0197655 0.00341 5.80 <.0001 Enrollment 0.2117829 0.133428 1.59 0.1159 Income 0.4131221 0.139552 2.96 0.0039 Distance -0.225258 0.178709 -1.26 0.2107
• Predict the average operating margin of an inn at a site with the following characteristics:– 3815 rooms within 3 miles,
– Closet competitor .9 miles away,
– 476,000 sq-ft of office space,
– 24,500 college students,
– $35,000 median household income,
– 11.2 miles distance to downtown center.
MARGIN = 38.14 - 0.0076(3815) +1.65(.9) + 0.020(476) +0.21(24.5) + 0.41(35) - 0.23(11.2) = 37.1%
Xm19-01
La Quinta Inns, Predictions