chapter4: classical normal linear regression model (cnlrm) · basic econometrics chapter4:...

Basic EconometricsBasic Econometrics

Chapter 4: Classical Normal Linear RegressionClassical Normal Linear Regression

Model (CNLRM)

Iris Wang

[email protected]

Sampling distributionsSampling distributions

• We have studied the expected value andWe have studied the expected value and variance of the OLS estimators

• In order to do inference we need to know theIn order to do inference, we need to know the full sampling distribution of the estimator

• To make this sampling distribution tractable we• To make this sampling distribution tractable, wenow assume that the unobserved error term (u) is normally distributed in the population.is normally distributed in the population.

This is often referred to as the normalityassumption (Assumption 10)assumption. (Assumption 10)

Assumption 10: NormalityAssumption 10: Normality

• We continue to make the assumptions introducedWe continue to make the assumptions introducedin the previous lecture (linear regression, no perfect collinearity, zero conditional mean homoskedasticity )zero conditional mean, homoskedasticity, …).

• And we add the following:

A i 10 N li Th l i• Assumption 10: Normality – The population erroru is independent of the explanatory variables x1,

d i ll di ib d i hx2,…,xk, and is normally distributed with zeromean and variance σ2: u ~ Normal(0, σ2)

Recap: The normal distributionRecap: The normal distribution

• The normal distribution is verywidely used in statistics & econometrics (one reason is thatnormality simplifies probabilitycalculations)calculations)

• A normal random variable is a continuous random variable thatcan take on any value.

• The shape of the probabilitydensity function (pdf) for the normal distribution is shown on the rightthe right.

• The mathematical formula for the pdf is as follows:

…where:

Why are we assuming normality?Why are we assuming normality?

• Answer: It implies that the OLS estimator followsAnswer: It implies that the OLS estimator followsa normal distribution too. And this makes it straightforward to do inference.

• Under the CLM assumptions (1‐7), conditional on the sample values of the independent variables,

The result that

implies that

• In words, this says that the deviation between the estimated value and the true parameter value, dividedby the standard deviation of the estimator is normallyby the standard deviation of the estimator, is normallydistributed with mean zero and variance equal to 1.

• On p.100On p.100

Th ti 1 7 ll d th l i l• The assumptions 1—7 are called the classicallinear model (CLM) assumptions.

• One immediate implication of the CLM assumptions is that, conditional on theassumptions is that, conditional on the explanatory variables, the dependent variable y has a normal distribution with constanty has a normal distribution with constantvariance, p.101.

How justify the normality assumption?How justify the normality assumption?

• Central limit theorem (CLT): the residual u is the sum ofCentral limit theorem (CLT): the residual u is the sum of many different factors; and by the CLT the sum of manyrandom variables is normally distributedy

• This argument is not without weaknesses (e.g. doesn’thold if u is not additive).)

• Whether normality holds in a particular application is an empirical matter – which can be investigatedp g

• Sometimes using a transformation – e.g. taking the log – yields a distribution that is closer to normal.y

Example: l dCEO Salary and Return on Equity

• Data: CEOSAL1 SAV (available on course• Data: CEOSAL1.SAV (available on coursewebsite)

• Salaries e pressed in tho sands of USD• Salaries expressed in thousands of USD.

• It would be interesting to do the sampledistributions of salary in the different scales.

Sample distributions of CEO salaries in l l & llevels & logs


Chapter 5: Interval Estimation and HypothesisInterval Estimation and Hypothesis

Testing

Iris Wang

[email protected]

Confidence intervalsConfidence intervals

• Once we have estimated the population parameter βp p p βand obtained the associated standard error, we caneasily construct a confidence interval (CI) for βj.

• has a t distribution with n‐k‐1 degreesof freedom (df).

• Define a 95% confidence interval for βj as

where the constant t0.025 is the 97.5th percentile in the t distribution.


(lower limit)

(upper limit)

Meaning of CI: in 95 out of 100 cases intervals like b ll h βEq. above, will contain the true βJ.

Confidence intervals


The width of the CI is proportional to the standard error of the estimator.

• the larger the Se, the larger is the width of the CI.

• the larger the , the greater is the uncertainty of estimating the true of the unknown parameter.

How is the confidence interval affected by an increase in the levelHow is the confidence interval affected by an increase in the levelof confidence (e.g. from 95% to 99%)? Why?

Don’t forget the CLM assumptions!Don t forget the CLM assumptions!

• Estimates of the confidence interval will not beEstimates of the confidence interval will not be reliable if the CLM assumptions do not hold.

Example:Example:

• Data: wage1.savg• These data were originally obtained from the 1976 Current Population Survey in the US.

• SPSS output:

Coefficientsa

Model

Unstandardized Coefficients

Standardized

Coefficients

t Sig.

95,0% Confidence Interval for B

B Std. Error Beta Lower Bound Upper Bound

1 (Constant) -,892 ,686 -1,300 ,194 -2,239 ,456

educ ,541 ,053 ,405 10,143 ,000 ,436 ,645

• Can you calculate these two CIs by yourself accordingto the formula?

educ ,541 ,053 ,405 10,143 ,000 ,436 ,645

a. Dependent Variable: wage

to the formula?

Hypothesis TestingHypothesis Testing

• In Chapter 3 we learned that Assumptions 1‐7In Chapter 3 we learned that Assumptions 1 7 (such as, linear regression, no perfect collinearity, zero conditional

mean, homoskedasticity) enable us to obtainmathematical formulas for the expected valueand variance of the OLS estimators

• To test a hypothesis, we need to know the full sampling distribution of the estimatorsampling distribution of the estimator

1. Sampling Distribution: Illustration1. Sampling Distribution: Illustration

• Suppose we want to make statements about a pppopulation consisting of (say) 10 million individuals.

• The model is as follows: Y = β0 + β1*x + u• Suppose we could draw (say) 100 samples from this population, where each sample consists of (say) 200 observations. Further suppose we would estimate 100observations. Further suppose we would estimate 100 different regressions (one for each sample)

• This would generate 100 different estimates of ourparameter of interest β1 – and they would form the distribution of our estimator.

Let’s do this!Let s do this!

• Let’s simulate 500 samples consisting of 200Let s simulate 500 samples consisting of 200 individuals. Our model is Y = β0 + β1*x + u, where u is normally distributed (and all otherwhere u is normally distributed (and all otherassumptions hold too).

• Since we are simulating data we can now• Since we are simulating data, we can nowchoose the true parameters (this wouldobviously not be the case for real empiricalobviously not be the case for real empiricalapplications). Let’s choose β0 = 0 and β1=0.

Here’s the distribution of the 500 d ff f βdifferent estimates of β1

64

Den

sity

02

0

-.2 -.1 0 .1 .2b1

Mean of b1: ‐0.005Std dev of b1: 0.072

2. Why do we need to know the sampling distribution of the OLS estimator?distribution of the OLS estimator?

• Recall the formula for the t statistic:

• In other words, the difference between the parameter estimate and a given (unknown) value of the true parameter, scaled by the standard error of the estimator, follows a

t di t ib tit‐distribution.

• This is very good news, because we know exactly what the t distribution looks like (statisticians have studied thisdistribution looks like (statisticians have studied this distribution for many years).

• In particular, we know exactly how to compute probabilitiesp , y p pusing a t distribution – and this will be very useful when testing hypothesis (more on this shortly)

• Here’s the answer to the question – if we don’tknow the sampling distribution of the OLS p gestimator, we can’t be sure that (beta_hat –beta)/se(beta hat) follows a t‐distribution. )/ ( _ )

• In that case, this quantity could follow anydistribution – in which case there’s no way ofdistribution – in which case there s no way of doing the probability analysis that underlies the hypothesis testinghypothesis testing.

Testing the null hypothesisTesting the null hypothesis

• In most applications, testingIn most applications, testing

is of central interest (j corresponds to any of the k independent variables in the model).

• Since βj measures the partial effect of xj on the expected value y after controlling for other factors, th ll h th i th t h ff tthe null hypothesis means that xj has no effect on the expected value of y.

Example: Wage equationExample: Wage equation

log(wage)=β +β education+ulog(wage)=β0 +β1 education+u

h ll h h i β h d i• The null hypothesis H0: β1=0 means that, educationhas no effect on hourly wage.

I thi i ll i t ti h th i ?• Is this an economically interesting hypothesis?

• Now let’s look at how we can carry out and interpret such a testsuch a test.

• The test statistic we use to test is calledthe t statistic or the t ratio of and is defined asthe t statistic or the t ratio of and is defined as

• As you can see, the t statistic is easy to compute: just divide your coefficient estimate by the standard y yerror.

• SPSS (and most other econometrics software) will do this for you.

• Since the se is always positive, the t statistic alwayshas the same sign as the coefficient estimate.

IntuitionIntuition

Two‐tailed testsTwo tailed tests

• Consider a null hypothesis like H0: βj=0 against a two‐id d lt ti lik H β ≠0sided alternative like H1: βj≠0.

• In words, H1 is that xj has a ceteris paribus effect on y which could be either positive or negativey, which could be either positive or negative.

Now let’s decide on a significance levelNow let s decide on a significance levelSignificance level = probability of rejecting H0 when it is in fact true (i.e. a mistake).Let’s decide on a 5% significance level (the most commonchoice): hence, we are willing to mistakenly reject H0when it is true 5% of the time.

Two‐sided (cont’d) ( )• To find the critical vale of t

(denote by c), we first specify the significancelevel, say 5%.

• Since the test is two‐Since the test is twotailed, c is then chosen tomake the area in each tailequal 2.5% ‐ i.e. c is theequal 2.5% i.e. c is the 97.5th percentile in the t distribution (again, with n‐k‐1 degrees of freedom).k 1 degrees of freedom).

• The graph shows that, ifdf=26, then c=2.06.

Econometric jargon: If H0: ßj=0 is rejected against a two‐sided alternative, we maysay that ”xj is statistically significant at the 5% level”. Thus we conclude that the effect of xj on y is not zero.

Testing against one‐sided alternativesTesting against one sided alternatives

• The rule for rejecting H0 depends on:The rule for rejecting H0 depends on:1. The alternative hypothesis (H1)

2. The chosen significance level of the test

• Let’s begin by looking at a one‐sided alternative of the form:

• Let’s assume we decide to apply a 5% significancelevel, that is, α=5%.

One‐tail testOne tail test

• Under H0 (βj=0 ), the t statistic has a t distribution.Under H0 (βj 0 ), the t statistic has a t distribution.

• Under H1 (βj>0), the expected value of the t statistic is positive. p

• Denote the critical value by c.

• On p.118On p.118

Rejection rule: H0 is rejected in favor of H1 at the 5% significance level if

We’ve seen how to obtain the t statistic.

But how do we obtain c?

To obtain c, we only need the significance leveland the degrees of freedom (df).

Example: For df = 28 and significance level 5%, c=1.701

If our t statistic is less than 1 701 we do notIf our t statistic is less than 1.701, we do not reject H0But if our t statistic is higher than 1.701, we do reject H0

A few points worth notingA few points worth noting

• As the significance level falls, the critical valueAs the significance level falls, the critical valueincreases. Why?

• If H0 is rejected at (say) the 5% level, it is 0 j ( y) ,automatically rejected at the 1% level too.

• What is the critical value c for o A 10% significance level with df=21?

o A 1% significance level with df=120?

• Confirm that, as the df gets large, the critical valuesfor the t‐distribution get very close to the criticall f h d d l di ib ivalues for the standard normal distribution.

Example: The wage equation( )(Data: WAGE1.SAV)

Model:

Based on the results below, test H0: β1=0 against H1: β1>0

Coefficientsa

Model

Unstandardized Coefficients

Standardized

Coefficients

t Sig.B Std. Error Beta

1 (Constant) - 892 686 -1 300 1941 (Constant) -,892 ,686 -1,300 ,194

educ ,541 ,053 ,405 10,143 ,000


Testing other hypotheses about βj

• Although H0: βj=0 is the most common hypothesis, wesometimes want to test whether βj is equal to somejother given constant. Suppose the null hypothesis is

• In this case the appropriate t statistic is:

• Now go back and test the hypothesis that the educcoefficient in the regression above is equal to 1 (against a two sided alternative)two‐sided alternative).

Computing p‐values for t testsComputing p values for t tests

• You have seen how the researcher chooses the significance level. There’s no ”correct” significancelevel.

• In practice, the 5% level is the most common one, but10% is also frequently used (especially for small datasets) as is 1% (more common for large datasets)datasets) as is 1% (more common for large datasets).

• Given the observed value of the t statistic, what is the smallest significance level at which the null hypothesissmallest significance level at which the null hypothesiswould be rejected?

• This level is known as the p‐valueThis level is known as the p‐value.

• Example: Suppose t = 1.85 and df=40.

• This results in a p‐value = 0.0718.

p‐values in SPSSp values in SPSS

• Correct interpretation: The p‐value is Coefficientsa

the probability of observing a t valueas extreme as we did if the nullhypothesis is true. ☺

• Wrong interpretation (not

Coefficientsa

Model

Unstandardized

Coefficients

Standardize

d

Coefficients

t Sig.B Std. Error Beta

• Wrong interpretation (not uncommon): ”The p‐value is the probability that the null hypothesis is true….”.

1 (Constant) -,892 ,686 -1,300 ,194

educ ,541 ,053 ,405 10,143 ,000


Thus, small p‐values are evidenceagainst the null hypothesis. If the p‐value is, say, 0.04, we might saythere’s significance at the 5% levelthere s significance at the 5% level(actually at the 4% level) but not at the 1% level (or 3% or 2% level).


Chapter 6: Extensions of the Two‐VariableExtensions of the Two Variable

Linear Regression Model

Iris wang

[email protected]

Log‐linear regression modelsLog linear regression models

• In many cases relationships betweenIn many cases relationships between economic variables may be non‐linear.

• However we can distinguish between• However we can distinguish between functional forms that are intrinsically non‐linear and those that can be transformed intolinear and those that can be transformed into an equation to which we can apply ordinary least squares techniquesleast squares techniques.


• Of those non‐linear equations that can beOf those non linear equations that can be transformed, the best known is the multiplicative power function formmultiplicative power function form (sometimes called the Cobb‐Douglas functional form) which is transformed into afunctional form), which is transformed into a linear format by taking logarithms.


Production functionsProduction functions

For example, suppose we have cross‐section data on firms in a particular industry withdata on firms in a particular industry with observations both on the output (Q) of each firm and on the inputs of labour (L) and capitalfirm and on the inputs of labour (L) and capital (K).

C id h f ll i f i l fConsider the following functional form


The parameters α and β can be estimated directly from a regression of the variable lnQ on lnL and lnK

chapter4: classical normal linear regression model (cnlrm) · basic econometrics chapter4:...

Documents