econometrics eviews simple easy presentation

8/18/2019 Econometrics Eviews simple easy presentation

1/53

Part 7: Estimating the Variance of b-1/53

Econometrics I

Professor William Greene

Stern School of Business

Department of Economics


2/53


Econometrics I

Part 7 – Estimating

the Variance of b


3/53


Context

The true ariance of b|X is σ!"X′X#-1 . Weconsi$er ho% to use the sample $ata to estimate

this matrix& The ultimate o'(ecties are to form

interal estimates for regression slopes an$ to

test h)potheses a'out them& Both re*uire

estimates of the aria'ilit) of the $istri'ution&

We then examine a factor %hich affects ho%+large+ this ariance is, multicollinearit)&


4/53


Estimating σ2

-sing the resi$uals instea$ of the $istur'ances:

The natural estimator: e′e./ as a samplesurrogate for ε′ε /n

Imperfect o'seration of εi, ei 0 εi 1 "β - b#′xi

Do%n%ar$ 'ias of e′e./& We o'tain the resultE2e′e|X3 0 "/14#σ!


5/53


Expectation of e′e

−

−

=

= −

= −

= = + = β + =

=

β ε ε ε

ε ε

= ε ε = ε ε = ε ε

1

1

( ' ) '

[ ( ' ) ']

( )

( '(

e y - Xb

y X X X X y

I X X X X y

My M X MX M M

e'e M M'M'M 'MM 'M


6/53

Part 7: Estimating the Variance of b-/53

5etho$ 6:

E[ ] E

E[ trace ( ) ] scalar = its trace

E[ trace ( ) ] permute in trace

[ trace E ( ) ] linear operators

e'e| X 'M | X

'M | X

M '| X

M '| X

[ε ε ]

ε ε

εε

εε

σ

σ

σ

2

2

2

[ trace E ( ) ] conditioned on X[ trace ] model assumption

[trace ] scalar multiplication and matrix

trace [ - ( ) ]-1

M '| XM I

M I

I X X'X X'

εε

σ

′

σ

σ

σ

2

2

2

2

{trace [ ] - trace[ ( ) ]}

{N - trace[( ) ]} permute in trace

{N - trace[ ]}

{N - K}

Notice tat E[ ! ] is not a

-1

-1

I

e

X X'X X'

X'X X'X

e

I

X

"unction o" #X


7/53Part 7: Estimating the Variance of b-7/53

Estimating !

The !nbiase" estimator is s! 0 e′e."/14#&

8Degrees of free$om correction9

Therefore, the unbiased estimator of σ! iss! 0 e′e."/14#


8/53Part 7: Estimating the Variance of b-#/53

5etho$ !: Some 5atrix lge'ra2E[ ] trace

$at is te trace o" % is idempotent& so its

trace euals its ran# ts ran euals te num*er

o" non+ero caraceristic roots#

,aracteric oots.

/i0nature o" a atrix = /pectral

σe'e| X MM M

ecomposition

= Ei0en (o3n) 4alue ecomposition

= ' 3ere= a matrix o" columns suc tat ' = ' =

= a dia0onal matrix o" te caracteristic roots

element

A C CC CC CC I

Λ

Λ

s o" ma5 *e +eroΛ


9/53Part 7: Estimating the Variance of b-$/53

Decomposing %

=

Λ = Λ

2 2

2 2

6se"ul esult. " = ' is te spectral

decomposition& ten ' (7ust multipl5)

= & so # 8ll o" te caracteristicroots o" are 1 or 9# :o3 man5 o" eac%

trace( ) = trace( ')=trace(

A C C

A C C

M MM

A C C

Λ

Λ

Λ

' )=trace( )

;race o" a matrix euals te sum o" its caracteristic

roots# /ince te roots o" are all 1 or 9& its trace is

7ust te num*er o" ones& 3ic is N-K as 3e sa3#

CC

M

Λ Λ


10/53Part 7: Estimating the Variance of b-1&/53

Example: Characteristic ;oots of a

Correlation 5atrix



6

1 iiλ

=′ ′= ∑ i i' ( )*) c c



Gasoline Data



X+X an$ its ;oots



Var2b



X’X

(X’X)-1

s2(X’X)-1



Stan$ar$ ;egression ;esults----------------------------------------------------------------------Ordinary least squares regression ........LHS=G Mean = 226.09444 Standard deviation = 50.59!2 "u#$er o% o$servs. = &6 Model si'e (ara#eters = ) *egrees o% %reedo# = 29+esiduals Su# o% squares = ))!.)022)

Standard error o% e = 5.!!) ,= sqr))!.)022)/&6 )13it +-squared = .99& dusted +-squared = .9!95--------------------------------------------------------------------- 7aria$le8 oe%%i:ient Standard ;rror t-ratio (8---------------------------------------------------------------------onstant8 -).)&9)5 49.9595 -.55 .!)!0

(G8 -5.&00!??? 2.42) -6.&! .0000 2.&66 @8 .02&65??? .00))9 &.0&) .0050 92&2.!6



Bootstrapping

Some assumptions that un$erlie it 1 the sampling mechanism

5etho$:

6& Estimate using full sample: 11= b

!& ;epeat ; times:

Dra% / o'serations from the n, %ith replacement

Estimate β %ith b"r#&

>& Estimate ariance %ith

V 0 "6.;#Σr 2b"r# 1 b32b"r# 1 b3?


18/53Part 7: Estimating the Variance of b-1#/53

Bootstrap pplication

matrbboot(init321&.0 tore res!ts here

namex(oneg 6efine X

regrhs(grhs(x )om!te b

caci(& )o!nter Proc 6efine roce"!re

regrhs(grhs(x!iet 8 'egression

matr9i(i:1;bboot


19/53Part 7: Estimating the Variance of b-1$/53

--------------------------------------------------------------------- 7aria$le8 oe%%i:ient Standard ;rror t-ratio (8---------------------------------------------------------------------onstant8 -)9.)5&5??? !.6)255 -9.96 .0000 @8 .0&692??? .00&2 2!.022 .0000 92&2.!6 (G8 -5.224??? .!!0&4 -!.042 .0000 2.&66---------------------------------------------------------------------o#Bleted 20 $ootstraB iterations.

----------------------------------------------------------------------+esults o% $ootstraB esti#ation o% #odel. Model Cas $een reesti#ated 20 ti#es. Means sCoDn $eloD are tCe #eans o% tCe $ootstraB esti#ates. oe%%i:ients sCoDn $eloD are tCe original esti#ates $ased on tCe %ull sa#Ble. $ootstraB sa#Bles Cave &6 o$servations.--------------------------------------------------------------------- 7aria$le8 oe%%i:ient Standard ;rror $St.;r. (8E8' Mean o% >--------------------------------------------------------------------- F008 -)9.)5&5??? !.&552 -9.545 .0000 -)9.5&29 F0028 .0&692??? .00&& 2).))& .0000 .0&6!2 F00&8 -5.224??? 2.0&50& -).4& .0000 -4.)654---------------------------------------------------------------------

;esults of Bootstrap Proce$ure


20/53Part 7: Estimating the Variance of b-2&/53

Bootstrap ;eplications

>! same res!t

?ootstrae" sameres!ts



@AS s& Aeast 'solute Deiations----------------------------------------------------------------------Least a$solute deviations esti#ator...............+esiduals Su# o% squares = 5&).5!60& Standard error o% e = 6.!25943it +-squared = .9!2!4--------------------------------------------------------------------- 7aria$le8 oe%%i:ient Standard ;rror $St.;r. (8E8' Mean o% >---------------------------------------------------------------------

8ovarian:e #atri $ased on 50 reBli:ations.onstant8 -!4.025!??? 6.0!64 -5.22& .0000 @8 .0&)!4??? .002) &.952 .0000 92&2.!6 (G8 -).0990??? 4.&)60 -&.9 .000 2.&66---------------------------------------------------------------------Ordinary least squares regression ............+esiduals Su# o% squares = 4)2.)9!&4 Standard error o% e = 6.6!059 Standard errors are $ased on3it +-squared = .9!&56 50 $ootstraB reBli:ations--------------------------------------------------------------------- 7aria$le8 oe%%i:ient Standard ;rror t-ratio (8---------------------------------------------------------------------onstant8 -)9.)5&5??? !.6)255 -9.96 .0000 @8 .0&692??? .00&2 2!.022 .0000 92&2.!6 (G8 -5.224??? .!!0&4 -!.042 .0000 2.&66---------------------------------------------------------------------


22/53


@!antie 'egressionA

Bication of ?ootstraEstimation


23/53


@!antie 'egression


24/53


Estimate" Variance for@!antie 'egression

8s5mptotic ;eor5

?ootstrap @ an ideal application


25/53


( )1 1

Model : , ( | , ) , [ , ] 0ˆˆResiduals: u

1Asymptotic Variance:

![" (0) ] !stimated #y

− −

′ ′= + α = α =′=

′

Asymptotic Theory Based Estimator of Variance of Q - REG

x | x

A C A

A xx

i i i i i i i i

i i i

u

y u Q y Q u y

N

βx βx-βx

[ ]1

$%

1 1 1ˆ1 | | &

& % &and'idt & can #e il*erman+s Rule o" um#:

ˆ ˆ( | $-.) ( | $%.)1$06 ,

1$/

(12 )

(12 ) [ ] !stimated #y

= ′

<

− ÷

α α′ ′α α

∑x x

C = xx

N

i i ii

i iu

u N

Q u Q u Min s

N

E N

( )1%3or $. and normally distri#uted u, tis all simpli"ies to $

%

−π ′α

But, this is an idea appication for !ootstrappin"

#

#

$

#

#u s


26/53


α 0 &!

α 0 &

α 0 &7


27/53



28/53

Part 7: Estimating the Variance of b-2#/53


29/53

Part 7: Estimating the Variance of b-2$/53

5ulticollinearit)

Cot Dshort ranF Ghich is a "eficienc in the mo"e.B characteristic of the "ata set Ghich affects the co,ariance matrix.

;egar$less,β

is un'iase$& Consi$er one of the un'iase$ coefficient estimatorsof β& E2'3 0 β

Var2b3 0 σ!"X+X#16 &The ariance of ' is the k th $iagonal element of σ!"X+X#16 &We can isolate this %ith the result in )our text& Aet HXI3 'e 2@ther xs, x3 0 2X6,x!3

"a conenient notation for the results in the text#&

We nee$ the resi$ual maer, %X&The general result is that the $iagonal element %e see is2I′%6I316 , %hich %e no% is the reciprocal of the sum of s*uare$ resi$uals inthe regression of I on X&


30/53

Part 7: Estimating the Variance of b-3&/53

I hae a sample of !! o'serations in a logit mo$el& T%o pre$ictors are highl) collinear"pair%aise corr &FH p&6#H if are a'out 6! for eachof themH aerage if is !&>H con$itionnum'er is 6&!H $eterminant of correlation matrix is &!66H the t%o lo%est eigen ales are&7F! an$ &!7& Centering.stan$ar$iJing aria'les $oes not change the stor)& /ote: most o's are Jeros for these t%o aria'lesH I onl) hae approx non1Jero o's forthese t%o aria'les on a total of !&! o's&

Both aria'le coefficients are significant an$ must 'e inclu$e$ in the mo$el "as perspecification#&

11 Do I hae a pro'lem of multicollinearit)KK11 Does the large sample siJe attenuate this concern, een if I hae a correlation of &FK11 What coul$ I loo at to ascertain that the conse*uences of multi1collinearit) are not apro'lemK11 Is there an) reference I might cite, to sa) that gien the sample siJe, it is not a pro'lemK

I hope )ou might help, 'ecause I am reall) in trou'leLLL


31/53


Variance of Aeast S*uares

1

%

%% 1

Model : 4 4

Variance o"c

Variance o" c is te lo'er ri5t element o" tis matri$

Var[c] [ ]7 7

'ere 7 te *ector o" residuals "rom te

−

−

γ

′ ′ = σ ÷ ′ ′

σ′σ =′#

y #% &

! # # # &

& # & &

& ' && &

&

ε

( )

( )

% %

n %

ii 1

n% %

ii 1

%% 1

n% %

ii 1

re5ression o" on $

7 7e R in tat re5ression is R 1 2 , so

(8 8)

7 7 1 R (8 8) $ ere"ore,

Var[c] [ ]1 R (8 8)

=

=

−

=

′

−

′ = − −

σ′σ =− −

∑∑

∑

&|#

#

&|#

& #

& &

& &

& ' &

z|X


32/53


5ulticollinearit)

( )

%% 1

n% %

ii 1

Var[c] [ ]1 R (8 8)

All else constant, te *ariance o" te coe""icient on rises as te "it

in te re5ression o" on te oter *aria#les 5oes up$ 9" te "it is

per"ect, te

−

=

σ′σ =

− −∑#

&|#

& ' &

&

&

( )%

*ariance #ecomes in"inite$

;etectin5 multicollinearity<

1Variance in"lation "actor: V93(8) $1 R − &|#


33/53


Gasoline 5aret+egression nalysislogG versus logIn:o#eJ log(G


34/53


Gasoline 5aret

+egression nalysis logG versus logIn:o#eJ log(GJ ...


35/53


Con$ition /um'er an$

Variance Inflation Mactors

Con$ition num'er

larger than > is

Nlarge&?

What $oes this

meanK


36/53



37/53


The Aongle) Data


38/53


/IST Aongle) Solution


39/53


Excel Aongle) Solution


40/53


Jhe CKJ >iiei Probem


41/53


Certifie$ Milipelli ;esults


42/53


5inita' Milipelli ;esults


43/53


Stata Milipelli ;esults


44/53


Een after $ropping t%o "ran$om

columns#, results are onl) correct to 6

or ! $igits&


45/53


;egression of x! on all other aria'les


46/53


-sing O; Decomposition


47/53


5ulticollinearit)

There is no 8cure9 for collinearit)& Estimating something else is not helpful"principal components, for example#&

There are 8measures9 of multicollinearit), such as the con$ition num'er of X an$ the ariance inflation factor&

Best approach: Be cogniJant of it& -n$erstan$ its implications for estimation&

What is 'etter: Inclu$e a aria'le that causes collinearit), or $rop the aria'lean$ suffer from a 'iase$ estimatorK5ean s*uare$ error %oul$ 'e the 'asis for comparison&Some generalities& ssuming X has full ran, regar$less of the con$ition,

b is still un'iase$Gauss15aro still hol$s


48/53


Specification an$ Munctional Morm:

/onlinearit)

% %

1 % / 1 % /

% / % /

%

%

=opulation !stimators

ˆ

[ | , ] ˆ% %

ˆ!stimator o" te *ariance o"

ˆ$ [ ] [ ]

x x

x

x

y x x z y b b x b x b z

E y x z x b b x

x

Est Var Var b x Va

= β + β + β + β + ε = + + +

∂δ = = β + β δ = +

∂δ

δ = + / % /[ ] [ , ]r b xCov b b+


49/53


Aog Income E*uation----------------------------------------------------------------------Ordinary least squares regression ............LHS=LOG@ Mean = -.5)46 ;sti#ated ov$J$2 Standard deviation = .4949 "u#$er o% o$servs. = 2)&22 Model si'e (ara#eters = ) *egrees o% %reedo# = 2)&5+esiduals Su# o% squares = 5462.0&6!6

Standard error o% e = .44))3it +-squared = .)2&)--------------------------------------------------------------------- 7aria$le8 oe%%i:ient Standard ;rror $St.;r. (8E8' Mean o% >--------------------------------------------------------------------- G;8 .06225??? .002& 29.!9 .0000 4&.52)2 G;S8 -.000)4??? .2424!2*-04 -&0.5)6 .0000 2022.99onstant8 -&.9&0??? .0456) -69.!!4 .0000 M++I;*8 .&25&??? .00)0& 45.)6) .0000 .)5!69

HHI*S8 -.&4??? .00655 -).002 .0000 .402)2 3;ML;8 -.0049 .00552 -.!!9 .&)&9 .4)!! ;*A8 .05542??? .0020 46.050 .0000 .&202--------------------------------------------------------------------- verage ge = 4&.52)2. ;sti#ated (artial e%%e:t = .066225 2/.000)414&.52)2 = .000!.;sti#ated 7arian:e 4.54)99e-6 4/4&.52)212/5.!)9)&e-01 4/4&.52)21/-5.2!5e-!1= ).4)550!6e-0!. ;sti#ated standard error = .0002)&4.


50/53


Specification an$ Munctional Morm:

Interaction Effect

1 % / 1 % /

% %

%

%

=opulation !stimators

ˆ

[ | , ] ˆ8

ˆ!stimator o" t(e *ariance o"

ˆ$ [ ] [ ]

x x

x

x

y x z xz y b b x b z b xz

E y x z b b z

x

Est Var Var b z Va

= β + β + β + β + ε = + + +

∂δ = = β + β δ = +

∂

δ

δ = + % [ ] % [ , ]r b zCov b b+


51/53


Interaction Effect----------------------------------------------------------------------Ordinary least squares regression ............LHS=LOG@ Mean = -.5)46 Standard deviation = .4949 "u#$er o% o$servs. = 2)&22 Model si'e (ara#eters = 4 *egrees o% %reedo# = 2)&!+esiduals Su# o% squares = 6540.459!! Standard error o% e = .4!9&3it +-squared = .00!96 dusted +-squared = .00!!5 Model test 3 &J 2)&! /Bro$1 = !2.4/.00001--------------------------------------------------------------------- 7aria$le8 oe%%i:ient Standard ;rror $St.;r. (8E8' Mean o% >---------------------------------------------------------------------onstant8 -.22592??? .0605 -)6.&)6 .0000 G;8 .0022)??? .000&6 6.240 .0000 4&.52)2

3;ML;8 .22&9??? .02&6& !.9!) .0000 .4)!! G;N3;M8 -.00620??? .00052 -.!9 .0000 2.2960---------------------------------------------------------------------*o Do#en earn #ore tCan #en /in tCis sa#Ble1


52/53



53/53

econometrics eviews simple easy presentation

Documents