1

Nonlinear Regression Functions(SW Chapter 8)

2

The TestScore – STR relation looks linear (maybe)…

3

But the TestScore – Income relation looks nonlinear...

4

The general nonlinear population regression function

5

6

Nonlinear Functions of a Single Independent Variable (SW Section 8.2)

7

1. Polynomials in X

8

Example: TestScore vs. Income

9

Estimation in STATA

10

Interpretation

. graph twoway scatter testscr avginc || connected yhat avginc, sort msymbol(none) || connected yhat2 avginc, sort msymbol(none)See Chapter 3 of Statistics with Stata, especially pages 79 & 118

11

Interpretation

12

13

Interpretation

14

Marginal effects in STATA

15

Estimation of a cubic in STATA

16

Plotting a cubic in STATA

. graph twoway scatter testscr avginc || connected yhat2 avginc, sort msymbol(none) || connected yhat3 avginc, sort msymbol(T)

. reg testscr avginc avginc2 avginc3, rob

. predict yhat3(option xb assumed; fitted values)

17

Marginal effects in STATA

18

Marginal effects in STATA

19

Ramsey’s RESET Test: REgression Specification Error Test

• Consider the model (1) • General test for misspecification of functional form• If LSA #1 holds, then no non-linear function of the X’s

should be significant when added to the model. • Consider (2) • Null hypothesis is that (1) is correctly specified• How many powers of predicted values to include?• Conduct F-test on powers of predicted values• J.B. Ramsey (1969), Tests for Specification Error in Classical Linear Least

Squares Regression Analysis. Journal of the Royal Statistical Society, Series B 31, 350–371

20

Ramsey’s RESET Test. reg test str avginc, r

Linear regression Number of obs = 420 F( 2, 417) = 132.65 Prob > F = 0.0000 R-squared = 0.5115 Root MSE = 13.349

------------------------------------------------------------------------------ | Robust testscr | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- str | -.6487401 .3533403 -1.84 0.067 -1.34329 .04581 avginc | 1.839112 .114733 16.03 0.000 1.613585 2.064639 _cons | 638.7292 7.301234 87.48 0.000 624.3773 653.081------------------------------------------------------------------------------

. estat ovtest (can just type . ovtest)

Ramsey RESET test using powers of the fitted values of testscr Ho: model has no omitted variables F(3, 414) = 18.36 Prob > F = 0.0000

21

Ramsey’s RESET Test. reg test str avginc avginc2, r

Linear regression Number of obs = 420 F( 3, 416) = 286.55 Prob > F = 0.0000 R-squared = 0.5638 Root MSE = 12.629

------------------------------------------------------------------------------ | Robust testscr | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- str | -.9099512 .3545374 -2.57 0.011 -1.606859 -.2130432 avginc | 3.881859 .2709564 14.33 0.000 3.349245 4.414474 avginc2 | -.044157 .0049606 -8.90 0.000 -.053908 -.034406 _cons | 625.2308 7.087793 88.21 0.000 611.2984 639.1631------------------------------------------------------------------------------

. estat ovtest

Ramsey RESET test using powers of the fitted values of testscr Ho: model has no omitted variables F(3, 413) = 2.48 Prob > F = 0.0605

22

Ramsey’s RESET Test. reg test str avginc avginc2 avginc3, r

Linear regression Number of obs = 420 F( 4, 415) = 207.23 Prob > F = 0.0000 R-squared = 0.5663 Root MSE = 12.608

------------------------------------------------------------------------------ | Robust testscr | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- str | -.9277523 .3562919 -2.60 0.010 -1.628114 -.2273905 avginc | 5.124736 .7045403 7.27 0.000 3.739824 6.509649 avginc2 | -.1011073 .0287052 -3.52 0.000 -.157533 -.0446815 avginc3 | .0007293 .0003414 2.14 0.033 .0000582 .0014003 _cons | 617.8974 7.926373 77.95 0.000 602.3165 633.4782------------------------------------------------------------------------------

. estat ovtest

Ramsey RESET test using powers of the fitted values of testscr Ho: model has no omitted variables F(3, 412) = 1.79 Prob > F = 0.1490

23

Ramsey’s RESET Test. reg test str el_pct meal_pct , r

Linear regression Number of obs = 420 F( 3, 416) = 453.48 Prob > F = 0.0000 R-squared = 0.7745 Root MSE = 9.0801

------------------------------------------------------------------------------ | Robust testscr | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- str | -.9983092 .2700799 -3.70 0.000 -1.529201 -.4674178 el_pct | -.1215733 .0328317 -3.70 0.000 -.18611 -.0570366 meal_pct | -.5473456 .0241072 -22.70 0.000 -.5947328 -.4999583 _cons | 700.15 5.56845 125.74 0.000 689.2042 711.0958------------------------------------------------------------------------------

. estat ovtest

Ramsey RESET test using powers of the fitted values of testscr Ho: model has no omitted variables F(3, 413) = 6.29 Prob > F = 0.0004

24

Ramsey’s RESET Test. reg test str el_pct meal_pct avginc , r

Linear regression Number of obs = 420 F( 4, 415) = 467.42 Prob > F = 0.0000 R-squared = 0.8053 Root MSE = 8.4477

------------------------------------------------------------------------------ | Robust testscr | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- str | -.5603892 .2550641 -2.20 0.029 -1.061768 -.0590105 el_pct | -.1943282 .0332445 -5.85 0.000 -.2596768 -.1289795 meal_pct | -.3963661 .0302302 -13.11 0.000 -.4557895 -.3369427 avginc | .674984 .0837161 8.06 0.000 .5104236 .8395444 _cons | 675.6082 6.201865 108.94 0.000 663.4172 687.7992------------------------------------------------------------------------------

. estat ovtest

Ramsey RESET test using powers of the fitted values of testscr Ho: model has no omitted variables F(3, 412) = 0.47 Prob > F = 0.7014

25

Ramsey’s RESET Test: replicated. predict yh(option xb assumed; fitted values)

. sum yh

Variable | Obs Mean Std. Dev. Min Max-------------+-------------------------------------------------------- yh | 420 654.1565 17.09817 614.9183 702.8387

. gen yhz = (yh-r(mean))/r(sd)

. sum yh*

Variable | Obs Mean Std. Dev. Min Max-------------+-------------------------------------------------------- yh | 420 654.1565 17.09817 614.9183 702.8387 yhz | 420 1.22e-09 1 -2.294882 2.847214

. gen yhz2=yhz*yhz

. gen yhz3=yhz^3

. gen yhz4=yhz^4

26

Ramsey’s RESET Test: replicated. reg test str el meal avginc yhz2 yhz3 yhz4

Source | SS df MS Number of obs = 420-------------+------------------------------ F( 7, 412) = 244.48 Model | 122595.145 7 17513.5921 Prob > F = 0.0000 Residual | 29514.4488 412 71.6370116 R-squared = 0.8060-------------+------------------------------ Adj R-squared = 0.8027 Total | 152109.594 419 363.030056 Root MSE = 8.4639

------------------------------------------------------------------------------ testscr | Coef. Std. Err. t P>|t| [95% Conf. Interval]-------------+---------------------------------------------------------------- str | -.5500585 .2336368 -2.35 0.019 -1.009327 -.0907896 el_pct | -.2170374 .0407058 -5.33 0.000 -.2970544 -.1370204 meal_pct | -.400967 .0289303 -13.86 0.000 -.4578364 -.3440976 avginc | .6476592 .1505253 4.30 0.000 .3517657 .9435527 yhz2 | .7652051 .915534 0.84 0.404 -1.034495 2.564906 yhz3 | -.0822669 .3243362 -0.25 0.800 -.7198272 .5552933 yhz4 | -.0650369 .1767693 -0.37 0.713 -.412519 .2824453 _cons | 675.8077 5.443279 124.15 0.000 665.1076 686.5077------------------------------------------------------------------------------

. test yhz2 yhz3 yhz4

( 1) yhz2 = 0 ( 2) yhz3 = 0 ( 3) yhz4 = 0

F( 3, 412) = 0.47 Prob > F = 0.7014

27

2. Logarithmic functions of Y and/or X

28

The 3 log specifications

29

I. Linear-log population regression function

30

Linear-log case, continued

31

Example: TestScore vs. ln(Income)

32

Linear-log vs. Cubic models

33

II. Log-linear population regression function

34

Log-linear case, continued

35

III. Log-log population regression function

36

Log-log case, continued

37

Example: ln( TestScore) vs. ln( Income)

38

Example: ln( TestScore) vs. ln( Income), ctd.

39

The log-linear and log-log specifications:

40

41

42

43

44

45

Other nonlinear functions (and nonlinear least squares) (SW App. 8.1)

46

Negative exponential growth

47

Nonlinear Least Squares

48

49