goodness-of-fit tests for parametric regression...

Goodness-of-Fit Tests for ParametricRegression Models

Jianqing Fan and Li-Shan Huang

Several new tests are proposed for examining the adequacy of a family of parametric models against large nonparametric alternativesThese tests formally check if the bias vector of residuals from parametric ts is negligible by using the adaptive Neyman test andother methods The testing procedures formalize the traditional model diagnostic tools based on residual plots We examine the ratesof contiguous alternatives that can be detected consistently by the adaptive Neyman test Applications of the procedures to the partiallylinear models are thoroughly discussed Our simulation studies show that the new testing procedures are indeed powerful and omnibusThe power of the proposed tests is comparable to the F -test statistic even in the situations where the F test is known to be suitable andcan be far more powerful than the F -test statistic in other situations An application to testing linear models versus additive models isalso discussed

KEY WORDS Adaptive Neyman test Contiguous alternatives Partial linear model Power Wavelet thresholding

1 INTRODUCTION

Parametric linear models are frequently used to describethe association between a response variable and its predic-tors The adequacy of such parametric ts often arises Con-ventional methods rely on residual plots against tted valuesor a covariate variable to detect if there are any system-atic departures from zero in the residuals One drawback ofthe conventional methods is that a systematic departure thatis smaller than the noise level cannot be observed easilyRecently many articles have presented investigations of the useof nonparametric techniques for model diagnostics Most ofthem focused on one-dimensional problems including Dette(1999) Dette and Munk (1998) and Kim (2000) The bookby Hart (1997) gave an extensive overview and useful refer-ences Chapter 5 of Bowman and Azzalini (1997) outlined thework by Azzalini Bowman and Haumlrdle (1989) and Azzaliniand Bowman (1993) Although a lot of work has focused onthe univariate case there is relatively little work on the multi-ple regression setting Hart (1997 section 93) considered twoapproaches one is to regress residuals on a scalar function ofthe covariates and then apply one-dimensional goodness-of- ttests the second approach is to explicitly take into account themultivariate nature The test by Haumlrdle and Mammen (1993)was based on the L2 error criterion in d-dimensions StuteGonzaacutelez Mantoiga and Presedo Quindimil (1998) investi-gated a marked empirical process based on the residuals andAerts Claeskens and Hart (1999) constructed tests basedon orthogonal series that involved choosing a nested modelsequence in the bivariate regression

In this article a completely different approach is proposedand studied The basic idea is that if a parametric family ofmodels ts data adequately then the residuals should havenearly zero bias More formally let 4x11 Y151 1 4xn1 Yn5 be

Jianqing Fan is Professor Department of Statistics Chinese Universityof Hong Kong and Professor Department of Statistics University ofNorth Carolina Chapel Hill NC 27599-3260 (E-mail jfanemailuncedu)Li-Shan Huang is Assistant Professor Department of BiostatisticsUniversity of Rochester Medical Center Rochester NY 14642 (E-maillhuangbstrocheste redu) Fanrsquos research was partially supported by NSA96-1-0015 and NSF grant DMS-98-0-3200 Huangrsquos work was partially sup-ported by NSF grant DMS-97-10084 The authors thank the referee and theassociate editor for their constructive and detailed suggestions that led tosigni cant improvement in the presentation of the article

independent observations from a population

Y D m4x5 C ˜1 ˜ N 401lsquo 251 (11)

where x is a p-dimensional vector and m4cent5 is a smooth regres-sion surface Let f 4cent1 ˆ5 be a given parametric family Thenull hypothesis is

H0 2 m4cent5 D f 4cent1 ˆ5 for some ˆ0 (12)

Examples of f 4cent1 ˆ5 include the widely used linear modelf 4x1 ˆ5 D xT ˆ and a logistic model f 4x1ˆ5 D ˆ0=41 Cˆ1 exp4ˆT

2 x55The alternative hypothesis is often vague Depending on the

situations of applications one possible choice is the saturatednonparametric alternative

H1 2 m4cent5 6D f4cent1 ˆ5 for all ˆ1 (13)

and another possible choice is the partially linear models (seeGreen and Silverman 1994 and the references therein)

H1 2 m4x5 D f 4x15 C x2T sbquo21 with f 4cent5 nonlinear1 (14)

where x1 and x2 are the covariates Traditional model diagnos-tic techniques involve plotting residuals against each covari-ate to examine if there is any systematic departure againstthe index sequence to check if there is any serial correlationand against tted values to detect possible heteroscedasticityamong others This amounts to informally checking if biasesare negligible in the presence of large stochastic noises Thesetechniques can be formalized as follows Let O be an esti-mate under the null hypothesis and let O D 4 O11 1 On5T

be the resulting residuals with Oi D Yi ƒ f4xi1 O5 Usually Oconverges to some vector ˆ0 Then under model (11) con-ditioned on 8xi9

niD1 O is nearly independently and normally

distributed with mean vector Dagger D 4Dagger11 1 Daggern5T where Daggeri Dm4xi5ƒf 4xi1 ˆ05 Namely any nite-dimensional components

copy 2001 American Statistical AssociationJournal of the American Statistical Association

June 2001 Vol 96 No 454 Theory and Methods

640

Fan and Huang Regression Goodness-of-Fit 641

of O are asymptotically independent and normally distributedwith mean being a subvector of Dagger Thus the problem becomes

H0 2 Dagger D 0 versus H1 2 Dagger 6D 01 (15)

based on the observations O Note that Oi will not be inde-pendent in general after estimating ˆ but the dependence ispractically negligible as will be demonstrated Thus the tech-niques for independent samples such as the adaptive Neymantest in Fan (1996) continue to apply Plotting a covariate Xj

against the residual O aims at testing if the biases in O alongthe direction Xj are negligible namely whether the scatter plot84Xj1Daggerj59 is negligible in the presence of the noise

In testing the signi cance of the high-dimensional prob-lem (15) the conventional likelihood ratio statistic is not pow-erful due to noise accumulation in the n-dimensional spaceas demonstrated in Fan (1996) An innovative idea proposedby Neyman (1937) is to test only the rst m-dimensional sub-problem if there is prior that most of nonzero elements lieon the rst m dimension To obtain such a qualitative priora Fourier transform is applied to the residuals in an attemptto compress nonzero signals into lower frequencies In test-ing goodness of t for a distribution function Fan (1996)proposed a simple data-driven approach to choose m basedon power considerations This results in an adaptive Neymantest See Rayner and Best (1989) for more discussions onthe Neyman test Some other recent work motivated by theNeyman test includes Eubank and Hart (1992) Eubank andLaRiccia (1992) Kim (2000) Inglot Kallenberg and Ledwina(1994) Ledwina (1994) Kuchibhatla and Hart (1996) andLee and Hart (1998) among others Also see Bickel andRitov (1992) and Inglot and Ledwina (1996) for illuminatinginsights into nonparametric tests In this article we adopt theadaptive Neyman test proposed by Fan (1996) to the testingproblem (15) It is worth noting that the test statistic stud-ied by Kuchibhatla and Hart (1996) looks similar to that ofFan (1996) but they are decidedly different See the secondparagraph of Section 3 for further discussions The adaptiveNeyman test of Kuchibhatla and Hart (1996) also will beimplemented to demonstrate the versatility of our proposedidea We would note that the adaptive Neyman tests in Fan(1996) and Kuchibhatla and Hart (1996) were proposed forindependent samples Herein the adaptive Neyman test of Fan(1996) will be justi ed to remain applicable for weakly depen-dent O in the current regression setting see Theorem 1

The power of the adaptive Neyman test depends on thesmoothness of 8Daggeri9

niD1 as a function of i Let us call this

function Dagger4cent5 namely Dagger4i=n5 D Daggeri The smoother the func-tion Dagger4cent5 the more signi cant are the Fourier coef cientson the low frequencies and hence the more powerful thetest will be see Theorems 2 and 3 When m4cent5 is com-pletely unknown such as in (13) there is no informationon how to make the function Dagger4cent5 smoother by ordering theresiduals Intuitively the closer the two consecutive covari-ate vectors xi and xiC1 the smaller the difference betweenm4xi5 ƒ f 4xi1 ˆ05 and m4xiC15 ƒ f4xiC11 ˆ05 and hence thesmoother the sequence 8m4xi5ƒf4xi1 ˆ059 indexed by i Thusa good proxy is to order the residuals 8˜i9

niD1 in a way that the

corresponding xirsquos are close as a sequence Note that when

there is only one predictor variable (ie p D 1) such orderingis straightforward However it is quite challenging to order amultivariate vector A method based on the principal compo-nent analysis is described in Section 22

When there is some information about the alternativehypothesis such as the partially linear model in (14) it issensible to order the residuals according to the covariate x1Indeed our simulation studies show that it improves a greatdeal the power of the generic ordering procedure outlined inthe last paragraph improved a great deal The parameters sbquo2

in model (14) can be estimated easily at the nƒ1=2 rate viaa difference based estimator for example that of Yatchew(1997) Then the problem is heuristically reduced to the one-dimensional nonparametric setting see Section 24 for details

Wavelet transform is another popular family of orthogo-nal transforms It can be applied to the testing problem (15)For comparison purposes wavelet thresholding tests are alsoincluded in our simulation studies See Fan (1996) andSpokoiny (1996) for various discussions on the properties ofthe wavelet thresholding tests

This article is organized as follows In Section 2 we pro-pose a few new tests for the testing problems (12) ver-sus (13) and (12) versus (14) These tests include theadaptive Neyman test and the wavelet thresholding tests inSections 21 and 23 respectively Ordering of multivari-ate vectors is discussed in Section 22 Applications of thetest statistics in the partially linear models are discussed inSection 24 and their extension to additive models is outlinedbrie y in Section 25 Section 26 outlines simple methods forestimating the residual variance lsquo 2 in the high-dimensionalsetting In Section 3 we carry out a number of numerical stud-ies to illustrate the power of our testing procedures Technicalproofs are relegated to the Appendix

2 METHODS AND RESULTS

As mentioned in the Introduction the hypothesis testingproblem (12) can be reduced to the high-dimensional prob-lem (15) based on the weakly dependent Gaussian randomvector O The adaptive Neyman test and the wavelet thresh-olding test proposed in Fan (1996) are adapted for the cur-rent regression setting The case of a multiple linear modelf 4x1 ˆ5 D xT ˆ is addressed here and its extension to a generalparametric model can be treated similarly

21 The Adaptive Neyman Test

Assume a parametric linear model under the null hypothe-sis

Y D xT ˆ C ˜1 ˜ N 401lsquo 251

and assume O is the resulting residual vector from a least-squares t Let O uuml D 4 O uuml

11 1 O uumln5T be the discrete Fourier

transform of the residual vector O More precisely we de ne

O uuml2jƒ1 D 42=n51=2

nX

iD1

cos42 ij=n5 Oi1

O uuml2j D 42=n51=2

nXiD1

sin42 ij=n5 Oi1 j D 11 1 6n=270

642 Journal of the American Statistical Association June 2001

When n is odd an additional term O uumln D 41=

pn=25

PniD1 Oi is

needed Note that for linear regression with an intercept thisterm is simply zero and hence can be ignored As mentionedin the Introduction the purpose of the Fourier transform isto compress useful signals into low frequencies so that thepower of the adaptive Neyman test can be enhanced Let Olsquo 1

be a nƒ1=2 consistent estimate of lsquo under both the null andthe alternative hypotheses Methods for constructing such anestimate are outlined in Section 26 The adaptive Neyman teststatistic is de ned as

T uumlAN1 1 D max

1micrommicron

1p2m Olsquo 4

1

mX

iD1

O uuml 2i ƒ Olsquo 2

1 1 (21)

and the null hypothesis is rejected when T uumlAN1 1 is large See

Fan (1996) for motivations and some optimal properties of thetest statistic Note that the factor 2lsquo 4 in (21) is the varianceof ˜2 with ˜ N 401lsquo 25 Thus the null distribution of thetesting procedure depends on the normality assumption Fornonnormal noises a reasonable method is to replace the factor2 Olsquo 4

1 by a consistent estimate Olsquo 22 of Var4˜2

i 5 This leads to thetest statistic

T uumlAN12 D max

1micrommicron

1p

m Olsquo 22

mXiD1


1 0 (22)

The null distribution of T uumlAN1 2 is expected to be more robust

against the normality assumption Following Fan (1996) wenormalize the test statistics as

TAN1 j Dp

2 log log nT uumlAN1 j

ƒ 82 log logn C 005 log log logn

ƒ 005 log44 59 for j D 1120 (23)

Under the null hypothesis (15) and the independent normalmodel with a known variance it can be shown that

P4TAN1 j lt x5 exp4ƒexp4ƒx55 as n ˆ0 (24)

See Darling and Erdos (1956) and Fan (1996) However theapproximation (24) is not so good Let us call the exact nulldistribution of TAN1 1 under the independent normal model witha known variance as Jn Based on a million simulations thedistribution of Jn was tabulated in Fan and Lin (1998) Wehave excerpted some of their results and they are presented inTable 1

We now show that the approximation (24) holds for test-ing linear models where 8 Oi9 are weakly dependent Forconvenience of technical proofs we modify the range of max-imization over m as 611 n=4log log n547 This hardly affects ourtheoretical understanding of the test statistics and has littleimpact on practical implementations of the test statistics

Theorem 1 Suppose that Conditions A1ndashA3 in theAppendix hold Then under the null hypothesis of (12) withf 4x1 ˆ5 D xT ˆ we have

P4TAN1j ltx5exp4ƒexp4ƒx55 as nˆ1 for j D1120

The proof is given in the Appendix As a consequence ofTheorem 1 the critical region

TAN1 j gt ƒ log8ƒ log41 ƒ 59 4j D 1125 (25)

has an asymptotic signi cance level Assume that O converges to ˆ0 as n ˆ Then f4cent1 ˆ05

is generally the best parametric approximation to the under-lying regression function m4cent5 among the candidate models8f4cent1 ˆ59 Let Daggeri D m4xi5 ƒf 4xi1 ˆ05 and let Dagger uuml be the Fouriertransform of the vector Dagger D 4Dagger11 1 Daggern5T Then we have thefollowing power result

Theorem 2 Under the assumptions in Theorem 1 andCondition A4 in the Appendix if

4log logn5ƒ1=2 max1micrommicron

mƒ1=2mX

iD1

Dagger uuml 2i ˆ1 (26)

then the critical regions (25) have an asymptotic power 1

We now give an implication of condition (26) Note thatby Parsevalrsquos identity

mƒ1=2mX

iD1

Dagger uuml 2i D mƒ1=2

nX

iD1

Dagger2i ƒ

nX

iDmC1

Dagger uuml 2i 0 (27)

Suppose that the sequence 8Daggeri9 is smooth so that

nƒ1nX

iDmC1

Dagger uuml 2i D O4mƒ2s5 (28)

for some s gt 0 Then the maximum value of (27) over m isof order

nƒ1nX

iD1

Dagger2i

4sC1 1=44s5

0

For random designs with a density h

nXiD1

Dagger2i D

nXiD1

8m4xi5 ƒ f 4xi1 ˆ0592

nZ

8m4x5 ƒ f 4x1 ˆ0592h4x5dx0

By Theorem 2 we have the following result

Theorem 3 Under the conditions of Theorem 2 if thesmoothness condition (28) holds then the adaptive Neymantest has an asymptotic power 1 for the contiguous alternativewithZ

8m4x5ƒf 4x1ˆ0592h4x5 dxDnƒ4s=44sC154loglogn52s=44sC15cn

for some sequence with liminfcn Dˆ

The signi cance of the preceding theoretical result is its adap-tivity The adaptive Neyman test does not depend at all on thesmoothness assumption (28) Nevertheless it can adaptivelydetect alternatives with rates O4nƒ2s=44sC154log logn5s=44sC155which is the optimal rate of adaptive testing (see Spokoiny1996) In this sense the adaptive Neyman test is truly adap-tive and is adaptively optimal


Table 1 Upper Quantile of the Distribution Jn

nn 10 20 30 40 60 80 100 120 140 160 180 200

001 5078 6007 6018 6022 6032 6037 6041 6041 6042 6042 6047 60430025 4057 4075 4082 4087 4091 4093 4095 4095 4095 4095 4095 4095005 3067 3077 3083 3085 3088 3089 3090 3090 3090 3091 3090 3089010 2074 2078 2081 2082 2085 2085 2086 2087 2086 2087 2087 2086

22 Ordering Multivariate Vector

The adaptive Neyman test statistics depend on the order ofthe residuals Thus we need to order the residuals rst beforeusing the adaptive Neyman test By Theorem 2 the power ofthe adaptive Neyman tests depends on the smoothness of thesequence 8m4xi5 ƒ f 4xi1 ˆ059 indexed by i A good orderingshould make the sequence 8m4xi5 ƒ f 4xi1 ˆ059

niD1 as smooth

as possible for the given function m so that its large Fouriercoef cients are concentrated on low frequencies

When the alternative is the partial linear model (14) wecan order residuals according to the covariate X1 so that thesequence 8m4xi5 ƒ f 4xi1 ˆ059

niD1 is smooth for given m For

saturated nonparametric alternative (13) there is little usefulinformation on how to order the sequence As discussed inthe Introduction a good strategy is to order the covariates8xi9

niD1 so that they are close to each other consecutively This

problem easily can be done for the univariate case (p D 1)However for the case p gt 1 there are limited discussions onordering multivariate observations Barnett (1976) gave manyuseful ideas and suggestions One possible approach is to rstassign a score si to the ith observation and then order theobservations according to the rank of si Barnett (1976) calledthis ldquoreduced orderingrdquo

What could be a reasonable score 8si9 We may considerthis problem from the viewpoint of principal component (PC)analysis (see eg Jolliffe 1986) Let S be the sample covari-ance matrix of the covariate vectors 8xi1 i D 11 1 n9 Denoteby lsaquo11 1 lsaquop the ordered eigenvalues of S with correspond-ing eigenvectors 11 1 p Then zi1 k D T

k xi is the score forthe ith observation on the kth sample PC and lsaquok can be inter-preted as the sample variance of 8z11 k1 1 zn1 k9 Note thatxi ƒ Nx D zi1 11 C cent cent cent C zi1 pp where Nx is the sample averageThus a measure of variation of xi may be formed by taking

si D1

n ƒ 1

pXkD1

lsaquokz2i1 k0

We call si the sample score of variation Also seeGnanadesikan and Kettenring (1972) for an interpretation of si

as a measure of the degree of in uence for the ith observationon the orientation and scale of the PCs A different orderingscheme was given in Fan (1997)

Ordering according to a certain covariate is another sim-ple and viable method It focuses particularly on testing ifthe departure from linearity in a particular covariate can beexplained by chance It formalizes the traditional scatterplottechniques for model diagnostics The approach can be pow-erful when the alternative is the additive models or partiallylinear models See Section 25 for further discussions

In the case of two covariates Aerts et al (1999) constructeda score test statistic and the power of which depends also onthe ordering of a sequence of models It seems that orderingis needed in the multiple regression setting whether choosingan ordering of residuals or a model sequence

23 Hard-Thresholding Test

The Fourier transform is used in the adaptive Neymantest We may naturally ask how other families of orthogo-nal transforms behave For comparison purposes we applythe wavelet hard-thresholding test of Fan (1996) that is basedon the ldquoextremal phaserdquo family of wavelets See Daubechies(1992) for details on construction of the discrete wavelet trans-form with various wavelets The Splus WaveThresh packageincludes many useful routines for wavelets

Assume that the residuals are ordered We rst standardizethe residuals as OS1 i D Oi= Olsquo 3 where Olsquo3 is some estimate of lsquo Let 8 OW 1 i9 be the empirical wavelet transform of the standard-ized residuals arranged in such a way that the coef cients atlow resolution levels correspond to small indexes The thresh-olding test statistic [equation (17) in Fan 1996] is de ned as

TH DJ0X

iD1

O2W 1 i C

nX

iDJ0C1

O2W 1 iI 4mdash OW 1 imdash gt bdquo5

for some given J0 and bdquo Dp

2 log4nan5 with an D logƒ24n ƒJ05 As in Fan (1996) J0 D 3 is used which keeps the waveletcoef cients in the rst two resolution levels intact Then itcan be shown analogously that TH has an asymptotic normaldistribution under the null hypothesis in Theorem 1 and the41 ƒ 5 critical region is given by

lsquo ƒ1H 4TH ƒ ŒH 5 gt ecircƒ141 ƒ 51

where ŒH D J0 Cp

2= aƒ1n bdquo41 C bdquoƒ25 and lsquo 2

H D 2J0 Cp2= aƒ1

n bdquo341 C 3bdquoƒ25 The power of the wavelet threshold-ing test can be improved further by using the following tech-niques owing to Fan (1996) Under the null hypothesis (15)the maximum of n independent Gaussian noises is on the orderof

p2 logn Thus replacing an in the thresholding param-

eter bdquo by min444maxi OW 1 i5ƒ41 logƒ24n ƒ J055 does not alter

the asymptotic null distribution Then under the alternativehypothesis the latter is smaller than an and hence the pro-cedure uses a smaller thresholding parameter automaticallyleading to better power of the wavelet thresholding test

24 Linear Versus Partially Linear Models

Suppose that we are interested in testing the linear modelagainst the partial linear model (14) The generic methods


outlined in Sections 21ndash23 continue to be applicable In thissetting a more sensible ordering scheme is to use the order ofX1 instead of the score si given in Section 22 Our simulationstudies show that this improves a great deal on the power ofthe generic methods Nevertheless the power can further beimproved when the partially linear model structure is fullyexploited

The basic idea is to estimate the coef cient vector sbquo2 rstand then to compute the partial residuals zi D Yi ƒ xT

i1 2Osbquo2

Based on the synthetic data 84xi11 zi59 the problem reducesto testing the one-dimensional linear model but with slightlydependent data Theoretically this dependence structure isasymptotically negligible following the same proof as that ofTheorem 1 as long as Osbquo2 is estimated at the root-n rate Wecan therefore apply the adaptive Neyman and other tests to thedata 84xi11 zi59

There is a large literature on ef cient estimation of coef -cients sbquo2 See for example Wahba (1984) Speckman (1988)Cuzick (1992) and Green and Silverman (1994) Most of themethods proposed in the literature involve choices of smooth-ing parameters For our purpose a root-n consistent estima-tor of sbquo2 suf ces This can be much more easily constructedthan the semiparametric ef cient estimator The basic idea isas follows

First the data are arranged according to the correspond-ing order of 8X19 yielding the sorted data 84x4i51 Y4i551 i D11 1 n9 Note that for a random sample x4iC1511 ƒ x4i511 isOP41=n5 Thus for the differentiable function f in (14) wehave

Y4iC15 ƒ Y4i5 D 4x4iC1512 ƒ x4i5125T sbquo2 C ei

C OP 4nƒ151 i D 11 1 nƒ 11 (29)

where 8ei9 are correlated stochastic errors with ei D ˜4iC15 ƒ˜4i5 Thus sbquo2 can be estimated using the ordinary least-squaresestimator from the approximate linear model (29) This ideawas proposed by Yatchew (1997) (and was independently pro-posed by us around the same time with the following improve-ments) Yatchew (1997) showed that such an estimate sbquo2 isroot-n consistent To correct the bias in the preceding approx-imation at nite sample particularly for those x4i511 at tails(hence the spacing can be wide) we t the linear regressionmodel

Y4iC15 ƒ Y4i5 D 4x4iC151 1 ƒ x4i51 15sbquo1 C 4x4iC151 2 ƒ x4i5125T sbquo2 C ei

into our implementation by using the ordinary least-squaresmethod This improves the performance of the estimator basedon the model (29)

25 Linear Models Versus Additive Models

As will be demonstrated in Section 33 when the data areproperly ordered the proposed tests are quite powerful To testlinear versus partially linear models we show in Section 33that the adaptive Neyman tests based on ordered residualsaccording to the variable X1 for model (14) are nearly aspowerful as those tests in Section 24 when the structure of

the partially linear model is taken into account This encour-aged us to extend the idea to testing a linear model againstthe additive model

Y D f14X15 C f24X25 C cent cent centC fp4Xp5 C ˜0

Let bTj be the normalized test statistic (23) when the residualsare ordered according to variable Xj Let bT D max1microjmicrop

bTj Reject the null hypothesis when bT gt Jn4=p5 with Jn4=p5the =p upper quantile of the distribution Jn in Table 1 Thisis simply the Bonferroni adjustment applied to the combinedtest statistics

26 Estimation of Residual Variance

The implementations of the adaptive Neyman test and othertests depend on a good estimate of the residual variance lsquo 2This estimator should be good under both the null and alter-native hypotheses because an overestimate (caused by biases)of lsquo 2 under the alternative hypothesis will signi cantly dete-riorate the power of the tests A root-n consistent estimatorcan be constructed by using the residuals of a nonparametrickernel regression t However this theoretically satisfactorymethod encounters the ldquocurse of dimensionalityrdquo in practicalimplementations

In the case of a single predictor (p D 1) there are manypossible estimators for lsquo 2 A simple and useful technique isgiven in Hall Kay and Titterington (1990) An alternativeestimator can be constructed based on the Fourier transformwhich constitutes raw materials of the adaptive Neyman testIf the signals 8Daggeri9 are smooth as a function of i then thehigh frequency components are basically noise even under thealternative hypothesis namely the discrete Fourier transformO uuml

j for j large has a mean approximately equal to 0 and vari-ance lsquo 2 This gives us a simple and effective way to estimatelsquo 2 by using the sample variance of 8 O uuml

i 1 i D In C 11 1 n9

Olsquo 21 D

1nƒ In

nX

iDInC1

O uuml 2i ƒ

1n ƒ In

nX

iDInC1

O uumli

2

0 (210)

for some given In (D 6n=47 say) Under some mild conditionson the smoothness of 8Daggeri9 this estimator can be shown to beroot-n consistent even under the alternative hypothesis Hart(1997) suggested taking In D mC1 where m is the number ofFourier coef cients chosen based on some data-driven criteriaSimilarly an estimate Olsquo 2

2 for Var4˜25 can be formed by takingthe sample variance of 8 O uuml 2

i 1 i D n=4 C 11 1 n9

Olsquo 22 D

1

n ƒ In

nX

iDInC1

O uuml 4i ƒ

1

nƒ In

nX

iDInC1

O uuml 2i

2

0 (211)

In the case of multiple regression (p gt 1) the foregoingmethod is still applicable However as indicated inSection 22 it is hard to order residuals in a way that theresulting residuals 8Daggeri9 are smooth so that E4 O uuml

j 5 D 0 for largej Thus the estimator (210) can possess substantial biasIndeed it is dif cult to nd a practically satisfactory esti-mate for large p This problem is beyond the scope of thisarticle Here we describe an ad hoc method that is imple-mented in our simulation studies For simplicity of descrip-tion assume that there are three continuous and one discrete


covariates X11 X21 X3 and X4 respectively An estimate Olsquo `

can be obtained by tting linear splines with four knots andthe interaction terms between continuous predictors

ˆ0 C4X

iD1

ˆiXi C3X

iD1

4XjD1

ˆi1 j4Xi ƒ ti1 j5C

C ƒ1X1X2 C ƒ2X1X3 C ƒ3X2X31

where ti1 j denotes the 420j5th percentile j D 1121 314 of theith covariate i D 11 21 3 (continuous covariates only) In thismodel there are 20 parameters and the residual variance isestimated by

Olsquo 2` D residual sum of squared errors=4nƒ 2050

3 SIMULATIONS

We now study the power of the adaptive Neyman tests(TAN1 1 and TAN1 2) and the wavelet thresholding test TH viasimulations The simulated examples consist of testing thegoodness of t of the linear models in the situations of onepredictor partially linear models and multiple regression Theresults are based on 400 simulations and the signi cance levelis taken to be 5 Thus the power under the null hypoth-esis should be around 5 with a Monte Carlo error ofp

0005 uuml 0095=400 1 or so For each testing procedure weexamine how well it performs when the noise levels are esti-mated and when the noise levels are known The latter mimicsthe situations where the variance can be well estimated withlittle bias An example of this is when repeated measurementsare available at some design points Another reason for usingthe known variance is that we intend to separate the perfor-mance of the adaptive Neyman test and the performance ofthe variance estimation For the wavelet thresholding test theasymptotic critical value 1645 is used The simulated criti-cal values in table 1 of Fan and Lin (1998) are used for theadaptive Neyman test if lsquo is known empirical critical valuesare taken when lsquo1 andor lsquo2 are estimated For simplicity ofpresentation we report only the results of the wavelet thresh-olding test when the residual variances are given

To demonstrate the versatility of our proposed testingscheme we also include the test proposed by Kuchibhatla andHart (1996) (abbreviated as the KH test)

Sn D max1micrommicronƒp

1m

mXjD1

2n O uuml 2i

lsquo 21

where p is the number of parameters tted in the null modelCompared to the adaptive Neyman test this test tends toselect a smaller dimension m namely the maximization in Sn

is achieved at smaller m than that of the adaptive Neymantest Therefore the KH test will be somewhat more power-ful than the adaptive Neyman test when the alternative is verysmooth and will be less powerful than the adaptive Neymantest when the alternative is not as smooth Again the results ofSn are given in cases when variance is known and when vari-ance is estimated The same variance estimator as the adaptiveNeyman tests is applied

31 Goodness of Fit for Simple Linear Regression

In this section we study the power of the adaptive Neymantest and other tests for univariate regression problems

H0 2 m4x5 D C sbquox versus H1 2 m4x5 6D C sbquox0

The sample size is 64 and the residuals Oi are ordered bytheir corresponding covariate The power of each test is eval-uated at a sequence of alternatives given in Examples 1ndash3The empirical critical values for TAN1 1 and TAN1 2 are 462 and474 respectively and 341 for the KH test when lsquo is esti-mated For comparison purposes we also include the paramet-ric F test for the linear model against the quadratic regressionsbquo0 C sbquo1x C sbquo2x

2

Example 1 The covariate X1 is sampled from uniform4ƒ2125 and the response variable is drawn from

Y D 1 C ˆX21 C ˜1 ˜ N 401151 (31)

for each given value of ˆ The power function for each testis evaluated under the alternative model (31) with a givenˆ This is a quadratic regression model where the F test isderived Nevertheless the adaptive Neyman tests and the KHtest perform close to the F test in this ideal setting whereasthe wavelet thresholding test falls behind the other three testssee Figure 1 (a) and (b) In fact the wavelet tests are notspeci cally designated for testing this kind of very smoothalternatives This example is also designated to show howlarge a price the adaptive Neyman and the KH tests have topay to be more omnibus Surprisingly Figure 1 demonstratesthat both procedures paid very little price

Example 2 Let X1 be N 40115 and

Y D 1 C cos4ˆX1 5 C ˜1 ˜ N 401150 (32)

This example examines how powerful each testing procedureis for detecting alternatives with different frequency compo-nents The result is presented in Figure 1 (c) and (d) Clearlythe adaptive Neyman and KH tests outperform the F test whenlsquo is given and they lose some power when lsquo is estimatedThe loss is due to the excessive biases in the estimation oflsquo when ˆ is large This problem can be resolved by settinga larger value of In in the high-frequency cases The waveletthresholding test performs nicely too As anticipated the adap-tive Neyman test is more powerful than the KH test for detect-ing a high-frequency component

Example 3 We now evaluate the power of each testingprocedure at a sequence of logistic regression models

Y D10

100C ˆ exp4ƒ2X15C ˜1 ˜ N 401150 (33)

where X1 N 40115 Figure 1 (e) and (f) depicts the resultsThe adaptive Neyman and KH tests far outperform the F testwhich is clearly not omnibus The wavelet thresholding testis also better than the F test whereas it is dominated by theadaptive Neyman and KH tests


Figure 1 Power of the Adaptive Neyman Tests the KH Test the Wavelet Thresholding Test and the Parametric F Test for Examples 1ndash3 Withn D 64 Key A the adaptive Neyman test with known variance B the adaptive Neyman test with estimated variance (210) F the parametricF-test statistic H the wavelet thresholding test with known variance R the robust version of the adaptive Neyman test with estimated variance(210) and (211) S the KH test with known variance T the KH test with estimated variance (210)


Figure 1 Continued

32 Testing Linearity in Partially Linear Models

In this section we test the linear model versus a partiallinear model

m4X5 D ˆ0 C ˆ1X1 C f 4X25 C ˆ3X3 C ˆ4x41

where X11 1X4 are covariates The covariates X1 X2 andX3 are normally distributed with mean 0 and variance 1 Fur-thermore the correlation coef cients among these three ran-dom variables are 5 The covariate X4 is binary independentof X1 X2 and X3 with

P4X4 D 15 D 04 and P4X4 D 05 D 060

The techniques proposed in Section 24 are employed here forthe adaptive Neyman and KH tests and the same simulatedcritical values for n D 64 as in the case of the simple linearmodel are used For comparison we include the parametricF test for testing the linear model against the quadratic model

sbquo0 C4X

iD1

sbquoiXi CX

1microimicrojmicro3

sbquoi1jXiXj 0

We evaluate the power of each testing procedure at two par-ticular partially linear models as follows

Example 4 The dependent variable is generated from thequadratic regression model

Y D X1 C ˆX22 C 2X4 C ˜1 ˜ N 401151 (34)

for each given ˆ Although the true function involves aquadratic of X2 the adaptive Neyman and KH tests do slightlybetter than the F test as shown in Figure 2 (a) and (b)

This is due partially to the fact that an overparameterizedfull quadratic model is used in the F test Again the waveletthresholding test does not perform as well for this kind ofsmooth alternatives Note that this model is very similar tothat in Example 1 and the power of the adaptive Neyman andKH tests behaves analogously to that in Example 1 This ver-i es our claim that the parametric components are effectivelyestimated by using our new estimator for sbquo2 in Section 24

Example 5 We simulate the response variable from

Y D X1 C cos4ˆX2 5 C 2X4 C ˜1 ˜ N 401lsquo 250 (35)

The results are depicted in Figure 2 (c) and (d) The adap-tive Neyman wavelet thresholding and KH tests perform wellwhen lsquo is known The F test gives low power for ˆ para 105which indicates that it is not omnibus Similar remarks tothose given at the end of Example 4 apply Note that whenˆ is large the residual variance cannot be estimated well byany saturated nonparametric methods This is why the powerof each test is reduced so dramatically when lsquo is estimatedby using our nonparametric estimate of lsquo This also demon-strates that to have a powerful test lsquo should be estimated wellunder both the null and the alternative hypotheses

33 Testing for a Multiple Linear Model

Two models identical to those in Examples 4 and 5 areused to investigate the empirical power for testing linearitywhen there is no knowledge about the alternative namely thealternative hypothesis is given by (13) The sample size 128is used If we know that the nonlinearity is likely to occur inthe X2 direction we would order the residuals according toX2 instead of using the generic method in Section 22 Thus


Figure 2 Power of the Adaptive Neyman Tests the KH test the Wavelet Thresholding Test and the Parametric F Test for Examples 4 and 5With Partial Linear Models as Alternative and n D 64 Key A the adaptive Neyman test with known variance B the adaptive Neyman test withestimated variance (210) F the parametric F-test statistic H the wavelet thresholding test with known variance R the robust version of adaptiveNeyman test with estimated variance (210) and (211) S the KH test with known variance T the KH test with estimated variance (210)


for the adaptive Neyman test we study the following fourversions depending on our knowledge of the model

1 ordering according to X2 and using the known variance(c D 3017 where c is the 5 empirical critical value)

2 ordering by X2 and using Olsquo ` (c D 3061)3 ordering according to si in Section 22 and using the

known variance (c D 3006)4 ordering by si and using Olsquo ` (c D 3054)

To demonstrate the versatility of our proposal we also appliedthe KH test in the following ways

1 ordering according to X2 and using the known variance(c D 2033)

2 ordering by X2 and using Olsquo ` (c D 2034)3 ordering according to the rst principal component and

using the known variance (c D 2054)4 ordering by the rst principal component and using Olsquo `

(c D 2038)

Hart (1997 section 93) proposed performing a one-dimensional test a couple of times along the rst coupleof principal components and adjusting the signi cance levelaccordingly For simplicity we use only one principal direc-tion in our implementation [see versions (3) and (4) of theKH test] The results of the wavelet thresholding test with theideal ordering (by X2) and known variance are reported Theresults of the conventional F test against the quadratic mod-els are also included The simulation results are reported inFigure 3

First of all we note that ordering according to X2 is morepowerful than ordering by the generic method or by the rstprincipal direction When ordering by X2 the nonparametricprocedures with estimated variance perform closely to the cor-responding procedures with a known variance except for theKH test at the alternative models (35) with high frequencyThis in turn suggests that our generic variance estimator per-forms reasonably well The F test in Example 4 performscomparably to the adaptive Neyman with ideal orderingwhereas for Example 5 the F test fails to detect high-frequency components in the model The results are encour-aging Correctly ordering the covariate can produce a test thatis nearly as powerful as knowing the alternative model

4 SUMMARY AND CONCLUSION

The adaptive Neyman test is a powerful omnibus test It ispowerful against a wide class of alternatives Indeed as shownin Theorem 3 the adaptive Neyman test adapts automaticallyto a large class of functions with unknown degrees of smooth-ness However its power in the multiple regression settingdepends on how the residuals are ordered When the residualsare properly ordered it can be very powerful as demonstratedin Section 33 This observation can be very useful for testingthe linear model against the additive model We just need toorder the data according to the most sensible covariate Esti-mation of residual variance also plays a critical role for theadaptive Neyman test With a proper estimate of the residualvariance the adaptive Neyman test can be nearly as powerfulas the case where the variance is known

APPENDIX

In this Appendix we establish the asymptotic distribution given inTheorem 1 and the asymptotic power expression given in Theorem 2We rst introduce some necessary notation that is for the linearmodel We use the notation sbquo instead of ˆ to denote the unknownparameters under the null hypothesis Under the null hypothesis weassume that

Yi D xTi sbquo C˜1 ˜ N401lsquo 251

where sbquo is a p-dimensional unknown vector Let X D 4xij5 be thedesign matrix of the linear model Let x uuml

1 1 1 x uumlp be respectively

the discrete Fourier transforms of the rst the pth column ofthe design matrix X We impose the following technical conditions

Conditions

A1 There exists a positive de nite matrix A such that nƒ1xT x A

A2 There exists an integer n0 such that

8n=4log log n549ƒ1n=4log logn54X

iDn0

x uuml 2ij D O4151 j D 11 1 p1

where x uumlij is the j th element of the vector x uuml

i A3 Olsquo 2

1 Dlsquo 2 COP 4nƒ1=25 and Olsquo 22 D 2lsquo 4 COP 84logn5ƒ19

A4 1n

PniD1 m4xi5xi b for some vector b

Condition A1 is a standard condition for the least-squares estima-tor Osbquo to be root-n consistent It holds almost surely for the designsthat are generated from a random sample of a population with a nitesecond moment By Parsevalrsquos identity

1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150

Thus Condition A2 is a very mild condition It holds almost surelyfor the designs that are generated from a random sample ConditionA4 implies that under the alternative hypothesis the least-squaresestimator Osbquo converges in mean square error to sbquo0 D Aƒ1b

Proof of Theorem 1

Let acirc be the n n orthonormal matrix generated by the discreteFourier transform Denote

X uuml D acircX and Y D 4Y11 1 Yn5T 0

Under the null hypothesis our model can be written as

Y D Xsbquo C˜0 (A1)

Then the least-squares estimate is given by Osbquo D 4XT X5ƒ1XT YDenote z D acirc˜ and o D X uuml 4 Osbquo ƒsbquo5 Then

O uuml D acirc˜ ƒacircX4 Osbquo ƒsbquo5 D z ƒ o and z N 401lsquo 2In50

Let zi and oi be the ith components of the vectors z and o respec-tively Then

mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)

We rst evaluate the small order term o2i By the CauchyndashSchwarz

inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92

micro ˜ Osbquo ƒsbquo˜2pX

jD1

x uuml 2ij 1 (A3)


Figure 3 Power of the Adaptive Neyman Tests the KH Test the Wavelet Thresholding Test and the Parametric F Test for Examples 4 and 5When the Alternative Is Fully Nonparametric and n D 128 Key A the adaptive Neyman test (ANT) ordered according to X2 and using the knownvariance B the ANT ordered according to si and using the known variance C the ANT test ordered by X2 and using Olsquo ` D the ANT test orderedby s i and using Olsquo ` F the parametric F-test statistic H the wavelet thresholding test ordered by X2 and using the known variance S the KH testwith known variance and ordered by X2 T the KH test with estimated variance Olsquo ` and ordered by X2 U the KH test with known variance andordered by rsquo rst PC V the KH test with estimated variance Olsquo ` and ordered by the rsquo rst PC


where Osbquoj and sbquoj are the jth components of the vectors Osbquo and sbquorespectively By the linear model (A1) and Condition A1 we have

E4 Osbquo ƒsbquo54 Osbquo ƒ sbquo5T Dlsquo 24XT X5ƒ1 D nƒ1Aƒ18lsquo 2 C o41590

This and Condition A2 together yield

mX

iDn0

o2i micro ˜ Osbquo ƒ sbquo˜2

pX

jD1

n=4log log n54X

iDn0

x uuml 2ij D OP 84log log n5ƒ490 (A4)

We now deal with the second term in (A2) Observe that

mƒ1mX

iD1

z2i microlsquo 2 C max

1micrommicronmƒ1

mX

iD1

4z2i ƒlsquo 25 D oP 4log log n50 (A5)

The last equality in (A5) follows from (A8) By the CauchyndashSchwarz inequality (A4) and (A5) we have

mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2

D oP 84log log n5ƒ3=290

This together with (A2) and (A4) entails

mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0

z2i CoP 84log log n5ƒ3=290 (A6)

Let

T uumln D max

1micrommicron42mlsquo 45ƒ1=2

mX

iD1

4z2i ƒlsquo 250

Then by theorem 1 of Darling and Erdoumls (1956) we have

P6p

2 log log nT uumln ƒ 82 log logn C 05 log log log n

ƒ 05 log44 59 micro x7 exp4ƒ exp4ƒx55 (A7)

Hence

T uumln D 82 log log n91=281 CoP 4159 (A8)

andT uuml

log n D 82 log log logn91=281 CoP 41590

This implies that the maximum of T uumln cannot be achieved at m lt

log n Thus by (21)

T uumlAN1 1 D max

1micrommicron=4log log n54

1p

2mlsquo 4

mX

iD1

4 O uuml 2i ƒlsquo 25 Cop 4log log n5ƒ3=2 0

Combination of this and (A6) entails

T uumlAN1 1 D T uuml

n COp84log log n5ƒ3=290

The conclusion for TAN1 1 follows from (A7) and the conclusion forTAN1 2 follows from the same arguments

Proof of Theorem 2

Let m uuml0 be the index such that

Dagger uumln D max

1micrommicron42lsquo 4m5ƒ1=2

mX

iD1

Dagger uuml 2i

is achieved Denote c D ƒ log8ƒ log41 ƒ59Using arguments similar to but more tedious than those in the

proof of Theorem 1 we can show that under the alternative hypoth-esis

T uumlAN1 j D T uuml

AN Cop84log log n5ƒ3=2 C 4log log n5ƒ3=24Dagger uumln51=291

where z in T uumlAN is distributed as N4Dagger uuml 1lsquo 2In5 Therefore the power

can be expressed as

P8TAN1 j gt c9 D P6T uumlAN gt 4log log n51=2

81 Co4159Co84Daggeruumln51=2970 (A9)

Then by (A9) we have

P8TAN1 j gt c9

para P 42m uuml0lsquo

45ƒ1=2m uuml

0X

iD1

4z2i ƒlsquo 25 gt 24log log n51=2


45ƒ1=2m uuml

0X

iD1

4z2i ƒ Dagger uuml 2

i ƒlsquo 25

gt 24log log n51=2 ƒ Dagger uumln 0 (A10)

The sequence of random variables

2mlsquo 4 C4lsquo 2mX

iD1

Dagger2i

ƒ1=2 mX

iD1

4z2i ƒDagger uuml 2

i ƒlsquo 25

is tight because they have the mean zero and standard deviation 1 Itfollows from (A10) and the assumption (28) that the power of thecritical regions in (25) tends to 1

[Received April 1999 Revised June 2000]

REFERENCES

Aerts M Claeskens G and Hart J D (1999) ldquoTesting Lack of Fit inMultiple Regressionrdquo Journal of the American Statistical Association 94869ndash879

Azzalini A and Bowman A N (1993) ldquoOn the Use of NonparametricRegression for Checking Linear Relationshipsrdquo Journal of the Royal Sta-tistical Society Ser B 55 549ndash557

Azzalini A Bowman A N and Haumlrdle W (1989) ldquoOn the Use of Non-parametric Regression for Model Checkingrdquo Biometrika 76 1ndash11

Barnett V (1976) ldquoThe Ordering of Multivariate Datardquo (with Discussion)Journal of the Royal Statistical Society Ser A 139 318ndash355

Bickel P J and Ritov Y (1992) ldquoTesting for Goodness of Fit ANew Approachrdquo in Nonparametric Statistics and Related Topics edA K Md E Saleh New York North-Holland pp 51ndash57

Bowman A W and Azzalini A (1997) Applied Smoothing Techniques forData Analysis London Oxford University Press

Cuzick J (1992) ldquoSemiparametric Additive Regressionrdquo Journal of theRoyal Statistical Society Ser B 54 831ndash843

Darling D A and Erdos P (1956) ldquoA Limit Theorem for the Maximum ofNormalized Sums of Independen t Random Variablesrdquo Duke MathematicalJournal 23 143ndash155

Daubechies I (1992) Ten Lectures on Wavelets Philadelphia SIAMDette H (1999) ldquoA Consistent Test for the Functional Form of a Regression

Based on a Difference of Variance Estimatorsrdquo The Annals of Statistics27 1012ndash1040

Dette H and Munk A (1998) ldquoValidation of Linear Regression ModelsrdquoThe Annals of Statistics 26 778ndash800

Eubank R L and Hart J D (1992) ldquoTesting Goodness-of-Fit in Regressionvia Order Selection Criteriardquo The Annals of Statistics 20 1412ndash1425

Eubank R L and LaRiccia V M (1992) ldquoAsymptotic Comparison ofCrameacuterndashvon Mises and Nonparametric Function Estimation Techniques forTesting Goodness-of-Fitrdquo The Annals of Statistics 20 2071ndash2086

Fan J (1996) ldquoTest of Signi cance Based on Wavelet Thresholding andNeymanrsquos Truncationrdquo Journal of the American Statistical Association 91674ndash688

(1997) Comments on ldquoPolynomial Splines and Their Tensor Productsin Extended Linear Modelingrdquo The Annals of Statistics 25 1425ndash1432

Fan J and Lin S K (1998) Test of Signi cance When Data Are CurvesrdquoJournal of the American Statistical Association 93 1007ndash1021


Gnanadesikan R and Kettenring J R (1972) ldquoRobust Estimates Residualsand Outlier Detection with Multiresponse Datardquo Biometrics 28 81ndash124

Green P J and Silverman B W (1994) Nonparametric Regression andGeneralized Linear Models A Roughness Penalty Approach LondonChapman and Hall

Haumlrdle W and Mammen E (1993) ldquoComparing Nonparametri c VersusParametric Regression Fitsrdquo The Annals of Statistics 21 1926ndash1947

Hall P Kay J W and Titterington D M (1990) ldquoAsymptotically OptimalDifference-Based Estimation of Variance in Nonparametric RegressionrdquoBiometrika 77 521ndash528

Hart J D (1997) Nonparametri c Smoothing and Lack-of-Fit TestsNew York Springer-Verlag

Inglot T Kallenberg W C M and Ledwina T (1994) ldquoPower Approxima-tions to and Power Comparison of Smooth Goodness-of-Fi t Testsrdquo Scan-dinavian Journal of Statistics 21 131ndash145

Inglot T and Ledwina T (1996) ldquoAsymptotic Optimality of Data-Driven Neyman Tests for Uniformityrdquo The Annals of Statistics 241982ndash2019

Jolliffe I T (1986) Principal Component Analysis New York Springer-Verlag

Kim J (2000) ldquoAn Order Selection Criteria for Testing Goodness of FitrdquoJournal of the American Statistical Association 95 829ndash835

Kuchibhatla M and Hart J D (1996) ldquoSmoothing-Based Lack-of-Fit TestsVariations on a Themerdquo Journal of Nonparametric Statistics 7 1ndash22

Lee G and Hart J D (1998) ldquoAn L2 Error Test with Order Selection andThresholdingrdquo Statistics and Probability Letters 39 61ndash72

Ledwina T (1994) ldquoData-Driven Version of Neymanrsquos Smooth Test of FitrdquoJournal of the American Statistical Association 89 1000-1005

Neyman J (1937) ldquoSmooth Test for Goodness of Fitrdquo SkandinaviskAktuarietidskrift 20 149ndash199

Rayner J C W and Best D J (1989) Smooth Tests of Goodness of FitLondon Oxford University Press

Speckman P (1988) ldquoKernel Smoothing in Partial Linear Modelsrdquo Journalof the Royal Statistical Society Ser B 50 413ndash436

Spokoiny V G (1996) ldquoAdaptive Hypothesis Testing Using Waveletsrdquo TheAnnals of Statistics 24 2477ndash2498

Stute W Gonzaacutelez Manteiga W and Presedo Quindimil M (1998) ldquoBoot-strap Approximations in Model Checks for Regressionrdquo Journal of theAmerican Statistical Association 93 141ndash149

Wahba G (1984) ldquoPartial Spline Models for the Semiparametric Estimationof Functions of Several Variablesrdquo in Analyses for Time Series TokyoInstitute of Statistical Mathematics pp 319ndash329

Yatchew A (1997) ldquoAn Elementary Estimator of the Partial Linear ModelrdquoEconomics Letters 57 135ndash143


of O are asymptotically independent and normally distributedwith mean being a subvector of Dagger Thus the problem becomes

H0 2 Dagger D 0 versus H1 2 Dagger 6D 01 (15)

based on the observations O Note that Oi will not be inde-pendent in general after estimating ˆ but the dependence ispractically negligible as will be demonstrated Thus the tech-niques for independent samples such as the adaptive Neymantest in Fan (1996) continue to apply Plotting a covariate Xj

against the residual O aims at testing if the biases in O alongthe direction Xj are negligible namely whether the scatter plot84Xj1Daggerj59 is negligible in the presence of the noise

In testing the signi cance of the high-dimensional prob-lem (15) the conventional likelihood ratio statistic is not pow-erful due to noise accumulation in the n-dimensional spaceas demonstrated in Fan (1996) An innovative idea proposedby Neyman (1937) is to test only the rst m-dimensional sub-problem if there is prior that most of nonzero elements lieon the rst m dimension To obtain such a qualitative priora Fourier transform is applied to the residuals in an attemptto compress nonzero signals into lower frequencies In test-ing goodness of t for a distribution function Fan (1996)proposed a simple data-driven approach to choose m basedon power considerations This results in an adaptive Neymantest See Rayner and Best (1989) for more discussions onthe Neyman test Some other recent work motivated by theNeyman test includes Eubank and Hart (1992) Eubank andLaRiccia (1992) Kim (2000) Inglot Kallenberg and Ledwina(1994) Ledwina (1994) Kuchibhatla and Hart (1996) andLee and Hart (1998) among others Also see Bickel andRitov (1992) and Inglot and Ledwina (1996) for illuminatinginsights into nonparametric tests In this article we adopt theadaptive Neyman test proposed by Fan (1996) to the testingproblem (15) It is worth noting that the test statistic stud-ied by Kuchibhatla and Hart (1996) looks similar to that ofFan (1996) but they are decidedly different See the secondparagraph of Section 3 for further discussions The adaptiveNeyman test of Kuchibhatla and Hart (1996) also will beimplemented to demonstrate the versatility of our proposedidea We would note that the adaptive Neyman tests in Fan(1996) and Kuchibhatla and Hart (1996) were proposed forindependent samples Herein the adaptive Neyman test of Fan(1996) will be justi ed to remain applicable for weakly depen-dent O in the current regression setting see Theorem 1

The power of the adaptive Neyman test depends on thesmoothness of 8Daggeri9

niD1 as a function of i Let us call this

function Dagger4cent5 namely Dagger4i=n5 D Daggeri The smoother the func-tion Dagger4cent5 the more signi cant are the Fourier coef cientson the low frequencies and hence the more powerful thetest will be see Theorems 2 and 3 When m4cent5 is com-pletely unknown such as in (13) there is no informationon how to make the function Dagger4cent5 smoother by ordering theresiduals Intuitively the closer the two consecutive covari-ate vectors xi and xiC1 the smaller the difference betweenm4xi5 ƒ f 4xi1 ˆ05 and m4xiC15 ƒ f4xiC11 ˆ05 and hence thesmoother the sequence 8m4xi5ƒf4xi1 ˆ059 indexed by i Thusa good proxy is to order the residuals 8˜i9

niD1 in a way that the

corresponding xirsquos are close as a sequence Note that when

there is only one predictor variable (ie p D 1) such orderingis straightforward However it is quite challenging to order amultivariate vector A method based on the principal compo-nent analysis is described in Section 22

When there is some information about the alternativehypothesis such as the partially linear model in (14) it issensible to order the residuals according to the covariate x1Indeed our simulation studies show that it improves a greatdeal the power of the generic ordering procedure outlined inthe last paragraph improved a great deal The parameters sbquo2

in model (14) can be estimated easily at the nƒ1=2 rate viaa difference based estimator for example that of Yatchew(1997) Then the problem is heuristically reduced to the one-dimensional nonparametric setting see Section 24 for details

Wavelet transform is another popular family of orthogo-nal transforms It can be applied to the testing problem (15)For comparison purposes wavelet thresholding tests are alsoincluded in our simulation studies See Fan (1996) andSpokoiny (1996) for various discussions on the properties ofthe wavelet thresholding tests

This article is organized as follows In Section 2 we pro-pose a few new tests for the testing problems (12) ver-sus (13) and (12) versus (14) These tests include theadaptive Neyman test and the wavelet thresholding tests inSections 21 and 23 respectively Ordering of multivari-ate vectors is discussed in Section 22 Applications of thetest statistics in the partially linear models are discussed inSection 24 and their extension to additive models is outlinedbrie y in Section 25 Section 26 outlines simple methods forestimating the residual variance lsquo 2 in the high-dimensionalsetting In Section 3 we carry out a number of numerical stud-ies to illustrate the power of our testing procedures Technicalproofs are relegated to the Appendix

2 METHODS AND RESULTS

As mentioned in the Introduction the hypothesis testingproblem (12) can be reduced to the high-dimensional prob-lem (15) based on the weakly dependent Gaussian randomvector O The adaptive Neyman test and the wavelet thresh-olding test proposed in Fan (1996) are adapted for the cur-rent regression setting The case of a multiple linear modelf 4x1 ˆ5 D xT ˆ is addressed here and its extension to a generalparametric model can be treated similarly

21 The Adaptive Neyman Test

Assume a parametric linear model under the null hypothe-sis

Y D xT ˆ C ˜1 ˜ N 401lsquo 251

and assume O is the resulting residual vector from a least-squares t Let O uuml D 4 O uuml

11 1 O uumln5T be the discrete Fourier

transform of the residual vector O More precisely we de ne

O uuml2jƒ1 D 42=n51=2

nX

iD1

cos42 ij=n5 Oi1

O uuml2j D 42=n51=2

nXiD1

sin42 ij=n5 Oi1 j D 11 1 6n=270



pn=25

PniD1 Oi is



T uumlAN1 1 D max

1micrommicron

1p2m Olsquo 4

1

mX

iD1


1 1 (21)





T uumlAN12 D max

1micrommicron

1p

m Olsquo 22

mXiD1


1 0 (22)



TAN1 j Dp



ƒ 005 log44 59 for j D 1120 (23)













mƒ1=2mX

iD1




mƒ1=2mX

iD1


nX

iD1

Dagger2i ƒ

nX

iDmC1



nƒ1nX

iDmC1



nƒ1nX

iD1

Dagger2i

4sC1 1=44s5

0


nXiD1

Dagger2i D

nXiD1

8m4xi5 ƒ f 4xi1 ˆ0592

nZ

8m4x5 ƒ f 4x1 ˆ0592h4x5dx0








nn 10 20 30 40 60 80 100 120 140 160 180 200

001 5078 6007 6018 6022 6032 6037 6041 6041 6042 6042 6047 60430025 4057 4075 4082 4087 4091 4093 4095 4095 4095 4095 4095 4095005 3067 3077 3083 3085 3088 3089 3090 3090 3090 3091 3090 3089010 2074 2078 2081 2082 2085 2085 2086 2087 2086 2087 2087 2086



niD1 as smooth









si D1

n ƒ 1

pXkD1

lsaquokz2i1 k0








TH DJ0X

iD1

O2W 1 i C

nX

iDJ0C1





where ŒH D J0 Cp


H D 2J0 Cp2= aƒ1










i1 2Osbquo2





C OP 4nƒ151 i D 11 1 nƒ 11 (29)














i 1 i D In C 11 1 n9

Olsquo 21 D

1nƒ In

nX

iDInC1

O uuml 2i ƒ

1n ƒ In

nX

iDInC1

O uumli

2

0 (210)



i 1 i D n=4 C 11 1 n9

Olsquo 22 D

1

n ƒ In

nX

iDInC1

O uuml 4i ƒ

1

nƒ In

nX

iDInC1

O uuml 2i

2

0 (211)






ˆ0 C4X

iD1

ˆiXi C3X

iD1

4XjD1





3 SIMULATIONS





1m

mXjD1

2n O uuml 2i

lsquo 21







2


Y D 1 C ˆX21 C ˜1 ˜ N 401151 (31)



Y D 1 C cos4ˆX1 5 C ˜1 ˜ N 401150 (32)



Y D10

100C ˆ exp4ƒ2X15C ˜1 ˜ N 401150 (33)





Figure 1 Continued





P4X4 D 15 D 04 and P4X4 D 05 D 060


sbquo0 C4X

iD1

sbquoiXi CX

1microimicrojmicro3

sbquoi1jXiXj 0



Y D X1 C ˆX22 C 2X4 C ˜1 ˜ N 401151 (34)



















(c D 2038)





APPENDIX






Conditions




iDn0



i A3 Olsquo 2


A4 1n



1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150


Proof of Theorem 1








mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)


inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92


jD1

x uuml 2ij 1 (A3)







mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES







































pn=25

PniD1 Oi is



T uumlAN1 1 D max

1micrommicron

1p2m Olsquo 4

1

mX

iD1


1 1 (21)





T uumlAN12 D max

1micrommicron

1p

m Olsquo 22

mXiD1


1 0 (22)



TAN1 j Dp



ƒ 005 log44 59 for j D 1120 (23)













mƒ1=2mX

iD1




mƒ1=2mX

iD1


nX

iD1

Dagger2i ƒ

nX

iDmC1



nƒ1nX

iDmC1



nƒ1nX

iD1

Dagger2i

4sC1 1=44s5

0


nXiD1

Dagger2i D

nXiD1

8m4xi5 ƒ f 4xi1 ˆ0592

nZ

8m4x5 ƒ f 4x1 ˆ0592h4x5dx0








nn 10 20 30 40 60 80 100 120 140 160 180 200

001 5078 6007 6018 6022 6032 6037 6041 6041 6042 6042 6047 60430025 4057 4075 4082 4087 4091 4093 4095 4095 4095 4095 4095 4095005 3067 3077 3083 3085 3088 3089 3090 3090 3090 3091 3090 3089010 2074 2078 2081 2082 2085 2085 2086 2087 2086 2087 2087 2086



niD1 as smooth









si D1

n ƒ 1

pXkD1

lsaquokz2i1 k0








TH DJ0X

iD1

O2W 1 i C

nX

iDJ0C1





where ŒH D J0 Cp


H D 2J0 Cp2= aƒ1










i1 2Osbquo2





C OP 4nƒ151 i D 11 1 nƒ 11 (29)














i 1 i D In C 11 1 n9

Olsquo 21 D

1nƒ In

nX

iDInC1

O uuml 2i ƒ

1n ƒ In

nX

iDInC1

O uumli

2

0 (210)



i 1 i D n=4 C 11 1 n9

Olsquo 22 D

1

n ƒ In

nX

iDInC1

O uuml 4i ƒ

1

nƒ In

nX

iDInC1

O uuml 2i

2

0 (211)






ˆ0 C4X

iD1

ˆiXi C3X

iD1

4XjD1





3 SIMULATIONS





1m

mXjD1

2n O uuml 2i

lsquo 21







2


Y D 1 C ˆX21 C ˜1 ˜ N 401151 (31)



Y D 1 C cos4ˆX1 5 C ˜1 ˜ N 401150 (32)



Y D10

100C ˆ exp4ƒ2X15C ˜1 ˜ N 401150 (33)





Figure 1 Continued





P4X4 D 15 D 04 and P4X4 D 05 D 060


sbquo0 C4X

iD1

sbquoiXi CX

1microimicrojmicro3

sbquoi1jXiXj 0



Y D X1 C ˆX22 C 2X4 C ˜1 ˜ N 401151 (34)



















(c D 2038)





APPENDIX






Conditions




iDn0



i A3 Olsquo 2


A4 1n



1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150


Proof of Theorem 1








mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)


inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92


jD1

x uuml 2ij 1 (A3)







mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES







































nn 10 20 30 40 60 80 100 120 140 160 180 200

001 5078 6007 6018 6022 6032 6037 6041 6041 6042 6042 6047 60430025 4057 4075 4082 4087 4091 4093 4095 4095 4095 4095 4095 4095005 3067 3077 3083 3085 3088 3089 3090 3090 3090 3091 3090 3089010 2074 2078 2081 2082 2085 2085 2086 2087 2086 2087 2087 2086



niD1 as smooth









si D1

n ƒ 1

pXkD1

lsaquokz2i1 k0








TH DJ0X

iD1

O2W 1 i C

nX

iDJ0C1





where ŒH D J0 Cp


H D 2J0 Cp2= aƒ1










i1 2Osbquo2





C OP 4nƒ151 i D 11 1 nƒ 11 (29)














i 1 i D In C 11 1 n9

Olsquo 21 D

1nƒ In

nX

iDInC1

O uuml 2i ƒ

1n ƒ In

nX

iDInC1

O uumli

2

0 (210)



i 1 i D n=4 C 11 1 n9

Olsquo 22 D

1

n ƒ In

nX

iDInC1

O uuml 4i ƒ

1

nƒ In

nX

iDInC1

O uuml 2i

2

0 (211)






ˆ0 C4X

iD1

ˆiXi C3X

iD1

4XjD1





3 SIMULATIONS





1m

mXjD1

2n O uuml 2i

lsquo 21







2


Y D 1 C ˆX21 C ˜1 ˜ N 401151 (31)



Y D 1 C cos4ˆX1 5 C ˜1 ˜ N 401150 (32)



Y D10

100C ˆ exp4ƒ2X15C ˜1 ˜ N 401150 (33)





Figure 1 Continued





P4X4 D 15 D 04 and P4X4 D 05 D 060


sbquo0 C4X

iD1

sbquoiXi CX

1microimicrojmicro3

sbquoi1jXiXj 0



Y D X1 C ˆX22 C 2X4 C ˜1 ˜ N 401151 (34)



















(c D 2038)





APPENDIX






Conditions




iDn0



i A3 Olsquo 2


A4 1n



1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150


Proof of Theorem 1








mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)


inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92


jD1

x uuml 2ij 1 (A3)







mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES








































i1 2Osbquo2





C OP 4nƒ151 i D 11 1 nƒ 11 (29)














i 1 i D In C 11 1 n9

Olsquo 21 D

1nƒ In

nX

iDInC1

O uuml 2i ƒ

1n ƒ In

nX

iDInC1

O uumli

2

0 (210)



i 1 i D n=4 C 11 1 n9

Olsquo 22 D

1

n ƒ In

nX

iDInC1

O uuml 4i ƒ

1

nƒ In

nX

iDInC1

O uuml 2i

2

0 (211)






ˆ0 C4X

iD1

ˆiXi C3X

iD1

4XjD1





3 SIMULATIONS





1m

mXjD1

2n O uuml 2i

lsquo 21







2


Y D 1 C ˆX21 C ˜1 ˜ N 401151 (31)



Y D 1 C cos4ˆX1 5 C ˜1 ˜ N 401150 (32)



Y D10

100C ˆ exp4ƒ2X15C ˜1 ˜ N 401150 (33)





Figure 1 Continued





P4X4 D 15 D 04 and P4X4 D 05 D 060


sbquo0 C4X

iD1

sbquoiXi CX

1microimicrojmicro3

sbquoi1jXiXj 0



Y D X1 C ˆX22 C 2X4 C ˜1 ˜ N 401151 (34)



















(c D 2038)





APPENDIX






Conditions




iDn0



i A3 Olsquo 2


A4 1n



1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150


Proof of Theorem 1








mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)


inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92


jD1

x uuml 2ij 1 (A3)







mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES








































ˆ0 C4X

iD1

ˆiXi C3X

iD1

4XjD1





3 SIMULATIONS





1m

mXjD1

2n O uuml 2i

lsquo 21







2


Y D 1 C ˆX21 C ˜1 ˜ N 401151 (31)



Y D 1 C cos4ˆX1 5 C ˜1 ˜ N 401150 (32)



Y D10

100C ˆ exp4ƒ2X15C ˜1 ˜ N 401150 (33)





Figure 1 Continued





P4X4 D 15 D 04 and P4X4 D 05 D 060


sbquo0 C4X

iD1

sbquoiXi CX

1microimicrojmicro3

sbquoi1jXiXj 0



Y D X1 C ˆX22 C 2X4 C ˜1 ˜ N 401151 (34)



















(c D 2038)





APPENDIX






Conditions




iDn0



i A3 Olsquo 2


A4 1n



1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150


Proof of Theorem 1








mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)


inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92


jD1

x uuml 2ij 1 (A3)







mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES








































Figure 1 Continued





P4X4 D 15 D 04 and P4X4 D 05 D 060


sbquo0 C4X

iD1

sbquoiXi CX

1microimicrojmicro3

sbquoi1jXiXj 0



Y D X1 C ˆX22 C 2X4 C ˜1 ˜ N 401151 (34)



















(c D 2038)





APPENDIX






Conditions




iDn0



i A3 Olsquo 2


A4 1n



1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150


Proof of Theorem 1








mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)


inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92


jD1

x uuml 2ij 1 (A3)







mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES
















































(c D 2038)





APPENDIX






Conditions




iDn0



i A3 Olsquo 2


A4 1n



1

n

nX

iD1

x uuml 2ij D

1

n

nX

iD1

x2ij D O4150


Proof of Theorem 1








mX

iD1

O uuml 2i D

mX

iD1

4z2i ƒ2zioi Co2

i 50 (A2)


inequality

o2i D

pX

jD1

x uumlij8

Osbquoj ƒsbquoj92


jD1

x uuml 2ij 1 (A3)







mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES











































mX

iDn0


pX

jD1

n=4log log n54X

iDn0



mƒ1mX

iD1


1micrommicronmƒ1

mX

iD1



mƒ1=2mX

iDn0

oizi micromX

iDn0

o2i

1=2

mƒ1mX

iDn0

z2i

1=2



mƒ1=2mX

iDn0

O2uumli D mƒ1=2

mX

iDn0


Let

T uumln D max


mX

iD1

4z2i ƒlsquo 250


P6p



Hence


andT uuml



log n Thus by (21)

T uumlAN1 1 D max


1p

2mlsquo 4

mX

iD1






Proof of Theorem 2


Dagger uumln D max


mX

iD1

Dagger uuml 2i






can be expressed as




P8TAN1 j gt c9


45ƒ1=2m uuml

0X

iD1



45ƒ1=2m uuml

0X

iD1


i ƒlsquo 25




iD1

Dagger2i

ƒ1=2 mX

iD1


i ƒlsquo 25



REFERENCES





































goodness-of-fit tests for parametric regression...

Documents