nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric...

14
Ann. Inst. Statist. Math. Vol. 56, No. 1, 73-86 (2004) @2004 The Institute of Statistical Mathematics NONPARAMETRIC REGRESSION UNDER DEPENDENT ERRORS WITH INFINITE VARIANCE LIANG PENG 1 AND QIWEI YAO 2 1School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160, U.S.A., e-maih [email protected] 2Department of Statistics, London School of Economics, London, WC2A 2AE, U.K., e-marl: [email protected] (Received May 7, 2002; revised March 31, 2003) Abstract. We consider local least absolute deviation (LLAD) estimation for trend functions of time series with heavy tails which are characterised via a symmetric stable law distribution. The setting includes both causal stable ARMA model and fractional stable ARIMA model as special cases. The asymptotic limit of the es- timator is established under the assumption that the process has either short or long memory autocorrelation. For a short memory process, the estimator admits the same convergence rate as if the process has the finite variance. The optimal rate of convergence n -2/5 is obtainable by using appropriate bandwidths. This is distinctly different from local least squares estimation, of which the convergence is slowed down due to the existence of heavy tails. On the other hand, the rate of convergence of the LLAD estimator for a long memory process is always slower than n -2/5 and the limit is no longer normal. Key words and phrases: ARMA, fractional ARIMA, heavy tail, least absolute devi- ation estimation, long memory, median, stable distribution, time series. i. Introduction and models A substantial literature now exists on using kernel-type smoothers to estimate a smooth trend in time series data. These methods are important, in part, because they allow estimation of a smooth trend without prior specification of the form of the trend. A popular setting which has attracted much attention in the last decade is a fixed-design regression with dependent 'errors', under which the observations Y1,... ,Yn follow the model (1.1) Yt = m(t/n) + ct, t = 1,...n, where m(.) is a smooth function defined on [0, 1], and {st} is a stationary process such as ARMA time series. If {et} are correlated but with only short-range dependence in the sense that its autocorrelation functions are absolutely summable, it has been proved that nonparametric regression estimators for m(.) are asymptotically normal at the same convergence rate as in the case of uncorrelated {st }, although the asymptotic variances have one more factor due to the dependence in the data; see Hall and Hart (1990). When there exists a long-range dependence, Hall and Hart (1990) shows that the estimators have a slower convergence rate for the long-range dependent normal errors; see also 73

Upload: others

Post on 25-Feb-2021

9 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

Ann. Inst. Statist. Math. Vol. 56, No. 1, 73-86 (2004) @2004 The Institute of Statistical Mathematics

NONPARAMETRIC REGRESSION UNDER DEPENDENT ERRORS WITH INFINITE VARIANCE

LIANG PENG 1 AND QIWEI YAO 2

1School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332-0160, U.S.A., e-maih [email protected]

2Department of Statistics, London School of Economics, London, WC2A 2AE, U.K., e-marl: [email protected]

(Received May 7, 2002; revised March 31, 2003)

A b s t r a c t . We consider local least absolute devia t ion (LLAD) es t ima t ion for t r end functions of t ime series wi th heavy tai ls which are character ised via a symmet r i c s table law dis t r ibut ion. The se t t ing includes bo th causal s table A R M A model and fract ional s table A R I M A model as special cases. The asympto t i c l imit of the es- t ima to r is es tabl ished under the assumpt ion t ha t the process has e i ther shor t or long memory autocorre la t ion. For a shor t memory process, the e s t ima to r admi t s the same convergence ra te as if the process has the finite variance. The op t ima l ra te of convergence n -2/5 is ob ta inab le by using appropr ia t e bandwidths . This is d i s t inc t ly different from local least squares es t imat ion, of which the convergence is slowed down due to the existence of heavy tails. On the other hand, the ra te of convergence of the LLAD es t imator for a long memory process is always slower t han n -2/5 and the l imit is no longer normal .

Key words and phrases: A R M A , fract ional ARIMA, heavy tail , least absolu te devi- a t ion es t imat ion , long memory, median, s table d is t r ibut ion, t ime series.

i . Introduction and models

A substantial literature now exists on using kernel-type smoothers to estimate a smooth trend in time series data. These methods are important, in part, because they allow estimation of a smooth trend without prior specification of the form of the trend. A popular setting which has attracted much attention in the last decade is a fixed-design regression with dependent 'errors', under which the observations Y1,... ,Yn follow the model

(1.1) Yt = m ( t / n ) + ct, t = 1 , . . . n ,

where m(.) is a smooth function defined on [0, 1], and {st} is a stationary process such as ARMA time series. If {et} are correlated but with only short-range dependence in the sense that its autocorrelation functions are absolutely summable, it has been proved that nonparametric regression estimators for m(.) are asymptotically normal at the same convergence rate as in the case of uncorrelated {st }, although the asymptotic variances have one more factor due to the dependence in the data; see Hall and Hart (1990). When there exists a long-range dependence, Hall and Hart (1990) shows that the estimators have a slower convergence rate for the long-range dependent normal errors; see also

73

Page 2: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

74 LIANG PENG AND QIWEI YAO

CsSrg5 and Mielniczuk (1995). Since the mean squared errors of the estimators axe different from those with independent data, research has been carried out to modify the standard kernel regression techniques, including the bandwidth selection procedures, to incorporate various dependence structures. This includes Altman (1990), Chiu and Marron (1991), Truong (1991), Hart (1991, 1994), Herrmann et al. (1992), Roussas et al. (1992), Tran et al. (1996) for short-range dependence data, and Ray and Tsay (1997) and Robinson (1994, 1997) for long-range dependence data. A common practice in all the aforementioned literature is to assume that st has the zero-mean and a finite variance.

In this paper, we also deal with the kernel regression estimation for function m(.) but with heavy tailed error terms such that E(st 2) = co or El~t I = oo. More specifically, we assume that in model (1.1) st is a linear process defined as

(1.2) st = ~ cjZt-j, j=o

where {Zt} are independent random variables sharing the same standard symmetric stable law distribution with index a E (0, 2), i.e. the characteristic function Zt has the form

E(e "z') ; e x p { - I t l

Under the condition

o o

(1.3) 0 < E IcJ]'~ < 0% j=0

the infinite sum in (1.2) is well-defined. Moreover we may say that {r has short o o memory or long memory according as Y~.j=o IcJl a/2 < oo or = oo respectively. Note

that Eletl = oo when a < 1 and E(s 2) = oc when a < 2, and m(t /n ) is the median (as well as mean when a > 1) of Yr. The setting (1.2)-(1.3) includes the causal stable ARMA model (Mikosch et al. (1995), and Klfippelberg and Mikosch (1996)) and the causal fractional stable ARIMA model (Kokoszka and Taqqu (1995, 1996)) as special cases. A causal infinite variance fractional ARIMA(p, d, q) time series may be defined as

(1.4) �9 (B)et = @(B)(1 - B ) -dz t ,

where d E (0, 1) is a se l f - s imi lar i ty parameter, B denotes the backshift operator, q'(.) and O(.) are polynomials with degrees p and q respectively, and all the roots of the equation O(z) = 0 are outside the unit circle. Because of the presence of d, the process {st} has infinite variance as well as long-range dependence. It in fact admits the MA(oo)-

o o representation (1.2) with Y~.j=o IcJ j~/2 = co. For further information on ARMA models with heavy tails and their applications, we refer to Adler et al. (1997) and Resnick (1997).

It is known that for regression models with heavy tailed noises, the conventional least squares estimators typically have slow convergence rates, are only consistent when the tail index a C (1, 2), and even then the limiting distribution is non-normal; see Davis etal. (1992) and references within for parametric regression, and Hall et al. (2002) for nonpaxametric regression. We consider in this paper the local linear least absolute deviations estimator for m(.). The asymptotic limit of the estimator is established for both short and long memory cases (Theorem 2.1 in Section 2 below). The limit is normal in case of short memory, however it is a stable law in case of long memory. The proof

Page 3: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

NONPARAMETRIC REGRESSION WITH INFINITE VARIANCES 75

is based on a combined use of the convex lamina (Pollard (1991)) and the asymptotic results for stable moving average processes of Hsing (1999), Koul and Surgailis (2001) and Surgailis (2002). When {ct} has short memory, the convergence rate as well as the first order asymptotic mean and variance are the same as if ct had a finite variance. We can reproduce the optimal rate of convergence n -2/5 by choosing the bandwidth of the order n -1/5, which is a folklore in conventional (one-dimensional) kernel regression. In this sense, the least absolute deviations estimation is adaptive to heavy tails. However when the process has a long memory, the convergence rate is always slower than n -~/5

and the limit is no longer normal. There is a substantial literature on nonparametric regression in the least absolute

deviation setting. Mallows (1980), Velleman (1980), Truong (1989) and Fan and Hall (1994) addressed local median smoothing for independent data, Tsybakov (1986) and Fan et al. (1994) developed robust methods for fitting local polynomials. In the time series context, Truong and Stone (1992) and Truong (1991, 1992a, 1992b) discussed robust nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional quantile estimation. All of them considered regression models with random designs and none of them addressed estimation with infinite-variance data. Hall et al. (2002) considered nonparametric least squares as well as least absolute deviations estimation for heavy tailed regressive models with random design under the assumption that the processes fulfill certain mixing conditions which rule out the possibility of long memory properties.

2. Estimators and main results

Let xt = t / n for t = 1 , . . . , n and x c (0, 1) fixed. The local linear least absolute deviations estimator is defined as rh(x) = &, where

n

(&,b) = argmin ~ [Yt - a - b(xt - x)IK (a,b) t=l

In the above expression, K(-) > 0 is a density function on R 1 and h > 0 is a bandwidth. We write ?Tt I (X) : D which is a n estimator for rh(x) _= d r n ( x ) .

In the sequel, we always assume that x E (0, 1) is fixed. Let p(.) denote the marginal density function of ~t, a~ = f u 2 K ( u ) d u , and D(~) = 1(4 > 0) - I(~ _< 0). We use C to denote some generic constant which may be different at different places.

(C1) For fixed x, rn(-) has second continuous derivative in a neighbourhood of x.

(C2) The kernel K is a symmetric, bounded and non-negative function with support [-1, 1]. Further I K ( Z l ) - K ( z 2 ) l < C l z x - z 2 1 for any zl, z2 E R 1.

(C3) h = h ( n ) ~ 0 and n h 3 ~ oo as n --~ oc.

THEOaEM 2.1. Let Condi t ions (C1) (C3) hold f o r the process def ined in (1.1)- (1.3).

I o~ (i) ~f E l = o Iczl < o~, then

Page 4: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

76 LIANG PENG AND QIWEI YAO

4 2 where ~(x) = ~ m ( x ) , ~,j = E L 0 cza_~, and

a 2 = lim E 1 D(c i , j )K .

The limit on the RHS of the above expression exists and is finite. (ii) Suppose c j / j - ' ---* bo as j ~ c~ for some ~ E (a -1, 1). Then

(nh)Z-1/~ { ~ ( x ) - m ( x ) - lh2a2ih(x)} d La,

where La is a stable law with characteristic function

ZeitL~ ~-exp{-]t]ab~/_-lcr ( / l_ lK(U)(U-V)- i3du)adv} .

(iii) Suppose cj/j -z ---* bo as j --+ oc for some Z C (1, 2 /a) . Let G denote the distribution function of ci. Then

(nh)l_]/(~) { ~ ( x ) _ m ( x ) _ l h 2 ~ h ( x ) } d c+ L+ c - L - p(O) ~ + p(o) ~ '

where

c+ = a* (/l_l K(S)dt) ( /o~(G~(+t) - G~(O))t-l-1/Zdt) ,

G~(x) = EG(x + Zi),

I I . = bg(a~- 1)

r(2 - Z , - - ~ - ap) cos --~---p

and L+~Z and L~Z are independent copies of a stable law L~ z with characteristic function

EeitL~ = exp {--I t l~ (1-- isgn(t)tan ~ ) } .

Remark. (i) If {~t} has the short range dependence, i.e. ~-~j=0 IcJl ~/~ < ~ , The- orem 2.10) indicates that the asymptotic distribution of the least absolute deviations estimator rh(.) is of the same form as if ~t had a finite variance. Note that the first order asymptotic approximation for the mean squared error of ~ ( x ) is

1 2 ~ h4ao4{~(x)} 2 + ~--~a .

Minimising this approximation over h, we obtain an optimum bandwidth of the order n -1/5, which is the same as for (one-dimensional) nonparametric regression estimation with finite variances. By using the optimum bandwidth, the estimator rh(x) converges at the rate 1/x/-n-h = 0(n-2/5).

(ii) In Theorem 2.1(ii) and (iii) the condition cj ~ boj -~ implies ~-~j=o IcJ I a/2 = cr The asymptotic stable law has been established for the subclass of long memory processes fulfilling this condition.

Page 5: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

NONPARAMETRIC REGRESSION WITH INFINITE VARIANCES 77

3. Proofs

In this section, we always assume the regularity conditions (C1)-(C3) hold. We introduce some notat ion first.

Let Yt* = Yt - r n ( x ) - ~ h ( x ) ( x t - x), zt = (1, x~-x~T Kt = K ( ~ - ~ ) , and h ] ' v/-n-h{rh(x) - re(x), h(rhi (x) - rh(x))} T. For 0 = (01,02) T, we define

r~

a(0) = ~{IYt* - OTzt/V~l- 15*l}Kt, t=l

or n 2 2 E ztD(Y**)Kt. R(o) = G(o) - p(O)(O~ + 0 ~ 0 ) + ~ - ~ ,:~

Obviously, 0" is the minimiser of G(O). We split the proof into several lemmas. Lemma 3.1 below follows easily from condition (1.3) and the proof of Lemma 3 of Hsing (1999).

LEMMA 3.1. The marginal density of et is positive and continuous at zero.

LEMMA 3.2. Let u and v be real numbers. For 0 < q < 1, lu + vIq <_ lu] q + Iv[ q. f ~ t h e r for I < q < ~ , ]u + vJq <_ 2q-l(Iutq + tvlq) and

IlU-~ Vl q --lUl q --Ivlql ~ lltl]Vl q-1 ~- [ulq--1]Vl.

PROOF. The first two inequalities follow from Lemma 2.7.13 of Samorodni tsky and Taqqu (1994). The last one follows from the inequalities

and

lU-}- Vl q ~ (l~l n a ]Vl)IU--~vlq -1 ~ ([ltl ~- IVI)(IUl q-1 -~- IV[ q - I )

lU-t-Vl q ~ ( l U l - IVl)(lUl q-1 --IVl q - l ) ~_ I lUl - IVll X [U-t-Vl q-1.

LEMMA 3.3. Let pt(x ,y) be the joint density of @l,~ t ) f o r t > 1. It holds that suPt>_2pt(O,O ) < 0(3.

PROOF. Note tha t the characteristic function of (C1, Cj+I) is

f ( t l , t2) - Ee i(tl~l+t2cj+l)

= E exp i tl C k Z l - k + t 2 k=0 k=0

= E exp i tick +t2ck+j)Zl_k

1

= exp - Itlck+t2c~+jl ~ - E = l=- j

+ i E t2cl+jZl-i l= - j

Page 6: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

78 LIANG PENG AND QIWEI YAO

We consider the case a E (0, 1] first. By the inverse formula and L e m m a 3.2, the densi ty funct ion pj+l (0, 0) is equal to

--1. { P ( e l < x, e j+l < x) - P ( e l < x, e j+l < O) lim x---*0 x ~

- P @ l < 0, c j + 1 < z ) J r - r @ l < 0, e j + l < 0 )}

l im 1 f f { _. x--*O 4 ~ X 2 e ~(tlx+t2x) _ e - i t l x _ e- i t2x + 1 } f ( t l , t 2 ) d t l d t 2 J , ] 1 } ff O([tl l)O([t2l)exp - ~-~[tlek + t2ck+3[ ~ - ~ [t2cl+jl a dtldt2

k=0 l = - j

< - / / O ( [ t t l ) O ( [ t 2 l ) e x p { - k=0~ [tl[~[ckl~

oo --1 / + E t ~ c 2 k+j - ~ [t2[~[cz+j[ ~ dhdt2

k=o t= - j

" f __ O(]t2[)exp -It2[ ~ [ck] ~ - _ _ Icl[ ~ dt2. J l=j

Note tha t (1.3) implies

j--1 (x~ oo

k=0 l=j k=0

for all large j ' s . Hence there exists j0 > 0 such tha t supj_>jo pj+l(O, 0) < co. W h e n a E (1, 2), the required inequali ty can be derived from

[tlek -}- t2Ck+j[ a > ] t l C k [ a - 2 [t2ck+~l~

in a similar manner and the above relat ion is implied by L e m m a 3.2. This completes the proof of L e m m a 3.3.

LEMMA 3.4. As n --* oc, E{R(O)} --* O.

PROOF. Let d t = O T z t / V ~ . Withou t loss of generality, we may assume tha t dt >_ O. T h e n

(3.1) {IYt* - O T z t / v ~ [ -- IYt*]}gt

= Kt[Yt*{I(Yt* > dr) - I(Yt* < dr) - I(Yt* > 0) + I(Yt* _< 0)}

+ d t { l (Y t* <_ dr) - I(Yt* > dt)}]

= -2KtYt* I ( 0 < Yt* <-. dr)

+ dtKt{X(Yt* <_ O) + I(0 < Yt* <- dr) - I(gt* > O) + I(0 < Yt* <- dr)}

= - d t K t D ( Y t * ) + 2Kt (d t - Yt*)I(0 < dt - Yt* < dr).

Page 7: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

NONPARAMETRIC REGRESSION WITH INFINITE VARIANCES

Note that (C2) implies dt ~ 0 as n ---+ oo. We have

E { ( d t - Yt*)I(0 _< dt - Yt* < dt)} = p(0)dt2{1 + o(1)}.

Combining the above equation with (3.1) we have

E{([Yt* - OT zt/v'~-~] -- ]Yt*] + d t D ( Y t * ) ) K t } = P(O)Ktd2t { 1 + o(1)}.

Thus

E G(O) + E d t D ( Y t * ) K t t = l

= p ( O ) E d 2 t K t { 1 +o(1)} t = l

2 2 ----+ p(O) (01 + 0 2 u ) 2 K ( u ) d u = p(0)(021 + 020"0),

c < )

since nh ---+ oo. This completes the proof of the lemma.

LEMMA 3.5. As n --* 0% R(O) converges to 0 in probability.

79

and

i = 1 i = 1

-- p2(O)- f(ol + 02u)4K2(u)du{1 + o(1)} --+ 0,

n- -1

Z ( n - - i ) IETITi+l l i = 1

n - - 1

= E ( n -- i)p/+l(0, O)Kld~Ki+ld~+l{1 + o(1)} i = 1

n 2 2 PROOF. Note that R(O) = ~-]~i=1 Tt -p(O)(O~ + 02a0) with

Tt = Kt[IYt* - OT zt/V'-~hl -- IY?l + OT z t D ( Y t * ) / v / - ~ ] �9

n T It follows from Lemma 3.4 that we only need to prove that ~ t = l ( t - E T t ) p O. Note that

P T i - E T ~ ) > e

n - 1 2

1 E(T i - ET i ) 2 + ~-~ Z ( n - i ) {E(T1T i+I ) - E T 1 E T i + I } . <-~ i = 1 i = 1

From (3.1) we have

~--~ET~= 4p(0)m {1 + o(i)} i = 1 i = 1

4 0 1 f = 5p( )~---s j (o l + 02~)aK2(u)du{1 + o(1)} --, 0,

n n

E ( E T / ) 2 = E p 2 ( O ) K ~ d ~ { 1 +0(1)}

Page 8: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

80 LIANG PENG AND QIWE1 YAO

n - 1

< supPJ+l(O'O)nKld2 E/(i+1d/2+1(1 -f- o(1)} j_>l i=1

< suppj+l(O,O)Ch-lK ( l / n - x ) / - - j> l h (01 + 02u)2K(u)du{1 + 0(1)}.

Note tha t by (C2) K((1/n - x - 1/(nh))/h) = 0

as n large enough. Hence by (C2) and (C3)

h _ l K ~l /n: - x~ _ ]K((1/n- x ) /h ) - K ( ( 1 / n - x - 1/(nh))/h)l < C/(nh3 ) 4 0 . \ h ] h

Thus by Lemma 3.3 n--1

E ( n - i)ET1Ti+I ~ O. i=1

By the same arguments as above we have

~( n -- i)ETIETi+I n--1

< n E IETIETi+I ] i=1 n--1

= n EpU(O)K, d2Ki+ld2+,{1 + o(1)} ~ 0. i=1

Hence the lemma is proven.

L E M M A 3 .6 . xfE =o Icily/= < then

lim E{D(el) - D(el,1)} 2 = 0.

PROOF. Let W1 = Cl,l and W2 = el - W1. Then W1 and W2 are independent. It follows from the symmetr ic distributions of el and cl,t tha t

E{D(e l ) - D(el,t)} 2 = 2P(el > 0, Cl,l < 0).

Let gw1 and Gw2 Then

Note tha t

denote the density of W1 and the distr ibution of W2, respectively.

f P(E" 1 > 0, g'l , / <: 0) - - - - gwl(y)[1 - G w 2 ( - y ) ] d y . ( X )

Gw2(Y) = P Z1 < Icjl ~ y

Page 9: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

N O N P A R A M E T R I C R E G R E S S I O N W I T H I N F I N I T E VARIANCES 81

and gw~ is uniformly bounded (see Lemma 3 of Hsing (1999)). Hence by Potter bounds (see Geluk and de Haan (1987))

F P(c1 > 0, el,/ < 0) = gwi(y)[1 - Gw~(-y)ldY + gw,(y)[1 - Gw2(Y)ldY oc 6

= o 1 - P Icjl +o(6) .

Therefore the lemma follows by letting l --* oc first, and then 5 ~ 0.

LEMMA 3.7. If ~t~=o [Ctl ~/2 < OC, then

1 )_~ D(ei)Ki ~ N(O, a 2)

V / - ~ i = l

where

exists and is finite.

PROOF. Let 9c_~,t be the a-field generated by {Zi , i <_ l}. Note that

D(ei)Ki - ~ D(ei,l)Ki i=1 i=1

) = E { D ( e ~ ) - D(ei, t)}KiI <_ 1 i=1

[n(x+h)] [n(x+h)] o~

= E { D ( e i ) - D(ei j )}Ki = E E KiUi,j,t, i=[n(x-h)] i=[n(x-h)] j = l

where

Ui,j,~ = { E ( D ( s i ) l Y_oo#_j) - Z(D(ei) I ~'_~,i--(j+l))} - {E(D(ei,t) I ~ - ~ , i - / ) - E(D(e{,I) I:F_~,i_(j+I))}I(j <_ l).

Then the lemma follows from Lemma 3.7, the boundedness of Ki and the proof of Theorem 1 of Hsing (1999).

LEMMA 3.8. Suppose cy/j -~ --~ bo > 0 as j ~ co, where/3 C (o1-1, 1). Then there exists 6o > 0 such that for any 61 > 0

P { sup(nh)- l+z-1/a i=1 Ki(I(ei <_ x) - P(ci < x) +p(x)ei) > (~1 ~- O((nh)-5~

PROOF. It is similar to the proof of Theorem 2.1 in Koul and Surgailis (2001) by n 1 replacing I ( - s l <_ j < n) and ~ t = l v j ( A (t - j ) -~0+n) ) in Lemma 4.3 of Koul and

Page 10: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

82 LIANG PENG AND QIWEI YAO

Surgailis (2001) by I ( - s l < j < n)nh and Et=lVj ( A ( t - j ) -~O+~)) I ( [n (x - h)] < t < [n(x + h)]), respectively.

LEMMA 3.9. Suppose c j / j -~ ~ bo as j ~ oc, where 3 E ( a - l , 1 ) . Then (nh) -1-1/~+~ }-]inl KiD(Ei) converges in distribution to a stable law with characteristic function

exp{-'t"(2p(O)bo)'f_-1oo(/_:K(u)(u-v)-'du)'dv}. PROOF. Define cj ---- 0 if j < 0. Note that

(nh) -1 -V~+z KiD(~i) = (nh) -1-1/~+z 2Ki I(ei <_ O) - , i=l i=1

(nh) -1-1/~ ~ 2p(O)Kici : (nh ) - l -1 / a+ /3

i=1

E nh)-l-1/c~+~ 2p(O)gic i - j l=-o~ i=1

~-~ ~-~ 2p(O)Kici- jZj , j=--oo i=1

( " )~ ~-- E (nh) - l -1 /a+ /3 E 2p(O)Kici- j

l<_nx-Snh i=1

+ ~2 (nh)- ' -" ~§ ~2p(0)Kic~_j l>nx-Snh /=1

= A 1 + A2.

It is easy to check that for any 8 > 1

A 1 --4 (2p(0)b0) a K(u)(u - v ) -Zdu dv oo

and ( nx+nh oo ~x2 <_ ~ 2;(O)(nh) -1-1/"+~ sup K(x) c. -~ 0.

l=nx--Snh j=0 /

Hence the limiting characterist ic function of rnh~-l-1/~+Z V "n 2p(0)Kizi is k '~! A.-~i= 1

Thus Lemma 3.9 follows from Lemma 3.8.

LEMMA 3.10. Suppose c j / j -~ ~ bo as j -~ oc, where/3 E ( 1 , 2 / a ) . Let G denote the distribution function of ~i. Then there exists 80 > 0 such that for any 81 > 0

{ �9 n ( P sup (nh) -1 / (~ ) E Ki I(ci <_ x) - G(x)

i=1

Page 11: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

NONPARAMETRIC REGRESSION WITH INFINITE VARIANCES

- } - ~ . ( a ( x - b j Z d - E G ( x - b j Z d ) > 51 = O { ( n h ) - e ~

j= l

83

and

E(I{0< -e~ ~(x)(x~ -x):/e+o(h~)}) =p(O)m(x)(~-x)2/2{1+o(1)}.

PROOF. It is similar to the proof of Lemma 2.3 of Surgailis (2002).

LEMMA 3.11. Suppose cj/ j -~ ~ bo as j ~ oc, where ~ E (1 ,2/a) . Let G denote the distribution function of ei. Then

n

(nh)-l/(~Z) E KiD(ci) d c+L +~ + c-L~z, i=1

where c • and L+an are defined in Theorem 2. l(iii).

PROOF. It follows from Lemma 2.4 of Surgailis (2002) and Theorem 3.1 of Kasa- hara and Maejima (1988) that

(nh) -1/(~) Ki E { G ( - c j Z i ) - EG(-cjZi)} d c+L+az + c-L~z. i=1 j = l

Hence Lemma 3.11 follows from Lemma 3.10 and the fact that

n n ( 1 ) ~_, KiD(ei) = 2 ~ Ki I(ei <_ O) - . i :1 i :1

PROOF OF THEOREM 2.1. Since R(O) P 0, the convex function

G(O) - (nh)-'/uo T E ziD(Yi*)Ki l< i<n

2 2 converges to p(0) (02 + 02 a 0). By the convexity lemma (Pollard (1991)), the convergence is uniform on compact sets in R 2. Using the arguments of Pollard ((1991), p. 193) we can show that the difference between the minimiser of 0 of G(O) and the minimiser of

n

L o T E ziD(Yi*)Ki + p(0)(0~ + 022a~) x/nh i=1

converges to 0 in probability. This implies that

1 (3.2) yrn--h{~h(x) - re(x)} - 2v/_n~p(O) E D(Yi*)Ki + Op(1).

i=1

We may assume tha t /h (x ) > 0. Then

D(Yt*) = D(et) + 2I{0 < - e t _< # ~ ( x ) ( z t - x ) 2 / 2 + o(h2)},

Page 12: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

84 LIANG PENG AND QIWEI YAO

Hence 1 n 1

nh 3 ~ K ~ E ( I { 0 < -~ t < ~(x)(x~ - x)2/2}) -~ ~ ( x ) p ( 0 ) . t = l

Let Wi = KJ(O < -ei < ih(x)(xi - x)2/2). Then

P - E W e ) >

i = 1

n n - -1 2 1 E E ( W i _ E W ~ ) 2 +r E(n_ i ) (E(W1Wi+I)_EW1EWi+I)" ~-- e2n2h6

i = 1 i = 1

Note that EWi = Kip(O)ii~(x)(xi - x)2/2. We have

n

1 E ( E W i ) 2 n~-h ~

i = 1

n

1 EP2(O)K2iit2(x ) ( x i - X)4h4{1 + 0(1)} n2h 6 -4--h~

i = 1

p(0)2~h2(x) ~ 1 2 4n 2 n-~h2Kh(X~ - 1)4{1 + o(1)}

i : 1

P(O)2ii~2(X)4nh ( f f K2(u)u4du) {1 + o(1)}--*0.

Similarly,

and

1 ~-~EW~-- 1 n2h 6 n2h 6 E K2p(O)rh(x)(xi - x)2/2{1 + 0(1)} i = 1 i = 1

p(O)ih(X)2nh 3 (JK2(u)u2du){1+o(1)}- -*0 ,

n- -1 1 n2h6 E ( n - i)EW1EWi+,

i = 1

n - - 1 1 - n2h 6 E K1Ki+lPi+l (0, 0)~t2(x) (Xl - 1) 2 (Xi+l - x) 2 {1 + o(1)} i=1 2 2

<-supp~+l(O'O)~h-lK~21 ( X l h X ) 1 ( / K ( u ) u 2 d u ) - n {1 +o(1)}

- - ~ 0 .

The last limit was ensured by Lemma 3.3. Combining all the above arguments together, we have that

1

2v ;(0) • D(r + Op(1). i = 1

Now the theorem follows from Lemma 3.7, Lemma 3.9 and Lemma 3.11 immediately. The proof is completed.

Page 13: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

NONPARAMETRIC REGRESSION WITH INFINITE VARIANCES 85

Acknowledgements

We thank Dr. Shi-Qing Ling for drawing our attention to the paper of Hsing (1999) and Professor Howell Tong for helpful comments. We thank a reviewer and an associate editor for their comments. QY's research was partially supported by an EPSRC grant.

REFERENCES

Adler, R. J., Feldman, R. E. and Gallagher, C. (1997). Analysing stable time series, A User's Guide to Heavy Tails: Statistical Techniques for Analysing Heavy Tailed Distributions and Processes (eds. R. J. Adler, R. E. Feldman and M. Taqqu), Birkh~user, Boston.

Altman, N. S. (1990). Kernel smoothing of data with correlated errors, Journal of the American Statistical Association, 85, 749-759.

Chiu, C. K. and Marron, J. S. (1991). Comparison of two bandwidth selectors with dependent errors, The Annals of Statistics, 19, 1906-1918.

CsSrg6, S. and Mielniczuk, J. (1995). Nonparametric regression under long-range dependent normal errors, The Annals of Statistics, 23, 1000-1014.

Davis, R. A., Knight, K. and Liu, J. (1992). M-estimation for autoregressions with infinite variances, Stochastic Processes and their Applications, 40, 145-180.

Fan, J. and Halt, P. (1994). On curve estimation by minimizing mean absolute deviation and its implications, The Annals of Statistics, 22, 867 885.

Fan, J., Hu, T.-C. and Trnong, Y. K. (1994). Robust nonparametric function estimation, Scandinavian Journal of Statistics, 21, 433-446.

Geluk, J. and de Haan, L. (1987). Regular variation, extensions and Tauberian theorems, CWI Tract, 40.

Hall, P. and Hart, J. D. (1990). Nonparametric regression with long-range dependence, Stochastic Processes and their Applications, 36, 339-351.

Hall, P., Peng, L. and Yao, Q. (2002). Prediction and nonparametric estimation for time series with heavy tails, Journal of Time Series Analysis, 23(3), 313-331.

Hart, J. D. (1991). Kernel regression estimation with time series errors, Journal of the Royal Statistical Society. Series B, 53, 173-187.

Hart, J. D. (1994). Automated kernel smoothing of dependent data by using time series crossing- validation, Journal of the Royal Statistical Society. Series B, 56, 529-542.

Herrmann, E., Gasser, T. and Kneip, A. (1992). Choice of bandwidth for kernel regression when residuals are correlated, Biometrika, 79(4), 783-795.

Hsing, T. (1999). On the asymptotic distributions of partial sums of functionals of infinite-variance moving averages, The Annals of Probability, 27, 1579-1599.

Kasahara, Y. and Maejima, M. (1988). Weighted sums of i.i.d, random variables attracted to integrals of stable processes, Probability Theory and Related Fields, "/8, 75-96.

Kliippelberg, C. and Mikosch, T. (1996). The integrated periodogram for stable processes, The Annals of Statistics, 24(5), 1855-1879.

Kokoszka, P. and Taqqu, M. S. (1995). Fractional ARIMA with stable innovations, Stochastic Processes and their Applications, 60, 19-47.

Kokoszka, P. and Taqqu, M. S. (1996). Parameter estimation for infinite variance fractional ARIMA, The Annals of Statistics, 24, 1880-1913.

Koul, H. and Surgailis, D. (2001). Asymptotics of empirical processes of long memory moving averages with infinite variance, Stochastic Processes and their Applications, 91, 309-336.

Mallows, C. L. (1980). Some theory of nonlinear smoothers, The Annals of Statistics, 8, 695-715. Mikosch, T., Gadrich, T., Klfippelberg, C. and Alder, R. J. (1995). Parameter estimation for ARMA

models with infinite variance innovations, The Annals of Statistics, 23, 305-326. Pollard, D. (1991). Asymptotics for least absolute deviation regression estimators, Econometric Theory,

7, 186-198. Ray, B. K. and Tsay, R. S. (1997). Bandwidth selection for kernel regression with long-range dependence,

Biometrika, 84, 791 802.

Page 14: Nonparametric regression under dependent errors with infinite … · 2010. 6. 22. · nonparametric regression for random-design models, Yao and Tong (1996) suggested robust conditional

86 LIANG PENG AND QIWEI YAO

Resnick, S. I. (1997). Heavy tail modeling and teletraffic data (with discussion), The Annals of Statistics, 25, 1805-1869.

Robinson, P. M. (1994). Rates of convergence and optimal bandwidth choice for long-range dependence, Probability Theory and Related Fields, 99, 443-473.

Robinson, P. M. (1997). Large-sample inference for nonparametric regression with dependent errors, The Annals of Statistics, 25, 2054-2083.

Roussas, G. G., Tran, L. T. and Ioannides, O. A. (1992). Fixed design regression for time series: Asymptotic normality, Journal of Multivariate Analysis, 40, 262-291.

Samorodnitsky, G. and Taqqu, M. S. (1994). Stable Non-Gaussian Random Processes, Chapman and Hall, New York.

Surgailis, D. (2002). Stable limits of empirical processes of moving averages with infinite variance, Stochastic Processes and their Applications, 100, 255-274.

Tran, L. T., Roussas, G. G., Yakowitz, S. and Truong Van, B. (1996). Fixed-design regression for linear time series, The Annals of Statistics, 24, 975 991.

Truong, Y. K. (1989). Asymptotic properties of kernel estimators based on local medians, The Annals of Statistics, 17, 606--617.

Truong, Y. K. (1991). Nonparametric curve estimation with time series errors, Journal of Statistical Planning and Inference, 28, 167-183.

Truong, Y. K. (1992a). Robust nonparametric regression in time series, Journal of Multivariate Anal- ysis, 41, 163-177.

Truong, Y. K. (1992b). A nonparametric approach for time series analysis, IMA Volumes in Mathematics and Its Applications, 45, 371-386.

Truong, Y. K. and Stone, C. J. (1992). Nonparametric function estimation involving time series, The Annals of Statistics, 20, 77-97.

Tsybakov, A. B. (1986). Robust reconstruction of functions by local-approximation method, Problems of Information Transmission, 22, 133-146.

Velleman, P. F. (1980). Definition and comparison of robust nonlinear data smoothing algorithms, Journal of the American Statistical Association, 75, 609-615.

Yao, Q. and Tong, H. (1996). Asymmetric least squares regression estimation: a nonparametric ap- proach, Journal of Nonparametric Statistics, 6, 273-292.