a quantile-copula approach to conditional density estimation. · · 2013-05-13introduction the...

IntroductionThe Quantile-Copula estimator

Comparison with competitors

A quantile-copula approach to conditional densityestimation.

Olivier P. Faugeras

Universite Paris VI - Laboratoire de Statistique Theorique et Appliquee

Universite Parix XIII - Laboratoire Lim&Bio

25/05/2008

Olivier P. Faugeras A quantile-copula approach to conditional density estimation.

Why estimating the conditional density?Two classical approaches for estimation

Outline

1 IntroductionWhy estimating the conditional density?Two classical approaches for estimation

2 The Quantile-Copula estimatorQuantile transform and copula functionConstruction of the estimatorAsymptotic results

3 Comparison with competitorsTheoretical comparisonFinite sample simulation

Setup and Motivation

Objective

observe a sample ((Xi, Yi); i = 1, . . . , n) i.i.d. of (X, Y ).predict the output Y for an input X at location x

with minimal assumptions on the law of (X, Y ) (Nonparametric setup).

Notation

(X, Y )→ joint c.d.f FX,Y , joint density fX,Y ;

X → c.d.f. F , density f ;

Y → c.d.f. G, density g.

Objective

Notation

Objective

Notation

Why estimating the conditional density ?

What is a good prediction ?

1 the regression function r(x) = E(Y |X = x),2 or the full conditional density f(y|x)?

Estimating the conditional density - 1

A first density -based approach

f(y|x) =fX,Y (x, y)

f(x)← fX,Y (x, y)

fX,Y , f : Parzen-Rosenblatt kernel estimators with kernels K, K ′,bandwidths h and h′.

The double kernel estimator

f(y|x) =

n∑i=1

K ′h′(Xi − x)Kh(Yi − y)

n∑i=1

K ′h′(Xi − x)

→ ratio shaped

f(y|x) =fX,Y (x, y)

f(x)← fX,Y (x, y)

f(y|x) =

n∑i=1

K ′h′(Xi − x)

→ ratio shaped

f(y|x) =fX,Y (x, y)

f(x)← fX,Y (x, y)

f(y|x) =

n∑i=1

K ′h′(Xi − x)

→ ratio shaped

f(y|x) =fX,Y (x, y)

f(x)← fX,Y (x, y)

f(y|x) =

n∑i=1

K ′h′(Xi − x)

→ ratio shaped

A regression strategy

Fact: E(1|Y −y|≤h|X = x

)= F (y + h|x)− F (y − h|x) ≈ 2h.f(y|x)

Conditional density estimation problem → regression framework

Transform the data:Yi → Y ′

i := (2h)−11|Yi−y|≤h

Yi → Y ′i := Kh(Yi − y) smoothed version

Nonparametric regression of Y ′i on Xis by local averaging methods

(Nadaraya-Watson, local polynomial, orthogonal series,...)

Nadaraya-Watson estimator

f(y|x) =

n∑i=1

K ′h′(Xi − x)

→ (same) ratio shape.

)= F (y + h|x)− F (y − h|x) ≈ 2h.f(y|x)

i := (2h)−11|Yi−y|≤h

f(y|x) =

n∑i=1

K ′h′(Xi − x)

)= F (y + h|x)− F (y − h|x) ≈ 2h.f(y|x)

i := (2h)−11|Yi−y|≤h

f(y|x) =

n∑i=1

K ′h′(Xi − x)

Ratio shaped estimators

Drawbacks

quotient shape of estimator is tricky to study;

explosive behavior when the denominator is small → numericalimplementation delicate (trimming);

minoration hypothesis on the marginal density f(x) ≥ c > 0.

How to remedy these problems?→ build on the idea of using synthetic data:find a representation of the data more adapted to the problem.

Ratio shaped estimators

Drawbacks

quotient shape of estimator is tricky to study;

explosive behavior when the denominator is small → numericalimplementation delicate (trimming);

minoration hypothesis on the marginal density f(x) ≥ c > 0.

How to remedy these problems?→ build on the idea of using synthetic data:find a representation of the data more adapted to the problem.

Quantile transform and copula functionConstruction of the estimatorAsymptotic results

Outline

The quantile transform and copula function

What is the “best” transformation of the data in that context ?

The quantile transform

(X1, . . . , Xn) (Y1, . . . , Yn)↓ ↓

(U1 = F (X1), . . . , Un = F (Xn)) (V1 = G(Y1), . . . , Vn = G(Yn))

then Ui, Vi = pseudo-sample with a prescribed uniform marginaldistribution.

→ leads naturally to the copula function:

Sklar’s theorem [1959]

FX,Y (x, y) = C(F (x), G(y)) , −∞ ≤ x, y ≤ +∞.

with C the c.d.f. of (U, V ).

(X1, . . . , Xn) (Y1, . . . , Yn)↓ ↓

FX,Y (x, y) = C(F (x), G(y)) , −∞ ≤ x, y ≤ +∞.

(X1, . . . , Xn) (Y1, . . . , Yn)↓ ↓

FX,Y (x, y) = C(F (x), G(y)) , −∞ ≤ x, y ≤ +∞.

A product shaped estimator

A product form of the conditional density

fY |X(y|x) =fXY (x, y)

f(x)= g(y)c(F (x), G(y))

with c(u, v) = ∂2C(u,v)∂u∂v the copula density.

→ a product estimator

The quantile copula estimator

fn(y|x) := gn(y)cn(Fn(x), Gn(y))

A product shaped estimator

A product form of the conditional density

fY |X(y|x) =fXY (x, y)

f(x)= g(y)c(F (x), G(y))

with c(u, v) = ∂2C(u,v)∂u∂v the copula density.

→ a product estimator

The quantile copula estimator

fn(y|x) := gn(y)cn(Fn(x), Gn(y))

Construction of the estimator

density of Y : g ← kernel estimator gn(y) := 1nhn

n∑i=1

(y−Yi

)c.d.f. F,G← Fn, Gn empirical c.d.f.pseudo sample (Ui, Vi)← (Fn(Xi), Gn(Yi)) empirical approximatecopula density c(u, v)← cn(u, v) kernel density (pseudo) estimatorbased on the pseudo sample (Ui, Vi)cn(u, v)← cn(u, v) kernel density estimator based on the empiricalapproximate (Fn(Xi), Gn(Yi))

The quantile-copula estimator

fn(y|x) :=

n∑i=1

(y − Yi

n∑i=1

(Fn(x)− Fn(Xi)

(Fn(y)−Gn(Yi)

)]Olivier P. Faugeras A quantile-copula approach to conditional density estimation.

n∑i=1

(y−Yi

fn(y|x) :=

n∑i=1

(y − Yi

n∑i=1

(Fn(x)− Fn(Xi)

(Fn(y)−Gn(Yi)

n∑i=1

(y−Yi

fn(y|x) :=

n∑i=1

(y − Yi

n∑i=1

(Fn(x)− Fn(Xi)

(Fn(y)−Gn(Yi)

n∑i=1

(y−Yi

fn(y|x) :=

n∑i=1

(y − Yi

n∑i=1

(Fn(x)− Fn(Xi)

(Fn(y)−Gn(Yi)

Asymptotic results

Under classical regularity assumptions on the densities and the kernels,with hn → 0, an → 0,

Consistency

weak consistency nhn →∞, na2n →∞ entail

fn(y|x) = f(y|x) + OP

(n−1/3

strong consistency nhn/(ln lnn)→∞ and na2n/(ln lnn)→∞

fn(y|x) = f(y|x) + Oa.s.

((ln lnn

asymptotic normality nhn →∞ and na2n →∞ entail√

(fn(y|x)− f(y|x)

(0, g(y)f(y|x)||K||22

hn ' n−1/5 and an ' n−1/6, we get

f(y|x) = f(y|x) + OP (n−1/3).

Theoretical comparisonFinite sample simulation

Outline

Theoretical asymptotic comparison

all estimators have the same order of convergence ≈ n−1/3

Asymptotic Bias

c of compact support : the “classical” kernel method to estimate thecopula density induces bias on the boundaries of [0, 1]2

→ use boundary-correction techniques (boundary kernels, beta kernels,reflection and transformation methods,.. )

Main terms in the asymptotic variance

Ratio shaped estimators: f(y|x)f(x) → explosive variance for small value

of the density f(x), e.g. in the tail of the distribution of X.

Quantile-copula estimator: g(y)f(y|x) → does not suffer from theunstable nature of competitors.

Asymptotic Bias

Finite sample simulation

Sample of n = 100 i.i.d. variables (Xi, Yi), from the following model:

X, Y is marginally distributed as N (0, 1)X, Y is linked via Frank Copula .

C(u, v, θ) =ln[(θ + θu+v − θu − θv)/(θ − 1)]

with parameter θ = 100.

Practical implementation:

Beta kernels for copula estimator, Epanechnikov for other.

simple Rule-of-thumb method for the bandwidths.

Conclusions

Summary

ratio type into the product → consistency and limit results whereobtained by simple combination of the previous known ones on(unconditional) density estimation,

nonexplosive behavior in the tails of the marginal density,

no need for trimming or clipping.

Conclusions

Perspectives and work-in-progress

Adaptive bandwidth choices to the regularity of the model

Efficient kernel estimation of the copula density byBoundary-corrected kernels

Designing an extreme-specific estimation method. Applications toinsurance, Risk theory, Environmental Sciences.

Application to prediction: functional of the density∫

φ(y)f(y|x)dy,mode, predictive sets.

Extension to time series by coupling arguments for Markovianmodels.

Alternative nonparametric methods of estimation: projections,wavelets,..

Bibliography

Reference

O. P. Faugeras. A quantile-copula approach to conditional densityestimation. Submitted, in revision, 2008. Available onhttp://hal.archives-ouvertes.fr/hal-00172589/fr/.

Related work:J. Fan and Q. Yao. Nonlinear time series. Springer Series inStatistics. Springer-Verlag, New York, second edition, 2005.Nonparametric and parametric methods.J. Fan, Q. Yao, and H. Tong. Estimation of conditional densitiesand sensitivity measures in nonlinear dynamical systems. Biometrika,83(1):189–206, 1996.L. Gyorfi and M. Kohler. Nonparametric estimation of conditionaldistributions. IEEE Trans. Inform. Theory, 53(5):1872–1879, 2007.R. J. Hyndman, D. M. Bashtannyk, and G. K. Grunwald. Estimatingand visualizing conditional densities. J. Comput. Graph. Statist.,5(4):315–336, 1996.

References

R. J. Hyndman and Q. Yao. Nonparametric estimation andsymmetry tests for conditional density functions. J. Nonparametr.Stat., 14(3):259–278, 2002.

C. Lacour. Adaptive estimation of the transition density of a markovchain. Ann. Inst. H. Poincare Probab. Statist., 43(5):571–597,2007.

M. Rosenblatt. Conditional probability density and regressionestimators. In Multivariate Analysis, II (Proc. Second Internat.Sympos., Dayton, Ohio, 1968), pages 25–31. Academic Press, NewYork, 1969.

M. Sklar. Fonctions de repartition a n dimensions et leurs marges.Publ. Inst. Statist. Univ. Paris, 8:229–231, 1959.

W. Stute. On almost sure convergence of conditional empiricaldistribution functions. Ann. Probab., 14(3):891–901, 1986.

a quantile-copula approach to conditional density estimation. · · 2013-05-13introduction the...

Documents