talk slides msast2016

47
Nonparametric Estimation of Distribution and Density Functions Using q-Bernstein Polynomials Yogendra P. Chaubey Department of Mathematics and Statistics Concordia University, Montreal, Canada H3G 1M8 E-mail:[email protected] Talk to be presented at the 6th International Conference of IMBIC, Kolkata, India, December 21-23, 2016 Acknowledgment: The computations were performed by Qi Zhang as part of his BSc (Honours) project. Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 1 / 47

Upload: ychaubey

Post on 12-Apr-2017

55 views

Category:

Presentations & Public Speaking


0 download

TRANSCRIPT

Nonparametric Estimation of Distribution and DensityFunctions Using q-Bernstein Polynomials

Yogendra P. Chaubey

Department of Mathematics and StatisticsConcordia University, Montreal, Canada H3G 1M8

E-mail:[email protected]

Talk to be presented at the 6th International Conferenceof IMBIC, Kolkata, India, December 21-23, 2016

Acknowledgment: The computations were performed by Qi Zhang aspart of his BSc (Honours) project.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 1 / 47

Abstract

Here we consider the smooth density estimation for the random variablesthat have a compact support. Without loss of generality, we restrict thesupport to the closed interval [0, 1], for which Bernstein polynomialestimator, originally proposed by Vitale (1973), is known to have somegood properties, such as being a genuine density estimator and being freefrom boundary bias (see Babu, Canty and Chaubey (2002) and Kakizawa(2004)). Generalized version of these estimators were proposed by PrakasaRao (2005) and Kakizawa (2011) by considering the generalized Bernsteinpolynomials introduced in Cao (1997). Some work on improvingconvergence rate of the Bernstein polynomial density estimator werecarried out in Leblanc (2009, 2012) and Igarshi and Kakizawa (2014). Inthis paper, we consider yet another generalization of the Bernsteinpolynomials introduced by Phillips (1997), that may have some advantageover the usual Bernstein polynomial estimator.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 2 / 47

1 Introduction2 Preliminaries3 Smooth Estimators of the Distribution and Density Functions4 Asymptotic Properties of Estimators

Asymptotic Properties of Fn,m,q

Asymptotic Properties of fn,m,q

5 Numerical StudiesCross Validation Method to Determine the Smoothing Parameter q

Likelihood Based Cross ValidationIntegrated Square Error Cross ValidationLeast Square Cross Validation

Illustration for Distribution Function EstimatorIllustration for Density EstimatorComparing Estimators by Average Squared Error

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 3 / 47

1. Introduction

Let h be a bounded continuous function defined on the interval [0, 1],then the corresponding Bernstein polynomial of degree m is defined as

Bm(x;h) =

m∑k=0

h

(k

m

)bk(x,m), (1.1)

where

bk(x,m) =

(m

k

)xk(1− x)m−k, k = 0, 1, .... (1.2)

This polynomial was proposed as an uniform approximation tobounded continuous functions defined on [0, 1] by S.N. Bernstein(1912), that provided an existential proof of Weierstrassapproximation theorem.

The monograph by Lorentz (1986) makes an excellent referenceregarding the properties of the Bernstein polynomials.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 4 / 47

1. Introduction

Assuming that the function h has a continuous derivative, thederivative B′m(x;h) given by

B′m(x;h) = m

m−1∑k=0

{h

(k + 1

m

)− h

(k

m

)}bk(x,m− 1) (1.3)

uniformly approximates the derivative h′(x).Given a random sample {X1, ..., Xn} from an absolutely continuousdistribution function F, we can estimate it by the empiricaldistribution function

Fn(x) =1

n

n∑i=1

I{Ti ≤ x}. (1.4)

Since the above function is a step function, it is not appropriate forfinding the density function f = F ′, thus smoothing of this functionis desired.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 5 / 47

1. Introduction

Replacing h by the empirical distribution function Fn, the Bernsteinpolynomial approximation mentioned earlier motivates the followingsmooth estimator of the distribution function

Fn,m(x) =

m∑k=0

Fn

(k

m

)bk(x,m). (1.5)

This estimator is a genuine distribution function and therefore itsderivative serves as a smooth density estimator given by

fn,m(x) = B′m(x;Fn)

= mm−1∑k=0

{Fn

(k + 1

m

)− Fn

(k

m

)}bk(x,m− 1)

(1.6)

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 6 / 47

1. Introduction

Bernstein polynomial density estimator: Originally proposed by Vitale(1973) and later investigated by many authors

Gowronski and Stadmuler (1981): Smoothing of histogram usingFeller’s Theorem

Stadtmuller, (1983): Convergence of the density estimator

Petrone (1999): Nonparametric Bayesian prior

Ghosal, 2001: Convergence of nonparametric posteriors

Babu, Canty and Chaubey (2002): Further studies

Leblanc (2009, 2012): Smooth distribution function estimator.

Prakasa Rao (2005) and Kakizawa (2011): Generalized Bernsteinpolynomials introduced by Cao (1997).

Another generalization is introduced by Phillips (1997) that is thesubject of this talk.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 7 / 47

2. Preliminaries

Let q > 0, then for any k ∈ N ∪ {0}, the q−integer [k]q andq−factorial [k]q! are respectively defined by

[k]q =

{(1− qk)/(1− q), for q 6= 1k, for q = 1,

(2.1)

and

[k]q! =

{[k]q[k − 1]q...[1]q, for k ≥ 11, for k = 0.

(2.2)

For integers 0 ≤ k ≤ m, the q−binomial coefficients are defined by[mk

]q

=[m]q!

[k]q![m− k]q!(2.3)

For q = 1, we clearly have [m]q = m, [m]q! = m! and

[mk

]q

=

(mk

),

q−factorials and q−binomial coefficients reduce to the usual factorialsand binomial coefficients, respectively.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 8 / 47

2. Preliminaries

Definition

(Phillips 1997). Let h ∈ C[0, 1]. The generalized Bernstein polynomialbased on the q−integers is

Bm,q(x;h) =

m∑k=0

h

([m]q[k]q

)bk,q(x,m) (2.4)

where

bk,q(x,m) =

[mk

]q

xkm−k−1∏s=0

(1− qsx), m = 1, 2, ...

and an empty product is taken to be equal to 1.

The polynomials defined in (2.4) reduce to the usual Bernsteinpolynomials for q = 1, and hence they are referred to as thegeneralized Bernstein polynomials, however, in the sequel we will referthem as the q−Bernstein polynomials.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 9 / 47

2. Preliminaries

The following analog of Bernstein’s Theorem for q−Bernsteinpolynomials was proved by Phillips (1997).

Theorem

Let a sequence {qm}∞m=1 satisfy 0 < qm < 1 and qm → 1 as m→∞.Then for any function h ∈ C[0, 1],

Bm,qm(x;h) ↪→ h(x) [x ∈ [0, 1];m→∞].

Here the expression gm(x) ↪→ g(x)[x ∈ [0, 1];m→∞] denotesuniform convergence of gm to g with respect to x ∈ [0, 1].

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 10 / 47

2. Preliminaries

The following theorem from Phillips (1996) gives convergence resultsof the derivative of the q−Bernstein polynomial

Theorem

Let h ∈ C1[0, 1] and let the sequence (qm) be chosen so that the sequence(εm) converges to zero from above faster than 1/3m, where

εm =m

1 + qm + q2m + ...+ qm−1m− 1.

Then the sequence of derivatives of the q−Bernstein polynomialsconverges uniformly on [0, 1] to h′(x).

Phillips (1996) showed that the condition on (εm) can be satisfied bychoosing (qm) such that

1− a−m

m≤ qm < 1 with a > 3.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 11 / 47

3. Estimators of the Distribution and Density Functions

Similar to the estimator given in (1.5) using the Bernstein polynomial,the q− Bernstein polynomial may be used to define the followingnon-parametric smooth estimator of F. Thus, the q−Bernsteinpolynomial estimator of the distribution function F is given by

Fn;m,q(x) =

m∑k=0

Fn

([k]q[m]q

)bk,q(x,m), x ∈ [0, 1], (3.1)

where

bk,q(x,m) =

[mk

]q

xkm−k−1∏s=0

(1− qsx). (3.2)

For q = 1, the above estimator reduces to the usual Bernsteinpolynomial smooth estimator of the distribution function studied inBabu, Canty and Chaubey (2004).

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 12 / 47

2. Estimators of the Distribution and Density Functions

By Euler identity (see Philips 1997) the product in (3.2) can bewritten as a sum and thus taking the derivative of Fn;m,q, thefollowing smooth estimator of f is obtained that will be calledq−Bernstein density estimator:

fn;m,q(x) =

m∑k=0

m−k∑s=0

Fn

([k]q[m]q

)ωs,k(x;m, q) (3.3)

where

ωs,k(x;m, q) =

[mk

]q

(−1)sqs(s−1)

2

[m− ks

]q

(k + s)xk+s−1.

This paper presents a comparative study of the resulting smoothestimators with Bernstein polynomial estimators while choosing thevalue of q by cross validation and the value of m = n/ log(n) assuggested in Babu, Canty and Chaubey (2004).

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 13 / 47

2. Estimators of the Distribution and Density Functions

Since, the approximation improves as m→∞, one may propose theuse of limiting q−Bernstein polynomials, that was studies in Il’inskiiand Ostrovska (2002). They noted that as m→∞ for 0 < q < 1

limm→∞

[k]q[m]q

= 1− qk

limm→∞

bk,q(x,m) =xk

(1− qk)[k]q!∞∏s=0

(1− qsx) =: bk,∞(x, q)

the limiting q−Bernstein polynomial becomes

B∞,q(x, h) =

{ ∑∞k=0 h(1− qk)bk,∞(x, q), if x ∈ [0, 1),

h(1) if x = 1.

Theorem

For any f ∈ C[0, 1],

B∞,q(x, h) ↪→ f(x)[x ∈ [0, 1]; q ↑ 1].

Theorem

Let 0 < α < 1. Then for any f ∈ C[0, 1],

Bm,q(x, h) ↪→ h(x)[x ∈ [0, 1]; q ∈ [α, 1];m→∞].

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 14 / 47

2. Estimators of the Distribution and Density Functions

Theorem

For any f ∈ C[0, 1],

B∞,q(x, h) ↪→ f(x)[x ∈ [0, 1]; q ↑ 1].

Theorem

Let 0 < α < 1. Then for any f ∈ C[0, 1],

Bm,q(x, h) ↪→ h(x)[x ∈ [0, 1]; q ∈ [α, 1];m→∞].

The above theorems basically recast Phillip’s results by explicitlyexhibiting the limiting form of Bm,q(x;h) as m→∞. So, for generalapproximation, the problem of determining the constants m and qreduces to determining q only. In our experience, the correspondingsmooth estimator of F is meaningful in practice, however, thecorresponding density estimator may not be very smooth.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 15 / 47

2. Estimators of the Distribution and Density Functions

In our experience, the corresponding smooth estimator of F ismeaningful in practice, however, the corresponding density estimatormay not be very smooth.

Thus, for numerical studies, we propose to consider m = n/ log(n)[the choice suggested in Babu, Canty and Chaubey (2004)] anddetermine q by cross-validation.

In the next section we provide the almost sure convergence results forthe smooth estimators similar to those obtained in Babu, Canty andChaubey (2004) using Bernstein polynomials. But, first some picturesof the estimators.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 16 / 47

Distribution Function Estimator for Beta(3, 3) density

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

q−Bernstein Smooth Distribution

Sample from Beta(3,3) Density, n=50x

Fn(x

)

Bernsteinq−Bernstein,q=.99

Figure: 1. Distribution Function Estimators for Beta(3, 3) density; n = 50.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 17 / 47

Distribution Function Estimator for Beta(3, 3) density

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

q−Bernstein Smooth Distribution

Sample from Beta(3,3) Density, n=50x

Fn(x

)

Bernsteinq−Bernstein,q=.99,m=200

Figure: 2. Distribution Function Estimators for Beta(3, 3) density; n = 50.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 18 / 47

Density Function Estimator for Beta(3, 3) density

q−Bernstein Smooth Desnity estimator

Sample from Beta(3,3) Density, n=50x

f n(x

)

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

Bernsteinq−Bernstein,q=.99

Figure: 3. Density Function Estimators for Beta(3, 3) density; n = 50.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 19 / 47

Density Function Estimator for Beta(3, 3) density

q−Bernstein Smooth Desnity estimator

Sample from Beta(3,3) Density, n=50x

f n(x

)

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0 Bernstein

q−Bernstein,q=.99,m=50

Figure: 4. Density Function Estimators for Beta(3, 3) density; n = 50.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 20 / 47

Density Function Estimator for Beta(3, 3) density

q−Bernstein Smooth Desnity estimator

Sample from Beta(3,3) Density, n=50x

f n(x

)

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0 Bernstein

q−Bernstein,q=.99,m=25

Figure: 5. Density Function Estimators for Beta(3, 3) density; n = 50.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 21 / 47

4. Asymptotic Properties of Estimators

Throughout this paper we use the notation

‖G‖ = supx∈[0,1]

|G(x)|,

for a bounded function G on [0, 1],

an = (n−1 log n)1/2, and bn,m = (n−1 log n)1/2(m−1 logm)1/4.(4.1)

The following theorem shows that Fn,m is strongly consistent.

Theorem

If m,n→∞, then ‖Fn,m,q − F‖ → 0 a.s. as q → 1.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 22 / 47

4.1 Asymptotic Properties of Distribution FunctionEstimator

Next theorem gives a result depicting the closeness of the smoothestimator with the empirical distribution function.

Theorem

Let F be continuous and differentiable on the interval [0, 1] with density f .If f is Lipschitz of order 1, then for n2/3 ≤ m ≤ (n/ log n)2, we have a.s.as n→∞ ,

limq↑1‖Fn,m − Fn‖ = O

((n−1 log n)1/2(m−1 logm)1/4

). (4.2)

For m = n, we have

limq↑1‖Fn,m − Fn‖ = O(n−3/4(log n)3/4) a.s.. (4.3)

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 23 / 47

4.2. Asymptotic Properties of Density Function Estimator

We now establish the strong convergence of fn(t) similar to that ofFn(t).

Theorem

Let F be continuous on the interval [0, 1] with a continuous density f .Then for 2 ≤ m ≤ (n/ log n), we have a.s. as n→∞ and q → 1,

‖fn,m,q − f‖ = O(m1/2an) +O(‖F ∗′m − f‖), (4.4)

a.s. as n→∞ , where F ∗′

m denotes the derivative of F ∗m. Consequently, ifm = o(n/ log n), then ‖fn,m,q − f‖ → 0 a.s. as m,n→∞.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 24 / 47

4.2. Asymptotic Properties of Density Function Estimator

The following theorem establishes the asymptotic normality of thefn,m,q(x).

Theorem

Under conditions in theorem 3.1, if f(x) > 0 then as q → 1

n1/2

m1/4(fn,m,q(x)− f(x)) D→ N

(0,

f(x)

2√πx(1− x)

)(4.5)

as m,n→∞ such that m ≤ (n/ log n).Remark: If mn−2/5 → δ−2 > 0, then as q → 1,

n(2/5)(fn,m,q(x)− f(x)) D→ N(0,

δf(x)

2√πx(1− x)

)(4.6)

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 25 / 47

4.3. Estimation for Other Support

The case of supports other than [0, 1] may be handled bytransformation.

1. For a finite support [a, b], transform the samplevalues to Y1, ...., Yn, where Yi = (Xi − a)/(b− a).Denoting the Gn(y) as the smooth distribution functionof the transformed sample, the smooth distributionFn(x) is given by

Fn(x) = Gn(y) where y = (x− a)/(b− a).2.For the interval (−∞,∞), the transformationYi = (1/2) + (1/π) tan−1Xi is useful. In this case

Fn(x) = Gn(y) where y = (1/2)(1/π) tan−1 x.

3. For the non-negative support [0,∞) :, we may usethe transformation: y = x/(1 + x)

Fn(x) = Gn(y) where y = x/(1 + x).

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 26 / 47

5. Numerical Studies

Here we illustrate the results for the estimation of distribution anddensity for the following three cases: (i) Beta (3,3) distribution, (ii)Beta(2,4) distribution and (iii) 0.65×Beta(2,10)+0.35×Beta(10,2)distribution.The three cases are chosen to depict three scenarios, namely that ofsymmetric unimodal distribution, asymmetric unimodal distributionand a bimodal distribution. We investigate the possibility of choosingq different than 1, for the choice of m = n/ log(n), a choicerecommended in Babu, Canty and Chaubey (2002) for Bernsteinpolynomials. We would also like to investigate the optimal choice of qusing the limiting form of the q−Bernstein polynomials, however, thisis not pursued here. For a given m, the choice of q is obtained bycross-validation that is explained below.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 27 / 47

5.1 Cross Validation

For the q−Bernstein polynomial CDF and PDF estimators, in order tochoose an appropriate parameter q, we introduce a numerical methodcalled Cross Validation, detail of relative equations are referencedfrom Chaubey and Sen (2009).Kullback-Liebler divergence between the estimated density fn and thetrue density f is given by

KL(fn; f) = E∫log

f(x)

fn(x)dF (x)

In practice, the optimum cross-validation method estimates suchdivergence from the data for a given smoothing parameter andchooses one which gives the smallest estimated divergence. Bowman(1984) shows that this procedure is equivalent to the minimization ofthe negative likelihood,

CVKL(q) = − log

n∏i=1

fn(Xi;Di) = −n∑

i=1

log(fn(Xi;Di)) (5.1)

where Di denotes data with Xi removed from D, which is the wholesample set. The solution of the above minimization problem will bedenoted by λnKL. When the whole sample is used in constructing thesmooth density estimator, we will simply denote the density by fn(x)

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 28 / 47

5.1 ISE Cross Validation

According to this criterion we determine λn that minimizes thecriterion related to the mean integrated squared error,

MISE(q) = E∫(fn(x)− f(x))2dx

Estimating this from the data and minimizing it is equivalent to theminimization of (see Silverman (1986),

CVISE(q) =

∫f2n(x; q,D)dx−

2

n

n∑i=1

fn−1(Xi; q,Di) (5.2)

The first part can be obtain by numerical integration, say compositeSimpson Method. And again, Di denotes data with Xi removed fromD.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 29 / 47

5.1 LS Cross Validation

The above cross validation works well if f is a density estimator. ForCDF, it is better to use least square cross validation (see Hardle,1991).

CVLS(q) =

n∑i=1

(Fn(Xi; q,D)− Fn(Xi; q,Di))2 (5.3)

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 30 / 47

Distribution Function-Illustration

For a given sample of size n we start with m = n and determine theoptimum value of the smoothing parameter as illustrated in Figures6-8 for a random sample generated from each of the distributions.We also investigate the effect of increasing the value of m. This isdone by selecting a grid of size n on the interval [0, 1], and we let Mdenote the value of m that gives the average error of estimationbetween the successive values of M and M + 1 less than ε, in orderto minimize the amount of computation. [In order to control theindividual errors, M will depend on the value of x also.].

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 31 / 47

0.0 0.2 0.4 0.6 0.8 1.0

0.05

0.10

0.15

0.20

0.25

qSu

m of S

quare

Erros

(SSE)

(a) n = 20, q = 0.94,m = 20

0.2 0.4 0.6 0.8

0.00.2

0.40.6

0.81.0

x

cdf

(b) M = 147, ε = 9.96× 10−5

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.8

q

Sum o

f Squa

re Err

os (SS

E)

(c) n = 50, q = 1,m = 50

0.2 0.4 0.6 0.8

0.00.2

0.40.6

0.81.0

x

cdf

(d) M = 1024, ε = 5.31× 10−5

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

1.2

q

Sum o

f Squa

re Err

os (SS

E)

(e) n = 100; q = 1,m = 100

0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(f) M = 801, ε = 7.56× 10−5

Figure 6: LSCV plot for q−Bernstein estimator (Left Panel); CDF Estima-tor for Beta(3, 3) Samples (Right Panel).Remark: red - true density; dash - q−Bernstein with m = n; blue -q−Bernstein with parameter q,M

12

Figure: 6. LSCV Cross Validation for Beta(3, 3) density

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 32 / 47

0.0 0.2 0.4 0.6 0.8 1.0

0.20.4

0.60.8

qSu

m of S

quare

Erros

(SSE)

(a) n = 20, q = 0.96,m = 20

0.1 0.2 0.3 0.4 0.5 0.6 0.7

0.00.2

0.40.6

0.81.0

x

cdf

(b) M = 206, ε = 9.60× 10−5

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

1.21.4

q

Sum o

f Squa

re Err

os (SS

E)

(c) n = 50, q = 0.99,m = 50

0.2 0.4 0.6 0.8

0.00.2

0.40.6

0.81.0

x

cdf

(d) M = 105, ε = 8.28× 10−5

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

2.02.5

3.03.5

q

Sum o

f Squa

re Err

os (SS

E)

(e) n = 100; q = 1,m = 100

0.0 0.2 0.4 0.6 0.8

0.00.2

0.40.6

0.81.0

x

cdf

(f) M = 611, ε = 6.64× 10−5

Figure 7: Smooth Bernstein(Left) and q−Bernstein(Right) CDF Estimatorfor Beta(2, 4) Samples.Remark: red - true density; dash - q−Bernstein with m = n; blue -q−Bernstein with parameter q,M

13

Figure: 7. LSCV Cross Validation for Beta(2, 4) density

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 33 / 47

0.0 0.2 0.4 0.6 0.8 1.0

0.20.4

0.60.8

1.01.2

qSu

m of S

quare

Erros

(SSE)

(a) n = 20, q = 0.99,m = 20

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(b) M = 73, ε = 6.88× 10−5

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

2.02.5

q

Sum o

f Squa

re Err

os (SS

E)

(c) n = 50, q = 1,m = 50

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(d) M = 355, ε = 4.41× 10−5

0.0 0.2 0.4 0.6 0.8 1.0

01

23

45

q

Sum o

f Squa

re Err

os (SS

E)

(e) n = 100; q = 1,m = 100

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(f) M = 371, ε = 8.98× 10−5

Figure 8: Smooth Bernstein(Left) and q−Bernstein(Right) CDF Estimatorfor 0.65 · Beta(2, 10) + 0.35 · Beta(10, 2) Samples.Remark: red - true density; dash - q−Bernstein with m = n; blue -q−Bernstein with parameter q,M

14

Figure: 8. LSCV Cross Validation for Mixture density

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 34 / 47

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

xcdf

(a) n = 20, q = 0.94,m = 7,M = 147

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(b) n = 20, q = 0.96,m = 7,M = 206

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(c) n = 50, q = 1,m = 13,M = 1024

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(d) n = 50, q = 0.99,m = 13,M = 105

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(e) n = 100, q = 1,m = 22,M = 801

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(f) n = 100, q = 1,m = 22,M = 611

Figure 9: Smooth Bernstein and q−Bernstein CDF Estimator forBeta(3, 3)(Left) and Beta(2, 4)(Right) Samples.Remark: red is theoretical, black is Bernstein ,dash is q−Bernstein withM = m, blue is q−Bernstein with parameter q,M

15

Figure: 9. Optimum m, q for Beta(3, 3) and Beta(2, 4) density

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 35 / 47

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

xcdf

(a) n = 20, q = 0.99,m = 7,M = 73

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(b) n = 50, q = 1,m = 13,M = 355

0.0 0.2 0.4 0.6 0.8 1.0

0.00.2

0.40.6

0.81.0

x

cdf

(c) n = 100, q = 1,m = 22,M = 371

Figure 10: Smooth Bernstein(Left) and q−Bernstein(Right) CDF Estimatorfor 0.65 · Beta(2, 10) + 0.35 · Beta(10, 2) Samples.Remark: red is theoretical, dash is q−Bernstein with M = m, blue isq−Bernstein with parameter q,M

16

Figure: 10. Optimum m, q for the mixture density

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 36 / 47

Density Function-Illustration

In Figures 11-12, we present the plots of density estimators correspondingto the three distributions based on a random sample for sample sizesn = 20, 50, 100, where m = n/ log(n) for different values of q ∈ [.85, 1].We will use the following colors to highlight specific q’s.We will use red for the theoretical density, while use green3 for q = 0.85,lightseagreen for q = 0.9, olivedrab for q = 0.95 and darkorange3 foroptimized q and blue for the Bernstein density estimator. For optimizedq ∈ {0.85, 0.9, 0.95}, we will display it in darkorange3. Note that we havenot considered q−Bernstein polynomial density estimator for the M, qcase, as in this situation the density shape is very rough. These areillustrated in Figures 9-10.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 37 / 47

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

xden

sity

(a) n = 20,m = 7, q = 0.85

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

x

density

(b) n = 20,m = 7, q = 0.85

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

x

density

(c) n = 50,m = 13, q = 0.96

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.5

1.0

1.5

2.0

x

density

(d) n = 50,m = 13, q = 0.96

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

x

density

(e) n = 100,m = 22, q = 0.99

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

2.0

x

density

(f) n = 100,m = 22, q = 0.99

Figure 11: Comparison of Bernstein and q−Bernstein PDF estimator withq ∈ [0.85, 1] for Beta(3, 3)(left) and Beta(2, 4)(right)

18

Figure: 11. Beta(3, 3) and Beta(2, 4) density

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 38 / 47

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

2.02.5

xcdf

(a) n = 20,m = 7, q = 0.95

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

2.02.5

x

cdf

(b) n = 50,m = 13, q = 0.99

0.0 0.2 0.4 0.6 0.8 1.0

0.00.5

1.01.5

2.02.5

x

cdf

(c) n = 100,m = 22, q = 1

Figure 12: Comparison of Bernstein and q−Bernstein PDF estimator withq ∈ [0.85, 1] for 0.65 · Beta(2, 10) + 0.35 · Beta(10, 2)

Figure: 12. Optimum m, q for the mixture density

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 39 / 47

Average Squared Error Comparison

The following tables display the average squared errors (ASE) based onB = 40 comparisons, where ASE(g) = 1

B

∑(g(xi)− g(xi))2, where g is

the true distribution or density, and g is its nonparametric estimator.

Distribution Sample size Bernstein q−Bernstein q

20 0.0020684 0.0004371 0.94Beta(3,3) 50 0.0037048 0.0043542 1

100 0.0003380 0.0001946 1

20 0.0036687 0.0021716 0.96Beta(2,4) 50 0.0049030 0.0041611 0.99

100 0.0013453 0.0006329 1

20 0.0014412 0.0036309 0.99Mixture 50 0.0012856 0.0032519 1

100 0.0041389 0.0046857 1

Table: Average Square Error for Bernstein and q−Bernstein CDF estimator withoptimized q

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 40 / 47

Average Squared Error Comparison

Distribution Sample size Bernstein q−Bernstein q

20 0.1388213 0.0760121 0.85Beta(3,3) 50 0.0278798 0.0669346 0.96

100 0.0182203 0.0185668 0.99

20 0.1669993 0.1340892 0.85Beta(2,4) 50 0.0488355 0.0780358 0.96

100 0.0221844 0.0219737 0.99

20 0.4132526 0.5370016 0.95Mixture 50 0.1991615 0.2336240 0.99

100 0.2113997 0.1938492 1

Table: Average Square Error for Bernstein and q−Bernstein PDF estimator withoptimized q

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 41 / 47

By looking at the figures, we see that as the sample size increase, the ASEhas a decreasing trend. The distribution function estimator may havesmaller errors specially for small sample sizes. For density estimators, q−Bernstein polynomial may have slight advantage over the Bernsteinpolynomial that too for small samples and unimodal distributions. Overall,we can say that the classical Bernstein polynomial estimator for estimatingdensity may still be considered superior over the general q− Bernsteinpolynomial estimator, however, the q−Bernstein polynomial estimator forthe distribution function might be potentially better. This requires furtherinvestigation, especially in the manner in which we have selected M. Thetypical question is that of finding optimal q when m =∞?

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 42 / 47

References

1 Babu, G.J., Canty, A.J. and Chaubey, Y.P. (2002). Application ofBernstein polynomials for smooth estimation of a distribution anddensity function. Journal of Statistical Planning and Inference 105,377-392.

2 Bernstein, S.N. (1912). Demonstration du theoreme de Weierstrass,fondee sur le calcul des probabilites. Commun. Soc. Math. Kharkov,No.1. Series XIII, 1–2.

3 Bowman, A.W. (1984). An alternative method of cross-validation forthe smoothing of density estimates.Biometrika 71, 353-360.

4 Cao, J.D. (1997). A generalization of the Bernstein polynomials.Journal Of Mathematical Analysis And Applications 209, 140–146.

5 Chaubey, Y. P., & Sen, P. K. (2009). On the selection of thesmoothing parameter in poisson smoothing of histogram estimator:Computational aspects. Pakistan Journal of Statistics 25, 385-401.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 43 / 47

References

6 Gawronski,W. and StadtMuller, U. (1981). Smoothing histograms bymeans of lattice- and continuous distributions. Metrika 28, 155164.

7 Ghosal, S. (2001). Convergence rates for density estimation withbernstein polynomials. The Annals of Statistics 29, 1264–1280.

8 Hardle, W. (1991). Smoothing Techniques with Implementation in S.Springer-Verlag: New York.

9 Igarashi, G. and Yoshihide Kakizawa, Y. (2014). On improvingconvergence rate of Bernstein polynomial density estimator. Journalof Nonparametric Statistics 26, 6184.

10 Il’inskii, A. and Ostrovska, S. (2002). Convergence of generalizedBernstein polynomials. Journal of Approximation Theory 116,100–112.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 44 / 47

References

11 Kakizawa, Y. (2004). Bernstein polynomial probability densityestimation. Journal of Nonparametric Statistics 16, 709–729.

12 Kakizawa, Y. (2011). A note on generalized bernstein polynomialdensity estimators. Statistical Methodology 8, 136153.

13 Leblanc, A. (2009). ChungSmirnov property for Bernstein estimatorsof distribution functions. Journal of Nonparametric Statistics 21,133142.

14 Leblanc, A. (2012). On estimating distribution functions usingBernstein polynomials. Ann Inst Stat Math 64, 919943.

15 Lorentz, G. G. (1986). Bernstein Polynomials (2nd ed.) New York:Chelsea Publishing.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 45 / 47

References

16 Phillips, G. M. (1997). Bernstein polynomials based on the q-integers.The heritage of P. L. Chebyshev: a Festschrift in honor of the 70thbirthday of T. J. Rivlin. Ann. Numer. Math. 14, 511–518.

17 Prakasa Rao, B. L. S. (2005). Estimation of distribution and densityfunctions by generalized Bernstein polynomials. Indian Journal ofPure and Applied Mathematics 36, 63–88.

18 Silverman, B. W. (1986). Density estimation for statistics and dataanalysis, Chapman and Hall, London.

19 StadtMuller, U. (1983). Asymptotic distributions of smoothedhistograms. Metrika 30, 145–158.

20 Vitale, R.A. (1973). A Bernstein polynomial approach to densityestimation. Comm. Statist. 2, 493–506.

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 46 / 47

Talk slides will be available on SlideShare:www.slideshare.net/ychaubey/talk-slides-msast2016-70014046

THANKS!!

Yogendra Chaubey (Concordia University) Department of Mathematics & Statistics December 21-23, 2016 47 / 47