bayesian inferences for two parameter weibull distribution · bayesian inferences for two parameter...

International Journal of Mathematics and Statistics Studies

Vol.4, No.3, pp.25-39, June 2016

___Published by European Centre for Research Training and Development UK (www.eajournals.org)

25

BAYESIAN INFERENCES FOR TWO PARAMETER WEIBULL DISTRIBUTION

Kipkoech W. Cheruiyot1, Abel Ouko2 and Emily Kirimi3

1Maasai Mara University, Kenya, Department of Mathematics and Physical Sciences 2The East African University, Kenya, Department of Mathematics

3Technical University of Kenya, Department of Mathematics and Statistics

ABSTRACT: In this paper, Bayesian estimation using diffuse (vague) priors is carried out for

the parameters of a two parameter Weibull distribution. Expressions for the marginal posterior

densities in this case are not available in closed form. Approximate Bayesian methods based

on Lindley (1980) formula and Tierney and Kadane (1986) Laplace approach are used to

obtain expressions for posterior densities. A comparison based on posterior and asymptotic

variances is done using simulated data. The results obtained indicate that, the posterior

variances for scale parameter obtained by Laplace method are smaller than both the

Lindley approximation and asymptotic variances of their MLE counterparts.

KEYWORDS: Weibull Distribution, Lindley Approximation, Laplace Approximation,

Maximum Likelihood Estimates

INTRODUCTION

The Weibull distribution is one of the most widely used distributions in reliability and survival

analysis because of various shapes assumed by the probability density functions (p.d.f) and the

hazard function. The Weibull distribution has been used effectively in analyzing lifetime data

particularly when the data are censored which is very common in survival data and life testing

experiments. The Weibull distribution was derived from the problem of material strength and

it has been widely used as a lifetime model. Weibull distribution corresponds to a family of

distribution that covers a wide range of distributions that goes from the normal model to the

exponential model making it applicable in different areas for instance in fatigue life, strength

of materials, genetic research, quality control and reliability analysis. The probability density

function (p.d.f) for the two parameter Weibull distribution is given by

1

( , , ) expT

x xf x

0x

(1.1)

Likelihood Based Estimation of Parameters of Weibull Distribution.

The Maximum Likelihood Estimation

Let 1 2, ,..., nX X X be independent random samples of size n from Weibull distribution with the

p.d.f given by (1.1). Differentiating with respect to and and equating to zero we obtain

and

0

dl

d

1

ln ln( ) ( ) ln( ) 0i i

n nx x

i

i i

nn x

(2.1)

http://www.eajournals.org/


Vol.4, No.3, pp.25-39, June 2016


26

0

dl

d

1

( ) 0i

nx

i

n

(2.2)

From (2.2) we obtained the maximum likelihood estimates of as

1

1 n

iin

x

(2.3)

Substituting (2.3) into (2.1), yields an expression in terms of

only as given by

1

1

1

ln1 1

ln

n

i i ni

ini

i

i

x x

xn

x

(2.4)

The maximum likelihood estimate for

is obtained from (2.4) with the aid of standard iterative

procedures.

Variance and Covariance Estimates

The asymptotic variance-covariance matrix of

and

are obtained by inverting information

matrix with elements that are negatives of expected values of second order derivatives of

logarithms of the likelihood functions. Cohen (1965) suggested that in the present situation it

is appropriate to approximate the expected values by their maximum likelihood estimates.

Accordingly, we have as the approximate variance-covariance matrix with elements

12 2

2

2 2

2

var( ) cov( )

cov( ) var( )

l l

l l

When and are independent, the covariance of the above matrix is zero.

When is known the asymptotic variances for

is obtained by

2

2

ˆvar( )

ˆn

Bayesian Estimation of Parameters of Weibull Distribution

Bayesian Approach.



Vol.4, No.3, pp.25-39, June 2016


27

Let 1 2, ,..., nX X X be a random sample from a population with a two parameter Weibull

distribution given by

( / , )f x =

1( )x

exp {-( ) }x

(3.1)

The likelihood function is the given by

1

11

( , ; ) exp{ ( ) }i

n nxn n

i

ii

L x x

The log likelihood function is given by

1 1

log log log ( 1) log ( )i

n nx

i

i i

l L n n x

Suppose that we are ignorant about the parameters ( , ) so that the diffuse (vague) prior

used is

1( , )

(3.2)

The joint posterior distribution is then given by

( , / ) ( / ) ( , )g x L x

11

1( , / ) exp{ ( ) }i

n nxn n

ii

g x

(3.3)

The marginal p.d.f of x is given by

1

0 011

( ) exp{ ( ) }i

n nxn n

ii

f x d d

(3.4)

Similarly, the marginal posterior p.d.f’s of and are required in order to compute the

corresponding posterior expectations of and as

( / ) ( )

( / ) ( / )( / ) ( )

L x

E x g xL x

(3.5)

and

( / ) ( )

( / ) ( / )( / ) ( )

L x

E x g xL x

(3.6)



Vol.4, No.3, pp.25-39, June 2016


28

respectively

Since the Bayes estimate for and involve evaluating ratios of two mathematically

intractable integrals, appropriate Bayesian approximations are applied. Assuming is known,

the likelihood function for is given by

1

( / ) exp{ ( ) }i

nxn

i

L x

(3.7)

The log likelihood function for is given by

1

log log ( )i

nx

i

l L n

Since is known, the prior density for is given by

1( )

The posterior density for is given by

( / ) ( / ) ( )g x L x

1

1( / ) ( ) exp{ }

nn

i

i

g x x

Therefore the posterior expectation for is obtained by

( / ) ( / )E x g x d

( / ) ( )

( / )( / ) ( )

L x

E xL x

(3.8)

Laplace Approximation

Since the Bayes estimate of involve ratio of two mathematically intractable integrals,

Tierney and Kadane, (1986) proposed to estimate (3.8) as follows

( / )

nL

nL

e dE x

e d

(3.9)

where

log ( )

lL

n



Vol.4, No.3, pp.25-39, June 2016


29

1

1

1log( ) log ( )i

nx

n

i

(3.10)

and

log ( ) logl

Ln

1

1

1log log( ) log ( )i

nx

n

i

(3.11)

The posterior mode

of L is obtained by differentiating L with respect to once and equating

to zero, that is,

0

dL

d

1

1( ) 0i

nx

in

Giving

in terms of as

1

1(1 )

n

i

i

xn

(3.12)

The posterior mode (local maximum) for L is obtained by differentiating L with respect to

and equating to zero to get

1

1

1log log( ) log ( )i

nx

n

i

1

( ) 0i

nx

in

Giving

in terms of as

1

1

1 n

i

i

xn

(3.13)

The 2 and

2 are equal to the minus the inverse of the second derivative of the log posterior

density at its mode given by



Vol.4, No.3, pp.25-39, June 2016


30

2

"

1

( )L

2

2 2 21

1 ( 1)( )i

nx

i

d L

d n

"

2 21

1 ( 1)( ) ( )i

nx

i

Ln

Hence

2

( 1)

12

1( 1)

n

in

i

x

(3.14)

Also

2

1

2( 2)

( 1)n

i

i

n xd L

dn

2

1"

1

( )L

2

12

1 2

1 n

i

i

xn

(3.15)

Thus, the Laplace approximation of (3.8) is given by

( / ) ( )exp{ ( ( ) ( ))}E x n L L

(3.16)

where

1

1(1 )

n

i

i

xn

and

1

1

1 n

i

i

xn



Vol.4, No.3, pp.25-39, June 2016


31

2

2

( / ) ( )

( / )( / ) ( )

L x

E xL x

(3.17)

2( / )

nL

nL

e dE x

e d

where

1

1 1log( ) log ( )i

nx

i

Ln

and

2

1

1 1log log( ) log ( )i

nx

i

Ln

The posterior mode of L and L are given by

1

1

1(1 )

n

i

i

xn

(3.18)

and

1

1

1( 1)

n

i

i

xn

(3.19)

respectively

The 2

1 and 2

1

of L and L respectively are given by

2

1"

1

( )L

"

2 21

1 ( 1)( ) ( )i

nx

i

Ln

Hence

2

( 1)

12

1( 1)

n

in

i

x

(3.20)



Vol.4, No.3, pp.25-39, June 2016


32

and

2

1"

1

1

( )L

"

1 21

1 ( 1)( )i

nx

i

Ln

Hence

2

12

1

( 1)

( 1)

n

i

i

xn

(3.21)

Thus, the Laplace approximation of (3.17) is

2

2 1 1( / ) ( )exp{ ( ( ) ( ))}E x n L L

(3.22)

Hence the posterior variance of is

2 2( / ) ( / ) [ ( / )]V x E x E x

(3.23)

Lindley (1980) Approximation

Lindley (1980) developed a multidimensional linear Bayes estimate of an arbitrary function as

an approximation of an asymptotic expansion of the ratio of two integrals which cannot be

expressed in a closed form given by

( ) ( )exp ( )

( ) /( )exp ( )

U V L d

E U xV L d

Expanding ( )L and ( ) ( )U V by Taylor’s series about

the MLE of , Lindley (1980)

Bayesian approximation of two parameter case is given by

21 1( ) / ( 2 ) of order n and smaller.

2 2ij j j ijkl ijkl

i j i j k l

E U x U U U L U terms

11 11 12 12 21 21 22 22 1 1 11 2 21 2 2 22 1 12

2 2 2

30 1 11 2 11 12 21 1 11 12 2 11 22 12 12 2 22 21 1 22 11 21

2

03 1 12 22 2 22

1

2

( ) [3 ( 2 )] [3 ( 2 )]1

2 ( )

U U U U U U U U U

L U U L U U L U U

L U U



Vol.4, No.3, pp.25-39, June 2016


33

(3.24)

all evaluated at MLE of (see, Lindley, 1980). For two parameter Weibull distribution we

have,

The MLE of ( , ) is ( , )

( ) log ( ) log ( )V

i j

ij i j

LL

2

; and Ui j ij

U U UU U

( , )th

ij i j element in the inverse of matrix { }ijL evaluated at ( , ), , 1,2i j

The quantities 'ijL s are the higher order derivatives of log-likelihood function given by

10

1

( )i

nx

i

l nL

(3.25)

2

2( 1)

202 21

( )i

nx

i

l nL

(3.26)

3

3( 1)( 2)

303 31

2( )i

nx

i

l nL

(3.27)

2

111

1 1

( ) ln( ) ( )i i i

n nx x x

i i

l nL

(3.28)

2 2 2

3( 1) ( 1)

212 21 1 1

( ) ( ) ln( ) ( )i i i i

n n nx x x x

i i i

l nL

(3.29)

2

2 2 2 2

42 2

4 22222 2

1 1 1 1 1

( ) ( ) ln( ) ( ) ln( ) ( ) ln( ) ( ) ln( )i i i i i i i i i

n n n n nx x x x x x x x x

i i i i i

lL

(3.30)

01ln ln( ) ( ) ln( )i i

n nx x

i

i i

l nn x L

(3.31)



Vol.4, No.3, pp.25-39, June 2016


34

22

022 21

( ) ln( )i i

nx x

i

l nL

(3.32)

33

033 31

2ln( )i i

nx x

i

l nL

(3.33)

32

2122

1 1

( ) ln( ) ( ) ln( )i i i i

n nx x x x

i i

lL

(3.34)

Fisher’s information matrix is given by

20 11

11 02

L LI

L L

Thus

20 111

11 0220 02 11 11

1 L LI

L LL L L L

11 12

21 22

(3.35)

where

2

2

( 1) 2 120 02 11 11 2 2

1 1 1 1

( ) (ln( )) ( ) ln( ) ( )i i i i i i

n n n nx x x x x x

i i i i

n n nL L L L

Next, we have

log ( ) ln( )

Hence

1 2

1 1 and

Since is known, we let

U

such that

1 21 ; 0 and 0 ; i,j=1,2ijU U U

The quantities 'ijL s are the higher order derivatives of log-likelihood function. Because is

known, the following derivatives are used to obtain Bayes estimates of

3

3( 1)( 2)

303 31

2( )i

nx

i

l nL



Vol.4, No.3, pp.25-39, June 2016


35

( , )th

ij i j is obtained by inverting minus second derivative of log likelihood function with

respect to evaluated at ( ), 1,2i

Therefore, Bayes estimate of using (3.24) is given by

211

30 11

1( / )

2E x L

(3.36)

Then

2 2 21130 11 11( / ) 2E x L

(3.37)

Hence the posterior variance for is given by

22( / ) / ( / )Var x E x E x (3.38)

RESULTS

Simulation experiments were carried out using R software to compare the performance of the

Bayes and MLE estimates of the two parameter Weibull distribution. We assumed the shape

parameter is known. We performed simulation experiment with different sample sizes (10,

30, 50 and 100), drawn from a Weibull distribution for different values of shape parameter

(1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5 and 6). We specified the true value of the scale parameter to

be fixed at one ( 1) for all the sample sizes. We computed estimates based on two different

Bayesian methods, that is, Tierney and Kadane, (1986) Laplace approximation method,

Lindley (1980) approximation and the method of Maximum likelihood estimation.



Vol.4, No.3, pp.25-39, June 2016


36

Figure 1: Graph of Estimates of against Lambda ( ) for a Sample of n=10 for affixed

1

Figure 2: Graph of Estimates of against Lambda ( ) for a sample of n=100 for affixed

1

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6

Esti

mat

es

Lambda

Estimates of Alpha against Lambda

MLE

LAPLACE

LINDLEY

0

0.2

0.4

0.6

0.8

1

1.2

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6

Esti

mat

es

Lambda

Estimates of Alpha against Lambda

MLE

LAPLACE

LINDLEY



Vol.4, No.3, pp.25-39, June 2016


37

Figure 3: Graph of variance for sample size 10 against values of Lambda ( ).

Figure 4: Graph of variance for sample size 100 against values of Lambda ( ).



Vol.4, No.3, pp.25-39, June 2016


38

DISCUSSIONS

Figure 1 show the estimates of under varying size of , Bayes estimates obtained by Lindley

(1980) and Tierney and Kadane, (1986) Laplace approximation are found to be larger than the

MLE counter parts. Lindley (1980) approximation is found to overestimate the scale parameter

; however the three methods demonstrate the tendency for their estimates and variances to

perform better for larger sample size. This is shown in Figure 2 when the sample size is

increased to 100. As the sample size increases the MLE and Bayes estimates becomes more

consistent and accurate.

Figure 3 show the variances of the estimates of for varying size of shape parameter for a

sample of 10. It is observed that posterior variances of estimates of are smaller than the

asymptotic variances of MLE hence more precise and accurate. However, both variances tend

to converge to zero as the value of get larger. For the sample of 10 Tierney and Kadane,

(1986) Laplace approximation is seen to perform slightly better than Lindley (1980), since it

stabilizes faster.

Figure 4 show that Tierney and Kadane, (1986) Laplace approximation performed better for

larger sample of 100. The variance of MLE are observed to stabilize faster when the sample

size and shape parameter increase.

It is also observed that the Bayesian method generally performed better for both small and

larger value of than the MLE counter parts. Lindley (1980) and Laplace methods tend to

perform almost similarly for smaller . But the Tierney and Kadane, (1986) Laplace

approximation is found to produce better results than both MLE and Lindley (1980) for larger

and larger samples sizes.

CONCLUSION

We have shown Bayesian techniques for estimating the scale parameter of the two parameter

Weibull distribution which produces estimates with smaller variances than the MLE. Tierney

and Kadane, (1986) Laplace approximation which requires the second derivatives in its

computation is found to be more accurate than the Lindley (1980) which requires third

derivatives in its computation. This is in line with Tierney et al (1989) findings, that Laplace

method is more accurate than the third derivative method of Lindley (1980). Even though the

two Bayesian methods are better than the MLE counter parts, they have their own limitations.

Lindley (1980) approximation requires existence of MLE in its computation. This appears as

if it is an adjustment to the MLE to reduce variability. On the other hand, Laplace

approximation requires existence of a unimodal distribution in its computation, hence difficult

to use in cases of a multi modal distribution.

RECOMMENDATION

In this study, it is noted that the posterior variances of Bayes estimates are smaller than

asymptotic variances. Comparing the two Bayesian methods, Tierney and Kadane, (1986)

Laplace approximation method has smaller variance than the Lindley (1980) approximation

technique hence more precise and accurate. Laplace approximation does not require explicit



Vol.4, No.3, pp.25-39, June 2016


39

third order derivatives in its computation which are required in Lindley (1980) approximation

method hence simple to compute. We therefore recommend further work to be done on two

parameter Weibull distribution when both scale and shape parameters are unknown to

investigate accuracy of the two Bayesian methods.

REFERENCES

Cohen, A.C. (1965): Maximum Likelihood Estimation in the Weibull Distribution Based on

Complete and Censored Samples. Technometrics, 7, 579-588.

Lindley, D.V. (1980). Approximate Bayesian Method, Trabajos Estadistica, 31, 223-237.

Tierney L, Kass, R.E. and Kadane, J.B. (1989): Fully exponential Laplace approximations to

expectations and variances of non-positive functions. Journal of American Statistical

Association, 84, 710-716.

Tierney L. and Kadane, J.B. (1986): Accurate Approximations for Posterior Moments and

Marginal Densities. Journal of American Statistical Association, 81, 82-86.


bayesian inferences for two parameter weibull distribution · bayesian inferences for two parameter...

Documents