Information Bias and Adjusted Profile Likelihood

Thomas J. DiCiccio, Michael A. MartinSteven E. Stern and G. Alastair Young

Technical Report No. 429June 1993

Assumptions:Let having probability distribution that depends on unknown parameters and Let denote the log likelihood function for based on.

Suppose is partitioned in the form of where is a -dimensional and is a nuance parameter.

Let be the overall maximum likelihood estimator of

And let be the constrained maximum likelihood function of given.

Inference for is often based on the profile log likelihood function

Example:

is maximize at and the profile likelihood ratio statistic is

is typically distributed at to error of order .

Likelihood Ratio test: approximately

R= no-interaction Model (or reduced model)F= interaction Model (or full model) 

The profile score statistic has a bias

(1)

and information

(2)

Typically they do not vanish but are of order.

Several authors including Cox (1979) have proposed additive adjustment

to the profile score statistic that reduces its bias to order .

Many proposed adjustments pertain to the case where is scalar, and the adjustment are usually obtained by replacing with an objective function of the form (3)

II - Calculation Of Information Bias

Necessary notation: Indices

a, b, c, … range over 1,…, p;

i, j, k, … range over p+1, …, p + q ;

r, s, t, …. range over 1, …., p + q ;

Differentiation is denoted by scripts such that

;

Also,

;

Let

;

All these quantity are of order

 

Finally, let be the matrix inverse of

and let be the inverse matrix of ,

the upper left hand submatrix of .

Set .

The Entries matrix are all 0, expect for the lower right-hand submatrix which is the inverse matrix of

The bias of the profile score statistic usually takes into account additive adjustment to , yielding

(4)

 

Where the quantities are order.

Define the expectation ;

Put ;

It is assumed that , , are of order .

In case, the derivative and do not necessarily coincide for and the existence of the function satisfying is not guarantee

Major Results

 

(5)

Then is of order.

 

Suppose that adjustment terms satisfy (5);

Then we consider another APSS:

(6)

is the Kroenke’s delta.

(10)

III - Information Bias for Specific Adjustments

Information Bias of APSS is calculated for additive adjustment proposed by Cox and Rei (1987).

When is scalar and orthogonal to the nuisance parameters, Cox and Reid (1987)profile log likelihood function from (3)

(13)

is a matrix of second order partial derivative of take with respect to the nuisance parameter.

Thus the infomation bias is given by

(14)

Profile Likelihood (Maximizer likelihood) - Revisited

Let θ be a vector parameter that can be decomposed as θ = (δ, ξ), where δ is a vector parameter of interest and ξ is a nuisance vector parameter. This is, you are interested only on some entries of the parameter θ. Then, the likelihood function can be written as 

Where f is the sampling model. An example of this is the case where f is a normal density, y consist of n independent observations, and say that you are interested on solely, then μ is a nuisance parameter.The profile likelihood of the parameter of interest is defined as

Sometimes you are also interested on a normalized version of the profile likelihood which is obtained by dividing this expression by the likelihood evaluated at the maximum likelihood estimator.

Questions