euclid.ss.1009211804

Statistical Science1999, Vol. 14, No. 1, 1–28

Integrated Likelihood Methods forEliminating Nuisance ParametersJames O. Berger, Brunero Liseo and Robert L. Wolpert

Abstract. Elimination of nuisance parameters is a central problem instatistical inference and has been formally studied in virtually all ap-proaches to inference. Perhaps the least studied approach is eliminationof nuisance parameters through integration, in the sense that this isviewed as an almost incidental byproduct of Bayesian analysis and ishence not something which is deemed to require separate study. Thereis, however, considerable value in considering integrated likelihood onits own, especially versions arising from default or noninformative pri-ors. In this paper, we review such common integrated likelihoods anddiscuss their strengths and weaknesses relative to other methods.

Key words and phrases: Marginal likelihood, nuisance parameters, pro-file likelihood, reference priors.

1. INTRODUCTION

1.1 Preliminaries and Notation

In elementary statistical problems, we try tomake inferences about an unknown state of natureω (assumed to lie within some set � of possiblestates of nature) upon observing the value X = xof some random vector X = �X1; : : : ;Xn� whoseprobability distribution is determined completelyby ω. If X has a density function f�x�ω�, strongarguments reviewed and extended in Berger andWolpert (1988) and Bjørnstad (1996) suggest thatinference about ω ought to depend upon X onlythrough the likelihood function L�ω� = f�x�ω�which is to be viewed as a function of ω for thegiven data x.

James O. Berger is Arts and Sciences Professor andRobert L. Wolpert is Associate Professor at the Insti-tute of Statistics and Decision Sciences, Duke Uni-versity, Box 90251, Durham, North Carolina 27708-0251 �e-mail: [email protected] and [email protected]�. Brunero Liseo is Associate Professorat the Dip. di Studi Geoeconomici, Statistici e Storiciper l’Analisi, Regionale, Universita di Roma “LaSapienza” �e-mail: [email protected]).

Rarely is the entire parameter ω of interest to theanalyst. It is common to select a parameterizationω = �θ; λ� of the statistical model in a way that sim-plifies the study of the “parameter of interest,” heredenoted θ, while collecting any remaining parame-ter specification into a “nuisance parameter” λ.

In this paper we review certain of the ways thathave been used or proposed for eliminating the nui-sance parameter λ from the analysis to achievesome sort of “likelihood” L∗�θ� for the parameterof interest. We will focus on integration methods,such as eliminating λ by simple integration (withrespect to Lebesgue measure), resulting in theuniform-integrated likelihood

�1� LU�θ� =∫L�θ; λ�dλ:

In justifying integration methods, we will occasion-ally refer to alternative maximization methods, suchas the profile likelihood

�2� L�θ� = supλL�θ; λ�:

(Typically the sup over λ is achieved at some valueλθ, which we will call the conditional mle.) However,no systematic discussion of nonintegration methodswill be be attempted.

1

2 J. O. BERGER, B. LISEO AND R. L. WOLPERT

Example 1. Suppose X1;X2; : : : ;Xn are i.i.d.normal random variables with mean µ and vari-ance σ2 �N�µ;σ2��. Suppose the parameter ofinterest is σ2 while µ is a nuisance parameter.(Thus, in the above notation, θ = σ2 and λ = µ.)Here, easy computations yield

LU�σ2� =∫L�σ2; µ�dµ

=∫ n∏i=1

1√2πσ2

exp{− 1

2σ2�xi − µ�2

}dµ

= 1�2πσ2��n−1�/2√n exp

{− 1

2σ2

n∑i=1

�xi− x�2};

L�σ2� = supµL�σ2; µ� = L�σ2; x�

= 1�2πσ2�n/2 exp

{− 1

2σ2

n∑i=1

�xi − x�2}y

note that x is the conditional mle. Since proportion-ality constants do not matter for likelihoods, LU�σ2�and L�σ2� differ only in the powers of σ2 in the de-nominators.

Notationally, we will write integrated likeli-hoods as

�3� L�θ� =∫L�θ; λ�π�λ � θ�dλ;

where π�λ�θ� is the “weight function” for λ. (In thispaper, we consider only examples with continuous λtaking values in Euclidean space, and consider onlyintegration with respect to Lebesgue measure.) Itis most natural to use Bayesian language, and callπ�λ�θ� the “conditional prior density of λ given θ,”although much of the paper will focus on nonsub-jective choices of π�λ�θ�. Also, we will have occasionto refer to a prior density, π�θ�, for the parameterof interest θ. As is commonly done, we will abusenotation by letting the arguments define the prior;thus π�θ� is the prior for θ, while π�λ� would be the(marginal) prior for λ.

1.2 Background and Preview

The elimination of nuisance parameters is a cen-tral but difficult problem in statistical inference. Ithas formally been addressed only in this centurysince, in the nineteenth century Bayes–Laplaceschool of “Inverse Probability” (see, e.g., Zabell,1989), the problem was not of particular concern;use of the uniform integrated likelihood LU�θ� wasconsidered “obvious.”

With the Fisher and Neyman rejection of theBayes–Laplace school, finding alternative waysto eliminate nuisance parameters was felt to be

crucial. Student’s derivation of the sampling distri-bution of the mean of a normal population whenthe variance is unknown and the derivation of thedistribution of the sample correlation coefficientof a bivariate normal population (Fisher, 1915,1921), are probably the first examples of a frequen-tist approach to the problem. Both were based onderivation of a pivotal quantity whose distributionis free of the nuisance parameters. Other famousexamples include the Bartlett (1937) test of homo-geneity of variances and the various solutions ofthe Behrens–Fisher problem (Fisher, 1935).

There have been numerous efforts to create a like-lihood approach to elimination of nuisance param-eters (Barnard, Jenkins and Winston, 1962). Thebeginnings of the “modern” likelihood school canperhaps be traced to Kalbfleisch and Sprott (1970,1974), who proposed systematic study of a varietyof methods for eliminating nuisance parameters (in-cluding integrated likelihood) and opened the way toa rich field of research.

Probably the simplest likelihood approach toeliminating nuisance parameters is to replace themwith their conditional maximum likelihood esti-mates, leading to the profile likelihood in (2); thiscan then be used as an ordinary likelihood. Manyexamples of misleading behavior of the profile like-lihood (Neyman and Scott, 1948; Cruddas, Cox andReid, 1989) have given rise to various “corrections”of the profile, which aim to account for the “error”in simply replacing λ by a point estimate. Amongthe advances in this area are the modified pro-file likelihood (Barndorff-Nielsen, 1983, 1988) andthe conditional profile likelihood (Cox and Reid,1987). These methods were primarily developed toprovide higher-order asymptotic approximations to(conditional) sampling distributions of statistics ofinterest, such as the maximum likelihood estima-tor or the likelihood ratio. As a by-product, theseapproximate distributions can be interpreted aslikelihood functions for the parameter of interestand/or used in a frequentist spirit via tail area ap-proximations. However, use of these methods tendsto be restricted to rather special frameworks (e.g.,exponential families or transformation groups). Ex-cellent references include Reid (1995, 1996), Fraserand Reid (1989); see also Sweeting (1995a, b, 1996)for a Bayesian version of these approaches. A dif-ferent way to adjust the profile likelihood, based onthe properties of the score function, is developed inMcCullagh and Tibshirani (1990).

Other likelihood approaches arise when one ormore components of the sufficient statistics havemarginal or conditional distributions which dependon θ, but not on λ. In these cases, such distributions

ELIMINATING NUISANCE PARAMETERS 3

are often used as the likelihood for θ, and are calledthe “marginal likelihood” or the “conditional likeli-hood.” Basu (1977) gives an interesting example ofconflicting marginal and conditional likelihoods forthe same problem, indicating that use of these tech-niques is likely to remain somewhat arbitrary.

Marginal and conditional likelihoods are specialcases of the more general partial likelihood (Cox,1975). If there exists a partition �y; z� of the datax, such that

�4� f�x � θ; λ� = h�x� f1�y � θ; λ�f2�z � y; θ�or

�5� f�x � θ; λ� = h�x� f1�y � θ�f2�z � y; θ; λ�;then the terms hf1 in (4) or hf2 in (5) are ignored,and the remaining factor is taken as the partial like-lihood for θ. Since the ignored term does depend onθ, there is some loss of information. Supporters ofthe approach suggest, however, that one loses onlythe information about θ which is inextricably tiedwith the unknown parameter λ. Basu (Ghosh, 1988,page 319) criticizes the idea of partial likelihood onthe ground that it usually cannot be interpreted interms of sampling distributions, and one is left onlywith the possibility of exploring the shape of theparticular observed partial likelihood.

From a subjective Bayesian point of view, theproblem has a trivial solution: simply integratethe joint posterior with respect to the nuisance pa-rameters and work with the resulting marginalposterior distribution of θ. From a philosophicalor foundational level, it would be difficult to addmuch to the fascinating articles (Basu, 1975, 1977)comparing subjective Bayesian and likelihood meth-ods for elimination of nuisance parameters. Thereis, however, considerable resistance to generalimplementation of subjective Bayesian analysis,centering around the fact that elicitation of a sub-jective prior distribution for multiple parameterscan be quite difficult; this is especially so for nui-sance parameters, whose choice and interpretationare often ambiguous.

Even if one is not willing to entertain subjectiveBayesian analysis, we feel that use of integratedlikelihood is to be encouraged. The integration mustthen be with respect to default or noninformativepriors. Our main goal will thus be to discuss and il-lustrate integrated likelihood methods based on dif-ferent choices of conditional noninformative priorsfor the nuisance parameters.

In Section 2, we argue in favor of the use of inte-grated likelihood on the grounds of simplicity, gener-ality, sensitivity and precision. In Section 3, we willillustrate the uses and interpretations of integrated

likelihood in various conditional approaches to in-ference, ranging from the pure likelihood to the fullyBayesian viewpoints. Section 4 reviews the variousintegrated likelihoods that have been considered;these primarily arise from different definitions ofconditional noninformative priors for the nuisanceparameters. The last section focuses on criticismsand limitations of integration methods.

Throughout the paper, we will repeatedly use sev-eral examples to illustrate and compare differentmethods. Several of these examples (Examples 4and 7, in particular) are simple to state but of nearimpossible difficulty to analyze, in the sense that de-fault methods of any type are questionable. In theseexamples at least, we will thus be asking effectively,“What is the best method of doing the impossible?”While firm conclusions cannot then be forthcoming,we feel that consideration of such extreme examplescan greatly aid intuition.

1.3 Some Philosophical Issues

In this section we discuss several issues that aresomewhat tangential to the main theme, but relateto overall perspective.

1.3.1 What is the likelihood function? Bayarri,DeGroot and Kadane (1988) argued that there canbe no unique definition of a “likelihood function,”rejecting as incomplete the usual (and usuallyvague) definitions such as the one Savage (1976)attributes to Fisher: “probability or density of theobservation as a function of the parameter.”

Such definitions give no guidance as to how totreat the value of an unobserved variable (for ex-ample, a future observation z): should we condi-tion on it, as we do for unknown parameters, lead-ing to what Bayarri, DeGroot and Kadane (1988)call the “observed” likelihood Lobs�θ; z� ≡ f�x�θ; z�?Or should we treat it like other observations, lead-ing to the “random variable” likelihood Lrv�θ; z� ≡f�x; z�θ�? They argue that likelihood-based infer-ence will depend on this choice and offer examplesillustrating that either choice may be the best onein different examples.

Here, we avoid this difficulty by assuming thatthe problem begins with a specified likelihood func-tion of the form

f�x; θ∗; λ∗ � θ; λ�;where (as before) x is the observed value of the datavector X, and θ∗ and λ∗ are unobserved variables ofinterest and nuisance variables, respectively, havingprobability distributions as specified by f. The vec-tor θ∗ could include a future observation z or some“random effects” that are of interest, for example;


thus any variables with known (conditional) distri-bution are put to the left of the bar, while thosewithout given distributions are placed on the right.Following Butler (1988) we recommend immediateremoval (by integration) of the nuisance variable λ∗,passing to

�6� f�x; θ∗ � θ; λ� =∫f�x; θ∗; λ∗ � θ; λ�dλ∗y

this then would be the likelihood we study, seekingto eliminate λ. [In most of our examples “θ∗” willbe absent and so we will just write f�x�θ; λ�.] Thiselimination of λ∗ by integration is noncontroversial,and should be acceptable to most statistical schools.See Bjørnstad (1996) for additional discussion.

Example 2. Random effects. Let Xi be indepen-dent N�µi;1� random variables, with unobservedmeans µi drawn in turn independently from theN�ξ; τ2� distribution; we wish to make inferenceabout θ = �ξ; τ2�, ignoring the nuisance parameterλ∗ = µ = �µ1; : : : ; µp� (the random effects). Here,(6) becomes (with no θ∗ or λ present)

f�x � θ� =∫Rp�2π�−p/2

· exp(−

p∑i=1

�xi − µi�22

)�2πτ2�−p/2

· exp(−

p∑i=1

�µi − ξ�22τ2

)dµ1 · · ·dµp

=(2π�1+ τ2�

)−p/2 exp(−

p∑i=1

�xi − ξ�22�1+ τ2�

)

∝ �1+ τ2�−p/2

· exp(− p �s2 + �x− ξ�2�

/2�1+ τ2�

);

(7)

where x is the sample mean and s2 =∑�xi−x�2/p.Again, most statistical schools would accept elim-

ination of µ by integration here; for instance, theusual alternative of maximizing over the unob-served parameters µi would lead instead to theprofile likelihood

L�θ� = supµ∈Rp

�2π�−p/2

· exp(−

p∑i=1

�xi − µi�22

)�2πτ2�−p/2

· exp(−

p∑i=1

�µi − ξ�22τ2

)

= �4π2τ2�−p/2 exp(−

p∑i=1

�xi − ξ�22�1+ τ2�

)

∝ τ−p exp(− p �s2 + �x− ξ�2�

/2�1+ τ2�

);

(8)

which differs from L�θ� by having a singularity atτ = 0 that strongly (and wrongly) suggests thatany data support an inference that τ2 ≈ 0. In sit-uations such as this it is often suggested that oneuse a local maximum of the likelihood. Interestingly,no local maximum exists if s2 < 4: Even if s2 ≥ 4,the local maximum is an inconsistent estimator ofτ2 as p→∞; for instance, if τ2 = 3, the local max-imum will converge to 1 as p → ∞. Figure 1 indi-cates the considerable difference between the like-lihoods; graphed, for p = 6, s2 = 4 and x = ξ areL�τ2� = f�x�τ2; ξ = x� and L�τ2; x�. Note that wehave “maximized” over ξ to eliminate that parame-ter for display purposes. Had we integrated over ξin f�x�θ�, the difference would have been even morepronounced.

1.3.2 The subjective Bayesian integrated likeli-hood. Since integrated likelihood methods can beviewed in a Bayesian light, it is useful to review thepurist Bayesian position. For subjective Bayesiansthere is no ambiguity concerning how to treat nui-sance parameters: all inference is based on thejoint probability distribution f�x; θ; λ� of all pa-rameters and variables, whether or not observed,which can be constructed from any of the condi-tional data distributions along with appropriatesubjective prior distributions. In problems withoutnuisance parameters, for example, the joint densityis f�x; θ� = f�x�θ�π�θ�, the product of the likeli-hood and the prior distribution π�θ� for θ; in thissetting, Bayes’ theorem is simply the conditionalprobability calculation that the posterior densityfor θ is

�9� π�θ � x� = f�x � θ�π�θ�∫f�x � θ�π�θ�dθ ∝ L�θ�π�θ�:

In the presence of a nuisance parameter λ, a subjec-tive Bayesian will base the analysis on a full priorπB�θ; λ�, which can also be factored as πB�θ; λ� =πB�θ�πB�λ�θ�, the product of the marginal and con-ditional prior densities of θ and λ (given θ), respec-tively. The subjective Bayesian still seeks π�θ�x�and would accept an integrated likelihood L�θ� ifit satisfied π�θ�x� ∝ L�θ�πB�θ�. It is easy to seethat the only L�θ� which satisfies this relationshipis given (up to a multiplicative constant) by

�10� LB�θ� =∫f�x � θ; λ�πB�λ � θ�dλ:

This thus defines the unique integrated likelihoodthat would be acceptable to a subjective Bayesian.

1.3.3 Why eliminate nuisance parameters? First,an explanation of the question: while, almost bydefinition, final inference about θ needs to be free


Fig. 1. Integrated and profile likelihoods for the random effects model.

of λ, it is not clear that one must pass through alikelihood function, L�θ�, that is free of λ. Mostnon-Bayesian analyses pass through some such in-termediary, but Bayesian analyses need not. Forinstance, many Bayesian analyses today proceed byMonte Carlo generation of a sequence of randomvariables �θ�1�; λ�1��; : : : ; �θ�m�; λ�m�� from the fullposterior distribution π�θ; λ�x�; inferences concern-ing θ then follow from direct use of the simulatedvalues θ�1�; : : : ; θ�m� (e.g., the usual estimate ofθ would be the average of these simulated val-ues). There is then no apparent need to explicitlyconsider an L�θ� that is free of λ. (Indeed, it iseven common for Bayesians to introduce artificialnuisance parameters to simplify the Monte Carloprocess.)

There are, nevertheless, several uses of L�θ� inBayesian analysis, which we list here.

1. Scientific reporting. It is usually considered goodform to report separately L�θ� and π�θ�x� (oftengraphically) in order to indicate the effect of theprior distribution. This also allows others to uti-lize their own prior distributions for θ.

2. Sensitivity analysis. It is often important to studysensitivity to π�θ�, and having L�θ� available forthis purpose is valuable. [Of course, sensitivity toπ�λ�θ� is also a potential concern, but frequentlythis is of less importance.]

3. Elicitation cost. It is typically very expensive (interms of time and effort) to obtain subjective

prior distributions. Under the frequent cost limi-tations with which we operate, elicitation effortsmay have to be limited. It is often cost effective toeliminate nuisance parameters in a default fash-ion, resulting in L�θ� and concentrate subjectiveelicitation efforts on π�θ�.

4. Objectivity. Although most statisticians are justi-fiably skeptical of the possibility of truly “objec-tive” analysis (cf. Berger and Berry, 1988), thereis an undeniable need, in some applications, foran analysis which appears objective. Using L�θ�,with default π�λ�θ�, can satisfy this need.

5. Combining likelihoods. If one obtains informa-tion about θ from different independent sources,and the information arrives as likelihoods, Li�θ�;then one can summarize the information by∏iLi�θ�. This is the basis of many important

meta-analysis techniques. One cannot, of course,simply multiply posteriors in this way. (But seeSection 5.2 for cautions concerning multiplica-tion of likelihoods.)

6. Improper priors. Focusing on integrated likeli-hood seems to reduce some of the dangers of us-ing improper priors. This is illustrated in Sec-tion 3.2.

2. ADVANTAGES OF INTEGRATED LIKELIHOOD

Once one departs from the pure subjective Bay-esian position, one cannot argue for integratedlikelihood solely on grounds of rationality or co-


Fig. 2. The likelihood surface for Example 3; when n = 1; x = 1 and y = 0.

herency. Here we present a mix of pragmatic andfoundational arguments in support of integratedlikelihood. Other advantages will be discussed aswe proceed.

2.1 Integration Versus Maximization

Most of the nonintegration methods are based onsome type of maximization over the nuisance pa-rameter. This can be very misleading if the like-lihood has a sharp “ridge,” in that the likelihoodalong this ridge (which would typically be that ob-tained by maximization) may be quite atypical ofthe likelihood elsewhere. Here is a simple example.

Example 3. SupposeX1; : : : ;Xn are i.i.d.N�θ;1�random variables, while Y is (independently)N�λ; exp�−nθ2��. Here θ and λ are unknown, withθ the parameter of interest. The joint density forX = �X1; : : : ;Xn� and Y is

f�x;y � θ; λ�

= �2π�−n/2 exp(−1

2

n∑i=1

�xi − θ�2)

· �2π exp�−nθ2��−1/2 exp(− �y− λ�2

2 exp�−nθ2�

)

= �2π�−�n+1�/2

· exp(−n

2�x2 − 2xθ� − �y− λ�2

2 exp�−nθ2�

);

(11)

where x is the sample mean. For n = 1 and thedata x = 1, y = 0; the overall likelihood L�θ; λ� =f�1;0�θ; λ� is graphed in Figure 2 for θ > 0. Notethe sharp ridge.

The profile likelihood is easy to compute, since theconditional mle for λ is just λθ = y. Thus

�12�L�θ� = f�x;y � θ; λθ�

∝ exp�nxθ�;ignoring factors that do not involve θ. Note that thisis a very strange “likelihood,” rapidly growing to in-finity as θ→∞ or θ→ −∞, depending on the signof x.

In contrast, the uniform integrated likelihood is

�13�LU�θ� =

∫f�x;y � θ; λ�dx

∝ exp(−n

2�x− θ�2

);

which is clearly proportional to a N�x;1/n� dis-tribution, just as if Y with its entirely unknownmean λ had not been observed. These two likeli-hoods would, of course, give completely different im-pressions as to the location of θ.

The integrated likelihood answer can also be pro-duced by a classical conditionalist. One can obtainthe marginal likelihood for X by integrating out Y;the answer is clearly LU�θ�. There would thus belittle disagreement as to the “correct” answer here,but the example does serve to indicate the dangerinherent in maximization.


2.2 Accounting for NuisanceParameter Uncertainty

The profile approach of replacing λ by its condi-tional mle, λθ, would appear to be dangerous in thatit ignores the uncertainty in λ. Example 1 demon-strates a very mild version of this problem; the pro-file likelihood has one more “degree of freedom” thanit should, given the replacement of µ by x. We willmention a more serious standard example of this inSection 3.4, but do not dwell on the issue because itis a well-recognized problem. Indeed, many of themodifications to profile likelihood that have beenadvanced have, as one of their primary goals, ad-justments to account for nuisance parameter uncer-tainty. It is important to emphasize that such mod-ifications can be crucial, especially because “raw”likelihood procedures will typically be anticonserva-tive, erring on the side of suggesting more accuracythan is actually warranted. Among the many dis-turbing examples of this is the exponential regres-sion model (see discussion in Ye and Berger, 1991,and the references therein).

In contrast, integration methods automaticallyincorporate nuisance parameter uncertainty, inthe sense that an integrated likelihood is an av-erage over all the possible conditional likelihoodsgiven the nuisance parameter. We do not claim that(default) integration methods are guaranteed toincorporate nuisance parameter uncertainty in asatisfactory way, but they certainly appear morenaturally capable of doing so. As a final comment,note that if the conditional likelihoods do not varysignificantly as the nuisance parameter λ changes,then the integrated likelihoods will be very insensi-tive to choice of π�λ�θ�.

2.3 Simplicity and Generality

In comparing the simplicity of integration ver-sus likelihood methods, it is difficult to draw firmconclusions because of the wide range of possiblemethods under each label. For instance, the pro-file likelihood in (2) is very simple to use while, onthe integration side, the simplest is the uniform-integrated likelihood in (1). The various adjustedprofile likelihoods and marginal likelihoods forman array of likelihood methods of modest to greatcomplexity. Correspondingly, “optimal” integratedlikelihoods can require difficult developments ofnoninformative priors. Computational considera-tions also come into play, although the historicalwisdom that Bayesian computations are harderhas today been reversed by the advent of MCMCcomputational techniques.

The key to comparison is judging the quality ofthe answers relative to the simplicity of the method.For instance, comparison at the simplest level, pro-file versus uniform-integrated likelihood, convincesmany that the latter is considerably more effectivein producing good answers in practice. Not manycomparisons have been done at higher levels ofcomplexity (Liseo, 1993, and Reid, 1996, are excep-tions).

A few general observations are worth mentioning.First, integration methods are all based on the samebasic idea; the only difference is in the prior distri-bution used to perform the integration. In contrast,the various profile, conditional and marginal likeli-hood approaches are based on very different ratio-nales, and obtaining a feel for when each approachshould be applied is not easy.

Second, the default Bayesian approaches havelately been used, with apparently great success,on a very large variety of complex applied prob-lems. There are fewer successes in complex appliedproblems for the likelihood methods. This could, ofcourse, be due to other factors, but is not irrelevantin terms of judging effectiveness versus difficulty.

A second relevant issue is generality of applica-tion. Likelihood methods are well known to havedifficulties with nonregular problems, such as prob-lems where the parameter is restricted to a certainrange and the sample size is modest (so that likeli-hood surfaces can have strange shapes). Even worseis when the range restriction is affected by the data(e.g., a model where xi > θ, i = 1; : : : ; n�, in whichcase standard asymptotics do not apply (Barndorff-Nielsen, 1991).

Another very difficult class of problems for likeli-hood methods is the Gleser–Hwang class, discussedin Section 5.1. A third difficult class is that consist-ing of problems involving discrete data and espe-cially discrete parameters. These are “difficult”because the discreteness very much reduces thepossibility of developing reasonable “adjustments”when basic methods are unreasonable. Here is aclassic illustration.

Example 4. Binomial �N;p�. Consider k inde-pendent success counts s = �s1; : : : ; sk� from abinomial distribution with unknown parameters�N;p�, and assume that N is the parameter ofinterest with p a nuisance parameter. This prob-lem has been discussed in Draper and Guttman(1971), Carroll and Lombard (1985), Kahn (1987),Raftery (1988), Aitkin and Stasinopoulas (1989),Lavine and Wasserman (1992) and the referencestherein. Most of the points we make have alreadybeen made, in some form, in these articles.


Fig. 3. Likelihoods for Nx L is the profile; LC is the conditional; LU is the uniform-integrated and LJ is the Jeffreys-integrated.

The likelihood function is

L�N;p� =[ k∏j=1

(Nsj

)]pT �1− p�Nk−T;

0 < p < 1; N ≥ smax;

where T= ∑kj=1 sj and smax= maxj sj. This likeli-

hood is difficult to deal with by likelihood methods.For instance, the profile likelihood is

L�N�=[ k∏j=1

(Nsj

)]�Nk−T�Nk−T�Nk�Nk ;

�14�N≥ smax;

and a “natural” conditional likelihood, the condi-tional distribution of �s1; : : : ; sk� given T and N, is

LC�N�=[ k∏j=1

(Nsj

)]/(kNT

);

�15�N ≥ smax:

For the data set s = �16;18;22;25;27�, Figure 3gives the graphs of L�N� and LC�N�. These arenearly constant over a huge range of N and areclearly nearly useless for inference. Such behaviorof L�N� and LC�N� is typical for this problem. [In-deed, LC�N� is often an increasing function of N.]

The uniform integrated likelihood is

�16�

LU�N� =∫ 1

0L�N;p�dp

=[ k∏j=1

(Nsj

)]0�kN−T+ 1�0�kN+ 2� ;

N ≥ smax:

This is also graphed in Figure 3 and appears tobe much more useful than either L�N� or LC�N�.There is more to be said here, and we will return tothis example several times.

2.4 Sensitivity Analysis

One of the considerable strengths of using inte-grated likelihood is that one has a readily availablesensitivity analysis: simply vary π�λ�θ�, and seehow L∗�θ� varies. This can be crucial in evaluatingthe robustness of the answer. Also, if considerablesensitivity is discovered, it is often possible to de-termine which features of π�λ�θ� are especiallycrucial, enabling either subjective elicitation ofthese features or the identification of additionaldata that could be collected to reduce sensitivity.

Example 4 (Continued). Use of LU�θ� corre-sponds to choice of a U�0;1� prior distribution forp. Another common noninformative prior is theJeffreys prior, π�p� ∝ p−1/2�1 − p�−1/2, which willyield an integrated likelihood we denote by LJ�θ�.


More generally, one could consider Beta �a; a� priordensities for p; note that a = 1 and a = 1/2 yieldthe uniform and Jeffreys priors, respectively. Cal-culation yields, for the integrated likelihood with aBeta �a; a� prior for p,

La�N� = ca[ k∏j=1

(Nsj

)]0�kN−T+ a�0�kN+ 2a� ;

�17�N ≥ smax;

where ca is the prior normalization constant ca =0�2a�/�0�a��2:

We also graph LJ�N� = L1/2�N� in Figure 3;La�N� for 1/2 < a < 1 can be shown to lie betweenLJ�N� andLU�N� = L1�N�, so that sensitivity overthis range can be judged effectively simply by com-paring LJ and LU. A useful general result here isthat

�18�

LU�N�LJ�N�

≈(1− T+�0:7�

kN+1

)1/2 π√kN+1

≈(1− 21:7

N

)1/2 �1:4�√N

(in the example);

so that the main difference is the more quickly de-creasing tail of LU. It is worth emphasizing thatall these reasonable “default” integrated likelihoodshave tails that are considerably sharper than thatof L�N� [or LC�N�], indicating that the extremelyflat tail of L�N� may be due to a “ridge” effect.

We later discuss the question of how to use in-tegrated likelihoods. For now, we simply report themodes of LU�θ� and LJ�θ� for the data s1 = �16;18;22;25;27�, along with those of L�θ� and LC�θ�. Ta-ble 1 gives these modes along with the correspond-ing modes for two other data sets, s2 = �16;18;22;25;28� and s3 = �16;18;22;25;26�. (The reason forconsidering such perturbations is that small errorsin collection of count data such as these are almostinevitable, and one hopes the answer is not overlysensitive to such errors.)

Table 1Modes of likelihoods for N

Data set

Likelihood type s1 s2 s3

Profile �L�N�� 99 191 69Conditional �LC�N�� ∞ ∞ ∞Uniform-integrated �LU�N�� 51 57 46Jeffreys-integrated �LJ�N�� 54 62 49

While modes alone are surely insufficient as asummary of likelihoods, the stability of those forthe integrated likelihoods, over both change in theprior and small perturbations in the data, is quiteappealing. For considerably more complete discus-sion of sensitivity of integrated likelihood in thisproblem, see Lavine and Wasserman (1992). We alsoshould mention that sensitivity analysis of othertypes is certainly possible; see Olkin, Petkau andZidek (1981) for an illustration.

3. INTERPRETATION AND USE OFINTEGRATED LIKELIHOOD

Since we are proposing use of integrated likeli-hood in general, and not only within the Bayesianparadigm, we need to discuss how it is to be in-terpreted and used. We discuss, in order, its use inlikelihood analysis, Bayesian analysis and empiricalBayes analysis. Note that, of course, one might sim-ply report the entire integrated likelihood function,leaving its interpretation and use to the consumer.

3.1 Use in Likelihood Analysis

Those likelihood methods which operate solely onthe given L�θ� can also be used with an integratedlikelihood. Examples of such methods are (1) usingthe mode, θ, of L�θ� as the estimate of θ; and (2) us-ing (if θ is a p-dimensional vector)

C ={θx − 2 log�L�θ�/L�θ�� ≤ χ2

p�1− α�}

as an approximate 100�1−α�% confidence set for θ,where χ2

p�1− α� is the �1− α�th quantile of the chi-squared distribution with p degrees of freedom. Thearguments which justify such methods for profile ormodified profile likelihoods will typically also ap-ply to integrated likelihoods (and can apply in moregenerality; see Sweeting, 1995a, b). Example 4 inSection 2.4 was one illustration. The following ex-ample, which will also be used later for other pur-poses, is another standard example.

Example 5. Suppose that, independently for i =1; : : : ; p, Xi∼N�µi;1�. The parameter of interest is

θ = 1p�m�2 = 1

p

p∑i=1

µ2i :

The usual choice of nuisance parameter here isλ = m/�m�, that is, the “direction” of m in Rp. Notethat λ can also be viewed as a point on the sur-face of the unit ball in Rp; hence it is natural toassign λ the uniform prior on this surface. The re-sulting uniform-integrated likelihood (see Changand Eaves, 1990 or Berger, Philippe and Robert,


1998) is

�19�LU�θ� ∝ θ−�p−2�/4 exp�−pθ/2�

· I�p−2�/2(√pθ�x�

);

where �x� = �∑pi=1 x2

i �1/2 and Iν is the modifiedBessel function of the first type of order ν.

For large p and assuming that θ stays boundedas p→∞, it can be shown that the mode of (19) isapproximately

�20� θ ≈ 1p�x�2 − �p− 1�

p− 1p

(2�x�2�p− 2� − 1

)−1

:

This is a sensible estimate. For instance, since��x�2/p − E�X�2/p� → 0 as p → ∞ by the law oflarge numbers, it is immediate that θ−θ→ 0. Thusθ is consistent for θ.

In contrast, the profile likelihood does not yield aconsistent estimator. Indeed, the profile likelihoodfor this problem can easily be seen to be

�21� L�θ� ∝ exp(−��x� −

√pθ�2

);

which has mode θ = �x�2/p: Clearly θ − θ → 1, es-tablishing the inconsistency. (There do exist “classi-cal” methods which would yield reasonable answers.One such is to look at the marginal distribution of∑pi=1 X2

i , which is a noncentral chi-square distri-bution with parameter pθ; the resulting likelihood,though complicated, will behave reasonably.)

3.2 Use in Bayesian Analysis

It is natural to seek to use an integrated likeli-hood, L�θ�, via the Bayesian approach of choosing aprior distribution, π�θ�, for the parameter of inter-est and obtaining the posterior distribution

π�θ � x� ∝ L�θ�π�θ�:This is completely legitimate when π�λ�θ� is aproper distribution, as observed in Section 1.3.2.When π�λ�θ� is improper, however, certain in-coherencies such as marginalization paradoxes(Dawid, Stone and Zidek, 1973) can creep in, mak-ing this practice questionable. Fortunately, thepractical effect of such incoherencies appears to beminor, and they can be minimized by appropriatechoice of π�λ�θ� [and π�θ�], such as the “referenceprior” (see Section 4.4).

In Section 1.3.3, we listed some of the ways inwhich L�θ� can be of direct value to a Bayesian.There is an additional, rather subtle but important,reason for Bayesians to consider approaching theproblem through integrated likelihood: one is con-siderably less likely to make a damaging mistakethrough use of improper prior distributions.

Example 5 (Continued). Bayesians need notclassify parameters as interesting or nuisance.Indeed, the common “naive” default Bayesianapproach to this problem would be to base infer-ence on the noninformative prior π�m� = 1 form = �µ; : : : ; µp�; the resulting posterior distribu-tion, given x = �x1; : : : ; xp�, is

�22� π�m � x� = �2π�−p/2 exp{−

p∑i=1

�µi − xi�2/2}:

If, now, one is interested in θ = �∑pi=1 µ2

i �/p,one can easily determine the posterior distributionπ�θ�x� for θ, since, from (22), pθ has a noncentralchi-square distribution with p degrees of freedomand noncentrality parameter �x�2 =∑p

i=1 x2i .

This posterior for θ is “bad.” For instance, a com-mon estimate of θ is the posterior mean, which herewould be θ = �x�2/p + 1. This is badly inconsistentas p → ∞, in the sense that θ − θ → 2. The pos-terior median and posterior mode also exhibit thisbehavior.

Thinking in terms of θ and the nuisance param-eters λ in Section 3.1 avoided this problem. It was“obvious” to use the uniform distribution on λ to in-tegrate out the nuisance parameters, and the ensu-ing integrated likelihood in (19) will work well withany sensible π�θ�. The “error” in use of π�m� = 1above can be seen by changing variables to �θ; λ�.Then π�m� = 1 is transformed into

π�θ; λ� = θ−�p−2�/2 π�λ � θ�;where π�λ�θ� is the uniform distribution on the sur-face of the unit ball, as before. Thus, π�m� = 1 hasunwittingly introduced a drastic and unreasonableprior distribution on θ. Separately considering λ andθ, through integrated likelihood, can help to avoidthis type of error. (It should be noted that sophisti-cated default Bayesian approaches, such as the ref-erence prior approach, automatically avoid this typeof error; hence formal consideration of integratedlikelihood is not strictly necessary if reference pri-ors are employed.)

We have not discussed which priors π�θ� shouldbe used with L�θ�. Subjective choices, when avail-able, are to be encouraged. Default choices and ex-amples of the possible importance of the choice arediscussed in Section 5.1.

3.3 Use in Empirical Bayes Analysis

In a variety of situations, including empiricalBayes analysis, it is common to estimate nuisanceparameters using integrated likelihood and then toreplace them in the likelihood with their estimates.This also goes under the name “Type II maximum


likelihood” (see Good, 1983). We content ourselveshere with an example.

Example 2 (Continued). Suppose now that θ =m = �µ; : : : ; µp� is of interest, with nuisance param-eter λ = �ξ; τ2�. The joint “likelihood” of all param-eters is

�23�L�θ; λ� ∝ exp

{−

p∑i=1

�xi − µi�2/2}

· exp{−

p∑i=1

�µi − ξ�2/�2τ2�};

which already includes the given prior distribution,π�m�ξ; τ2�. In any case, to eliminate ξ and τ2 thestandard method is to form the integrated likelihoodfrom (7),

L�λ� = L�ξ; τ2� =∫L�θ; λ�dθ

∝ �1+ τ2�−p/2 exp{−p�s2 + �x− ξ�2�

2�1+ τ2�

};

estimate �ξ; τ2� by the mode

ξ = x; τ2 = max�0; s2 − 1�and plug back into (23). After simplification, the re-sult is

�24� L�θ� ∝ exp{− 1

2v

p∑i=1

�µi −mi�2};

where v = τ2/�1 + τ2� and mi = vxi + �1 − v�x.This is actually typically interpreted directly as theposterior distribution of m.

Bayesians would argue that a superior approachis to integrate directly,

�25� L�θ� =∫L�θ; λ�π�λ�dλ

[note that it would be incorrect to use the con-ditional prior π�λ�θ� here, since L�θ; λ� alreadycontains π�θ�λ� as the second factor in (23); onlythe marginal prior for λ is still needed]. A commonchoice for π�λ� is π�λ� = π�ξ; τ2� = 1 [althoughBerger and Strawderman, 1996, suggest thatπ�ξ; τ2� = �1+ τ2�−1 is better]. In a sense, the supe-riority of (25) over (24) is almost obvious, since (e.g.)if τ2 = 0, then v = 0 and mi = x, so that (24) wouldimply that all µi = x with absolute certainty. Thisis another example of the potential inadequacy offailing to incorporate the uncertainty in nuisanceparameters.

Although the direct integrated likelihood in (25)is arguably superior to (24), it is worth noting thatestimation of λ in L�θ; λ� by Type II MLE is at

least better than using profile likelihood. Indeed,the profile likelihood can be easily seen to be

�26�L�θ� ∝

( p∑i=1

�µi − µ�2)−p/2

· exp{−

p∑i=1

�µi − xi�2/2}:

It is hard to know what to do with L�θ�; thefirst term has a singularity along the line m =�c; c; : : : ; c�, and it is far from clear which, if any, ofthe other modes are reasonable.

4. VERSIONS OF INTEGRATED LIKELIHOOD

As mentioned in the introduction, any conditionalprior density π�λ�θ� can be used to define an in-tegrated likelihood. In this section, we review themore common choices of default or noninformativeconditional priors and discuss their strengths andweaknesses.

4.1 Proper Conditional Priors

When π�λ�θ� is naturally a proper conditionaldensity, certain of the concerns with interpret-ing and using the resulting integrated likelihooddisappear. In particular, the resulting integratedlikelihood can unambiguously be interpreted as alikelihood and can be combined with any (proper)prior density for θ to produce a true posterior.Example 5 provides an illustration of this, withπ�λ�θ�—the uniform distribution on the sphere ofrotations—clearly being proper. Example 4 offersanother illustration, with both the Jeffreys and theuniform distributions proper on the unit interval.

Today it is popular, for computational reasons, touse “vague” proper priors (often “vague” conjugatepriors). Unfortunately, the use of such does not re-ally provide protection against the concerns thatarise with use of improper conditional priors; if adifficulty would arise in using an improper condi-tional prior, the same difficulty would manifest itselfthrough great sensitivity to the degree of “vague-ness” chosen.

4.2 Uniform Conditional Prior

The uniform choice of π�λ�θ� = 1 has already beenmentioned in the introduction and in several of theexamples. It is still the most commonly used de-fault conditional prior and is an attractive choicewhen nothing else is available. (When λ is a vec-tor, many even prefer the uniform prior to the Jef-freys prior because of concerns with the behavior ofthe Jeffreys prior in higher dimensions.) There are,


however, well-documented difficulties with the uni-form prior, perhaps the most well known being itslack of invariance to reparameterization. (Using theuniform prior for a positive parameter λ will yielda different integrated likelihood than using the uni-form prior for its logarithm λ∗ = log λ, for exam-ple.) While this tends not to be a serious issue inpractice (Laplace, 1812, suggested that parameteri-zations are typically chosen so that a uniform prioris reasonable), it does suggest that a uniform priorcannot be the final answer.

A more serious potential problem with the uni-form conditional prior is that the resulting inte-grated likelihood may not exist. Here is a simpleexample.

Example 6. Suppose X1 and X2 are independentN�θ; σ2� and the uniform prior π�σ2�θ� = 1 is usedfor the nuisance parameter σ2. Then

LU�θ� =∫ ∞

0

12πσ2

· exp{− 1

2σ2��x1− θ�2+�x2− θ�2�

}dσ2

= ∞:In contrast, use of the usual conditional priorπ�σ2�θ� = 1/σ2 (see Section 4.4) would yield

�27� LR�θ� = 1π��x1 − θ�2 + �x2 − θ�2�

:

Interestingly, this latter integrated likelihood co-incides with the profile likelihood and with themarginal likelihood.

4.3 Right Haar Measure

If f�x�θ; λ� is invariant with respect to anamenable group whose action on the parame-ter space leaves θ unchanged, then the compellingchoice for π�λ�θ� is the induced right invariant Haardensity for λ (see Berger, 1985, and Eaton, 1989, fordefinitions). Virtually all default Bayesian meth-ods recommend this conditional prior, as do various“structural” and even frequentist approaches.

Example 1 (Continued). The model is invariantunder a location shift, and the right invariant Haardensity for θ is the uniform density π�θ�σ2� = 1.

Example 5 (Continued). The model is invariantunder rotations of x and m, and the rotation groupleaves θ = �m�2/p unchanged. The right invariantHaar density (actually the Haar density here, sincethe rotation group is compact and hence unimodu-lar) induced on λ = m/�m� is the uniform density onthe unit sphere discussed in Section 3.1.

For formal discussion and other examples of useof the right Haar density as the conditional prior,see Chang and Eaves (1990) and Datta and Ghosh(1995a). The chief limitation of this approach is therarity of suitable invariance. A secondary limitationis that, if the group is not amenable, use of the re-sulting right Haar density can be problematical.

4.4 Conditional Reference Integrated Likelihood

The reference prior algorithm (Bernardo, 1979;Berger and Bernardo, 1989, 1992; Bernardo andSmith, 1994) is a quite general and powerful tool forobtaining “automatic” priors to be used in Bayesiananalysis. It is motivated by trying to find that priordistribution which is least informative, in the senseof maximizing (in an asymptotic sense) the expectedKullback–Liebler divergence between the prior dis-tribution and the posterior distribution. (Intuitively,such a prior distribution allows the data to “speakmost loudly.”) The reference prior is typically thesame as the Jeffreys prior in the one-dimensionalcase; when the parameter space is multivariate, thereference algorithm takes into account the order ofinferential importance of the parameters, by parti-tioning the parameter vector ω = �ω1;ω2; : : : ; ωp�into several blocks of decreasing interest to the in-vestigator. Berger and Bernardo suggest using theone-at-time reference prior which corresponds topartitioning ω into p one-dimensional blocks.

Since the reference prior depends on whichparameters are of primary interest, it is usuallydifferent from the Jeffreys prior. In numerous mul-tivariate examples, it has been shown to performconsiderably better than the Jeffreys prior. Also, itseems to typically yield procedures with excellentfrequentist properties (Ghosh and Mukerjee, 1992;Liseo, 1993; Sun, 1994; Datta and Ghosh, 1995a, b).

In the standard Bayesian approach, the referenceprior is used to produce the joint posterior distribu-tion for �θ; λ�; then λ is integrated out to obtain themarginal posterior for θ. In this approach, there isno need to develop the notion of a likelihood for θ.Indeed, any attempt to do so directly, via an expres-sion such as (10), is made difficult by the typicalimpropriety of reference priors; one cannot defineconditional and marginal distributions directly froman improper distribution.

Therefore, to produce a reference-integrated like-lihood, we need to slightly modify the reference prioralgorithm. In this paper, we only discuss the “twogroups” case, where the parameter vector is splitin the parameters of interest, θ, and the nuisanceparameters, λ; extensions to several groups is im-mediate. For simplicity of notation, we will take θand λ to be scalars.


Under certain regularity conditions (see, e.g.,Bernardo and Smith, 1994), basically the existenceof a consistent and asymptotically normal estimatorof the parameters, the reference prior for λ, when θis known, is defined to be

�28� π∗�λ � θ� ∝√I22�θ; λ�;

where I22�θ; λ� is the lower right corner of the ex-pected Fisher information matrix. Since direct useof this, as in (10), to obtain an integrated likeli-hood is problematical due to its typical impropriety(Basu, 1977), we employ the idea of the Berger andBernardo (1992) reference prior algorithm and con-sider a sequence of nested subsets of � = 2 × 3,�1;�2; : : :, increasing to � and over which (28) canbe normalized; then the conditional reference prior,and associated integrated likelihood, will be definedas the appropriate limit. Indeed, defining 3m�θ� =�λx �θ; λ� ∈ �m� and the normalizing constant overthis compact set as Km�θ�−1 =

∫3m�θ� π

∗�λ � θ�dλ,the conditional reference prior is defined as

�29� πR�λ � θ� = h�θ�π∗�λ � θ�;where

�30� h�θ� = limm→∞

Km�θ�Km�θ0�

;

assuming the limit is unique up to a proportionalityconstant for any θ0 in the interior of 2. The corre-sponding integrated likelihood is then given by

�31� LR�θ� =∫3f�x � θ; λ�πR�λ � θ�dλ;

which is in the “standard” form of an integratedlikelihood.

This definition of the conditional reference priorcan be shown to be consistent with the definitionof the joint reference prior πR�θ; λ� in Berger andBernardo (1992), providing the joint reference priorexists and (30) holds, in the sense that then

�32� πR�θ; λ� = πR�λ � θ�πR�θ�for some function πR�θ�. We will then define πR�θ�to be the marginal reference prior for θ. The condi-tional reference prior can be shown to share many ofthe desirable properties of the joint reference prior,such as invariance to reparameterization of the nui-sance parameters. Note that this conditional refer-ence prior was also considered in Sun and Berger(1998), although for different purposes.

Example 7. The coefficient of variation. Let X1;X2; : : : ;Xn be n iid random variables with distri-bution N�µ;σ2�. The parameter of interest is thecoefficient of variation θ = σ/µ and λ = σ is the

nuisance parameter. The expected Fisher informa-tion matrix, in the �θ; λ� parameterization, is

�33� I�θ; λ� =

1θ4

− 1λθ3

− 1λθ3

2θ2 + 1λ2θ2

:

The above algorithm gives

�34� π∗�λ � θ� ∝ 1λ

√2θ2 + 1θ2

:

A natural sequence of compact sets in the �µ;σ�parameterization is given by

�m ={�µ;σ�x − am < µ < am;

1bm

< σ < bm

}

for increasing sequences am and bm that diverge toinfinity. Then the resulting 3m�θ� sequence is

3m�θ� =

�1/bm; bm�; �θ� > bmam;

�1/bm; �θ�am�;1

ambm< �θ� < bm

am;

\; �θ� < 1ambm

:

Therefore

K−1m =

2

√2θ2 + 1θ2

log bm; �θ� > bmam;

√2θ2 + 1θ2

log��θ�ambm�;

1ambm

< �θ� < bmam;

0; �θ� < 1ambm

;

and

h�θ� = limm→∞

Km�θ�Km�θ0�

∝√

θ2

2θ2 + 1:

Thus the conditional reference prior and integratedlikelihood are πR�λ � θ� = 1/λ and

�35�

LR�θ� =∫ ∞

0

1λf�x � θ; λ�dλ

∝ exp[− n

2θ2

(1− x2

D2

)] ∫ ∞0zn−1

· exp[−n

2D2(z− x

D2θ

)2]dz;

where x is the sample mean and D2 = ∑x2i /n.

[Note, also, that the problem is invariant to scale


changes, and π�λ � θ� = 1/λ is the resulting rightinvariant Haar density.] This example will be fur-ther discussed in Section 5.1.

Example 5 (Continued). When θ = �∑p1 µ

2i �/p is

the parameter of interest, then the conditional ref-erence prior for the nuisance parameters λ is simplythe uniform prior on the surface of the unit ball, asdiscussed in Section 3.1.

Example 6 (Continued). Unlike the conditionaluniform prior, the conditional reference prior forthis model, namely πR�λ � θ� ∝ 1/λ, yielded a finiteintegrated likelihood in (27). Of course, with threeor more observations, LU�θ� would also be finitehere, but the example is indicative of the commonlyobserved phenomenon that reference priors virtu-ally always yield finite integrated likelihoods, whilethe uniform prior may not.

4.5 Other Integrated Likelihoods

In one sense, there are as many integrated likeli-hoods as there are priors. For instance, any methodwhich yields a noninformative prior, πN�θ; λ�, leadsto an integrated “likelihood”

�36� L�θ� ∝∫L�θ; λ� πN�θ; λ�dλ:

Since πN�θ; λ� is a joint prior distribution on both θand λ, however, this is not formally in the spirit of(3); equation (36) would actually yield the proposedmarginal posterior distribution for θ and must benormalized by dividing by the marginal πN�θ� =∫πN�θ; λ�dλ (if finite) to yield a likelihood.Reference noninformative priors offer a ready so-

lution to this dilemma, since the reference prior al-gorithm itself suggested a suitable πR�λ�θ� for usein (3). While it is easy to normalize proper priorscorrectly, other noninformative priors are more dif-ficult to modify to produce an integrated likelihood.We illustrate the possibility by considering how tomodify the Jeffreys prior approach to yield an in-tegrated likelihood, since the Jeffreys prior is prob-ably the most widely used default prior distribu-tion. (The motivation for the Jeffreys prior can befound in Jeffreys, 1961, and was primarily the de-sire to construct default priors such that the result-ing Bayesian answers are invariant under reparam-eterization.)

The Jeffreys prior for �θ; λ� is

πJ�θ; λ� ∝√

det�I�θ; λ��;where I�θ; λ� is the expected Fisher informationmatrix. To obtain a reasonable πJ�λ�θ�, perhaps themost natural option is to simply treat θ as given,

and derive the Jeffreys prior for λ with θ given. Itis easy to see that the result would be

�37� π∗�λ � θ� ∝√

det�I22�θ; λ��;where I22�θ; λ� is the corner of I�θ; λ� correspondingto the information about λ. This, however, also hasambiguities. In Example 7, for instance, we foundfrom (34) that π∗�λ�θ� ∝ λ−1

√2+ θ−2 but, since θ

is now to be viewed as given, the factor√

2+ θ−2 isjust a proportionality constant which can be ignored(the problem does not arise for proper priors, sincesuch a factor would disappear in the normalizingprocess).

The net result of this reasoning suggests thatthe correct definition of πJ�λ�θ� is as given in (37),but ignoring any multiplicative factors which onlyinvolve θ. Thus, in Example 7, we would obtainπJ�λ�θ� = 1/λ which is the same as πR�λ�θ�. Wehave not explored the quality of integrated likeli-hoods based on πJ�λ�θ�, but suspect that they wouldtypically be satisfactory.

The other prominently studied default priors arethe probability matching priors, designed to produceBayesian credible sets which are optimal frequen-tist confidence sets in a certain asymptotic sense.Literature concerning these priors can be accessedthrough the recent papers by Berger, Philippeand Robert (1998), Datta and Ghosh (1995a, b)and Ghosh and Mukerjee (1992). We have not ex-plored the use of these priors in defining integratedlikelihood.

5. LIMITATIONS OF INTEGRATED LIKELIHOOD

5.1 The Posterior Distribution May Be Needed

As in the univariate case without nuisanceparameters, the integrated likelihood function con-tains the information provided by the data (filteredby the conditional prior on λ) and often can be useddirectly for inferential purposes. In some cases,however, L∗�θ� needs to be augmented by a priordistribution for θ (possibly noninformative), yield-ing a posterior distribution for θ, before it can serveas a basis for inference. (We are envisaging thatthe posterior distribution would be used in ordi-nary Bayesian ways, to construct error estimates,credible sets, etc.)

Example 4 (Continued). In Section 2.4, we usedonly the modes of LU�N� and LJ�N� as point es-timates for N. How can we do more, for example,convey the precision of the estimates?

Classical approaches have not made much head-way with this problem, and the “standard” non-informative prior Bayesian approach also fails.


Fig. 4. Posterior distributions proportional to LU�N�π1�N�;LJ�N�π1�N�; LU�N�π2�N� and LJ�N�π2�N�.

The standard noninformative prior for N would beπ�N� = 1. But La�N� behaves like cN−a for largeN (see Kahn, 1987), as does the resulting poste-rior, so for a ≤ 1 the posterior will be improperand Bayesian analysis cannot succeed. (In a sensethe difficulty here is that sophisticated noninfor-mative prior methodology does not really exist fordiscrete N; see the next example for an illustrationof what such methodology can do for a continuousparameter.)

Subjective Bayesians would argue the importanceof introducing proper subjective prior distributionshere, and it is hard to disagree. This could be doneeither by keeping the (vague) π�N� = 1, but intro-ducing a Beta �a; b� prior for θ with a > 1 (in whichcase Kahn, 1987, can be used to show that the pos-terior is proper), or by choosing an appropriatelydecreasing π�N�.

One could still, however, argue for the benefits ofhaving a default or conventional analysis availablefor the problem. To go along with the default LU�N�or LJ�N� [see (16) and (17)], one might consider, asdefault priors, either π1�N� = 1/N (Raftery, 1988;Moreno and Giron, 1995; de Alba and Mendoza,1996) or the Rissanen (1983) prior

π2�N� ∝kN∏i=0

�log�i��N��−1;

where log�0��N�=N and log�k+1��N�= log log�i��N�,with kN the largest integer such that log�kN��N� >1. This latter prior is, in some sense, the vaguest

possible proper prior. Both π1 and π2 can easily beshown to yield proper posteriors when paired witheither LU�N� or LJ�N�. Figure 4 shows the fourresulting posterior distributions for the data s =�16;18;22;25;27�.

Inferential conclusions are straightforward fromthese proper posteriors. For instance, the quartilesof the four posteriors are given in Table 2. (Provid-ing quartiles is more reasonable than providing mo-ments when, as here, the distributions have largeand uncertain tails.)

Compare this with the much more limited (andquestionable) inferences provided in Table 1. Whilethe answers in Table 2 vary enough that a subjectiveBayesian analysis might be deemed by many to benecessary, a case could also be made for choosing, asa conventional analysis, the LUπ1 or LJπ2 posteriordistributions.

The Gleser–Hwang class. Example 4 is a ratherspecial situation, because the parameter is discrete.However, similar phenomena occur for a large

Table 2Quartiles of the posterior distributions for N

Posterior First quartile Median Third quartile

LU�N� π1�N� 51 80 158LJ�N� π1�N� 59 112 288LU�N� π2�N� 44 63 106LJ�N� π2�N� 47 74 152


class of problems, which includes Fieller’s problem,errors-in-variables models, and calibration models.We begin discussion of this class by presenting anexample and then discuss the characterization ofthe class given by Gleser and Hwang (1987).

Example 7 (Continued). The use of the condi-tional reference prior, π�λ�θ� ∝ 1/λ, for the nuisanceparameter λ = σ in the coefficient of variation prob-lem leads to the integrated likelihood (35). It can beeasily seen that this likelihood does not go to zeroas �θ� goes to infinity, making direct inferential useof LR�θ� difficult. To progress, one can bring in themarginal reference prior

�38� πR�θ� ∝ 1

�θ�√θ2 + 1/2

:

This leads to the reference posterior distribution,

�39�

πR�θ � x�

∝ 1

�θ�√θ2 + 1/2

exp[− n

2θ2

(1− x2

D2

)]

·∫ ∞

0zn−1 exp

[−n

2D2(z− x

D2θ

)2]dz;

which, interestingly, is proper, and hence can beused for inference. This is an example of the ratheramazing property of reference (and Jeffreys) priorsthat they virtually always seem to yield proper pos-terior distributions, at least in the continuous pa-rameter case.

Of course, one might object to using (39) for in-ference, either because a Bayesian analysis is notwanted or because the introduction of the referenceprior seems rather arbitrary. Indeed, it is difficultto come up with auxiliary supporting arguments foruse of the reference prior here. For instance, the re-sulting Bayesian inferences will often not have par-ticularly good frequentist properties, which is one ofthe commonly used arguments in support of refer-ence priors.

It is important, however, to place matters in per-spective. For this situation, there simply are nomethods of “objective” inference that will be viewedas broadly satisfactory. Consider likelihood meth-ods, for instance. The profile likelihood for θ can beshown to be

L�θ� ∝(

θ

g�x; θ�

)n

· exp[− n

2θ2+ 2ng2�x; θ��xg�x; θ�− θ

2D2�];

where g�x; θ� = −x+�sgnθ�√x2 + 4D2θ2. It is easy

to see that, as �θ� → ∞, this approaches a positive

constant. The correction factor for the profile likeli-hood given by the Cox–Reid (1987) method (see theAppendix) yields the conditional profile likelihood

LC�θ�

∝{�θ��x sgn θ+

√x2+4D2θ2�

}

·{√

θ2+1/2

·√

4θ2D4+x2(x sgn θ+√x2+4D2θ2 )2

}−1

· L�θ�:

(40)

It is easy to see that this, too, is asymptotically con-stant.

The similarities between the three different like-lihoods [LR�θ�, L�θ� and LC�θ�] are better appre-ciated in the special case where x = 0 and D = 1.Then

�41�LR�θ� ∝ L�θ�

∝√θ2 + 1/2�θ� LC�θ� ∝ exp

(− n

2θ2

):

These are graphed in Figure 5, along with πR�θ�x�.Note that the conditional profile likelihood is vir-tually indistinguishable from the other likelihoods.There is no appealing way to use these likelihoodsfor inference.

Frequentists might argue that this simply revealsthat no type of likelihood or Bayesian argumentis appealing for this example. However, frequen-tist conclusions are also very problematical here,since this example falls within the class of prob-lems, identified in Gleser and Hwang (1987), wherefrequentist confidence procedures must be infinitesets (and often the whole parameter space) with pos-itive probability. [Saying that a 95% confidence setis �−∞;∞� is not particularly appealing]. First, werestate the Gleser and Hwang result in our nota-tion.

Theorem 1 (Gleser and Hwang, 1987). Considera two parameter model with sampling densityf�x�θ; λ�, θ ∈ 2, λ ∈ 3, on the sample space X . Sup-pose there exists a subset 2∗ of 2 and a value of λ∗

in the closure of 3 such that θ has an unboundedrange over 2∗ and such that for each fixed θ ∈ 2∗and x ∈ X ,

�42� limλ→λ∗

f�x � θ; λ� = f�x � λ∗�

exists, is a density on X and is independent of θ.Then every confidence procedure C�X� for θ withpositive confidence level 1− α will yield infinite setswith positive probability.


Fig. 5. Reference, profile and conditional likelihoods and reference posterior for the coefficient of variation problem, with x = 0; n = 5and D = 1.

To show that Example 7 is an example of thistype, we must slightly generalize Theorem 1, as fol-lows.

Theorem 1′. Assume there exist a sequence�θm; λm� and a value λ∗; with λm → λ∗ and�θm� → ∞; such that

�43� limm→∞

f�x � θm; λm� = f�x � λ∗�;

for some density f�x � λ∗�. Then the conclusion ofTheorem 1 holds.

The proof in Gleser and Hwang (1987) can be fol-lowed exactly, using the new condition in obviousplaces.

Example 7 (Continued). If θ = σ/µ and the nui-sance parameter is chosen to be λ = µ, then definingλm = 1/�θm� with �θm� → ∞ yields

limm→∞

f�x � θm; λm� =1

�2π�n/2 exp{−nD

2

2

};

so that (43) is satisfied. Hence frequentist confi-dence sets must be infinite with positive probability.

Another interesting fact is that the class of densi-ties satisfying (43) appears to be related to the classof densities for which the profile likelihood does notgo to zero at infinity.

Lemma 1. Under the conditions of Theorem 1′, theprofile likelihood does not converge to zero.

Proof. From condition (43),

limm→∞

supλ∈3

f�x � θm; λ� ≥ limm→∞

f�x � θm; λm�

= f�x � λ∗� > 0:

Recall that this class of densities, which are veryproblematic from either a likelihood or a frequentistperspective, includes a number of important prob-lems (in addition to the coefficient of variation prob-lem) such as Fieller’s problem, errors-in-variablesmodels and calibration models. One can make a con-vincing argument that such problems are simply notamenable to any “objective” analysis; prior informa-tion is crucial and must be utilized. If, however, oneis unwilling or unable to carry out a true subjec-tive Bayesian analysis, then a case can be made forhaving standard “default” analyses available. TheBayesian analyses with reference noninformativepriors, such as that leading to πR�θ�x� in Example 7,are arguably the best candidates for such “default”analyses.

Visualization. We have argued that, for certainimportant classes of problems, inference appearsimpossible based on L�θ� alone. A counterargumentis sometimes advanced, to the effect that formalinference is not necessary; it may suffice simply


to present L�θ� itself as the conclusion, with con-sumers learning to interpret likelihood functionsdirectly.

One difficulty with this argument is that likeli-hood functions can appear very different, dependingon the parameterization used.

Example 7 (Continued). The conditional refer-ence likelihood, LR�θ�, for x = 1, n = 2 and D = 2,is presented in Figure 6a. Suppose, instead, thatthe parameterization ξ = θ/�1+ �θ�� had been used.The conditional reference likelihood for ξ can theneasily be seen to be

LR�ξ� = LR�ξ/�1− �ξ��;

which is plotted in Figure 6b over �−1;1�, the rangeof ξ. Visually, LR�ξ� and LR�θ� appear to conveymarkedly different information: for LR�ξ� the “lo-cal” mode appears to dominate, while for LR�θ� itappears to be insignificant relative to the huge tail.Yet these two functions clearly reflect the same in-formation.

Introducing π�θ� removes this difficulty [provid-ing that the method used for obtaining π�θ� is in-variant under reparameterization of θ], because theJacobian will correct for the influence of the changeof variables. Subjective priors will be suitably in-variant, as are the Jeffreys and reference priors (seeDatta and Ghosh, 1996).

Example 7 (Continued). The marginal referenceprior, πR�θ� in (38), results in the proper posteriorin (39). If, instead, the ξ parameterization had beenused, the marginal reference prior for ξ would be

πR�ξ� = πR�ξ/�1− �ξ�� · �1− �ξ��−2;

which is obtainable from πR�θ� by straightforwardchange of variables. Because of this invariance ofthe prior, it is clear that the two posteriors, πR�θ �x� ∝ LR�θ�πR�θ� and π�ξ � x� ∝ LR�ξ�πR�ξ�, aresimple transformations of each other. Figure 7aand 7b graphs these two posteriors for the samesituation as in Figure 6. These seem to be visu-ally satisfactory, in the sense of conveying similarinformation.

One possibility for “correcting” likelihoods to pre-vent this visualization problem is to mimic theBayesian approach by including a “Jacobian” termto account for reparameterization. Various sug-gestions to this effect have been put forth in thelikelihood literature, from time to time, but noneseem to have caught on.

5.2 Premature Elimination ofNuisance Parameters

Any summary reduction of data runs the risk oflosing information that might later be needed, andnuisance parameter elimination is no exception. Wereview a few of the ways in which this loss of in-formation can occur. The most basic situation thatcan cause problems is if more data are later to beobserved.

Example 1 (Continued). Defining S21=∑ni=1�xi−

x�2/n, the uniform integrated likelihood for σ2 wasLU1 �σ2� ∝ σ−�n−1� exp�−nS2

1/2σ2�. Suppose ad-

ditional data xn+1; : : : ; xn+m become available.The corresponding uniform integrated likeli-hood for σ2 from these data alone is LU2 �σ2� ∝σ−�m−1� exp�−mS2

2/2σ2�, with S2

2 being the cor-responding sum of square deviations. To find theoverall likelihood for σ2, it is tempting to multiplyLU1 �σ2� and LU2 �σ2�, since the two data sets wereindependent (given the parameters). Note, however,that, if all the data were available to us, we woulduse the integrated likelihood

LU3 �σ2� ∝ σ−�n+m−1� exp{−�n+m�S2

3

2σ2

};

where S23 is the overall sum of square deviations,

and this is not the product of LU1 and LU2 . Indeed,from knowledge only of LU1 and the new data, LU3cannot be recovered. (To recover LU3 , one would alsoneed x from the original data.)

A second type of loss of information can occur inmeta-analysis, where related studies are analyzed.

Example 2 (Continued). We generalize this ex-ample slightly, supposing the observations to beXij ∼ N�µi; σ2

i �; j = 1; : : : ; ni and i = 1; : : : ; p.Of interest here are the study-specific µi or theoverall �ξ; τ2�, where µi ∼ N�ξ; τ2�; i = 1; : : : ; p.Such data might arise from studies at p differ-ent sites, with the means at the different sitesbeing related to the overall population mean, ξ,as indicated. Each site might choose to eliminatethe nuisance parameter σ2

i by integration [using,say, πR�σ2

i � = 1/σ2i ], resulting in the report of the

integrated likelihoods

�44� LRi �µi� ∝[S2i + �xi − µi�2

]−ni/2;

where xi =∑nij=1 xij/ni and S2

i =∑nij=1�xij−xi�2/ni,

i = 1; : : : ; p. One might then be tempted to use theproduct of these likelihoods [along with the knowl-edge that µi ∼N�ξ; τ2�] in the meta-analysis.


Fig. 6. Integrated likelihoods for Example 7 when x = 1; n = 2 and D = 2. (a) θ parameterization; (b) ξ parameterization.

If the σ2i were completely unrelated, such an anal-

ysis would be reasonable. Typically, however, the σ2i

would themselves be closely related, as in Hui andBerger (1983), and their “independent” eliminationby integration would then not be appropriate. In-terestingly, in this situation the original likelihoodcan be recovered (assuming the original model is

known), in that the sufficient statistics ni, xi andS2i can all be found from (44). But one would have

to use these sufficient statistics to reconstruct thefull likelihood and not use the LRi �µi� directly.

A third situation in which one should not use anintegrated likelihood for θ is when prior informa-


Fig. 7. The reference posteriors for the situation of Figure 6; (a) θ parameterization; (b) ξ parameterization.

tion is available in the form of a conditional priordistribution π�θ�λ�. If λ is first eliminated from thelikelihood, then one cannot subsequently make useof π�θ�λ�. [One possible default treatment of this sit-uation is described in Sun and Berger (1998): firstfind the reference marginal prior for λ, based onπ�θ�λ�, and then find the resulting posterior for θ.]

While one should be aware of the potential prob-lems in premature elimination of nuisance param-eters, the situation should be kept in perspective.Problems arise only if the nuisance parameters con-tain a significant amount of information about howthe data relate to the quantities of interest; in sucha situation one should delay integrating away the


nuisance parameters as long as possible. In the ex-amples discussed above, however, the nuisance pa-rameters hold little enough information about thequantities of interest that the answers obtained bysimply multiplying the integrated likelihoods arereasonably close to the answers from the “correct”likelihood. Indeed, much of statistical practice canbe viewed as formally or informally eliminating nui-sance parameters at different stages of the analysis,then multiplying the resulting likelihoods. The trickis to recognize when this is a reasonable approxima-tion and when it is not.

APPENDIX

To derive a conditional profile likelihood for thecoefficient of variation in Example 7, one first needsto obtain an orthogonal parameterization. It can beshown that ξ = �

√2σ2 + µ2�−1 is orthogonal to θ.

Cox and Reid (1987) defined the conditional profilelikelihood as

LC�θ� = L�θ��jξ;ξ�θ; ξθ��−1/2;

where �jξ;ξ�θ; ξθ�� is the lower right corner of theobserved Fisher Information matrix, and ξθ is theconditional maximum likelihood estimate. Calcula-tions show that

�jξ;ξ�θ; ξθ��−1/2

∝ �θ�(x+

√x2 + 4D2θ2

)√θ2 + 1/2

√4D4θ2 + x2(x+

√x2 + 4D2θ2

)2 :

Together with the fact that the profile likelihood isinvariant with respect to the choice of the nuisanceparameters, one obtains expression (40).

ACKNOWLEDGMENTS

Supported in part by NSF Grants DMS-93-03556,DMS-96-26829 and DMS-9802261, and the ItalianConsiglio Nazionale delle Ricerche. The authors arealso grateful to referees and Jan Bjørnstad for point-ing out errors (and their corrections!) in the paper.

REFERENCES

Aitkin, M. and Stasinopoulos, M. (1989). Likelihood analysisof a binomial sample size problem. In Contributions to Prob-ability and Statistics (L. J. Gleser, M. D. Perlman, S. J. Pressand A. Sampson, eds.) Springer, New York.

Barnard, G. A., Jenkins, G. M. and Winsten, C. B. (1962).Likelihood inference and time series (with discussion).J. Roy. Statist. Soc. Ser. A 125 321–372.

Barndorff-Nielsen, O. (1983). On a formula for the distribu-tion of the maximum likelihood estimator. Biometrika 70343–365.

Barndorff-Nielsen, O. (1988). Parametric Statistical Modelsand Likelihood. Lecture Notes in Statist. 50. Springer, NewYork.

Barndorff-Nielsen, O. (1991). Likelihood theory. In StatisticalTheory and Modelling: In Honour of Sir D.R. Cox. Chapmanand Hall, London.

Bartlett, M. (1937). Properties of sufficiency and statisticaltests. Proc. Roy. Soc. London Ser. A 160 268–282.

Basu, D. (1975). Statistical information and likelihood (with dis-cussion). Sankhya Ser. A 37 1–71.

Basu, D. (1977). On the elimination of nuisance parameters.J. Amer. Statist. Assoc. 72 355–366.

Bayarri, M. J., DeGroot, M. H. and Kadane, J. B. (1988). Whatis the likelihood function? In Statistical Decision Theory andRelated Topics IV (S. S. Gupta and J. O. Berger, eds.) 2 3–27.Springer, New York,

Berger, J. O. (1985). Statistical Decision Theory and BayesianAnalysis. Springer, New York.

Berger, J. O. and Bernardo, J. M. (1989). Estimating a productof means: Bayesian analysis with reference priors. J. Amer.Statist. Assoc. 84 200–207.

Berger, J. O. and Bernardo, J. M. (1992). Ordered group ref-erence priors with applications to a multinomial problem.Biometrika 79 25–37.

Berger, J. O. and Berry, D. A. (1988). Statistical analysis andthe illusion of objectivity. American Scientist 76 159–165.

Berger, J. O., Philippe, A. and Robert, C. (1998). Estimation ofquadratic functions: noninformative priors for non-centralityparameters. Statist. Sinica 8 359–376.

Berger, J. O. and Strawderman, W. (1996). Choice of hierar-chical priors: admissibility in estimation of normal means.Ann. Statist. 24 931–951.

Berger, J. O. and Wolpert, R. L. (1988). The Likelihood Princi-ple: A Review, Generalizations, and Statistical Implications,2nd ed. IMS, Hayward, CA.

Bernardo, J. M. (1979). Reference posterior distributions forBayesian inference (with discussion). J. Roy. Statist. Soc.Ser. B 41 113–147.

Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory.Wiley, New York.

Bjørnstad, J. (1996). On the generalization of the likelihoodfunction and the likelihood principle. J. Amer. Statist. As-soc. 91 791–806.

Butler, R. W. (1988). A likely answer to “What is the likelihoodfunction?” In Statistical Decision Theory and Related Top-ics IV (S. S. Gupta and J. O. Berger, eds.) 2 21–26. Springer,New York.

Carroll, R. J. and Lombard, F. (1985). A note on N estimatorsfor the Binomial distribution. J. Amer. Statist. Assoc. 80 423–426.

Chang, T. and Eaves, D. (1990). Reference priors for the orbitin a group model. Ann. Statist. 18 1595–1614.

Cox, D. R. (1975). Partial likelihood. Biometrika 62 269–276.Cox, D. R. and Reid, N. (1987). Parameter orthogonality and

approximate conditional inference (with discussion). J. Roy.Statist. Soc. Ser. B 49 1–39.

Cruddas, A. M., Reid, N. and Cox, D. R. (1989). A time series il-lustration of approximate conditional likelihood. Biometrika76 231.

Datta, G. S. and Ghosh, J. K. (1995a). Noninformative priorsfor maximal invariant parameter in group models. Test 495–114.

Datta, G. S. and Ghosh, J. K. (1995b). On priors providingfrequentist validity for Bayesian inference. Biometrika 8237–45.


Datta, G. S. and Ghosh, J. K. (1996). On the invariance of non-informative priors. Ann. Statist. 24 141–159.

Dawid, A. P., Stone, M. and Zidek, J. V. (1973). Marginaliza-tion paradoxes in Bayesian and structural inference. J. Roy.Statist. Soc. Ser. B 35 180–233.

de Alba, E. and Mendoza, M. (1996). A discrete model forBayesian forecasting with stable seasonal patterns. In Ad-vances in Econometrics II (R. Carter Hill, ed.) 267–281. JAIPress.

Draper, N. and Guttman, I. (1971). Bayesian estimation of thebinomial parameter. Technometrics 13 667–673.

Eaton, M. L. (1989). Group Invariance Applications in Statis-tics. IMS, Hayward, CA.

Fisher, R. A. (1915). Frequency distribution of the values of thecorrelation coefficient in samples from an indefinitely largepopulation. Biometrika 10 507.

Fisher, R. A. (1921). On the “probable error” of a coefficient ofcorrelation deduced from a small sample. Metron 1 3–32.

Fisher, R. A. (1935). The fiducial argument in statistical infer-ence. Ann. Eugenics 6 391–398.

Fraser, D. A. S. and Reid, N. (1989). Adjustments to profile like-lihood. Biometrika 76 477–488.

Ghosh, J. K., ed. (1988). Statistical Information and Likelihood.A Collection of Critical Essays by D. Basu. Springer, NewYork.

Ghosh, J. K. and Mukerjee, R. (1992). Noninformative priors.In Bayesian Statistics 4 (J. O. Berger, J. M. Bernardo, A. P.Dawid and A. F. M. Smith, eds.) 195–203. Oxford Univ. Press.

Gleser, L. and Hwang, J. T. (1987). The nonexistence of 100�1−α�% confidence sets of finite expected diameter in errors-in-variable and related models. Ann. Statist. 15 1351–1362.

Good, I. J. (1983). Good Thinking: The Foundations of Probabil-ity and Its Applications. Univ. Minnesota Press.

Hui, S. and Berger, J. O. (1983). Empirical Bayes estimationof rates in longitudinal studies. J. Amer. Statist. Assoc. 78753–760.

Jeffreys, H. (1961). Theory of Probability. Oxford Univ. Press.Kahn, W. D. (1987). A cautionary note for Bayesian estimation

of the binomial parameter n. Amer. Statist. 41 38–39.Kalbfleisch, J. D. and Sprott, D. A. (1970). Application of like-

lihood methods to models involving large numbers of param-eters. J. Roy. Statist. Soc. Ser. B 32 175–208.

Kalbfleish, J. D. and Sprott, D. A. (1974). Marginal and con-ditional likelihood. Sankhya Ser. A 35 311–328.

Laplace, P. S. (1812). Theorie Analytique des ProbabilitiesCourcier, Paris.

Lavine, M. and Wasserman, L. A. (1992). Can we estimate N?Technical Report 546, Dept. Statistics, Carnegie Mellon Univ.

Liseo, B. (1993). Elimination of nuisance parameters with refer-ence priors. Biometrika 80 295–304.

McCullagh, P. and Tibshirani, R. (1990). A simple method forthe adjustment of profile likelihoods. J. Roy. Statist. Soc.Ser. B 52 325–344.

Moreno, E. and Giron, F. Y. (1995). Estimating with incompletecount data: a Bayesian Approach. Technical report, Univ.Granada, Spain.

Neyman, J. and Scott, E. L. (1948). Consistent estimates basedon partially consistent observations. Econometrica 16 1–32.

Olkin, I., Petkau, A. J. and Zidek, J. V. (1981). A comparison ofn estimators for the binomial distribution. J. Amer. Statist.Assoc. 76 637–642.

Raftery, A. E. (1988). Inference for the binomial N parameter:a hierarchical Bayes approach. Biometrika 75 223–228.

Reid, N. (1995). The roles of conditioning in inference. Statist.Sci. 10 138–157.

Reid, N. (1996). Likelihood and Bayesian approximation meth-ods. In Bayesian Statistics 5 (J. O. Berger, J. M. Bernardo,A. P. Dawid and A. F. M. Smith, eds.) 351–369. Oxford Univ.Press.

Rissanen, J. (1983). A universal prior for integers and estima-tion by minimum description length. Ann. Statist. 11 416–431.

Savage, L. J. (1976). On rereading R. A. Fisher. Ann. Statist. 4441–500.

Sun, D. (1994). Integrable expansions for posterior distributionsfor a two parameter exponential family. Ann. Statist. 221808–1830.

Sun, D. and Berger, J. O. (1998). Reference priors with partialinformation. Biometrika 85 55–71.

Sweeting, T. (1995a). A framework for Bayesian and likelihoodapproximations in statistics. Biometrika 82 1–24.

Sweeting, T. (1995b). A Bayesian approach to approximate con-ditional inference. Biometrika 82 25–36.

Sweeting, T. (1996). Approximate Bayesian computation basedon signed roots of log-density ratios. In Bayesian Statistics5 (J. O. Berger, J. M. Bernardo, A. P. Dawid and A. F. M.Smith, eds.) 427–444. Oxford Univ. Press.

Ye, K. and Berger, J. O. (1991). Non-informative priors for infer-ence in exponential regression models. Biometrika 78 645–656.

Zabell, S. L. (1989). R.A. Fisher on the history of inverse prob-ability. Statist. Sci. 4 247–263.


CommentJan F. Bjørnstad

This paper by Berger, Liseo and Wolpert dealswith the important theoretical and practical prob-lem of eliminating nuisance parameters in the like-lihood function. The authors are to be commendedfor taking up this difficult task in a pragmatic andpractical approach, really trying to find out whatworks and what doesn’t work, especially in exam-ples where regular statistical analyses are hard toattain. There is a vast literature on various sug-gestions for partial likelihood for the parameter ofinterest, θ, in the case of nuisance parameters, λ, asindicated by the list of references in the paper, eventhough the bibliography is not complete, of course.Much of the effort so far has been based on one ofthe following operations on the likelihood L�θ; λ�:maximization (giving us the profile likelihood), con-ditioning on a sufficient statistic for λ or findingpivots or components of sufficient statistics with dis-tribution independent of λ. Integration of the like-lihood with respect to λ has for the most part beenregarded as a Bayesian approach, needing prior in-formation about λ. In this paper, Berger, Liseo andWolpert argue, I think convincingly, for the use ofintegrated likelihood, where integration is with re-spect to default or noninformative priors, as a gen-eral likelihood method, whether or not one is willingto assume a Bayesian model.

Integrated likelihood methods are discussed andillustrated by using several examples. I especiallyfound Examples 4, 5 and 7 interesting and illumi-nating, covering cases that are very difficult to an-alyze. It certainly seems that integrated likelihoodmethods have more promise in complicated prob-lems than methods based on alternative partial like-lihood like the profile likelihood, not overlooking theproblems of nonuniqueness and lack of parameterinvariance.

The use of integrated likelihood is consideredboth in a Bayesian and a non-Bayesian perspec-tive. I shall concentrate my discussion mostly onnon-Bayesian issues, but let me just make a briefcomment on Bayesian aspects. A Bayesian analysiswith available prior for �θ; λ� does not in principle

Jan F. Bjørnstad is Professor of Statistics at theAgricultural University of Norway, Department ofMathematical Sciences, P.O. Box 5035, N-1432 As,Norway �e-mail: [email protected]�.

need L�θ�, of course. However, I find that Berger,Liseo and Wolpert argue convincingly that thereare several reasons why L�θ�, althoughnot neces-sary, can be very useful in Bayesian analysis. Themost important of these, it seems to me, deals withmodels with a large number of nuisance parame-ters where it may be difficult to construct priors forthe nuisance parameters, noninformative or proper.One can then modify L�θ� with Bayes (or empiricalBayes) information about θ alone, without worryingabout nuisance parameters. A promising alter-native approach to integrated likelihood for thisproblem is the fiducially related concept of impliedlikelihood introduced by Efron (1993).

As noted by Berger, Liseo and Wolpert, in non-Bayesian inference one cannot argue for integratedlikelihood solely on grounds of rationality and co-herency . However, one can at least say that asa likelihood method, integrated likelihood satisfiesthe likelihood principle, and is a proper likelihood.That is, if two likelihood functions for �θ; λ� are pro-portional, then so are the corresponding integratedlikelihoods for θ. The main inferential issue witheliminating nuisance parameters λ from the likeli-hood function is to be able to take into account theuncertainty in λ. It is well known that replacingλ by an estimate or even conditional to θ estimate(giving us the profile likelihood) ignores the uncer-tainty in λ. This can be especially serious if the di-mension of λ is large. The resulting L�θ� can thenbe much too accurate, giving the impression that wehave more information about θ than is warranted. Ithink that the single most important reason for us-ing an integrated likelihood is, as emphasized in thepaper, that this partial likelihood automatically andnaturally takes into account parameter uncertaintyin λ.

A central theme in the paper is the comparisonof the operations of integration and maximization.One of the main messages I read from the paper isthat any reasonable integrated likelihood will typ-ically outperform the profile likelihood. It seemsquite clear that integration is a safer operationthan maximization, so if it is obvious what kindof noninformative π�λ�θ� to use, integration wouldclearly be preferable. In fact, it seems to me thatmaybe the best thing is to do what Laplace sug-gested, choose parametrizations of the nuisance


parameters such that the uniform prior is rea-sonable. Then at least there is no question whatweight function to use. The question is now howto define such parametrizations. Outside simplelocation-scale families, this seems rather difficult.Maybe one should simply try to develop a “com-plete” theory for the uniform integrated likelihood,LU�θ� =

∫L�θ; λ�dλ, possibly restricting integra-

tion to a finite interval so that the uniform prior isequivalent to a proper flat distribution for λ.

One of the examples used to illustrate the differ-ent consequences of maximization and integrationis Example 4, estimating the binomial size N.Here, the uniform integrated likelihood LU andbeta-prior based Bayesian likelihood Lbeta giveseemingly much more “accurate” information aboutN than the profile Lprofile and the conditional like-lihood LC (given total number of successes). This isa rather strange case, contrary to the typical caseof being too accurate, here Lprofile (and LC) in factseems to be too inaccurate; they are nearly constantover a huge range of N and are nearly useless forinference, according to Berger, Liseo and Wolpert.Even though I tend to agree with this, it cannot beruled out that in fact Lprofile here gives a correctpicture about the lack of information on N that thedata exhibits in the likelihood, as is also indicatedby the instability of the maximum likelihood esti-mate (MLE). A nearly flat likelihood over a largeregion may then be a proper illustration of this. Itis not clear that LU or some Lbeta is to be preferredjust because it gives sharper inference. It wouldhave been nice to see a graph of L�N;p� to be ableto judge this further. Also, it is worth noticing thatby looking at the shapes of LU and LJ, it is clearthat the mode can lead to serious underestimatingof N, even assuming that these likelihoods give anearly “correct” impression of how likely differentN-values are in light of the data.

Berger, Liseo and Wolpert seem to view refer-ence priors as the most important noninformativepriors for λ. Reference priors are invariant underreparametrizations of λ and they do seem to give an-swers generally, always leading to finite integratedlikelihood. However they are not uniquely defined,it seems, and can be rather complicated to deter-mine. In general, the problem of choosing, and therobustness in the choice of, a noninformative priorfor λ can be studied using a sensitivity analysis byvarying π�λ�θ� and see how L�θ; λ� varies. It is ar-gued that this is a considerable strength of inte-grated likelihood. To me, this rather illustrates theproblem with the nonuniqueness of integrated like-lihood: that it is, in fact, necessary with such a sen-sitivity analysis in order to trust the resulting like-

lihood. This can also make the integrated likelihoodmethod more complicated to use. So I don’t see thisas a strength of the integrated likelihood method,but rather a necessity because of a weakness in themethod.

In Section 5.1, Berger, Liseo and Wolpert claimthat sometimes it is necessary to perform aBayesian analysis with L�θ� in order to makeinferences like error estimates and confidence sets.A nice illustration of such a case is Example 7on estimating the coefficient of variation wherefrequentist confidence sets must be infinite withpositive probability. This is an example showingthe power of using reference priors. Another il-lustration they use is Example 4—estimating thebinomial N. I do not think this is such a good ex-ample, since the only justification for the choice ofprior on N seems to be to achieve a proper poste-rior distribution. Also, the priors suggested seemrather unreasonable to me. Say we decide to use abeta-integrated likelihood with π�p� = π�1− p�,

Lbeta�N� = c∫L�N;p�pa�1− p�a dp:

Since little is known of the distribution of Lbeta�N�,the question of finding a confidence interval for Nremains to be addressed. Standard noninformativeBayes analysis with π�N� = 1 also fails, and it issuggested that one has to use a default prior likeπ�N� = 1/N or the Rissanen prior. The problemwith these priors is that they are strongly decreas-ing in N and indeed very informative. For exam-ple, π�N� = 1/N says that a priori one assumesthat N = 10 is three times as likely as N = 30. I’msure there are cases where this may be a reasonableassumption, but it must be impossible to assumethis in general as a default assumption. I can’t helpthinking that these priors are bound to lead to se-rious underestimation of N and are simply tools toget a confidence interval. It cannot be enough thata method is capable of producing a result. We musthave reason to believe that the resulting inferenceis valid. Maybe the only reasonable way to solve thisproblem is to study the distribution of Lbeta�N�.

Finally, let me just mention an alternative inte-grating approach, borrowing an idea from predictivelikelihood by Harris (1989). Instead of integratingL�θ; λ� with respect to a conditional noninforma-tive π�λ�θ�, one can use a data-based weight func-tion for λ, for example the distribution of the MLEλ at �θ; λ�,

Lest�θ� =∫L�θ; λ�fθ; λ�λ = λ�dλ


or

L∗est�θ� =∫L�θ; λ�fθ; λ�λ = λ�dλ:

It could be worthwhile to study this type of inte-

grated likelihood. I would expect that, typically,these likelihoods would not be in closed form.Then probably only L∗est is obtainable by using asimulation-based estimate of fθ; λ�λ = λ�.

CommentEdward Susko

The authors give a collection of thought-provokingexamples contrasting the use of likelihood meth-ods with integrated likelihood methods when deal-ing with nuisance parameters. Since profile like-lihood methods may be the most frequently usednon-Bayesian method for dealing with nuisance pa-rameters, comparison of the profile likelihoods andintegrated likelihoods are of primary interest. Thetwo methods are closely related in many of the situ-ations of interest. When the number of observationsis large, the model is regular and the prior distribu-tion is proper, integrated likelihood is, with a rela-tive error of O�n−3/2�; proportional to

L�θ; λθ�π�λθ�θ�/√

det�J22�θ; λ��;(1)

where J22�θ; λ� is the observed information ma-trix corresponding to the information about λ (cf.Leonard, 1982; Tierney and Kadane, 1986; Reid,1996). Thus, in the case that h�θ� = c in (24),π�λθ�θ� and

√det�I22�θ; λ�� are equal up to first or-

der and thus the integrated and profile likelihoodsare the same up to first order. Hence for the refer-ence priors used here, one expects similar profileand integrated likelihoods with large samples andregular models. Appropriately, the examples con-sidered involve small samples or irregular models.The comments here will be restricted primarily toExamples 3, 4 and 5 with some additional com-ments about likelihood based methods in modelswith large numbers of nuisance parameters.

Example 3 illustrates a general situation whereprofile likelihood methods can be expected toperform poorly. The problem with using profile

Edward Susko is Assistant Professor at the De-partment of Mathematics and Statistics, DalhousieUniversity, Chase Building, Halifax, Nova ScotiaB3H 3J5 �e-mail: [email protected]�.

likelihood methods arises because of the over-parameterization in the contribution to the likeli-hood given by

�2π exp�−nθ2��−1/2 exp(− �y− λ�2

2 exp�−nθ2�

):

However, even with additional information aboutthe value of θ; the problems created by the un-certainty about λ are so severe that no sensibleanswers can be directly obtained through auto-matic use of the profile likelihood. In contrastintegrated likelihoods perform well. Whenever aproduct of very different likelihoods is being consid-ered, as might be the case in some meta-analyses,a badly behaved contribution can lead to inappro-priate conclusions.

In Example 4, if the true value of p is 0.00001, thedata are not inconsistent with large N ≥ 250, yetthe integrated likelihoods given suggest that suchvalues of N are not very plausible. Here, probabil-ities obtained at a particular parameter of interestare highly dependent upon the nuisance parameterand there is very little information about the nui-sance parameter. This would appear generally to bethe type of situation where the use of integratedlikelihoods might lead to conclusions not supportedby the data.

Example 2 illustrates that variations of inte-grated likelihood methods are a part of conventionallikelihood methods. However, as the use of Greekletters and the suggestion about the alternative ofmaximizing over the µi indicates, the distinctionbetween unobservable variables and parameters isoften not very clear. Without the random effectsassumption, this is a model where the number ofnuisance parameters increases without bound asthe number of observations increase. Models withlarge numbers of nuisance parameters arise fre-quently in practice and often give profile likelihoodswith poor behavior. In many such models, randomeffects assumptions are natural. However, as the


integrated likelihood L�τ2� in Example 2 indicates,the profile likelihoods that result from a random ef-fects model can be viewed as integrated likelihoods,where the definition in (3) is extended to allow datadependent priors, without the initial random ef-fects assumption as a part of the model. Randomeffects integrated likelihoods incorporate informa-tion about the nuisance parameters directly andusually provide estimates of the parameter of inter-est that are consistent for most (in the almost suresense) sequences of nuisance parameters that couldbe generated from the random effects distribution(Kiefer and Wolfowitz, 1956; for a dscussion aboutefficiency issues see also Lindsay, 1980).

As a different example of the use of random ef-fects integrated likelihoods, consider Example 5.The problems associated with the use of profile like-lihood methods appear to be due to the relativelylarge number of nuisance parameters. A randomeffects assumption seems natural here given theparameter of interest. Consider the same randomeffects model as in Example 2: the µi are i.i.d.N�ζ; τ2�: In this case, λ = �ζ; τ�; and θ is a randomvariable (θ∗ in the notation of Section 1.3.1). Theauthors’ recommendation of (6) as a likelihood inthis setting is closely connected with the discussionabout empirical Bayes methods in Section 3.3. If (6)is taken as the likelihood, the profile likelihood forθ = ∑

µ2i /p would be obtained by substituting an

estimate λθ into (6). It is more common to substi-tute an estimate of the λ based upon the likelihoodfor λ: the function of λ obtained by integrating overθ∗ in (6). In this case the estimate for λ is

ζ = x; τ2 = max�0; s2 − 1�:Substituting this estimate of λ into (6) gives an inte-grated likelihood that is proportional to an estimate

of the conditional density of θ given the observeddata. Since the µi�x are independently N�mi; v�;where v = τ2/�1 + τ2� and mi = vxi + �1 − v�x; theestimate of the conditional distribution for pθ/v isa noncentral χ2 distribution with p degrees of free-dom and noncentrality parameter v−1∑m2

i /2: Theadditional random effects assumption is used in theabove derivation but, even with the the original as-sumption that the µi are a fixed sequence, assumingthat θ converges to a limit, one can show that thisconditional distribution converges to a point massat this limit.

Statistical models with a large number of nui-sance parameters arise frequently and profile like-lihoods for these models often give unreasonableparameter of interest inferences. Often a naturalalternative to a model with a large number of nui-sance parameters is a random effects or mixturemodel. The analysis arising from the use of a ran-dom effects model can be viewed as an integratedlikelihood method and frequently gives reasonableparameter of interest inferences that are robust tothe assumption of randomness about the nuisanceparameter. In many other statistical models, pro-file and integrated likelihoods tend to be similar.However, in models with badly behaved likelihoodcontributions, as in Example 3, profile likelihoodscan give unreasonable parameter of interest infer-ences where integrated likelihoods give reasonablesolutions. In models with a large amount of uncer-tainty about the nuisance parameter, as in Exam-ple 4, integrated likelihoods can give misleading re-sults. Thus the differences between the results forthe two methods is informative in itself, which sug-gests that in important and complex problems itmight be of value to integrate the two as a formof sensitivity analysis.

RejoinderJames O. Berger, Brunero Liseo and Robert L. Wolpert

We thank the discussants for their considerableinsights; if we don’t mention particular points fromtheir discussions, it is because we agree with thosepoints. We begin by discussion of an issue raised byboth discussants and then respond to their individ-ual comments.

Dr. Bjørnstad and Dr. Susko both express con-cerns about the integrated likelihood answers in

the situation of estimating a binomial N in Exam-ple 4, wondering if the integrated likelihoods mightlead to conclusions not supported by the data. Thisis a situation in which the answer does changemarkedly depending upon one’s prior beliefs. Forinstance, as Dr. Susko notes, when p is small thelikelihood is extremely flat out to very large valuesof N; thus someone who viewed small p as being


likely might well find the profile likelihood morereasonable than the integrated likelihoods we fa-vor. Conversely, someone who viewed large p asbeing likely would find the likelihood for N con-centrated on much smaller values of N than theintegrated likelihoods. Faced with this extreme de-pendence on prior opinions about p (or, alternately,about N) one could thus argue that there is no “ob-jective” answer inherent in the data, and that theprofile likelihood properly reflects this whereas theintegrated likelihoods do not.

Our premise, however, is that inference con-cerning N is to be done and that the (subjectiveBayesian) analysis suggested by the above reason-ing is not going to be performed. We are simplyarguing that, under these circumstances, the an-swers obtained via the integrated likelihood routeare the most reasonable default answers. This can,indeed, be given some theoretical support, if one iswilling to view a default likelihood as a surrogatefor an (unknown) posterior density. In Example 4,for instance, suppose we are willing to agree thatthe prior, though unknown, is in the very largeclass of priors for which p and N are a priori inde-pendent, the prior density for N is nonincreasingand the prior distribution for p is a Beta distri-bution symmetric about 1/2. Liseo and Sun (1998)have shown that the Jeffreys-integrated likelihoodis closer (in a Kullback–Leibler sense) to any pos-terior arising from a prior in this class than isthe profile likelihood. In summary, while we haveno quarrel with those who “refuse to play thegame” (i.e., those who refuse to give a nonsubjectiveanswer to this problem), we do not see any betteralternatives for “players” than the recommended in-tegrated likelihood (or, even better, noninformativeprior Bayesian) answers.

Reply to Dr. Bjørnstad. We seem to be basically inagreement concerning the value of integrated likeli-hood and, indeed, Dr. Bjørnstad emphasizes severalof the particularly valuable features of integratedlikelihood. We would add a slight qualification to hiscomment that the integrated likelihood approachhas the desirable feature of following the likelihoodprinciple. This is certainly true if the prior distribu-tion is chosen independently of the model, but werecommend use of the (conditional) reference non-informative prior which can depend on the model(and not on only the observed likelihood function).In practice, however, this is just a slight violation ofthe likelihood principle.

Concerning the issue of choice of the prior, Dr.Bjørnstad wonders if it might be worthwhile to de-velop a complete theory of uniform integrated likeli-

hood, finding parameterizations in which a constantprior is reasonable. In a sense, noninformative priortheory began by following this path, but this pathled to what we view as more powerful modern the-ories, such as the reference prior approach.

Dr. Bjørnstad expresses two related concerns overthe choice of the prior for integrated likelihood: thenonuniqueness of the choice and the difficulty ofsophisticated choices, such as the conditional refer-ence prior. One could define an essentially uniquechoice, such as the Jeffreys prior, and we would ar-gue that such a unique choice would generally givebetter answers than any “unique” nonintegratedlikelihood method. One can do better, however, andwe see no reason not to utilize the more sophisti-cated choices of priors if they are available (e.g.,if the possibly difficult derivation of a conditionalreference prior has already been performed). Wehope that, for common problems, the statisticalcommunity will eventually identify and recommendspecific sophisticated priors for “standardized” use.Our comment about the possible value of havingseveral integrated likelihoods was simply that, ifone is in a new situation with no well- studied in-tegrated likelihoods available, it can be comfortingif one obtains essentially the same answer fromapplication of several integrated likelihoods.

We do not have any definitive opinion about theweighted likelihood derived by following Harris(1989). We are somewhat concerned with the ap-parent double use of the data in the definitions ofLest�θ� and L∗est�θ� (using the data once in the like-lihood function and again in the weight function forintegration). Indeed, these weighted likelihoods aresomewhat hard to compute. We were able to com-pute L∗est�θ� for Example 3, and it turned out tobe the same as the (inadequate) profile likelihood,which does not instill confidence in the method.

Reply to Dr. Susko. The asymptotic equivalencebetween profile likelihood and integrated likelihoodis indeed interesting to contemplate. As the jus-tification for use of profile likelihood is primarilyasymptotic, it might be tempting for a “likelihoodanalyst” to view integrated likelihood as simply atool to extend the benefits of profile likelihood tosmall sample situations. Interesting issues arise inany attempt to formalize this, not the least of whichis the fact that (conditional) reference priors oftenhave nonconstant h�θ� in (29) and, having such, canconsiderably improve the performance of the inte-grated likelihood.

The “random effects” profile likelihood analysisconsidered by Dr. Susko for Example 5 is interest-ing. We note, first, that the actual profile likelihood


is not computed but, rather, the likelihood basedon the “Type II maximum likelihood estimates” ofthe nuisance parameters. Of course, as discussedin Section 3.3, we certainly support this modifica-tion. Dr. Susko mentions that the resulting “profile”likelihood yields consistent estimates of θ, even ifthe random effects assumption does not hold. Theuniform integrated likelihood in (19) also possessesnice (frequentist) confidence properties; it wouldbe interesting to know if the same can be said forthe “profile” likelihood. Finally, deriving the “pro-file” likelihood here required a strong additionalassumption (the random effects assumption) andtwo uses of integrated likelihood (first to integrateout the random effects parameters and then to ob-tain the Type II maximum likelihood estimatesof the nuisance parameters). While the resultinganswer may well be sensible, the direct uniform in-tegrated likelihood in (19) seems more attractive,both in terms of the simplicity of the answer andthe simplicity of the derivation.

Dr. Susko finishes by suggesting that it might beuseful to compute both profile and integrated likeli-hood answers in application, as a type of sensitivitystudy. It is probably the case that, if the two an-swers agree, then one can feel relatively assured inthe validity of the answer. It is less clear what to

think in the case of disagreement, however. If theanswers are relatively precise and quite different,we would simply suspect that it is a situation witha “bad” profile likelihood. On the other hand, it canbe useful to know when the profile likelihood is es-sentially uninformative, as in Example 4. While onecould argue that the same insight (and more) can beobtained by investigation of sensitivity of integratedlikelihood with respect to the noninformative prior,it is certainly reasonable to look at the profile like-lihood if it is available.

ADDITIONAL REFERENCES

Efron, B. (1993). Bayes and likelihood calculations from confi-dence intervals. Biometrika 80 3–26.

Harris, I. (1989). Predictive fit for natural exponential families.Biometrika 76 675–684.

Kiefer, J. and Wolfowitz, J. (1956). Consistency of the maxi-mum likelihood estimator in the presence of infinitely manyincidental parameters. Ann. Math. Statist. 27 887–906.

Leonard, T. (1982). Comment on “A simple predictive densityfunction” by M. Lejeune and G. D. Faulkenberry. J. Amer.Statist. Assoc. 77 657–658.

Liseo, B. and Sun, D. (1998). A general method of comparisonfor likelihoods. ISDS discussion paper, Duke Univ.

Tierney, L. J. and Kadane, J. B. (1986). Accurate approxima-tions for posterior moments and marginal densities. J. Amer.Statist. Assoc. 81 82–86.

euclid.ss.1009211804

Documents

likelihood function

value x

integrated likelihood

marginal likelihood

entire parameter

justifying integration

x onlythrough

profile likelihood