objective bayesian analysis of spatial data with measurement error

Objective Bayesian Analysis of Spatial Data with Measurement ErrorAuthor(s): Victor de OliveiraSource: The Canadian Journal of Statistics / La Revue Canadienne de Statistique, Vol. 35, No. 2(Jun., 2007), pp. 283-301Published by: Statistical Society of CanadaStable URL: http://www.jstor.org/stable/20445254 .

Accessed: 14/06/2014 03:31

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Statistical Society of Canada is collaborating with JSTOR to digitize, preserve and extend access to TheCanadian Journal of Statistics / La Revue Canadienne de Statistique.

http://www.jstor.org

This content downloaded from 195.78.108.60 on Sat, 14 Jun 2014 03:31:52 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=ssc

http://www.jstor.org/stable/20445254?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


The Canadian Journal of Statistics 283 Vol. 35, No. 2, 2007, Pages 283-301 La revue canadienne de statistique

Objective Bayesian analysis of spatial data with measurement error Victor DE OLIVEIRA

Key words and phrases: Frequentist properties; Jeffreys prior; nugget effect; reference prior.

MSC 2000: Primary 62F15, 62M30; secondary 62M40.

Abstract: The author shows how geostatistical data that contain measurement errors can be analyzed ob jectively by a Bayesian approach using Gaussian random fields. He proposes a reference prior and two versions of Jeffreys' prior for the model parameters. He studies the propriety and the existence of moments for the resulting posteriors. He also establishes the existence of the mean and variance of the predictive dis tributions based on these default priors. His reference prior derives from a representation of the integrated likelihood that is particularly convenient for computation and analysis. He further shows that these default priors are not very sensitive to some aspects of the design and model, and that they have good frequentist properties. Finally, he uses a data set of carbon/nitrogen ratios from an agricultural field to illustrate his approach.

Analyse bayesienne objective de donnees spatiales entachees d'erreurs de mesure Resum=: L'auteur montre comment des donnees g6ostatistiques entachees d'erreurs de mesure peuvent etre analysees objectivement par une approche bay6sienne a l'aide de champs aleatoires gaussiens. I1 propose une loi a priori de r6ference et deux versions de la loi de Jeffreys pour les param6tres du modele. I1 etudie l'integrabilite et l'existence des moments des lois a posteriori correspondantes. II demontre aussi l'existen ce de l'esp6rance et de la variance des lois previsionnelles deduites de ces lois a priori objectives. Sa loi a priori de ref6rence d6coule d'une representation de la vraisemblance integree qui est tres commode aux fins de calcul et d'analyse. II montre par ailleurs que ces lois a priori sont plutot insensibles a certaines caracte ristiques du plan d'experience et du modele, en plus de bien se comporter au plan frequentiste. Enfin, il se sert de donnees sur le rapport carbone/azote d'une terre agricole pour illustrer son propos.

1. INTRODUCTION

The Bayesian approach for the analysis of geostatistical data was pioneered by Kitanidis (1986), Le & Zidek (1992) and Handcock & Stein (1993), while more recent developments include De Oliveira, Kedem & Short (1997) and Ecker & Gelfand (1997). These works specify prior distributions for model parameters using a combination of intuition and ad hoc methods. More recently, Berger, De Oliveira & Sans6 (2001) provided an extensive discussion on theoretical issues involved in the Bayesian analysis of Gaussian random fields, where in particular it is shown that specification of the prior distribution is a somewhat delicate issue. On the one hand, it is difficult to carry out subjective elicitation of the prior distribution, either because of a lack of prior information or the difficulty in interpreting some of the parameters, while on the other hand, a naive specification of the prior distribution may give rise to an improper posterior.

An attractive strategy to overcome these difficulties and one that has seen significant devel opments in recent years is the use of "objective Bayesian analysis." This is based on default or automatic prior distributions derived by formal rules that use the structure of the problem at hand, but do not require subjective prior elicitation. Two of the most popular of these methods are the Jeifreys prior (Jeifreys 1961) and the reference prior (Bernardo 1979; Berger & Bernardo 1992; Bemnardo & Smith 1994). An objective Bayesian analysis of Gaussian random fields was under taken by Berger, De Oliveira & Sanso (2001) for the analysis of geostatistical data containing no measurement errors.

The present work deals with default Bayesian analysis of Gaussian random fields used to



284 DE OLIVEIRA Vol. 35, No. 2

model geostatistical data that contain errors of measurement. A reference and two versions of the Jeffreys prior are computed for these models and several properties of these priors and re sulting posteriors are studied. Results regarding the propriety of posterior distributions based on these default priors are obtained, as well as results regarding the existence of moments of the posterior and predictive distributions. For the derivation and study of the reference prior, a representation for the integrated likelihood of the covariance parameters is used that is particu larly convenient for computation and analysis. This representation has been previously used for the study of restricted maximum likelihood estimation in certain Gaussian linear mixed models (Dietrich 1991; Kuo 1999), but its use for deriving and analyzing default priors appears to be new.

Reference and Jeffreys priors would, in general, depend on some aspects of the experimental design (such as sample size), an issue sometimes raised as a critique to these priors. It is shown through numerical exploration that for the model considered here, the sensitivity of the reference and Jeffreys priors on several aspects of the design and model is very mild (with one exception), so the above issue bears little or no practical relevance.

A numerical experiment is performed to investigate frequentist properties of Bayesian infer ences based on the above default priors. It is found that frequentist properties of the reference prior and one version of Jeffreys prior are adequate and similar to each other, while frequentist properties of Bayesian inferences based on the second version of Jeffreys are inadequate when the model has nonconstant mean or strong dependence.

Finally, a data set of carbon-nitrogen ratios from an agricultural field is used to illustrate the Bayesian analysis based on the reference prior, as well as a comparison with a standard analysis based on kriging. The proofs of the results are given in the Appendix.

2. THE MODEL

The quantity of interest is assumed to vary continuously over a region of space D c RC, e > 1, and this variation is modelled by a Gaussian random field {Y(s), s C D} with

p E{Y(s)} = Zjfj(s) and cov{Y(s),Y(u)} = u2K(s, u),

j=1

where f(s) = (fi(s), . S. ,fp(s))T are known location-dependent covariates, /3 =

(1 .,P)T Rp are unknown regression parameters, 2 = var{Y(s)} > 0 is unknown

and K(s, u) is a correlation function in Re. Commonly used models use isotropic parametric correlation functions, where K(s, u) = K+(d), with d = s- ull (Euclidean distance) and 4

is equal to b1 or (01, b2). An example is given by the Matern family of isotropic correlation functions

K+p(d) 2+2 F (02) (d1) d 2 ( ), ) 1 > , 02 > 07 (1)

where r is the gamma function and K02 iS the modified Bessel function of second kind and order 02. For this and many other families, b1 controls how fast the correlation decays with distance (the so-called range parameter), and ?'2 controls geometric properties of the random field, such as differentiability (the so-called smoothness or roughness parameter).

The data consist of noisy measurements of the random field taken at known sampling loca tions sl, . . ., s,, E X, denoted by Y = (Yl,obs, . .. , Yn,obs)T, where

Yi,obS =Y(si) +si , i =1, .. .,n; (2)

here {62}=l_ ld N(O, a2) represent measurement errors distributed independently of the random field Y(.*) and o2 ? 0 is the so-called nugget effiect.



2007 OBJECTIVE BAYESIAN ANALYSIS 285

Berger, De Oliveira & Sanso (2001) derived reference and Jeffreys priors for models having

the range parameter as the only unknown correlation parameter and no nugget effect, so 02 iS known and o2 = 0. This work considers models where b is assumed known, but have an

unknown nugget effect. If the covariance matrix of the observed data is parameterized in terms of o2 and 9 = a2/a2 the so-called signal-to-noise ratio, it follows that

(Y 1 ,a2,

0)Nn(l3aQ) (3)

where (X)ij = fj (si) is a known n x p design matrix of rank p and Qo = In + OH, with In being the n x n identity matrix and (H)ij = K(si, sj) a known n x n positive definite matrix;

the spatial model is then parameterized by rq = (,/, or, 0) E RP x (0, 1) X [0 c).

Geostatistical models used in practice often include unknown parameter(s) in the correlation function K (such as q in (1)), in which case H is unknown and depends nonlinearly on these

extra parameter(s). A default Bayesian analysis for these models is quite challenging. A possible way to proceed in this case is a two-stage approach: First estimate the parameter(s) in K, say

by least squares or restricted maximum likelihood, and then perform a default Bayesian analysis on the resulting model when the parameter estimates in the first stage are assumed known; see

Section 8. A similar treatment of parameters in K has been advocated by Nychka (2000) who

claimed that spatial predictors are often more sensitive to 0 than to parameters in K.

3. INTEGRATED LIKELIHOODS

The likelihood of the parameters T based on the observed data y is given by

L(Ti; y) xy (a2Y-'2IQeK112 eXP{-j-2 (Y-X/3)TQl(y-Xi3)}, (4)

where IAl denotes the determinant of matrix A. I consider integrated likelihoods with respect to the class of prior distributions given by

7r(n) cc -(9) (5) (or2)a'

where a E R is a hyperparameter and 7r(0) is loosely interpreted as the marginal prior of 0. The relevance of this class of priors will become clear in the next section when it is shown that several default priors are members of this class.

From standard calculations it follows that the log-integrated likelihood of (ao, 9) with respect to the prior (5) is given, up to an additive constant, by

?I(a,2 9; Y) =2-2{(n-p) log(o2) + log(lQO e) + log(IXTIQlXI) + }X (6)

where S2 - T (QI -

_ Ql -X(XTQ lX)-IXT l)y. Likewise, the integrated likelihood of 0 with respect to the prior (5) is given by

L'(9;y) (C (IQO I XTQ 1XI) /22S< +a-1) (7)

The computation of integrated likelihoods and reference priors in Berger, De Oliveira & Sanso (2001), as well as thie study of their properties, is based on expressions (6) and (7). These expressions are not particularly convenient for computation and analysis, so we will consider simpler alternative representations based on the following result (Verbyla 1990; Dietrich 1991; Kuo 1999).




LEMMA 1. Let X be a full rank n x p matrix, with n > p, and E an n x n symmetric positive define matrix. Then, there exist a full rank n x (n - p) matrix L satisfying LTX = 0 and LTL = In_,p for which the following also hold:

(i) E - _-lx(xTE-lX)-lXTE-1 = L(LTEL)'LT.

(ii) log(IEI) + log(jXTE-lXj) = log(ILTELj) + c, where c depends on X but not on E.

(iii) LT>EL = D, where D is an (n - p) x (n - p) diagonal matrix with positive diagonal elements.

Applying Lemma 1 with E = Qo, we have LTQOL - In_p ?+LTHL, and hence by part (ii)

n-p

log(IQOI) + log(IXTQ lXl) = E log(1 + 0() ? c, (8) j=1

where 6> > * n* > 0 are the ordered eigenvalues of LTHL. Also, from (3) and part (iii)

LTY Nn_P(O, o6diag (1 + 0*(j)),

where o2* and 0* are the true values of, respectively, a 2 and 0, so from this and part (i)

S2= (LTY)T (LTQoL)-1 (LTY) E ((LTY) )2 j=1

d n-p1 1I + 0* d a2 E ( 1 + v1,3 )Zj2 (9)

j=1 o '

where {j }=n l ld N(0, 1) and denotes equality in distribution. Plugging (8) and (9) into (6)

we have that, up to an additive constant, for any a2 and 0

r 7L~~~~-P( ~ 2'1 9 \ l

tI ('<2>?; y)- [(-dlo(2) + E log(, + 0(j) + 1e (I +0*T))j2} (1 ?'(cr~,9;Y) (n+ - p+ log ()aI. (10)

This alternative representation for the log-integrated likelihood of (Cr2o 0) is more amenable for computation and analysis than the standard representation (6).

It follows that the posterior distribution of r based on prior (5) is proper if and only if

0 < J L'(0;y)ir(0)dO < o, (11)

so posterior propriety is determined by the behaviour of LI'(; y) and 7r(0) on [0, oo). The behaviour of LI (H; y) is described next.

PROPOSITION 1. Consider model (3) with prior distribution (5). Then, LI (0; y) is a continuous

fiunction on [0, oo) and LI (0; y) = O(9a- ) as 0 -* oo.

The above result and (11) provide the basis for determiniing whether a prior distribution from class (5) yields a proper posterior distribution. For instance, when prior (5) with a = 1 is used, L' (9; y) is bounded away from zero everywhere, so the resulting posterior distribution is proper if and only if 7r(9) is integrable on [0, oo). In this case naive default choices such as ir(9) oc 1 or ir(9) oc 1/9 produce improper posterior distributions, so sensible default priors are needed.




4. DEFAULT PRIORS

4.1. Reference prior.

In this section a reference prior for the parameters of model (3) is derived, where we con sider (or, 9) as the parameter of interest and /3 as the nuisance parameter. For that we use the two-step reference prior algorithm that uses exact marginalization, as described in Berger,

De Oliveira & Sanso (2001). The first step is to factor the joint prior distribution as 7r(,q) =

rR(O j 2, 0)9rR(a2, 0) and use irR(3 1 2, 9) )C 1, since this is the conditional Jeffreys-rule (or reference) prior for ,3 for model (3) when (a2, 0) is known (Bernardo & Smith 1994). Second, 7R(a2, 0) is computed using the Jeffreys-rule algorithm based on the marginal model provided by the integrated likelihood of (a2, 9).

THEOREM 1. Consider model (3). Then, the reference prior of rq, as described above, is of the form (5) with

a=1I and r-rR(9)c OCE ~i~ (12) ( ) [(1 +/) n-p{( + 0y)

What follows provides the main properties of the above reference prior and those of its cor responding reference posterior.

PROPOSITION 2. Suppose that {,j }j= are not all equal. Then, the marginal reference prior of 9 given in (12) is a continuous function on [0, oo) satisfying:

(i) irR(9) is strictly decreasing on [0, cc).

(ii) rR(9) = o(9-2) as 0 -- oo.

COROLLARY 1. Consider model (3) and let k > 1. Then:

(i) The marginal reference prior irR(9) and joint reference posterior TR(ri I y) are both

proper:

(ii) The marginal reference posterior irR(9 i y) does not have moments of any order k.

(iii) The marginal reference posterior rrR(of2 I y) has a finite moment of order k if n > p + 2k + 1.

Remark 1. The derivation of the reference prior in Theorem 1 uses exact marginalization in the reference prior algorithm instead of the more typical asymptotic marginalization, as described in Berger & Bernardo (1992). Using the latter, van der Linde (2000) derived a reference prior for a model similar to model (3) which differs from the reference prior in (12). As it turns out, van der Linde's reference prior [her equation (7)] is the same as the independence Jeffreys prior, as given in Theorem 2 of the next section.

4.2. Jeffreys priors. The Jeffreys-rule prior is given by 1rr(r1) cc II(r,)I1/2, where I(n~) is the Fisher information matrix with (i, j) entry

(I(77))ij = -E { (Or,9 1i,og(L(tl; Y))) }.@




In this section the Jeffreys-rule prior and the independence Jeffreys prior, are derived, where the latter is obtained by assuming that ,3 and (c2c, 0) are independent a priori and computing the

marginal prior of each parameter using Jeffreys-rule when the other parameter is known. In what follows A1 > ... > An > 0 denote the ordered eigenvalues of H.

THEOREM 2. Consider model (3). Then the independence Jeffreys prior and the Jeffreys-rule prior of rq, to be denoted by irJ1 (r) and gr J2 (r), are of the form (5) with, respectively,

a=1and 7rrA9)i Ai/2, (13) d l

tJ() (:x E 1 + OAi

) n {(1 + OAi )}](3

and

a P and rrJ2 (9) () / ) (14)

What follows provides the main properties of the independence Jeifreys and Jeifreys-rule priors, and those of their corresponding posterior distributions.

PROPOSITION 3. Suppose that {Ai}.=1 are not all equal. Then, the marginal independence

Jeffreys and Jeffreys-rule priors of 9 given in, respectively, (13) and (14) are continuous functions on [0, oo) satisfying:

(i) irhj(9) and iXJ2 (0) are strictly decreasing on [0, xc).

(ii) 7rJ1(0) = O(9-2) as 0 -* oo.

(ii) 7rJ2(0) = Q(0-(2+2)) as 0 -* oo.


(i) The marginal independence Jeffreys prior 7rJ1 (0) and joint independence Jeffreys poste rior irJl (77 I Y) are both proper.

(ii) The marginal independence Jeifreys posterior 7rjl (91 y) does not have moments of any order k.

(iii) The marginal independence Jeffreys posterior 7rJ1 (U2 y) has a finite moment of order k if n > p+2k+ 1.


(i) The marginal Jeffreys-rule prior IrJ2(0) and joint Jeffreys-rule posterior 7rJ2(r1 I y) are both proper.

(ii) The marginal Jeifreys-rule posterior 1rr2(9 I y) has a finite moment of order k if k < 1 ? p/2.

(iii) The marginal Jeifreys-rule posterior wrJ2(af6 e y) has a finite moment of order k if n > p +2k?+ 1.




Remark 2. The above corollaries show that Bayesian inference about 0 using reference or Jeffreys priors cannot be summarized using posterior moments, as is commonly done in practice, so posterior quantiles or modes should be used instead. In this regard, an estimate computed in van

der Linde (2000, p. 255) of 0 [o in that paper's notation] is invalid. Using the independence Jeffreys prior (13) 0 was estimated by E (9 I y), but by Corollary 2 this does not exist.

Remark 3. Because of part (i) of Propositions 2 and 3, the above default priors can be considered conservative regarding their information content about 0, in that these priors assign substantial probability to values of 0 close to zero that represent little or no spatial association..

5. PREDICTIVE INFERENCE

Spatial prediction is the most common goal in the analysis of geostatistical data. If so E D is a location where prediction is sought, Bayesian inference about Yo = Y(so) is based on the

predictive density function

lr(yo I = JP(Yo I ', y)lr(? I Y) dq. (15)

Predictive inference is commonly summarized using the mean and variance of this predictive distribution, where existence of these predictive summaries is almost always taken for granted. But it is not clear in general whether or not these exist. The following result provides sufficient conditions for existence of these predictive summaries under the class of priors (5).

PROPOSITION 4. Consider model (3) with n > p + 3. Suppose prior (5) is used with a E R and ir(9) a continuous function on [0, oo), with 7r(9) = Q(9-b) as 0 - oo for some b E ]R. Then for any so E 2D, r(yo y y) has afinite second moment if a < b.

COROLLARY 4. Consider model (3) with n > p + 3. For any so E D the predictive distribu tions of Y(so) based on the reference, independence Jeifreys and Jeffreys-rule priors have finite variance.

6. COMPUTATION AND SIMULATION

6.1. Computation. The computation of the reference and Jeffreys-rule priors requires the matrix L, which can be efficiently and stably computed by the following two steps. First, compute the QR decomposition of the matrix X (Dietrich 1991; Schott 2005)

x =(Qi Q2)()

where R is a p x p upper triangular matrix and (Ql Q2) is an n x n orthogonal matrix, with Ql n x p and Q2 n x (n - p), whose columns form orthonormal bases for, respectively, the column space of X and its orthogonal complement. Second, compute the spectral decomposition of

Q2THQ2, Q2THQ2 = UAUT, where U is an (n - p) x (n - p) orthogonal matrix, UTU UUT = In_p and A = diag (1, .. , np) the latter holds since Q2THQ2 and LTHL have the same eigenvalues. Then L = Q2U. satisfies all the properties stated in Lemma 1.

6.2. Simulation. Posterior inference about the model parameters would rely on a Markov chain Monte Carlo algorithm based on factoring the posterior distribution as

ir(f3, as,9Iy) = ir(f3j2,90, y>(as2 |9, y)ir(91|y),




where from (4) and (5)

r(f3jo O2,y) = Np(23e,cT(XTQolXyl), (16)

ir(o2I9,y) = IG ( P +a-11 ) S) (17)

(0 I(Y) o (IQoI IxT 21xI) r/2(s (2+a-) r(), (18)

where ,38 = (XTQlX)lXTQ -ly and IG(a, b) denotes the inverse gamma distribution with

mean b/(a - 1). Simulation from (16) and (17) is straightforward, while simulation from (18)

would be accomplished using the adaptive rejection Metropolis sampling (ARMS) algorithm proposed by Gilks, Best & Tan (1995).

It is worth noting that assuming H known and the availability of formulas (8) and (9) make

it possible to perform posterior inference about covariance parameters with computational com

plexity 0(n), instead of the standard 0(n3) that is required for general geostatistical models. This allows Bayesian fitting of this model to large data sets that are not feasible for general

models.

(a) (b)

prior dergni o - c _ reference MrKign ' ---- indep. Jeffreys ---- regular

_Jefreys-rule ..... red

0 5 10 Is 0 5 10 15

a H

FIGURE 1: (a) Marginal reference, independence Jeffreys and Jeffreys-rule priors of 0, and (b) Marginal

reference priors of 0 corresponding to the sampling designs in Figure 2.

7. FURTHER PROPERTIES

This section compares the proposed default priors and provides some additional properties. Un less stated otherwise, all the computations and simulations of this section use model (3) with

n = 100 randomly selected sampling locations within 2D = [0,1] x [0,1]. Also, all displayed

reference priors of 9 are normalized to make them probability density functions.

Figure 1(a) displays the marginal reference, independence Jeffreys and Jeffreys-rule priors of 0 for the model with E {Y(s)} = fl + 32x + 38y, s = (x, y) E D, and K(d) = exp(-d/0.25).

The reference and independence Jeffreys priors are very close to each other, while the JefFreys rule prior differs more substantially from the above two. For many other models and designs

the same pattern was found, namely, when E {Y(s)} is not constant, ,rR(0) and 7rJ' (0) are very

close to each other, while 7rJ2(0) differs from the above two. Also, 7rJ2(0) is very conservative in that it assigns most of its mass to values of 9 close to zero, while rr11 (9) and rrR(9) are less conservative. For instance, for the above model PJ2(9 < 1) t 0.86, while pJl (9 < 1) t 0.49 and PR(0 K 1) 0.46. When E {Y(s)} is constant, the three default priors are close to each other.




7.1. Sensitivity to design and model.

The reference and Jeffreys priors obtained in the previous sections depend, in principle, on sev eral features of the selected design and model, such as sampling design, sample size, design matrix and correlation function. This section describes a numerical exploration to determine how sensitive these default priors are to the above features.

. .. I. . . .r....

z 1 ;~~~~~. . . .; . . ... I s1 1 . . .

0l * ** 0 * ** I o 1 1 ol * *s-.

FIGURE 2: Sampling designs within D =[0, 1] x [O, 1]: (a) random, (b) regular, (c) clustered.

---- 10_ 0---3

40 501 0 6s05 0

a 00~~0

FIGURE 3: Marginal reference priors of 0 corresponding to (a) different sample sizes, and (b) different design matrices.

Sampling design. Consider the model with E {Y(s)} = 31 + 32x + 33y and K(d) = exp(-d/0.25), and the sampling designs shown in Figure 2. Figure l(b) displays the marginal reference priors of 0 corresponding to these sampling designs, showing that they are very close to each other.

Sample size. Consider the model witi E {Y(s)} = i31 and K(d) = exp(-d/0.25), and nested sampling designs with sample sizes n = 50, n = 100 and n = 500. Figure 3(a) displays the marginal reference priors 7rR(9) corresponding to these sample sizes, showing that they are very close to each other.

Design matrix. Consider the models with K(d) = exp(-d/0.25) and mean functions E {Y(s)} =31 (p = 1), E{Y(s)} = 31 +/:2X?+/33Y(p= 3) and E{Y(s)} =31?+1:2x?+ /5338 ? /34x2 ? [5y2 + /36xy (p-=6). Figure 3(b) displays the marginal reference priors rrR(9) corresponding to these models, showing that they are very close to each other.




Correlation function. Consider the models with E {Y(s)} = 0f1 and correlation functions from the Matern family, as given in (1). Figure 4(a) displays marginal reference priors 7rR(0) corre sponding to 012 = 0.5 and range parameters 41 0.1, 0.3 and 0.6. These priors are very close to each other. Figure 4(b) displays marginal reference priors 7rR(9) corresponding to 41 = 0.25 and smoothness parameters 412 = 0.5,1.5 and 2.5. Once again, these priors are very close to each other.

(a) (b)

smoothness mangein|......9 m -~~~~~~~~~~~~~0.5

_ '5' 0.6

---- 0.301.5 X I~~~~~~~.... 0. .... 2

0 5 10 15 0 5 10 15

0 0

FIGURE 4: Marginal reference priors of 0 corresponding to the Matern correlation function with (a) 012 = 0.5 and three range parameters, and (b) q$ 0.25 and three smoothness parameters.

Similar comparisons as the ones described above were done to investigate sensitivity of the independence Jeffreys and Jeffreys-rule priors (not shown). Both priors displayed the same lack of sensitivity to all of the above design and model features, except in one case: the marginal Jeffreys-rule prior rJ2 (0) displayed substantial sensitivity to changes in the design matrix X.

7.2. Frequentist properties.

This section presents results of a simulation experiment to study some of the frequentist prop erties of Bayesian inferences based on reference, independence Jeffreys and Jeffreys-rule priors. These properties are often proposed as a way to evaluate and compare default priors. The focus of interest is on the covariance parameters, and the frequentist properties to be considered are frequentist coverage of Bayesian credible intervals and mean absolute error of Bayesian estima tors. For or, we use 95% equal-tailed credible intervals and the (marginal) posterior median as estimator, while for 0, we use 95% highest probability density credible intervals and the (mar ginal) posterior mode as estimator. The latter are used because the posterior density of 0 is highly skewed with a very heavy right tail.

The factors to be varied in the experiment are E {Y(s)}, or2a and 0. We consider E {Y(s)} to be 0.15 (p = 1) or 0.15-0.65x-0.ly+0.9x2+1.2y2 -xy (p = 6), a2 is 0.1 or 2, and 0 is 0.1, 1 or 10; all the data were simulated from models with correlation function K(d) = exp(-d/0.25). This setup provides a range of different scenarios in terms of trend, variability and signal-to-noise ratio. For each of the 12 (2 x 2 x 3) possible scenarios, 3000 data sets were simulated and for each data set a posterior sample of the model parameters of size m = 4000 was generated by the algorithm described in Section 6.2.




TABLE 1: Frequentist coverage and [average length] of Bayesian equal-tailed 95% credible interVals for or.

P= 1 P=6

0 0.1 1 10 0.1 1 10

2 0.1

Reference .943 .929 .942 .921 .940 .940

[.064] [.088] [.184] [.067] [.091] [.191]

Ind. Jeffreys .945 .929 .930 .940 .941 .938

[.064] [.089] [.185] [.067] [.094] [.194]

Jeffreys-rule .950 .928 .940 .940 .879 .858

[.063] [.090] [.190] [.058] [.090] [.237]

a2 = 2

Reference .938 .934 .940 .926 .936 .940

[1.283] [1.770] [3.664] [1.344] [1.843] [3.805]

Ind. Jeffreys .941 .940 .942 .939 .939 .938

[1.288] [1.777] [3.675] [1.354] [1.891] [3.873]

Jeffreys-rule .949 .937 .939 .939 .856 .861

[1.266] [1.791] [3.769] [1.157] [1.827] [4.743]

TABLE 2: Frequentist coverage and [average length] of Bayesian highest probability density 95% credible intervals for 0.

p= p =6

0 0.1 1 10 0.1 1 10

a2 = 0.1

Reference .966 .910 .937 .947 .932 .931

[.786] [3.638] [55.770] [1.470] [4.087] [57.684]

Ind. Jeffreys .973 .910 .933 .973 .906 .926

[.785] [3.601] [55.710] [1.267] [4.547] [56.715]

Jeffreys-rule .984 .884 .922 .957 .476 .779

[.596] [3.145] [52.474]- [.317] [1.651] [37.840] 2

Reference .962 .920 .941 .945 .935 .932

[.786] [3.524] [56.432] [1.527] [4.818] [58.323]

Ind. Jeffreys .963 .915 .938 .972 .920 .930

[.760] [3.500] [56.255] [1.339] [4.581] [57.475]

Jeffreys-rule .978 .882 .931 .959 .488 .779

[.595] [3.050] [53.032] [.320] [1.658] [38.123]

Table 1 shows frequentist coverage and average length of Bayesian 95% credible intervals for a2 corresponding to the three default priors. The coverage of the three credible intervals are similar to each other under most scenarios, being. slightly below the nominal .95, except w.hen E {Y(s)} is not constant and 0 is large. In this case Jeifreys-rule credible intervals have coverage




well below nominal. The average lengths of the three credible intervals are also similar to each other in most scenarios, except when 0 is large, in which case Jeffreys-rule credible intervals tend to be slightly wider than the other two.

Table 2 shows frequentist coverage and the average length of Bayesian 95% credible intervals for 0 corresponding to the three default priors. The coverage of reference and independence Jeffreys credible intervals are reasonably close to the nominal .95 in all scenarios, while the coverage of Jeffreys-rule credible intervals is well below nominal when E {Y(s)} is not constant and 0 is large. The average lengths of reference and independence Jeffreys credible intervals are similar to each other in all scenarios, while Jeffreys-rule credible intervals tend to be narrower than the other two, substantially so when E {Y(s)} is not constant. This reduction in length comes at the expense of having poor frequentist coverage when E {Y(s)} is not constant or 0 is large.

Tables 3 and 4 show the mean absolute error of Bayesian estimators for, respectively, ac2 and 0, corresponding to the three default priors. For both parameters the mean absolute errors of the three estimators are close to each other in most scenanos, except when E {Y(s)} is not constant and 9 is large. In this case the mean absolute error of JefFreys-rule estimator is substantially larger than those of the other two estimators.

TABLE 3: Mean absolute error of the posterior median of o,.

p=1 p=6

0 0.1 1 10 0.1 1 10

2 0.1

Reference .013 .018 .041 .015 .018 .042

Ind. Jeffreys .013 .018 .041 .014 .019 .043

Jeffreys-rule .012. .019 .042 .012 .023 .067 2 2_

Reference .259 .371 .797 .297 .379 .833

Ind. Jeffreys .257 .373 .800 .278 .388 .853

Jeffreys-rule .249 .379 .827 .240 .472 1.347

TABLE 4: Mean absolute error of the posterior mode of 0.

p=l p=6

0 0.1 1 10 0.1 1 10

0 .1

Reference .087 .514 4.599 .120 .608 4.856

Ind. Jeffreys .091 .521 4.608 .113 .670 4.923

Jeffreys-rule .090 .549 4.795 .097 .913 6.427

- = 2

Reference .091 .499 4.600 .121 .609 4.864

Ind. Jeffreys .094 .505 4.610 .113 .672 4.936

Jeifreys-rule .093 .534 4.807 .097 .909 6.460

In summary, frequentist properties of Bayesian inferences based on the reference and in dependence Jeifreys priors are quite similar and moderately good, but frequentist properties of




inferences based on the Jeffreys-rule prior are inadequate when E {Y(s)} is not constant or 0 is

large.

8. EXAMPLE

This section presents a Bayesian analysis, based on the reference prior, of a data set analyzed by

Schabenberger & Gotway (2005, ch. 4) as well as comparisons with a classical analysis. The data

set consists of ratios of total soil carbon and total soil nitrogen percentages (C/N ratios) measured

at 195 locations in an agricultural field of about 500 x 300 feet; the sampling locations are

displayed as dots in Figure 5. The exploratory data analysis done by Schabenberger & Gotway

suggests that a reasonable model for these data is a Gaussian random field with constant mean,

isotropic exponential covariance function and nugget effect, where the latter is interpreted here as measurement error. That is, the data follow (2) with

E {Y(s)} = /3 and cov{Y(s), Y(u)} = u2 exp _ d

The maximum likelihood (ML) and restricted maximum likelihood (REML) estimates of the model parameters are given in Table 5. These estimates are very close to each other, with the

exception of the estimates for q1 which, as is commonly the case, is the most difficult to estimate.

TABLE 5: Maximum likelihood and restricted maximum likelihood parameter estimates.

y )1

ML 10.850 0.202 47.200 0.113

REML 10.859 0.215 57.331 0.118

Using the reference prior (12), a posterior sample of size 20000 for the model parameters

was obtained using the algorithm described in Section 6.2. To implement the algorithm q1 was

set equal to its REML estimate, q1 = 57.331. Graphical exploration of the Markov chain Monte

Carlo output (not shown) reveals that the algorithm produces well-mixed chains with low auto

correlations. The posterior distribution of 13 is essentially symmetric, the posterior distribution of o2 is just mildly asymmetric, while the posterior distribution of 0 is highly asymmetric with a

very heavy right tail.

TABLE 6: Posterior quantiles of the model parameters based on the reference prior for several choices

of+1.

q5 = 57.331 1= 47.874 q1 = 66.787

2.5% 50% 97.5% 2.5% 50% 97.5% 2.5% 50% 97.5%

la 10.598 10.858 11.128 10.616 10.850 11.083 10.566 10.865 11.172 2 0.078 0.118 0.168 0.070 0.112 0.161 0.082 0.122 0.172

0 0.817 1.826 4.388 0.841 1.874 4.646 0.803 1.863 4.310

Table 6 (left) provides posterior summaries for the model parameters where it is noted that,

for this data set, Bayesian and likelihood-based estimates are very close (9m1 - 1.788, 9reml = 1.822). To assess the sensitivity of inferences about model parameters to the choice of 4i, the above Bayesian reference analysis was done for two other values of +b1 corresponding to the end points of a large sample (approximate) 90%o confidence interval: 57.331 ?t 1.645A/33.04661 = (47.874, 66.787). Table 6 (center and right) provides posterior summaries for these analyses




which show that for this data set, inference about the model parameters is not very sensitive to the choice of q1. This gives additional credence to the results and suggests that the particular choice of 41 has little or no practical relevance for inference about the other parameters.

(a) (b)

1 105 ID 11 1ts 010. 0 S3504

I~~~~~~~~~~~~~~

I

|_ 'lit l_ ............~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~... . ..... .:.

-, , =g1s_ ..~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~... ... ...... . ,- ..

I1 11< i1H1 I|rI 1Ii

0 100 200 300 400 S0o 0 100 200 300 400 Soo

x-ooed (l" X-00o4. "no

FIGURE 5: Bayesian predictive summaries based on the reference prior. (a) Map of predicted C/N ratios, and (b) Map of uncertainty measure.

To investigate the spatial variation of C/N ratios throughout the region, the mean and stan dard deviation of the predictive distribution of Y(so) were computed for each location so in a 51 x 30 regular grid (every 10 feet). Samples from these predictive distributions are straight forward to simulate using (15) and the sample of the posterior distribution of the parameters. Figure 5(a) displays the map of predicted C/N ratios and Figure 5(b) displays the map of predic tion uncertainty. For comparison, maps of the ordinary kriging predictor and ordinary kriging standard deviation were also computed on the same grid (not shown). These kriging summaries are given, respectively, by (22) and the square root of (23) with a2, 0 and q1 set at their REML estimates. Inspection of these maps reveals that Bayesian and kriging maps are quite similar. The absolute value of the difference between Bayesian and kriging predictions is smaller than .025 E {Y(so) I y} for all locations. As expected, Bayesian predictive standard deviations are

larger than kriging standard deviations almost everywhere (99% of locations), but by a small amount: the absolute value of the difference between Bayesian and kriging standard deviations is smaller than .025 sd {Y(so) I y} in about 84% of locations.

To assess the sensitivity of predictive inferences to the choice of 41, the mean and standard deviation of the predictive distribution of Y(so) were computed on the same grid for the two other values of q1 considered above (not shown). Inspection of the predictive summary maps obtained for f1 = 57.331 and 01 = 47.874 reveals that they are quite similar. The absolute value of the difference between the two Bayesian predictions is smaller than .025 E {Y(so) I y, (k =

57.331} for all locations. The predictive standard deviations for f1 = 57.331 are smaller than those for /1 = 47.874 almost everywhere, but by a small amount: the absolute value of the difference between the two standard deviations is smaller than .05 sd {Y(so) I y, 41 = 57.331} in about 96% of locations. Finally, comparisons between predictive summary maps obtained for q51 = 57.331 and k1 = 66.787 are quite similar to those obtained for q1 = 57.331 and

ol = 47.874. The only qualitative difference is that in this case the predictive standard deviations for e1 = 57.331 are larger than those for e1 = 66.787 almost everywhere.

9. CONCLUSIONS This work provides default priors for the parameters of Gaussian random fields based on geosta tistical data that contain errors of measurement and it derives several properties of thie resulting




posterior and predictive distributions. These default priors yield proper posteriors and display little sensitivity to several aspects of the design and model. In addition, the reference and in dependence Jeffreys priors are in general similar to each other and Bayesian inferences based on them have moderately good frequentist properties. Any of these is equally recommended for use in practice. In contrast, Bayesian inferences based on the Jeffreys-rule prior has inadequate frequentist properties when the model has nonconstant mean or strong dependence. The priors proposed here and those in Berger, De Oliveira & Sanso (2001) may serve as building blocks for tackling the much more challenging problem of performing default Bayesian analysis of models with general isotropic covariance functions.

A possible critique to the Bayesian methodology proposed here is its reliance on a two-stage approach in which parameters in K(*) are treated as known. The results from the data analysis of C/N ratios suggest that this might not be of much practical concern when dealing with range parameters (such as b1), as predictive inference did not display much sensitivity to the choice of q1. But the same is not expected to hold when dealing with smoothness parameters (such as 02 in the Matern family); an extensive analysis and discussion of this issue is given by Stein

(1999, ch. 3).

APPENDIX

Proof of Proposition 1. Direct inspection of (7) shows that LI (0; y) is continuous on [0, oc). From (8) we have that as 0 -4oo

n-p

IQjI IXT01X lx| (1 + 09) - Q(9n p) j=1

since JJn-i (j > 0. Also from (9), we have that as 0 - oo

Eoc _ + 0 - ) o (= ) (19)

since .=(1 + 0* )Z? /1j > 0. The result then follows.

Proof of Theorem 1. The reference prior of r, described in Section 4.1 is given by 7rR(?,) oc III(a2, 9)11/2, where II(c2, 9) is the Fisher information matrix with (i, j) entry

{ 2 (I I(ae2 0))ij =-E {ag a I

'(0,2 0;Y)} '1 =a 2, '02 = O-20 aV ajI(9.\L1 {6, 9 (20)

Using the representation for the log-integrated likelihood given in (10) we have

(II(' 0))ii =

(II(O 0))12 = ,2E j=1 ( + 16

where the expectations in (20) were evaluated for acr* - a2 and90*=S. Then

(II c2 _ A n I




1 np ( 2 1 n-p( 2g } 1/2

LEMMA 2. If a, < a2 < * ? ar and b1 < b2 < ... < br are monotone sequences of positive numbers, then aj) (E 1 bj) < r . a3b3. Equality holds only if aj = a or bj = b

for all j =1,...,r.

Proof. Valliant, Dorfman & Royall (2000, p. 128).

Proof of Proposition 2. Direct inspection shows that irR(0) is continuous on [0, oo). To show (i), we have for 9(0) = ( 7rR(0))2 that

d(9) n2 [{nP( n-p 2)}{n( -) 32} - (n ( )3]

Now note that for any 0 > 0, aj = .n-p+i-j/(l + ?O,np+1-j) and bj = a?, j = 1 ... . n-p, are increasing sequences of positive numbers so dg(9)/d9 < 0 on [0, oo) follows from Lemma 2; this implies (i).

Now let,'i3 = 1/(j, j = 1,. . .,X n-p and c = E in- Kj;/(n-p). To show (ii) we use the

second-order Taylor series expansion of the functions x-1 and x-2, both around 1. Evaluating these at x = 1 + rj /9, it follows that as 0 oo

7rR(0) C x E 1 + E

1 + - 1

=~~ ~ 9?{192?o(93)}]13 -nPEP{ 1 +j+(l } 2] 1/2

= {(j _ R)2 + 0 ( 02

Proof of Corollary 1. Results (i) and (ii) follow directly from (11), and Propositions 1 and 2.

To show (iii), we note that E {(I2)k ( y} exists if E {(oJ)k I 9, y} exists and is integrable with

respect to 7rR(O y). By factoring the joint posterior distribution, we have

~~R(a219,.y) = rR(j3 U, 0 1y)

rrl a2' 9, y)Ce R j y)

LL(f3, 42 9; y)/U2 rR( 02 70 ( y)

cX (Ce2 (a2 +1)exp

and the last identity follows since

ir(3 I 49 , y) = Np(p9, o2 (XBQ)1X)-)




where'-' = (XTQ lX)-lXTQ ly. Then

7rR(a2 1 0, Y) = IG SO

where IG (a, b) denotes the inverse gamma distribution with mean b/(a - 1). [The statement of this result in van der Linde (2000, p. 252) has a mistake. In that paper's notation and equation numbering, the exponent of C2 in equations (10), (13) and (16) should be-(E=E + 1) rather than -N/2.] Hence direct calculation shows that

E {(a0 )k 9 0, y} = C(SO2)k < 0, (21)

provided n > p + 2k + 1, where c > 0 does not depend on (9, y). Finally, note that both S0 and 7rR(9 I y) are continuous functions on [0, oo). From (19) and Propositions 1 and 2, we have that as9-+ cc, S02

= O(9-') and iR(9 y) - Q(92), so E {(0f)k 9 0, y} is integrable with

respect to 7rR(9 I y)

Proof of Theorem 2. From Berger, De Oliveira & Sanso (2001, Th. 5), we have for the spatial model (3) that the independence Jeffreys prior and Jeffreys-rule prior are both of the form (5) with, respectively,

1/2 a=1 and 7r'l(9)oc [tr(Ue2)--{tr(Uo)}2J

and a = 1 + and rJ2(O) oc IXTQ lXll/27rJl(9)

where Us = (OQo0/O)Q 1 = H(I, + OH)-'. Since the eigenvalues of Uo are we have

tr(U) = E A) and tr(Uo2)= E Z(1A )

Also, from part (ii) of Lemma 1

IL n.= ( l + Oi )

and the results follow.

Proof of Proposition 3. Result (i) regarding strict monotonicity of 7rJ1 (9) and (ii) are proved in

essentially the same way as in the proof of Proposition 2. To show the strict monotonicity of

rJ2(9), we have for h(9) = {i=1iif(i

? 9(1)}/{fL t,(1 + OAi)} that

d n-p ( & n Ai) TOlog (h(0)) = + E 1+O

j= (1 +j 0( =1 + >) =-+l1+fA

Now by the Poincare separation theorem (Schott 2005, p. 111), (j < A3 for j = 1,... ., ni- p, so d log(h(9))/d9 <0O on [0, cc). The strict monotonicity of irJ2(9) follows from this and the strict




monotonicity irjl (0) since 7rJ2 (0) is the product of two strictly decreasing positive functions. To show (iii), we note that

h (0) = f l Th1 (? z (= ) 1 + 0Aj ) i n-p+l

I + 0Ai

= O( ) as 0 -oo,

since fl;n? ((j/Aj) > 0 and Hl=n p+i (i/As) > 0. The result follows from this and (ii).

Proof of Corollary 2. The same as the proof of Corollary 1, by using Propositions 1 and 3, since

rJ1 (0) and 7rR(0) have the same behaviour at infinity.

Proof of Corollary 3. The same as the proof of Corollary 2.

Proof of Proposition 4. It holds that E (Yo2 I y) exists if E (Y2 U

o2, 0, y) exists and

is integrable with respect to ir(o2, 9 1 y). Let xo = (fi(so), *... , fp (so))', ko =

(K(so, si), ... , K(so Sn))T and Qo = 0-'Qo. From the standard properties of multivariate

normal distributions, we have

E(Yo I4 0, y) (Oko + X(XTp lX)-l (xo _9XT-lko ))TQ_1

= (ko + X(XTB 1X)<1(xo - XT lko))T?2 y = 0(1) as -+oo, (22)

and

var(Yojo ,,2y) = 2 _(0_2kTQ lko

+ (Xo XTQ-l'ko)T(XTQlxy)-l(xo- XTQB 0ko))

= or (1-k T 1lko

+ (Xo -XTT4lko)T(XTQ4lx)l(xo1 - XTQ1-ko))

= oQ2ovO(9), say = a200(1) as 0 - oo. (23)

Noting that E {Yo a2, 0, y} does not depend on or, we have

j j {E (Yo c27, 9, y)}27r(_, 0 j y) d,2 dO = j f{E (Yo y, y)}2r(9 I y) dO,

and from (23), (21) and n > p + 3, we have

J j var(Yo I oQ, 9, y)7r(a2,9 H y) dU2 dO o J Ovo(9)S Sr(0 y) dO.

Since 7r(9 I y) oc LI (9; y)7r(9), it follows from Proposition 1, (22), (19) and (23) that the last two integrals are finite if a < b. The result follows since

E (Yo2IoQ2) ,9y) = var(YoI|a2,0, y)?+{ E(YoI as,a, Y)}2

Proof of Corollary 4. Follows immediately from Propositions 2, 3 and 4.




ACKNOWLEDGEMENTS I thank the Editor, Associate editor and referee for their comments and suggestions that lead to an improved article. I also thank Angelika van der Linde for her meticulous reading of a previous version of the article and numerous comments and suggestions. This work was partly supported by a U.S. National Science Foundation grant.

Received 30 May 2006 Victor DE OLIVEIRA: [email protected] Accepted 28 February 2007 Department of Management Science and Statistics

The University of Texas at San Antonio San Antonio, TX 78249, USA

REFERENCES J. O. Berger & J. M. Bernardo (1992). On the development of the reference prior method. In Bayesian Sta

tistics 4: Proceedings of the Fourth Valencia International Meeting, April 15-20,1991 (J. M. Bernardo, J. O. Berger, A. P. Dawid & A. F. M. Smith, eds.), Oxford University Press, pp. 35-60.

J. O. Berger, V. De Oliveira & B. Sans? (2001). Objective Bayesian analysis of spatially correlated data. Journal of the American Statistical Association, 96,1361-1374.

J. M. Bernardo (1979). Reference posterior distributions for Bayes inference (with discussion). Journal of the Royal Statistical Society Series B, 41, 113-147.

J. M. Bernardo & A. F. M. Smith (1994). Bayesian Theory. Wiley, New York.

V. De Oliveira, B. Kedem & D. A. Short (1997). Bayesian prediction of transformed Gaussian random fields. Journal of the American Statistical Association, 92,1422-1433.

C. R. Dietrich (1991). Modality of the restricted likelihood for spatial Gaussian random fields. Biometrika, 78,833-839.

M. D. Ecker & A. E. Gelfand (1997). Bayesian variogram modeling for an isotropic spatial process. Journal

of Agricultural, Biological and Environmental Statistics, 2,347-369. W. R. Gilks, N. G. Best & K. K. C. Tan (1995). Adaptive rejection Metropolis sampling within Gibbs

sampling. Applied Statistics, 44,455-472. M. S. Handcock & M. L. Stein (1993). A Bayesian analysis of kriging. Technometrics, 35,403-410. H. Jeffreys (1961). Theory of Probability. Oxford University Press.

P. K. Kitanidis (1986). Parameter uncertainty in estimation of spatial functions: Bayesian analysis. Water Resources Research, 22,449-507.

B.-S. Kuo (1999). Asymptotics of ML estimator for regression models with a stochastic trend component. Econometric Theory, 15, 24?49.

N. D. Le & J. V. Zidek (1992). Interpolation with uncertain spatial covariances: A Bayesian alternative to

kriging. Journal of Multivariate Analysis, 43, 351-374.

D. W. Nychka (2000). Spatial-process estimates as smoothers. In Smoothing and Regression: Approaches,

Computation, and Application (M. G. Schimek, ed.), Wiley, New York, pp. 393-424.

O. Schabenberger & C. A. Gotway (2005). Statistical Methods for Spatial Data Analysis. Chapman & Hall, London.

J. R. Schott (2005). Matrix Analysis for Statistics, Second edition. Wiley, New York.

M. L. Stein (1999). Interpolation of Spatial Data. Springer, New York.

R. Valliant, A. H. Dorfman & R. M. Royall (2000). Finite Population Sampling and Inference: A Prediction

Approach. Wiley, New York.

A. van der Linde (2000). Reference priors for shrinkage and smoothing parameters. Journal of Statistical

Planning and Inference, 90,245-274.

A. P. Verbyla (1990). A conditional derivation of residual maximum likelihood. Australian Journal of Statistics, 32,227-230.



objective bayesian analysis of spatial data with measurement error

Documents