emission tomography and bayesian inverse problemsmapjg/slides/bernoulli... · 2016. 8. 12. ·...

Emission tomographyand Bayesian inverse problems

Peter Green

University of Bristoland UTS, Sydney

Bernoulli Lecture

12 July 2012, Istanbul

Peter Green (Bristol/UTS) Bayesian inverse problems Istanbul, July 2012 1 / 70

Outline

1 The Bernoullis

2 Inverse problems from a Bayesian perspective

3 Single-photon emission computed tomography

4 Concentration and approximation of posterior in non-regular inverseproblems


The Bernoullis Family

The Bernoulli family



The Bernoulli family and the Bernoulli Society

The Bernoulli Society was formed in 1975 out of the InternationalAssociation for Statistics in Physical Sciences, an autonomous sectionof the ISI, the International Statistical Institute.

.. not out of an ISI section on Mathematical Statistics!




Jacob: ArsConjectandi – Law of large numbers. e




Johann: Calculus & li i& application to physical problems




Daniel: fluid mechanics, St ,Petersburg paradox




Hesse: The Glass Bead Game


The Bernoullis Inverse problems

Principia (1687)



Central force motion

In edition 1 of the Principia (1687),Newton solved the direct problem ofcentral forces: he proved that if a bodyfollows a curve that is a conic section, itmust be subject to an inverse-square lawof force towards a focus of the curve.

(This is proved by an essentiallygeometric argument, starting from a proofthat Kepler’s area law holds if and only ifthe force is central.)



The inverse problem of central forces

Newton also asserts the converse,the solution to the inverse problemof central forces: a body subject toan inverse-square law of forcetowards a fixed point traverses aconic section with focus at thatpoint.

But his 1687 explanation wasincomplete!




In 1709, in time for the secondedition, he had filled in the details –with an argument based onuniqueness




Johann Bernoulli took Leibniz’s side in the notorious controversy withNewton over priority in the calculus – and more importantly onmethods of proof, the role of models, and the relationship betweencalculus and dynamics.

By the time of Newton’s second edition, Johann had published (1710)two different and complete proofs. Newton’s corollary was “asupposition not a demonstration”.


Inverse problems from a Bayesian perspective Basic formulation

Linear inverse problem with exact data

In modern language, the linear inverse problem is to solve y = Ax for xgiven data y ; typically the problem is ill-posed: the columns of A arenot linearly independent.

Given y , there is an infinite number of solutions; to specify a particularsolution, it is common to take a minimum norm approach:

x† = argminAx=y ||x − x0||B,

Conventionally x0 = 0 and B = I, then x† = A†y , where A† is theMoore-Penrose inverse of A.



Linear inverse problem with noise

More generally, to specify a particular solution:

x? = argminAx=ypen(x),

In practice, observe y with error; a statistical approach assumes adistribution p(y |x):

x = argmin− log p(y |x) + νpen(x).

Can be interpreted as a MAP (maximum a posteriori) estimate ofposterior distribution with prior density ∝ exp−νpen(x):

p(x |y) =p(y |x)p(x)

p(y).

A full Bayesian view allows also assessment of uncertainty.



Why take a Bayesian view?

Imposing smoothness or other ‘regular’ behaviour of the solution to aninverse problem, is based on a prior assumption on the unknown x ,known or assumed prior to the data being collected – using thisaccepts that the required solution combines data with this priorinformation.

Formulating a statistical inverse problem as one of inference in aBayesian model has great appeal, notably for what this brings in termsof

coherence,the interpretability of regularisation penalties,the integration of all uncertainties, andbecause it supports principled elaboration of the model (e.g.allowing measurement error, indirect observation).



Why take a Bayesian view?

The Bayesian formulation comes close to the way that most scientistsintuitively regard the inferential task, and in principle allows the freeuse of subject knowledge in probabilistic model building.

Mathematical analysis of inverse problems is usually focussed on howwell the true solution can be recovered, in the presence of noise – in aBayesian analysis we correspondingly focus on the behaviour of theposterior distribution.


Inverse problems from a Bayesian perspective Applications

Cell microscopy & segmentationFrom Fox talk o o aStatistical Solutions to Inverse Problems: some examples

Al-Awadhi, Jennison and Hurn



Electrical impedance tomographyElectrical impedance tomographyElectrical impedance tomography (Colin Fox and Geoff Nicholls)

E i t l tExperimental set-up, MAP reconstruction, posterior mean andposterior mean and variance, under circular inclusions prior

Colin Fox and Geoff Nicholls



Deconvolution of Chandra X-ray images

Overlay of the Hubble optical image first with the raw Chandra data and second with the posterior mean reconstruction, highlighting black holes.

van Dyk



Remote sensingFrom Fox talk o o aStatistical Solutions to Inverse Problems: some examples

Josiane Zerubia, Xavier Descombes, C. Lacoste, M. Ortner, R. Stoica



Ocean circulation from tracer dataOcean circulation from tracer dataOcean circulation from tracer data

Posterior mean for advection fieldfield superimposed on reconstructed oxygen andoxygen and salinity concentrations (left) compared(left) compared to interpolated data

McKeague, Nicholls, Speer and Herbei, J. Marine Research, 2005


Inverse problems from a Bayesian perspective Theory

Some recent themes in inverse problems theory

`1 penalisation for sparse high dimensional linear inverseproblems, Candes & Tao (2007), Bickel, Ritov & Tsybakov (2009)Cavalier & Tsybakov (2002) – sharp adaptationHofinger and Pikkarainen (2007, 2009) – convergence of posteriordistribution in linear inverse problems, via Ky Fan metricLasanen (2007, 2012) – Bayesian formalism for infinitedimensional inverse problems (more generally, on Banach spaces)Knapik, van der Vaart, van Zanten (2011) – Bayesian inverseproblems with Gaussian priors; Recovery of initial conditions forthe heat equation (Today at 3pm!)Pokern, Stuart, van Zanten (2012) – drift estimation for sde’sAgapiou, Larsson, Stuart (2012) – Bayesian nonparametric linearinverse problems in a separable Hilbert space


Single-photon emission computed tomography Principles

SPECT



SPECT

SPECT is a medical imaging technique for creating a 3-dimensionalvisualisation of the pattern of concentration of a tracer of interest withinthe human body. As such, it images ‘function’ not ‘form’. It iscomplementary to other techniques such as CT and MRI/fMRI; usefulfor specific studies, and comparatively very cheap.

The patient is injected with/ingests/inhales a tracer whose pattern ofuptake within body tissue has a known relationship to physiologicalfunction. The tracer has been radioactively labelled. Some of thephotons subsequently emitted are detected in a gamma camera forsubsequent processing and interpretation.



SPECT50 Journal of the American Statistical Association, March 1997

x,y,z

position/pulse-height module

photomultiplier tubes

scintillation crystal collimato r ,JI'1111111111EIII.

patient's head 3 4

56

Figure 1. Gamma Camera and Photon Interactions. (1) A photon rejected by the collimator; (2) a direct count; (3) a scattered undetected count; (4) an absorbed photon; (5) a scattered detected count; (6) an undetected count.

ats depend on various factors: the geometry of the detec- tion system, the strength of the isotope and exposure time, and the extent of attenuation and scattering between source and detector. Weir and Green (1994) discussed and compared two methods of constructing these weights; the first method uses simple geometric and physical arguments, and the second is an extension of the empirical model of Geman, Manbeck, and McClure (1991). The first method is found to be preferable and is used here. The weight modeling aims to be sufficiently accurate for reasonable reconstruction of x, without the need for auxiliary transmission experiments. The values involve only known physical constants and easily measured dimensions, and they do not depend on the data obtained during the scanning of the patient. It is assumed that the rate of elimination of the radiopharmaceu- tical from the organ is negligible within the duration of the data acquisition and that scattering can be neglected. Each ats term is assumed to be the product of three factors:

1. the proportion of radioactivity that has not decayed by the time at which photons are collected in detector t

2. the proportion of emissions that survive attenuation 3. the solid angle of view of the center of pixel s into

detector t, which is treated as a cylindrical tube of known length and radius.

Attenuation is assumed to be at a constant rate per unit distance within an idealised elliptical boundary represent- ing the patient's body. The dimensions of the ellipse can be easily measured from the patient, or estimated from a trial reconstruction from the data, presuming that the body outline can be readily distinguished. In my experience, five EM iterations are sufficient to determine the approximate center and length of axes of the ellipse.

The joint probability function of the data given the isotope concentration is given by

p(ylx) = I exp(- Zs atsxs)(Z8 atsxs)Yt (1

and Shepp and Vardi (1982) proposed using the EM algorithm for obtaining its maximum likelihood estima- tor (MLE) solution. However, MLE solutions have the unattractive feature of being very noisy in appearance because the reconstruction problem is ill-posed. To overcome this, Vardi, Shepp, and Kaufman (1985) suggested using a slightly smoothed version of the MLE or running a limited number of EM iterations from a uniform start. With the lat- ter approach, the progression from smooth to noisy can be stopped at a point that yields a satisfactory reconstruction.

Bayesian reconstruction from SPECT data was proposed by Geman and McClure (1985), following the general paradigm of Bayesian image analysis introduced by Be- sag (1986) and Geman and Geman (1984). This involves constructing a probability distribution on the space of true patterns of isotope concentration. Geman and McClure proposed a pairwise difference Gibbs prior with a potential function that incorporates a smoothing and scale parameter. Reconstruction can then be accomplished by considering the posterior distribution that follows from Bayes's theorem on the two model components. Various reconstruction methods have been used. Geman and McClure [1985, 1987, 1991 (with Manbeck)] used iterated conditional modes (ICM) and Markov chain Monte Carlo (MCMC) methods, and Green (1990) proposed the one-step-late (OSL) algorithm as an adaptation of the EM algorithm, which is impractical for posterior maximization and MCMC methods (1996). Regardless of the method used for reconstruction, a Bayesian method could be fatally misleading with a bad choice of prior. A simple approach to choosing appropriate prior parameter values is by experimentation using training data sets. In earlier work (Weir 1993), I used simulated training sets from a suitably designed phantom and found the transition from simulated to real data effortless. A case can be made for using fixed parameters in clinical prac-

Camera bins

Gamma camera T

Caer is Roae Thog Ange_



Cartoon of SPECT data acquisition

Sinogram: raw data from a single slice, SPECT scan of pelvis


Single-photon emission computed tomography Modelling

Statistical modelling of SPECT

The Poisson linear model y ∼ Poisson(Ax) (componentwise,independently) is close to reality (there are some dead-time effectsand other artifacts in recording).

x represents the spatial distribution of the isotope in question,typically discretised on a square/cubic grid, x = xj,y the array of detected photons, also discretised y = yi by therecording process,the array A = (aij) - discrete Radon transform - quantifies theemission, transmission, attenuation, decay and recording process;aij is the mean number of photons recorded at i per unitconcentration at pixel/voxel j .



Bayesian analysis of SPECT

The reconstruction problem is to infer the pattern of concentration x forthe array of detected photon counts y ; a statistical inverse problem,linear, but with (approximately) Poisson variation.

Earlier work: Geman & McClure (Proc ASA, 1985); PJG (JRSSB,1990; IEEE-TMI, 1990); PJG & Weir (J. Appl. Stat., 1994); Weir (JASA,1997) discussed:

modelling of SPECT – particularly building the matrix A fromphysical, geometric and data-analytic considerationsBayesian methods of reconstruction, using both EM- andsampling-based computational methodsassessment on simulated data and real gamma camera images ofboth physical phantoms and actual patients.



Pelvis data

Sinogram: raw datafrom a single slice,SPECT scan ofpelvis

10 20 30 40 50 60

1020

3040

50

angle

dete

ctor



Ill-posed character of SPECT

matrix A: Singleslice, SPECT scanof pelvis



Eigenvalues

Single slice, SPECTscan of pelvis.Small scaleproblem: 24× 24body space, 26× 32detector space,Gaussianapproximation

0 100 200 300 400 500

−3

−2

−1

01

log1

0(ei

genv

alue

s)


Single-photon emission computed tomography Reconstruction

Bayesian SPECT reconstruction

In practical work, we have typically used non-Gaussianpairwise-interaction Markov random field priors:

p(x) ∝ exp

(−βδ(1 + δ)

∑s∼s′

log cosh((xs − xs′)/δ)

)

This has attractive propertieslog-convexpenalises less reconstructions x with physically-realistic abruptboundaries (e.g. between tissue types), which are smoothed overwith Gaussian priorsbridges Gaussian (δ →∞) and Laplace (δ → 0) Mrf’s

and β and δ can also be inferred.




Somedemonstrationreconstructions...approx mle

approx mle




Somedemonstrationreconstructions...approx MAP usinglog cosh prior

osl iteration 30




Somedemonstrationreconstructions...approx MAP usinglog cosh prior, finerbody-spaceresolution

osl iteration 30


Single-photon emission computed tomography Current practice

Current practice

Standard clinical gamma-camera systems still supplied with(non-probabilistic) filtered back-projection algorithmsSmoothed/penalised EM methods also now supplied as standard,increasingly used given attenuation information (e.g. from CT)In PET, EM (in Hudson’s ‘ordered subsets’ variant) is standard,due to improved noise characteristics in low count regionsInterest in 4D problems that incorporate motion or tracer kineticsModelling approaches have particular advantage over FBP withlow counts and/or irregular acquisition protocolsIn all these areas, algorithms rely on OSL to implementpenalised/Bayesian methods.See Qi & Leahy, 2006; Hutton, 2011.



SPECT-CT

Buck et al, 2008.



SPECT-MRI phantom study

Recent advances in iterative reconstruction 855

SPECT) [54,55]. Also novel acquisition geometries are being explored for specifi c applications where either angular sampling is limited or truncation unavoidable; examples are in interventional sys-tems, organ-specifi c designs where complete rota-tion is impractical or in direct dose verifi cation during radiotherapy using cone-beam CT [e.g. 56]. In these cases iterative reconstruction tends to

be more immune to artefact than analytical reconstruction. As in the case of emission tomog-raphy prior information can be incorporated (e.g. as a noise constraint) or additional information derived from additional measurement (e.g. motion [57 – 59]). Clearly due to the larger matrix sizes typically used in CT this places even greater demands on computational speed.

Figure 3. Brain simulation illustrating (top to bottom) raw emission data; co-registered MR; conventional MLEM reconstruction; reconstructions using a conventional Bowsher prior (BP) and a modifi ed prior with non-local weighting (NLBP) (image courtesy Daniil Kazantsev, UCL).

Act

a O

ncol

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f B

rist

ol o

n 04

/11/

12Fo

r pe

rson

al u

se o

nly.

Recent advances in iterative reconstruction 855

SPECT) [54,55]. Also novel acquisition geometries are being explored for specifi c applications where either angular sampling is limited or truncation unavoidable; examples are in interventional sys-tems, organ-specifi c designs where complete rota-tion is impractical or in direct dose verifi cation during radiotherapy using cone-beam CT [e.g. 56]. In these cases iterative reconstruction tends to

be more immune to artefact than analytical reconstruction. As in the case of emission tomog-raphy prior information can be incorporated (e.g. as a noise constraint) or additional information derived from additional measurement (e.g. motion [57 – 59]). Clearly due to the larger matrix sizes typically used in CT this places even greater demands on computational speed.

Figure 3. Brain simulation illustrating (top to bottom) raw emission data; co-registered MR; conventional MLEM reconstruction; reconstructions using a conventional Bowsher prior (BP) and a modifi ed prior with non-local weighting (NLBP) (image courtesy Daniil Kazantsev, UCL).

Act

a O

ncol

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Uni

vers

ity o

f B

rist

ol o

n 04

/11/

12Fo

r pe

rson

al u

se o

nly.

Hutton, 2011.


Concentration and approximation of posterior Framework

Theoretical analysis of SPECT

This section is joint work with Natalia Bochkina (Edinburgh)

There is plenty of empirical evidence of the value of taking a Bayesianapproach to inverse problems in emission tomography – whatmathematical statements can we make about the results?

Our framework includes:studying frequentist properties of a Bayesian methodthe likelihood part of model assumed to be true (cf nonparametricanalysis of van der Vaart etc.)treating the prior as the construct of the analyst, not a statementabout naturea treatment that addresses non-gaussianity, non-regularity andconstraints



Theoretical analysis of SPECT

Using SPECT reconstruction as an example for an asymptotic study,there are three directions in which it is very natural in practical terms togo to a limit:

exposure time becomes largeresolution of data becomes finerresolution of reconstruction becomes finer

We will concentrate on the first of these – thus we hold the dimensionsof data and reconstruction fixed, but allow relative noise levels todecrease towards 0.

If the exposure time is extended by a factor T , the model becomesT y ∼ Poisson(T Ax), T → ∞.



Small-variance SPECT

Our model for SPECT is an example of an ill-posed generalised linearinverse problem – p(y |x) depends on x only through the linearpredictor Ax where A has (numerically) linearly dependent columns.

We study inference for x given observed y , in the limit as a noiseparameter σ2 (here, 1/T ) goes to 0. We assume an ‘identity linkfunction’, so that y becomes concentrated on Ax as σ2 → 0.

Because of the ill-posed/ill-conditioned character of the problem, wecannot expect consistency in inference about x based on the likelihoodalone. Even as σ2 → 0, so that y converges to ‘exact data’yexact = Axtrue, we will not be able to distinguish betweenx : Ax = Axtrue.



Model formulation: GLIP

Rather than restricting to the SPECT model, our theoretical analysis isset in a framework of generalised linear inverse problems: specifically,

p(y |x) = cy (σ2) exp(−fy (Ax)/σ2)

for a given n × p matrix A and appropriate functions fy and cy . There isa continuous bijective link function G, and we write G(yexact) = Axtrue.

We suppose the data are generated from this distribution, withx = xtrue, and attempt to recover xtrue as the dispersion parameterσ2 → 0. We assume that for all µ such that G(µ) ∈ AX ,

if y has the above distribution with Ax = G(µ), then yp→ µ as

σ → 0fµ(η) has a unique minimum over η ∈ AX at η = G(µ)



Prior modelling

One of the roles of the prior in the Bayesian approach is to resolve thisambiguity (as well as generally improve reconstruction through‘regularisation’, even without σ2 → 0).

Our results apply for a wider class of suitably smooth priors, but for thistalk we simply assume

p(x) ∝ exp(−1/(2γ2)||x − x0||2B) = exp(−1/(2γ2)(x − x0)T B(x − x0))

subject to x ∈ X = x ∈ Rp : xj ≥ 0 ∀ j.

Thus the posterior is proportional to

p(y |x)× exp(−1/(2γ2)||x − x0||2B) subject to x ∈ X .



Small-variance SPECT

The posterior is proportional to

p(y |x)× exp(−1/(2γ2)||x − x0||2B) subject to x ∈ X .

We study this in the σ → 0 limit; to obtain convergence to a point, wewill need γ → 0 as well (though, as we will see, at a slower rate thanσ).

In particular, we will show that as σ → 0 and γ → 0 in such a way thatσ/γ → 0, the posterior converges to the point

x? = argminx∈X :Ax=yexact||x − x0||2B


Concentration and approximation of posterior Geometry

Basic geometry

Suppose that A is a real n × p matrix, and B a real symmetricnon-negative definite p × p matrix, both possibly of deficient rank.

Assume that the p × (n + p) block matrix [AT ...B] has full rank p.

Thenx? = argminx∈X :Ax=yexact

||x − x0||2Bis unique, and if σ → 0, γ → 0 and σ/γ → 0, then approaching the limitthe posterior is a truncated Gaussian, with variance scaling differentlyin different directions.

If q is the rank of A, then asymptotically the variance of the posteriordistribution (before truncation) has q eigenvalues scaling like σ2 andthe remaining (p − q) like the (larger) γ2.



Geometry

Visualisation ofposterior when

n = 1, p = 2,q = 1.



Geometry


n = 1, p = 3, q = 1.In this case x? lies

internal to X .



Geometry



internal to X .σ and γ getting

smaller.



Geometry


n = 1, p = 3, q = 1.In this case x? lieson boundary of X .



Geometry


n = 1, p = 3, q = 1.In this case x? lieson boundary of X .

σ and γ gettingsmaller.



Karush–Kuhn–Tucker

We are interested in the solution to the constrained minimisationproblem

x? = argminx∈X :Ax=yexact||x − x0||2B

By the Karush–Kuhn–Tucker theory, for this particular problem, it isnecessary and sufficient to find (x?, ζ, λ) ∈ Rp ×Rp ×Rn such that

B(x? − x0)− ζ + ATλ = 0x? ≥ 0

Ax? = yexact

ζ ≥ 0for all j , ζj = 0 or x?j = 0



Karush–Kuhn–Tucker

Geometricalinterpretation ofKKT conditions

when n = 1,p = 2, q = 1,

in case B = I.


Concentration and approximation of posterior Concentration of posterior

Metrics and convergence

To study convergence of the posterior simultaneously over ally = y(ω), we need a metric that metrises convergence in probability.

The Ky Fan metric between two random variables ξ1 and ξ2 in a metricspace (Y,dY) is

ρK(ξ1, ξ2) = infε > 0 : pdY(ξ1(ω), ξ2(ω)) > ε < ε.

Weak convergence of the posterior µpost = p(x ∈ ·|y(ω)) as a randomvariable to the point mass δx? is equivalent to its convergence in the KyFan metric, where (Y,dY) is a space of distributions with the Prohorovmetric.



Concentration

We are able to give explicit upper bounds on ρK(µpost, δx?), dependingon the spectral properties of the asymptotic information matrix andprior precision at x?, on ρK(y , yexact), and on the likelihood and priordispersion parameters σ2 and γ2.

For example, in the interior point case, under conditions,

ρK(µpost, δx?) ≤ max2ρK(y , yexact),

c1ρK(y , yexact) + c2ν

+

[− τ

λmin(Hν)log(

CP

(τ

λmin(Hν)

)κP)]1/2

(1 + ∆?,K (δ))



Concentration

Ky Fan metrics can be readily evaluated for specific distributions.If Yt ∼ σ2Poisson(µt/σ

2), then

ρK(Y , µ) ∼√−σ2M log(σ2M)

as σ2 → 0, where M =∑

t µt .If Y ∼ Np(µ,Σ) then there exist constants Cp, θp such that for anyΣ with ||Σ|| < θp,

ρK(Y , µ) ≤√−||Σ|| logCp||Σ||max1,p−2

If Y ∼ Exp(λ/σ2) then ρK(Y ,0) ∼ −(σ2/λ) log(σ2/λ) (and isalways ≤) as σ → 0.



Concentration in the Bayesian SPECT model

If x? is an interior point of X , then for small enough σ2 and γ2,

ρK(µpost, δx?) ≤[C1

√−σ2 logσ + C2

σ2

γ2

+ C3σ1−αγα

√− log(σ1−αγα)

](1 + o(1))

where α = 0 is AT∇2 fyexact(Ax?)A is of full rank, and α = 1 otherwise.

If α = 0 (well-posed case), the fastest rate is σ√− logσ, with

γ ≥ σ1/2[− logσ]−1/4

If α = 1 (ill-posed case), the fastest rate is σ2/3√− logσ, with

γ = σ2/3[− logσ]−1/6

If x? is on the boundary of X , there are additional terms inρK(µpost, δx?).



Geometry



internal to X .



(Lack of) consistency

In summary, the posterior concentrates at the point

x? = argminx≥0:Ax=yexact||x − x0||2B

– but this is usually not equal to the truth xtrue. We do have

Ax? = Axtrue

but (I − PAT )x? is determined solely by the prior and the constraints.


Concentration and approximation of posterior Approximation of posterior

Bernstein–von Mises

Can we quantify probabilistically the variation of p(x |y) about theconcentration point x?? More concretely, can x − x? be (transformedand) scaled so that it has a non-trivial limit posterior law?

Such questions are traditionally the subject of the Bernstein–von Mises(BvM) theorem, in various versions. Here we need a version of theBvM theorem that deals, as well as singularity of the prior, with thenon-regular aspects of our set-up, namely

positivity constraintsill-posed-ness of A (so x not identifiable)likelihood non-regularity, e.g. from the possibility of Poissoncounts of 0



BvM: transformation of coordinates

Recall that the point of concentration of the posterior is

x? = argminx≥0:Ax=yexact||x − x0||2B

Let J = j : x?j = 0. When we scale x − x? to get a non-degeneratelimit as σ, γ → 0, the support of the scaled posterior isXJ = x : xJ ≥ 0. We can decompose

XJ =W0 ⊕W1 ⊕W+2 ⊕W

+3

to focus on the different characters of the limit in different directions:different scalings and different shapes




We can decompose

XJ =W0 ⊕W1 ⊕W+2 ⊕W

+3

The four subsets can be characterised as follows:

identi- projectiondim fiable? onWk scaling

subspace W0 p0 yes interior σsubspace W1 p1 no interior γ

polyhedral cone W+2 p2 yes boundary σ2

polyhedral cone W+3 p3 no boundary γ2

Any of pk can be 0, depending on the details of the problem, butp0 + p1 + p2 + p3 = p.




W0 is the whole space when the problem is regularW1 appears in an ill-posed linear Gaussian problemW+

3 arises from the positivity constraints on x? being activeW+

2 arises with an irregular likelihood (zero Poisson counts)



BvM theorem for non-regular GLIP

Visualisation of acase with

n = 1, p = 3,

in whichx?1 , x

?2 > 0, x?3 = 0.

p0 =p1 =p3 =1,p2 =0.

x33

x2

x1





n = 1, p = 3,in which

x?1 , x?2 > 0, x?3 = 0.

p0 =p1 =p3 =1,p2 =0.

x33

x2

X*

x1






x?1 , x?2 > 0, x?3 = 0.

p0 =p1 =p3 =1,p2 =0.

x33

x2

x1






x?1 , x?2 > 0, x?3 = 0.

p0 =p1 =p3 =1,p2 =0.

x33

x2

W3+

W1

W0x1




Supposewe fill the p × pk matrix Vk with column vectors forming a basis forWk (W+

k ) for each k ,

we set V = [V0...V1

...V2...V3],

we assume σ → 0, γ → 0, σ = o(γ) and σ = O(γ2),

we write S =(σV0 γV1 σ2V2 γ2V3

)−1,then

S(x − x?) | y → µ? = N(a0,Ω−100 )× N(0,B−1

11 )× Exp(a2)× Exp(a3)

in total variation norm.




The parameters in this limiting distribution are given by:

Ω00 = V T0 ∇2fyexact(x

?)V0, a2 = V T2 ∇fyexact(x

?),

B11 = V T1 BV1, a3 = V T

3 ζ,

a0(ω) = Ω−100 lim

σ→0[1/σV T

0 ∇fy(ω)(x?) + σ/γ2V T0 B(x? − x0)].



Assumptions

We have to exclude degeneracy in the convex optimisation:for all j , either ζj = 0 and x?j = 0 but not both;

for all i , either ∇i fyexact(yexact) = 0 or yi = 0 but not both.

and also assume thatΩ00 and B11 are of full rank;a0(ω) <∞ for all ω;γ → 0, σ/γ → 0 as σ → 0, and c = limσ→0 σ/γ

2 <∞.



Assumptions

We need an assumption about the concentration of the posterior:

For some δk → 0 but with (δ0, δ1, δ2, δ3) (σ, γ, σ2, γ2), resp.,

Px 6∈ B(x?, δ)|y → 0 as σ → 0.

where B(x?, δ) = x? + S−1 B2,0(δ0)× B2,1(δ1)× B∞,2(δ2)× B∞,3(δ3)

and Br ,k (δ) = x ∈ Rpk : ||x ||r < δ.

We need continuity at x?: y P→ yexact, ∇k fy (x)P→ ∇k fyexact(x),

k = 0,1,2,3, as σ → 0.

and smoothness: fy ,fyexact both have bounded 3rd derivatives onB(x?, δ).



Idea of proof

0. Split the total variation norm in 3 parts:

||PS(x−x?)|y − µ?||TV ≤ ||PS(x−x?)|y1B − µ?1B||TV

+||PS(x−x?)|y − PS(x−x?)|y1B||TV + ||µ? − µ?1B||TV ,

where 1B represents renormalised truncation onto the rescaledlocal neighbourhood B(x?, δ) of x?.

1. Use quadratic approximation of the log posterior distribution on aneighbourhood of x? to bound

||PS(x−x?)|y1B − µ?1B||TV .

OnW0 andW1, the leading term is quadratic, and onW+2 andW+

3the leading term is linear.



Idea of proof

2. Bound the effect of the local neighbourhood truncation

||PS(x−x?)|y1B − PS(x−x?)|y ||TV

using the concentration assumption.3. Note that µ? outside of the local neighbourhood vanishes since B

converges to the whole space.



Practical implications of the BvM approximation

Of course, on a realistic practical scale we cannot hope to constructand manipulate p × p matrices such as V . However, the theorem canbe used to calculate approximate posterior probabilities using posteriormeans and covariances estimated by (e.g.) MCMC.

54 Journal of the American Statistical Association, March 1997

Table 2. Brain Phantom Major MCMC Run: Estimates of Posterior Means, Integrated Auto-Correlation Times, and Standard Errors (SE's)

Parameter Mean T(f) M ;(f)/N SE

High 188.29 12.392 38 .01377 1.6785 Medium 93.524 8.1245 25 .00903 .8502 Low 41.905 12.653 38 .01406 1.4670

3.6189 x 10-3 6.0526 19 .00673 2.2004 x 10-4 3 2.3384 x 10-2 11.003 34 .01223 9.6926 x 10-5

4f. The fully Bayesian posterior mean is not as aesthetically pleasing as the posterior mode from the OSL using the fixed "best" prior parameters. The former is slightly noisier and has RMSE value of 6.3670, compared to 5.7003 from the fixed parameter posterior mode. The RMSE value of the fully Bayesian reconstruction is substantially less than the least possible obtainable by ML/EM (8.2179). The MCMC run took 940.3 seconds on a Sun SPARC 10/41 worksta- tion, compared to 30.0 seconds for OSL. Therefore, the comparative computational cheapness of OSL when "good" fixed values of the prior parameters are available discour- ages the use of MCMC to obtain reconstructions. However, in reality the variety of organs imaged and range of photon counts recorded may mean that these are not available. Us- ing other parameter values, the OSL algorithm invariably produced grossly oversmoothed or noisy reconstructions. Thus the automation and quality of the fully Bayesian reconstruction is admirable.

Furthermore, the fully Bayesian method also allows one to go beyond mere reconstruction, because functionals of the body space can be easily obtained as a by-product of the MCMC. For instance, fully Bayesian posterior pixelwise modes could be obtained. These may be useful for examin- ing a specific region of interest. The OSL fixed-parameter solution would not be capable of this, as it is a mode for the whole body space. Fully Bayesian posterior medians also

a a o c

0 .

CL~~~~~~~~

0.

0 5 10 15 20 25 30 0 5 10 15 20 25 30 pixel pixel

(a) (b)

o? o? 0- a LO a LO

Cu C

..z. *.....

0.0 o) 0

0 5 10 15 20 25 30 0 5 10 15 20 25 30 pixel pixel

(c) (d)

Figure 5. Brain Phantom: (a) Row 25 Truth and Posterior Mean ( phantom; ---, Mean); (b) Row 25 Truth and Credibility Bounds (---, 90% bounds; (---, 95% bounds); (c) Row 16 Truth and Posterior Mean ( , Phantom; ---, Mean); (d) Row 16 Truth and Credibility Bounds (---, 90% bounds; ---, 95% bounds).

can be easily obtained; here the reconstruction composed of pixelwise medians was found to be very similar in appearance to the posterior mean. Bayesian interval estimators are also easily obtained. Here I report pixelwise quantiles; however, Besag et al. (1995) outlined the construction of si- multaneous bounds for the whole vector x. Figure 5 graphs two transects of the brain phantom that cross distinct regions of interest. The posterior mean and 90% and 95% pixelwise credibility bounds are superimposed on these plots. It can be seen that the posterior mean faithfully reproduces the general truth shape, the major departure being overes- timation of the low metabolic activity region. However, the credibility bounds fully bracket the truth. In general, there may be situations where the credibility bounds do not give full information about the truth. For instance, the ability to successfully reconstruct small area hot spots and edges will be affected by low count/noisy data sets, the sampling bin widths, the reconstruction discretization used, and the accuracy of the ats weight modeling.

We can also use output from the MCMC to get posterior density estimates of the prior parameters. The density estimates do not have pathological worth but do give insight into appropriate prior parameter values that could, for instance, be used to gain fixed-parameter OSL posterior mode reconstructions.

3.2 Real Data

The data relate to a horizontal slice through a brain and comprise 119,405 photon counts recorded by a system of 64 detectors at 64 angles. The filtered back-projection reconstruction provided by the Bristol Royal Infirmary is on a 64 x 64 grid of .48 cm squares. Figure 6a displays the central 48 x 48 pixels. The presence of spurious radial artifacts is a well-documented deficiency of filtered back-projection. Figures 6b and 6c show ML/EM reconstructions after 50 and 100 iterations. These reconstructions define the outline of the brain better, but they are noisier. Figure 6d displays the OSL solution using the prior parameter values chosen for the simulated data and reveals a clear isotope pattern that comprises several well-defined regions. Fig- ures 6e and 6f present the final realization from the fully Bayesian MCMC run and the resulting estimate of the posterior mean. The fully Bayesian posterior mean is similar to the OSL fixed-parameter solution but is not as smooth and suggests greater contrast between several regions. Fig- ure 7 displays the posterior mean and credibility bands of a cross-section; the widths of the credibility bounds are not constant indicating the varying confidence in parts of the solution.

4. DISCUSSION

The work presented here has demonstrated the ability to obtain representative Bayesian reconstructions of SPECT data without the need for prior parameter selections. The process is thus fully automated and can be applied with no supervision. The reconstructions are of comparable quality to OSL ones when "good" fixed prior parameter values are used and better when inappropriate values are used.



Practical implications of the BvM approximation

Of course, on a realistic practical scale we cannot hope to constructand manipulate p × p matrices such as V . However, the theorem canbe used to calculate approximate posterior probabilities using posteriormeans and covariances estimated by (e.g.) MCMC.

Inferential questions of real interest, such asquantitative inference about amounts of radio-labelled tracerwithin specified regions of interest, ortest for significance of apparent hot- or cold-spots

can be answered using approximate posterior distributions for linearcombinations of x .

We can use the MAP estimate x = argmaxxp(x |y) to identify x? andhence J; we anticipate p0 p1,p2,p3 in practical situations.


Acknowledgements/contact

I am particularly indebted to Natalia Bochkina, and also grateful to myearlier collaborators Iain Weir and Malcolm Hudson, and to BrianHutton for his update on modern SPECT reconstruction.

To follow up: contact [email protected]


emission tomography and bayesian inverse problemsmapjg/slides/bernoulli... · 2016. 8. 12. ·...

Documents