inside-out: characterisation of computed tomography noise ...sip/miniproject1/report_4.pdf ·...

Inside-Out: Characterisation ofComputed Tomography Noise inProjection and Image Space with

Applications to 3D Printing

OxWaSP 2015-16 Warwick Cohort - Mini-project 1

Sherman Ip

Supervisor: Dr. J. Brettschneider (Statistics, Warwick)

Supervisor: Prof. T. Nichols (Statistics, Warwick)

10th June 2016

Abstract

I did this. This happened. I should of done that.

Acknowledgements

None

Contents

1 Introduction 3

2 Literature Review 42.1 X-Ray Production . . . . . . . . . . . . . . . . . . . . . . . . 52.2 X-Ray Interactions . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.4 Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Exploratory Data Analysis 93.1 Summary Statistics . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1.1 Histogram . . . . . . . . . . . . . . . . . . . . . . . . . 93.1.2 Time Series . . . . . . . . . . . . . . . . . . . . . . . . 93.1.3 Distribution of Individual Pixels . . . . . . . . . . . . . 12

3.2 Principal Component Analysis . . . . . . . . . . . . . . . . . . 123.2.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.3 Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 233.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4 Mean and Variance Relationship 284.1 Weighted Least Squares . . . . . . . . . . . . . . . . . . . . . 28

4.1.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.1.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 304.1.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.2 Subsampling Least Squares . . . . . . . . . . . . . . . . . . . . 31

1

4.2.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 314.2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.3 Mixture of Linear Regression . . . . . . . . . . . . . . . . . . . 314.3.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.3.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 374.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.4 Distance Weighted Least Squares . . . . . . . . . . . . . . . . 384.4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 384.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5 Model for Intensity 415.1 Rate Density . . . . . . . . . . . . . . . . . . . . . . . . . . . 415.2 Attenuation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435.3 Intensity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455.4 Hierarchical Model . . . . . . . . . . . . . . . . . . . . . . . . 46

5.4.1 Small example . . . . . . . . . . . . . . . . . . . . . . . 52

Bibliography 55

2

Chapter 1

Introduction

Mini-project for OxWaSP.

3

Chapter 2

Literature Review

Computed tomography (CT) scanning is a 3D imaging technique. It doesthis by reconstructing the geometry of the sample through a series of 2DX-ray images of the sample. The sample rotates after each image taken.

Figure 2.1 shows a diagram on how CT scanning works. A 2D image istaken by projecting X-ray photons onto the stationary sample. The photonsare then scattered or absobred by the sample. Some of these photons arethen detected by an X-ray detector on the other side of the sample, whichproduces an image. After an image has been taken, the object rotates andanother image is taken. Finally after a number of images, a 3D reconstructionof the object can be estimated [1].

CT scanning was invented by G. Hounsfield [2] in the 1980’s and it wasmainly used for medical imaging. The setup for CT scanning is differentwhen scanning patients because the detector and X-ray source rotates aroundthe patient [1]. Recently CT has been used industrially for non-destructivetesting in manufacturing [1]. One possible application would be inspecting3D printed (additive layer manufactured) samples.

There are many sources of error in CT scanning [1] and this can causeproblems when reconstructing the geometry of the sample. Sources of errorinclude: defects in the detector, environmental noise and the behaviour ofphotons.

This chapter aims to give brief description on how CT scanning worksand a short discussion on the sources of error.

4

Figure 2.1: X-ray computed tomography reconstructs the sample by project-ing X-ray photons onto a rotating sample. The photon are then detected bythe detector. Source: http://www.phoenix-xray.com/

2.1 X-Ray Production

X-ray photons in CT scanning are produced in an X-ray tube. In an X-raytube, a cathode, consisting of a heated filament, fires projectile electronsthrough an electric potential to a target which forms the anode [3], as shownin Figure 2.2. Most of the kinetic energy of the projectile electrons is con-verted into heat however some is converted into electromagnetic radiation.This depends on how the projectile electrons interact with the atoms in theanode [1].

Bremsstrahlung radiation is the result of projectile electrons deacceler-ating due to the electrostatic field produced by nucleus of the target. Thekinetic energy of the projectile electrons is then converted to electromag-netic radiation to produce X-ray radiation. As a result, the photon energiesin bremsstrahlung radiation is a continuous spectrum and can range up tothe maximum kinetic energy of the projectile electrons [3].

Characteristic radiation is due to projectile electrons colliding with elec-trons in the target atom and ionizing them. This produce vacancies in theelectron shell and emits X-ray photons when the electrons in the target atomdrops down back to the ground state. The energy of the emitted radiation ismonoenergetic and depends on the binding energy of the target’s atoms [3].

5

Figure 2.2: An X-ray tube produces X-ray photons by firing projectile elec-trons from a cathode to an anode. Source: G. Michael (2001) [3]

A typical energy spectrum of X-ray photons emitted from an X-ray tube isas shown in Figure 2.3. The energy spectrum consist of both bremsstrahlungand characteristic radiation [3].

The voltage, the energy per charge, and current, the rate of charge, canbe varied in the X-ray tube to produce different energy spectrums and rateof X-ray emittion. This can vary the results produced when collecting CTdata [1]. Another important factor is the focal spot size because smaller spotsizes produce sharper edges. Larger spot sizes produce unsharp results andthis is know as the penumbra effect, as shown in Figure 2.4. However spotsizes too small can produce concentrated heat [4].

2.2 X-Ray Interactions

X-ray radiation emitted by the X-ray tube are projected onto the sample andinteracts with it in a number of ways [1].

The sample can effectively absorb X-ray radiation via the photoelectriceffect or pair production [1]. In the photoelectric effect, the X-ray photonstransfers all its energy to a bounded electron and ejects it from the atom [5].In pair production, the X-ray photons converts into an electron-position pair

6

Figure 2.3: A typical energy spectrum of X-ray photons emitted from an X-ray tube. The continuous spectrum is the result of Bremmsstrahlung radia-tion. The peaks are the result of characteristic radiation. Source: G. Michael(2001) [3]

Figure 2.4: Larger focal spot sizes produces unsharp results. This is know asthe penumbra effect. Source: F. Welkenhuyzen et al. (2009)[4]

7

by interacting with the Coulomb field of the sample’s atomic nucleus [6].X-ray photons can be scattered by the sample. This happens when X-ray

photons collide inelastically with and transfers it’s energy to the sample’selectrons. This process is know as Compton scattering [7].

The attenuation (decrease in X-ray intensity from I0 to I1) when prop-agating though a material with attenuation coefficient µ and distance x isgiven as

I1 = I0e−µx (2.1)

where it was assumed the X-ray photons are monoenergetic [3]. This can beextended to a continuous spectrum of X-ray energies by making µ dependenton the energy of the X-ray photons [1]. In general low energy photons aremore likely to be absorbed than high energy photons, this increases the av-erage energy of the attenuated photons and can be a source of error in CTscanning. This is referred to beam hardening [3]. This can be reduced byplacing a thin sheet of filter to absorb low energy photons [4] or by correctingit in the data analysis stage [3].

2.3 Detector

Most X-ray detectors are scintillator-photodiode detectors. The X-ray pho-tons interact with the scintiallator material and produces visible light. Thevisible light is then detected by photodiodes and coverts it into electricicalcurrent [3]. The detectors used in CT scanning are flat bed scanner whichconsist of an array of panels of photodiodes [1].

2.4 Reconstruction

The scale of the 2D images are calibrated with the use of reference standards[8]. The method used to reconstruct the 3D geometry of the sample is calledthe filtered back-projection [3]. The mathematics can be reviewed in [9].

8

Chapter 3

Exploratory Data Analysis

3.1 Summary Statistics

A dataset was given for this project. It consists of 100 realizations of anX-ray image of a cuboid with its vertices in the middle, taken using an X-raydetector. These images were 1 996×1 996 pixels in size and have a colourdepth of 16 bits. A sample of them can be viewed in Figure 3.1.

3.1.1 Histogram

The distribution of the grey values of all the pixels in the data set is shown inFigure 3.2. It was observed that there were 3 distinct peaks in the histogramat ∼ 2 × 104, ∼ 4 × 104 and ∼ 5 × 104. By doing a threshold, as shown inFigure 3.2, it was clear that the main 3 sources of the grey values were thesample, the background and the foam holder at the bottom of the image.

3.1.2 Time Series

The mean and standard error of the grey values for each of the 100 imageswere taken and plotted as a time series as shown in Figure 3.4. There wasevidence of dependence between samples by looking the sample autocorrela-tion plots as shown in Figure 3.5. Because there was a peak at lag 1 in thesample partial autocorrelation plot, the time series could be modelled usingan autoregressive model with one parameter.

9

Figure 3.1: The 1st, 50th and 99th image in the dataset of an X-ray imageof a cuboid.

Figure 3.2: Histogram of the grey values of all the pixels in the 100 imagesin the dataset. The grey values range from 18 286 to 60 045.

10

(a) Threshold pixels (b) Histogram

Figure 3.3: A threshold was selected to highlight pixels with different greyvalues. Blue: grey values less than 34 765. Green: grey values more than40 269.

Observation number

10 20 30 40 50 60 70 80 90 100

Mean g

rey v

alu

e

×10 4

3.81

3.82

3.83

3.84

3.85

3.86

3.87

3.88

3.89

3.9

Figure 3.4: The mean and standard error of all the pixel grey values for eachimage.

11

Lag

0 5 10 15 20

Sam

ple

Auto

corr

ela

tion

-0.2

0

0.2

0.4

0.6

0.8

1

(a) Sample autocorrelation

Lag

0 5 10 15 20

Sam

ple

Part

ial A

uto

corr

ela

tions

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

(b) Sample partial autocorrelation

Figure 3.5: Sample autocorrelation and partial autocorrelation of the meangrey values time series.

3.1.3 Distribution of Individual Pixels

The mean and standard deviation for each pixel’s grey value was estimatedand as shown in Figure 3.6. Both supervised and unsupervised learning canbe used to model the mean and variance relationship of the grey values.

A Normal distribution was fitted on each pixel’s grey values and the χ2

goodness of fit test was conducted, as shown in Figure 3.7. When consideringthe Bonferoni correction [10] for multiple hypothesis testing, the Normaldistribution appeared to be a good fit because only 9 pixels have p valuesless than 10%. In addition, these significant pixels did not seem to clusterand appear random in space. When ignoring the Bonferoni correction, therewas a suspicious patch of low p values on the bottom right.

3.2 Principal Component Analysis

By estimating the covariance matrix of the individual pixel’s grey values,sources of (orthgonal) variance can be estimated. This can be done inves-tigating eigendecomposition representation of the sample covariance matrix.A lot of the linear algebra can be reviewed in undergraudate maths text booksuch as [11].

12

Sample grey value mean (AU)×10 4

2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e s

td (

AU

)

150

200

250

300

350

400

450

500

550

600

650

700

0

0.5

1

1.5

2

2.5

3

3.5

Figure 3.6: Frequency density of the 1 9962 pixel’s grey values mean andstandard deviation.

13

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(a) p values

-25

-20

-15

-10

-5

(b) log10 p values

Figure 3.7: p values from the χ2 goodness of fit test on fitting the Normaldistribution on the grey values for each pixel. In b) circled are pixels withp values less than 10%, corrected for multiple hypothesis testing using theBonferroni correction.

14

3.2.1 Theory

Suppose the p× p covariance matrix Σ has eigenvalues λi and eigenvector φi

for i = 1, 2, ··· , p. Then the eigenvectors and eigenvalues are related by

Σφi = λiφi . (3.1)

A closure can be form by pre-multiplying both sides by φTj so that

φTj Σφi = λiφ

Tj φi(

ΣTφj

)Tφi = λiφ

Tj φi .

But given the covariance matrix is symmetric, Σ = ΣT, and λj is the corre-sponding eigenvalue of the eigenvector φj, then(

λjφj

)Tφi = λiφ

Tj φi

λjφTj φi = λiφ

Tj φi .

As long as λi 6= λj for i 6= j, then φTj φi = 0. Therefore all eigenvalues in the

covariance matrix are orthogonal to each other.Equation (3.1) can be extended to include all eigenvectors and eigenvalues

such that

Σ

↑ ↑ ↑φ1 φ2 · · · φp

↓ ↓ ↓

=

↑ ↑ ↑λ1φ1 λ2φ2 · · · λpφp

↓ ↓ ↓

. (3.2)

Transposing both sides← φ1 →← φ2 →

...← φp →

Σ =

λ1 0 · · · 00 λ2 · · · 0...

.... . .

0 0 · · ·λp

← φ1 →← φ2 →

...← φp →

but because all eigenvectors are orthogonal

Σ =

↑ ↑ ↑φ1 φ2 · · · φp

↓ ↓ ↓

λ1 0 · · · 00 λ2 · · · 0...

.... . .

0 0 · · ·λp

← φ1 →← φ2 →

...← φp →

.

15

This is just a weighted sum of the outer products of the eigenvectors, thereforethe eigendecomposition is

Σ =

p∑i=1

λiφiφTi . (3.3)

Because this is a weighted sum, the eigenvectors with the biggest eigenvaluewill be most responsible for the covariance matrix. φi are called the principlecomponents.

3.2.2 Methods

The 100 images were shrunk with averaging to a size of 100 × 100 pixelsusing ImageJ. The sample covariance matrix, of the pixel’s grey values, wascalculated using the maximum likelihood estimator so that the 6 eigenvectorswith the biggest eigenvalues can be obtained. Using 1 600 bootstrap samples,the sampling distribution of the 6 eigenvalues were sampled.

3.2.3 Results

Figure 3.9 shows the first 6 of the diagonal of the outer product of the indi-vidual principle components, they represent the normalised main sources ofvariance. It was observed that on the first and second principle components,there are sources of variance in the bottom right and top right of the image.This could imply that the sources of variance are not uniformly distributedin the background.

The sampling distribution of the eigenvalues are shown in Figure 3.9.Quite clearly there were 2 significant sources of variance because the eigen-values of the first 2 principle components were much larger than zero whencomparing it with its interquartile range.

3.3 Factor Analysis

A probabilistic latent model can be used to model the pixel’s grey values.These latent variables then can be used to describe the underlining structure,and perhaps sources of variance.

16

PC1 ×10-4

1

2

3

4

PC2 ×10-4

1

2

3

4

PC3 ×10-4

2

4

6

8

PC4 ×10-4

2

4

6

8

PC5 ×10-4

2

4

6

8

10

PC6 ×10-4

2

4

6

8

10

12

Figure 3.8: Normalised variance due to the principle components (1 to 6).The values of all the pixels in the figures equal to 1.

Eigenvalue number

1 2 3 4 5 6

Eig

envalu

e (

AU

)

×10 8

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Figure 3.9: Eigenvalues of the sample covariance matrix. 1 600 bootstrapsamples were used.

17

3.3.1 Theory

Let Y be a random vector containing k latent variables, called factors, and

Y ∼ N (0, 1) . (3.4)

Let e be a random p vector modelling the uncorrelated intrinsic noise suchthat

e ∼ N (0,Ψ) (3.5)

where Ψ is a diagonal matrix.Given a p× k loading matrix Λ, the observable data X can be modelled

throughX = ΛY + e (3.6)

thusX ∼ N

(0,ΛΛT + Ψ

). (3.7)

As a result there are two sources of variance, factor noise from the ΛΛT

component and the instrinic noise Ψ. The model can be extended for non-zero mean easily by adding an arbitrary constant.

The conditional distribution of X given Y can be calculated to be

X|Y ∼ N (ΛY,Ψ) . (3.8)

Using Bayes’ theorem, the conditional distribution of Y given X can becalculated

pY|X(y|x) ∝ pX|Y(x|y)pY(y)

pY|X(y|x) ∝ exp

[−1

2(x− Λy)T Ψ−1 (x− Λy)

]exp

[−1

2yTy

]pY|X(y|x) ∝ exp

[−1

2

(−xTΨ−1Λy − yTΛTΨ−1x + yTΛTΨ−1Λy + yTy

)]pY|X(y|x) ∝ exp

[−1

2

(yT(ΛTΨ−1Λ + 1

)y − 2yTΛTΨ−1x

)]and comparing it with the density function of a general multivariate Normaldistribution

Y|X ∼ N((

ΛTΨ−1Λ + 1)−1

ΛTΨ−1X,(ΛTΨ−1Λ + 1

)−1). (3.9)

18

Suppose X = x(i) was observed for i = 1, 2, ··· , n, then the likelihood is

L =n∏i=1

1

‖2π (ΛΛT + Ψ) ‖1/2exp

[−1

2

(x(i))T (

ΛΛT + Ψ)−1

x(i)

]. (3.10)

The log likelihood is then

lnL = −n2

ln ‖2π(ΛΛT + Ψ

)‖ − 1

2

n∑i=1

(x(i))T (

ΛΛT + Ψ)−1

x(i)

and by using the trace and its properties

lnL = −n2


)‖ − 1

2

n∑i=1

Tr[(x(i))T (

ΛΛT + Ψ)−1

x(i)]

lnL = −n2


)‖ − 1

2

n∑i=1

Tr[(

ΛΛT + Ψ)−1

x(i)(x(i))T]

lnL = −n2


)‖ − 1

2Tr

[(ΛΛT + Ψ

)−1 n∑i=1

x(i)(x(i))T]

.

(3.11)

The loading matrix Λ and intrinsic noise Ψ can be estimated by maximisingthe log likelihood. This is best done using the expectation maximisation(EM) algorithm. The EM algorithm is an iterative procedure conducting anE step followed by a M step many times till some convergence conditions aremet.

The E step estimates the values of the latent variables, y(1),y(2), ··· ,y(n),

using the expectation of the condition distribution of the latent variablesgiven the observed data x(1),x(2), ··· ,x

(n). The parameters Λ and Ψ are as-sumed to be know in this step and are fixed. By using Equation (3.9), eachlatent variable are assigned by

y(i) ←(ΛTΨ−1Λ + 1

)−1ΛTΨ−1x(i) (3.12)

for i = 1, 2, ··· , n. The covariance of the latent variables are estimated by

ΣY ←(ΛTΨ−1Λ + 1

)−1(3.13)

In the E step, the parameters Λ and Ψ are varied to maximise the loglikelihood, assuming the latent variables are known and fixed. The objective

19

to maximise is the condition expectation of the log joint likelihood, this isgiven as

H = EY|X

[n∑i=1

ln pX,Y(x(i),y(i)

)]. (3.14)

This can be expressed as

H = EY|X

[n∑i=1

(ln pX|Y

(x(i)|y(i)

)+ ln pY

(y(i)))]

Because the marginal distribution of the latent variables do not depend onΛ and Ψ, this can be ignored. Using Equation (3.8)

H = EY|X

[n∑i=1

ln

(1

‖2πΨ‖1/2exp

(−1

2

(x(i) − Λy(i)

)TΨ−1

(x(i) − Λy(i)

)))]+c1

H = EY|X

[n∑i=1

(−1

2ln ‖2πΨ‖ − 1

2

(x(i) − Λy(i)

)TΨ−1

(x(i) − Λy(i)

))]+ c1

H = −1

2EY|X

[n∑i=1

(ln ‖2πΨ‖+

(x(i))T

Ψ−1Λy(i) − 2(x(i))T

Ψ−1Λy(i) +(y(i))T

ΛTΨ−1Λy(i))]

+c1

By defining

EY|X[y(i)]

= µ(i) =(ΛTΨ−1Λ + 1

)−1ΛTΨ−1x(i) (3.15)

andCovY = ΣY =

(ΛTΨ−1Λ + 1

)−1(3.16)

then when taking the conditional expectation

H = −1

2

n∑i=1

[ln ‖2πΨ‖+

(x(i))T

Ψ−1x(i) − 2(x(i))T

Ψ−1Λµ(i)

+ Tr(ΛTΨ−1ΛΣY

)+(µ(i))T

ΛTΨ−1Λµ(i)]

+ c1

H = −1

2

n∑i=1

[ln ‖2πΨ‖+

(x(i))T

Ψ−1x(i) − 2(x(i))T

Ψ−1Λµ(i)

+ Tr(ΛTΨ−1ΛΣY

)+ Tr

(ΛTΨ−1Λµ(i)

(µ(i))T)]

+ c1

20

H = −1

2

n∑i=1

[ln ‖2πΨ‖+

(x(i))T

Ψ−1x(i) − 2(x(i))T

Ψ−1Λµ(i)

+ Tr(

ΛTΨ−1Λ(

ΣY + µ(i)(µ(i))T))]

+ c1

H = −n2

ln ‖Ψ‖ − 1

2

n∑i=1

(x(i))T

Ψ−1x(i) +n∑i=1

(x(i))T

Ψ−1Λµ(i)

− 1

2

n∑i=1

Tr(

ΛTΨ−1Λ(

ΣY + µ(i)(µ(i))T))

+ c2

Taking the derivative with respect to Λ

DΛH = DΛ

[n∑i=1

(x(i))T

Ψ−1Λµ(i) − 1

2

n∑i=1

Tr(

ΛTΨ−1Λ(

ΣY + µ(i)(µ(i))T))]

DΛH = DΛ

[n∑i=1

Tr((

µ(i))T

ΛTΨ−1x(i))− 1

2

n∑i=1

Tr(

ΛTΨ−1Λ(

ΣY + µ(i)(µ(i))T))]

DΛH = DΛ

[n∑i=1

Tr(

ΛTΨ−1x(i)(µ(i))T)− 1

2

n∑i=1

Tr(

ΛTΨ−1Λ(

ΣY + µ(i)(µ(i))T))]

DΛH =n∑i=1

Ψ−1x(i)(µ(i))T − n∑

i=1

Ψ−1Λ(

ΣY + µ(i)(µ(i))T)

.

Setting the derivative to zero

0 =n∑i=1

Ψ−1x(i)(µ(i))T − n∑

i=1

Ψ−1Λ(


.

0 =n∑i=1

x(i)(µ(i))T − Λ

(nΣY +

n∑i=1

µ(i)(µ(i))T)

Λ =

(n∑i=1

x(i)(µ(i))T)(

nΣY +n∑i=1

µ(i)(µ(i))T)−1

. (3.17)

21

The objective can be re-parametrised using T = Ψ−1 so that

H = −n2

ln ‖T−1‖ − 1

2

n∑i=1

(x(i))T

Tx(i) +n∑i=1

(x(i))T

TΛµ(i)

− 1

2

n∑i=1

Tr(

ΛTTΛ(

ΣY + µ(i)(µ(i))T))

+ c2

H = −n2

ln ‖T−1‖ − 1

2

n∑i=1

(x(i))T

Tx(i) +n∑i=1

(x(i))T

TΛµ(i)

− 1

2

n∑i=1

Tr(

ΛTTΛ(

ΣY + µ(i)(µ(i))T))

+ c2 .

‖T−1‖ = 1/‖T‖ and using the properties of the trace

H =n

2ln ‖T‖ − 1

2

n∑i=1

Tr(

Tx(i)(x(i))T)

+n∑i=1

Tr(

TΛµ(i)(x(i))T)

−n∑i=1

1

2Tr(

TΛ(


ΛT)

+ c2 .

Taking the derivative with respect to Ψ

DΨH =n

2T−1 − 1

2

n∑i=1

x(i)(x(i))T

+n∑i=1

Λµ(i)(x(i))T

−n∑i=1

1

2Λ(


ΛT .

Setting the derivative to zero

0 =n

2Ψ− 1

2

n∑i=1

x(i)(x(i))T

+n∑i=1

Λµ(i)(x(i))T

− n

2ΛΣYΛ− 1

2Λ

n∑i=1

µ(i)(µ(i))T

)ΛT

22

0 =n

2Ψ− 1

2

n∑i=1

((x(i) − Λµ(i)

) (x(i) − Λµ(i)

)T)− n

2ΛΣYΛ

Ψ =1

n

n∑i=1

((x(i) − Λµ(i)

) (x(i) − Λµ(i)

)T)+ ΛΣYΛ . (3.18)

Because Ψ is a diagonal matrix, only the diagonal entries need to be updated.

3.3.2 Methods

The data was centred so that the mean of each pixel’s gray value was zero.By doing this, factor analysis focuses on modelling the variance of the pixel’sgray values.

For a given k number of factors, the EM algorithm was initialized byrandomly assigning each element of Λ to have distribution N(0, (103)2). Inaddition, each diagonal entry of Ψ was initialized by sampling from theGamma(1, 10−3) distribution. This was chosen to have arbitrary large vari-ance so that it covers as much of the parameter space as possible.

The EM algorithm was run many times with different initial values. Be-cause evaluating the log likelihood was expensive, the stopping condition ismet when a certain amount of E and M steps were taken. The parameterswith the lowest log likelihood was recorded.

The Bayesian information criterion (BIC) was used to select the best kto use. The BIC is given as

BIC = −2 lnL+ p(k + 1) lnn . (3.19)

The data was bootstrapped to assess the uncertainty in the model selection.

3.3.3 Results

k = 1, 2, ··· , 15 were investigated. The EM algorithm was repeated 81 timesfor each k with a stopping condition of 100 EM steps. The BIC for each kis as shown in Figure 3.10. The number of factors with the lowest BIC wasfound to be 8.

The 8 factor variances were isolated and as shown in Figure 3.11, togetherwith the intrinsic variance as shown in Figure 3.12. It was interesting to notethat in Figure 3.11, there are clusters of high variance in most of the factors.This suggest clusters of high variance can appear in the data randomly, thus

23

Number of factors

2 4 6 8 10 12 14

BIC

(nat)

×10 6

8

8.5

9

9.5

10

10.5

Figure 3.10: BIC for fitting different number of factors onto the data. 81different initials were used for each k with a stopping condition of 100 EMsteps.

the variance of each of the pixel’s gray values can depend on each otherspatially. In Figure , the shape of the sample can be seen represented by thelow intrinsic variance. This was evidence that they may well be a mean andvariance relationship.

The uncertainty of model selection was assessed by investigating how theBIC and optimal k varied for different bootstrap samples. The BIC for the99 bootstrap samples is shown in Figure 3.13. Only one initial value wasused for each k to capture the uncertainty if the EM algorithm converges toa global maxima or not. From the figure, quite clearly there was a lot ofvariability in the BIC and can cause imprecise decisions on model selection.Over all 99 bootstrap samples, the mean and standard deviation optimal kwas found to be 10± 2 factors.

24

×108

0.5

1

1.5

2

2.5

3

Figure 3.11: The 8 individual factor variances with the lowest log likelihoodfound.

25

200

400

600

800

1000

1200

Figure 3.12: The intrinsic variances with the lowest log likelihood found.

26

Number of factors

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

BIC

(nat)

×10 6

8.5

9

9.5

10

10.5

Figure 3.13: BIC for fitting different number of factors onto the data. 99bootstrap samples were used and only 1 initial value was used for each k. Astopping condition of 100 EM steps was used.

The main source of error was down to the small sample size. The ex-periments can be improved if the analysis was done on the full size imagerather than downsizing it. When downsizing, there is a risk of tampering thevariance of the pixel’s grey values.

27

Chapter 4

Mean and VarianceRelationship

The aim of this chapter was to do supervised learning on predicting thevariance of the pixel’s grey values given its mean. By training a classifier,a single CT scan can be used to quantify the uncertainty on each pixel’sgrey values. This uncertainty is then carried forward when doing volumetricreconstruction.

4.1 Weighted Least Squares

4.1.1 Theory

The sample variance of the grey values is a random variable, thus has varianceor in other words uncertainty. Weighted least squares can be used to takeinto consideration the uncertainty of the sample variance. This can be doneby giving low weights to data points with high uncertainty on the grey valuevariance.

Let S2i and σ2

i be the sample variance and true variance, respectively,of the ith pixel’s grey value for i = 1, 2, ··· ,m. The sample variance has asampling distribution such that

Ui =(n− 1)S2

i

σ2i

∼ χ2n−1 (4.1)

where n is the number of samples used in estimating the variance. The

28

variance is then

2(n− 1) =(n− 1)2

σ4i

Var(S2i

)(4.2)

thus

Var(S2i

)=

2σ4i

n− 1. (4.3)

A good weight for pixel i, wi, would be one which is proportional to theprecision, or reciprocal of the variance, of S2

i . σ2i is generally unknown but

can be estimated using S2i . By doing so, the weights for each grey value was

chosen to be

wi ∝1

(S2i )

2 . (4.4)

The weights can be introduced into the least squares problem. LetY1, Y2, ··· , Ym be the sample variance of the grey values and x1,x2, ··· ,xmbe the feature vectors of the sample mean of the grey values. The weightedleast squares is an optimization problem where the objective is

T =

∑mi=1wi

(Yi − (xi)

T β)2∑m

i=1wi. (4.5)

This can be expanded

T =

∑mi=1wi

(Yi − 2Yiβ

Txi + βTxi (xi)T β)

∑mi=1wi

. (4.6)

Using the properties of the trace

T =

∑mi=1wi

(Yi − 2Yi Tr

(βTxi

)+ Tr

(βTxi (xi)

T β))

∑mi=1wi

(4.7)

the differential with respect to β is

∇βT =−2∑m

i=1wiYixi + 2∑m

i=1wixi (xi)T β∑m

i=1wi. (4.8)

Setting the differential to 0

β =

[m∑i=1

wixi (xi)T

]−1 m∑i=1

wixiYi . (4.9)

29

Assuming Y1, Y2, ··· , Ym are independent and have constant variance σ2,the covariance of the weighted least squares is

Cov(β) = σ2

[m∑i=1

wixi (xi)T

]−1 [ m∑i=1

wixi

][m∑i=1

wi (xi)T

][m∑i=1

wixi (xi)T

]−1.

(4.10)σ2 can be estimated using the weighted mean squared error

σ2 =

∑mi=1wi

[Yi − (xi)

T β]2∑m

i=1wi. (4.11)

After estimating β and σ2 and given a data point xtest, then the predictedvariance is

Y = (xtest)T β + εtest (4.12)

with expectation

E(Y)

= (xtest)T β (4.13)

and varianceVar

(Y)

= (xtest)TCov(β)xtest + σ2 . (4.14)

4.1.2 Methods

The sample mean and sample variance were calculated for all 3 984 016 pixel’sgrey values. Polynomial features were constructed, up to order k, using thesample mean. The weighted linear regression was then fitted.

Before fitting the linear regression, the weighted were normalized to havea maximum of 1. Furthermore, each feature was normalized to have mean 0and standard deviation 1. Afterwards, a constant of 1 was appended to thefeature vector to model the intercept of the linear regression.

As in previous experiments, the BIC was used to do model selection.

4.1.3 Results

The weighted least squares with polynomial features were fitted successfullyas shown in Figure 4.1. It can be observed that the weighted least squaresgave less weights to data points with high sample variance, thus in a waytreating them like outliers.

30

The BIC, as shown in Figure 4.2, favoured high order polynomials. Be-cause the BIC favoured more complicated models, this suggested that fittingpolynomial features was not a good model. The BIC aimed to seek more andmore polynomials so that the model become more similar to the true model.

4.2 Subsampling Least Squares

4.2.1 Methods

From the previous chapter, it was clear there were 3 materials, the sample,background and foam, which contributed to the peaks in the histogram ofthe grey values.

The sample mean sample variance pairs were sub-sampled from each ofthe 3 materials. A 1st order linear regression was fitted for each sub-sample.

4.2.2 Results

It is clear the gradients are material dependent.

4.3 Mixture of Linear Regression

4.3.1 Theory

In order to consider the 3 different materials, a mixture of linear regressionwas used of the form

Yi ∼

N(

(xi)T β1, σ

21

)if Si = 1

N(

(xi)T β2, σ

22

)if Si = 2

N(

(xi)T β3, σ

23

)if Si = 3

(4.15)

where Si is a latent variable and P (Si = r) = πr for i = 1, 2, ··· ,m.The likelihood is

L =m∏i=1

3∑j=1

pY |S (yi|Si = j) πj (4.16)

31


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

aria

nce (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(a) Order 1


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

aria

nce (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(b) Order 2


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

ariance (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(c) Order 3


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

ariance (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(d) Order 4


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

ariance (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(e) Order 5


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

ariance (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(f) Order 6

Figure 4.1: Frequency density plots of the sample mean and variance of thegrey values. Weighted least squares with polynomial features was fitted. The(red) dotted line represent the 95% prediction interval.

32

Polynomial order

1 2 3 4 5 6

BIC

(nat)

×10 7

9.23

9.24

9.25

9.26

9.27

9.28

9.29

9.3

Figure 4.2: Weighted BIC when for fitting weighted least squares with poly-nomial features on the sample mean sample variance pairs.

thus the log likelihood is

lnL =m∑i=1

ln

[3∑j=1

pY |S (yi|Si = j) πj

]. (4.17)

Such a log likelihood can be maximised by using the EM algorithm, treatingSi as a random latent variable.

In the E step, the posterior distribution is used to put a distribution onSi for i = 1, 2, ··· ,m given the parameters βj, σ

2j and πj for j = 1, 2, 3. The

posterior distribution is given as

ri,j = P (Si = j|Yi = yi) =pY |S (yi|Si = j)πj∑3j′=1 pY |S (yi|Si = j′) πj

(4.18)

and just a quantity is called the responsibility of the ith data belonging tothe jth model.

In the M step, the parameters βj, σ2j and πj for j = 1, 2, 3 are updated

given the posterior distribution of Si for i = 1, 2, ··· ,m. Consider some arbi-trary parameter θj which model j depends on. Then

∂ lnL

∂θj=

m∑i=1

πj∑3j′=1 pY |S (yi|Si = j′) πj

∂pY |S (yi|Si = j)

∂θj(4.19)

33


2 2.5 3 3.5 4 4.5 5

Sa

mp

le g

rey v

alu

e v

aria

nce

(A

U2)

×10 5

0.5

1

1.5

2

2.5×10

-3

0

1

2

3

4

5

6

(a) Sample


4.6 4.7 4.8 4.9 5 5.1

Sa

mp

le g

rey v

alu

e v

aria

nce

(A

U2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

×10-3

0

0.5

1

1.5

2

(b) Background


3.6 3.7 3.8 3.9 4 4.1 4.2

Sa

mp

le g

rey v

alu

e v

aria

nce

(A

U2)

×10 5

1

1.5

2

2.5

3

3.5

4

4.5

×10-4

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

(c) Foam

Figure 4.3: A linear regression was fitted on a subsample of sample mean-variance pairs for each material.

34


2 2.5 3 3.5 4 4.5 5

Sa

mp

le g

rey v

alu

e v

aria

nce

(A

U2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

Figure 4.4: Frequency density of all the sample mean-variance pairs. Thedotted lines represent the 95% prediction interval of the least squares regres-sion for each sub-sample, these lines ranges so that they do not extrapolate.

∂ lnL

∂θj=

m∑i=1

pY |S (yi|Si = j) πj∑3j′=1 pY |S (yi|Si = j′) πj

∂ ln[pY |S (yi|Si = j)

]∂θj

(4.20)

which is just∂ lnL

∂θj=

m∑i=1

ri,j∂ ln

[pY |S (yi|Si = j)

]∂θj

. (4.21)

Using the above result

∇βjlnL =

m∑i=1

ri,j∇βjln

1√2πσj

exp

−1

2

(yi − (xi)

T βjσj

)2 (4.22)

∇βjlnL =

m∑i=1

ri,j∇βj

−1

2ln(2π)− 1

2ln(σ2j

)− 1

2

(yi − (xi)

T βjσj

)2 .

(4.23)Expanding it out

∇βjlnL =

m∑i=1

ri,j∇βj

[− 1

2σ2j

(−2yiβ

Tj xi + βT

j xi (xi)T βj

)+ c1

](4.24)

35

and using the properties of the trace

∇βjlnL =

m∑i=1

ri,j∇βj

[− 1

2σ2j

(−2yi Tr

(βTj xi)

+ Tr(βTj xi (xi)

T βj

))+ c1

](4.25)

∇βjlnL =

m∑i=1

− ri,j2σ2

j

(−2yixi + 2xi (xi)

T βj

). (4.26)

Setting this to zero

m∑i=1

ri,jyixi =m∑i=1

ri,jxi (xi)T βj (4.27)

βj =

[m∑i=1

ri,jxi (xi)T

]−1 [ m∑i=1

ri,jyixi

](4.28)

and this is just weighted least squares where the weights are the responsibil-ities on model j.

By re-parametrise 1/σ2j = τj

∂ lnL

∂τj=

m∑i=1

ri,j∂

∂τj

[−1

2ln(2π) +

1

2ln (τj)−

τj2

(yi − (xi)

T βj

)2](4.29)

∂ lnL

∂τj=

m∑i=1

ri,j

[1

2τj− 1

2

(yi − (xi)

T βj

)2]. (4.30)


σ2j =

∑mi=1 ri,j

(yi − (xi)

T βj

)2∑mi=1 ri,j

(4.31)

and this is just the weighted mean squared error.To obtain the maximum log likelihood estimator for πj, a Lagrange mul-

tiplier must be used to put a constraint that∑3

j=1 πj = 1. The objective isthen

T = lnL+ λ

(3∑j=1

πj − 1

)(4.32)

36

∂T

∂πj=

∂

∂πj

m∑i=1

ln

[3∑

j′=1

pY |S (yi|Si = j′) πj′

]+ λ (4.33)

∂T

∂πj=

m∑i=1

pY |S (yi|Si = j)∑3j′=1 pY |S (yi|Si = j′) πj′

+ λ (4.34)

∂T

∂πj=

m∑i=1

ri,jπj

+ λ . (4.35)


πj =m∑i=1

ri,jλ

. (4.36)

λ can be determined by putting a constraint on the estimators of πj forj = 1, 2, 3

3∑j=1

m∑i=1

ri,jλ

= 1 (4.37)

m∑i=1

3∑j=1

ri,j = λ (4.38)

m∑i=1

1 = λ (4.39)

m = λ (4.40)

therefore

πj =m∑i=1

ri,jm

. (4.41)

This is the average responsibility.All steps in the EM algorithm have been derived.

4.3.2 Methods

The EM algorithm was initialised by assigning random responsibilities for allmodels and data points in the E step.

37

Number of EM steps

5 10 15 20 25 30 35 40 45 50

Log lik

elih

ood (

nat)

×10 4

-1.25

-1.2

-1.15

-1.1

-1.05

-1

-0.95

Figure 4.5: The log likelihood of the mixture of linear regressions at everyEM step for 10 different initial values.

4.3.3 Results

It was verified that the EM algorithm always increased the log likelihood atevery EM step, as can be seen in Figure 4.5.

The resulting fit, as shown in Figure 4.6, did not capture the 3 differentmaterials. This is because the mixture probabilities, π1, π2, π3, do not dependon the sample-mean of the grey values. For example, low grey values are morelikely to be from the sample.

4.4 Distance Weighted Least Squares

4.4.1 Methods

To consider the 3 different sources, weighted least squares can be used wherethe weights depends on some distance from some particular sample-meangrey value. 3 different weighted least squares were used where the weightsare

wi ∝ exp

[−1

2

(xi − µjσj

)2]

(4.42)

µj was found using k-means in the sample-mean grey value space and σj ispredefined constant, called the Gaussian width.

38


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

ariance (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

Figure 4.6: Fitting a mixture of 3 linear regressions on the sample mean-variance data.

4.4.2 Results

The model captured the sample and foam well because the regression cap-tured the shape of the histogram. However for the background, the regressioncan be negative as a result of being influenced by lower sample grey valuesfrom the foam and sample.

For higher Gaussian widths, the regression becomes an ordinary leastsquares regression because all the data points approximately equal weights.

39


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

arian

ce (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(a) σ = 102


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

arian

ce (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(b) σ = 103


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

ariance (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(c) σ = 104


2 2.5 3 3.5 4 4.5 5

Sam

ple

gre

y v

alu

e v

ariance (

AU

2)

×10 5

0.5

1

1.5

2

2.5

3

3.5

4×10

-3

0

1

2

3

4

5

6

(d) σ = 105

Figure 4.7: 3 independent Gaussian weighted least squares regressions. Theweights depend on the distance from the k-means sample mean grey valuesand some width σ.

40

Chapter 5

Model for Intensity

5.1 Rate Density

Consider photons emitted from an X-ray tube. These photons have a boardenergy spectrum as a result of Bremsstrahlung and characteristic radiation.An experiment can be conducted by measuring the energy of each photon,over some exposure time, to obtain a histogram of photon energies, similarto the one in Figure 5.1.

In order to model the number of photons detected in some exposure timeas a Poisson process, the rate of photons emitted is an important variable.However it is nonsense to talk about the rate of photons with energy exactlyE. This is because energy is a continuous variable and cannot be modelleddiscretely. Instead the rate of photons with energy E1 < E < E2 should beconsidered.

To take into consideration the continuous nature of energy, the histogramof photon energies should be normalised in such a way that the area ofthe histogram is the rate of photons, with any energy, detected. This willproduce an approximation to the rate density function λ(E). The rate densityfunction has units of s−1 · keV−1.

The rate density function has the following properties.

rate of photons with energy E1 < E < E2 =

∫ E2

E1

λ(E)dE (5.1)

rate of photons with any energy =

∫ ∞0

λ(E)dE (5.2)

41

Figure 5.1: A typical energy spectrum of X-ray photons emitted from an X-ray tube. The continuous spectrum is the result of Bremmsstrahlung radia-tion. The peaks are the result of characteristic radiation. Source: G. Michael(2001) [3]

42

The expected energy can be calculated by splitting the rate density func-tion into n equal intervals, of size ∆E, E1, E2, ··· , En. A weighted mean canbe used to estimate the expected energy

E[E] =

∑ni=1Eiλ(Ei)∆E∑ni=1 λ(Ei)∆E

. (5.3)

Taking the limit ∆E → 0 and n→∞, the sum becomes an integral and canbe generalised to

E[E] =

∫∞0Eλ(E)dE∫∞

0λ(E)dE

. (5.4)

This can be extended to the expected function of energy, for example

E[E2] =

∫∞0E2λ(E)dE∫∞

0λ(E)dE

. (5.5)

The variance of the energy then can be computed

Var[E] = E[E2]− (E[E])2 . (5.6)

The intensity I is the rate of energy per unit area. Again, by splitting therate density function into n equal intervals, the intensity can be approximatedby summing energy × rate for each interval

I =1

A

n∑i=1

Eiλ(Ei)∆E (5.7)

where A is the area of the detector. Taking the limit ∆E → 0 and n→∞,the sum becomes an integral and can be generalised to

I =1

A

∫ ∞0

Eλ(E)dE . (5.8)

Thus intensity is proportional to the average energy.

5.2 Attenuation

Consider an incident pencil beam of X-ray photons, with intensity I0, randomenergy E0 and rate density λ0(E). The beam penetrates through an object

43

with attenuation constant µ and length L. It is given that the intensity ofthe transmitted beam I1 is

I1 = I0e−µL . (5.9)

Then

I1 =e−µL

A

∫ ∞0

Eλ0(E)dE (5.10)

where A is the area of the pencil beam. Substituting the rate density for thetransmitted beam λ1(E)∫ ∞

0

Eλ1(E)dE = e−µL∫ ∞0

Eλ0(E)dE (5.11)∫∞0Eλ1(E)dE∫∞

0λ1(E)dE

= e−µL∫∞0Eλ0(E)dE∫∞

0λ1(E)dE

(5.12)

E[E1] = e−µL∫∞0Eλ0(E)dE∫∞

0λ1(E)dE

(5.13)

E[E1] = e−µL∫∞0λ0(E)dE∫∞

0λ1(E)dE

E[E0] . (5.14)

Thus the expected photon energy will shift by a factor when transmittingthrough an object.

Unfortunately Beer’s law only seem to capture the shift in expectation.It would be interesting to see how the total rate

∫∞0λ1(E)dE and variance

varied after transmission.It is given that the attenuation varies with different photon energies.

To consider this, the attenuation should be a function of energy, that isµ = µ(E). By splitting the rate density function into n equal intervals, theintensity of the transmitted beam is

I1 =1

A

n∑i=1

e−µ(Ei)LEiλ0(Ei)∆E . (5.15)

Taking the limit ∆E → 0 and n→∞, the sum becomes an integral and canbe generalised to

I1 =1

A

∫ ∞0

e−µ(E)LEλ0(E)dE (5.16)

and

I1 =1

A

∫ ∞0

λ0(E)dE × E[E0e

−µ(E0)L]. (5.17)

Such a relationship is considerably more complicated.

44

5.3 Intensity

Let Ui be the energy of photon i. Let T be the sum of all the N photons’energy detected after some exposure time t. These can be modelled as

N ∼ Poisson

(t

∫ ∞0

λ1(E)dE

)(5.18)

T |N = U1 + U2 + ··· +UN (5.19)

where Ui are i.i.d. and has distribution

pUi(u) =

λ1(u)∫∞0λ1(E)dE

(5.20)

with mean µU and variance σ2U .

The expectation of T is

E[T ] = E[E[T |N ]] (5.21)

E[T ] = E[NµU ] (5.22)

E[T ] = µU t

∫ ∞0

λ1(E)dE . (5.23)

The variance of T is

Var[T ] = Var[E[T |N ]] + E[Var[T |N ]] (5.24)

Var[T ] = Var[NµU ] + E[Nσ2U ] (5.25)

Var[T ] =(µ2U + σ2

U

)t

∫ ∞0

λ1(E)dE . (5.26)

The transmitted intensity I1 is the rate of energy per unit area. This canbe modelled as

I1 =α

tT + ε (5.27)

where α is some constant with dimensions[L−2]

and ε is some zero-meannoise with variance β2. The expectation of I1 is

E[I1] = αµU

∫ ∞0

λ1(E)dE . (5.28)

The variance of I1 is

Var[I1] =α2

t

(µ2U + σ2

U

) ∫ ∞0

λ1(E)dE + β2 . (5.29)

45

5.4 Hierarchical Model

Suppose n images have been obtained and each image has p pixels. Each pixelhas a corresponding intensity Xi for i = 1, 2, ··· , p. The following hierarchicalmodel was proposed to model the intensity of each pixel where α and τ areknown constants.

Yi ∼ Poisson(νiτ) (5.30)

Ui|Yi ∼ N(Yiµi, Yiσ

2i

)(5.31)

Xi|Ui ∼ N(ατUi, β

2i

)(5.32)

The full joint probability density function is

pXi,Ui,Yi (xi, ui, yi) = pXi|Ui(xi|ui)pUi|Yi(ui|yi)P(Yi = yi) (5.33)

pXi,Ui,Yi (xi, ui, yi) =1√

2πβiexp

[−1

2

(xi − αui/τ

βi

)2]

1√

2π√yiσ2

i

exp

−1

2

(ui − yiµi√

yiσ2i

)2 e−νiτ

(νiτ)yi

yi!. (5.34)

By marginalising out Ui, a one layer latent variable model can be obtainedand the unknown parameters νi, µi, σ

2i , β

2i can be estimated using the EM

algorithm. This can be shown by integrating the full joint probability densityfunction with respect to ui

pXi,Yi (xi, yi) =

∫ ui=∞

ui=−∞

1√2πβi

exp

[−1

2

(xi − αui/τ

βi

)2]

1√

2π√yiσ2

i

exp

−1

2

(ui − yiµi√

yiσ2i

)2 e−νiτ

(νiτ)yi

yi!dui (5.35)

to get

pXi,Yi (xi, yi) =e−νiτ (νiτ)yi

yi!

1√

2π√β2i + α2yiσ2

i /τ2

exp

−1

2

(xi − yiµiα/τ√β2i + α2yiσ2

i /τ2

)2 . (5.36)

46

By inspection the one layer latent model is

Yi ∼ Poisson(νiτ) (5.37)

Xi|Yi ∼ N

(αµiτYi,

α2σ2i

τ 2Yi + β2

i

). (5.38)

The marginal expectation and variance can be calculated

E[Xi] = E[E[Xi|Yi]] (5.39)

E[Xi] = E[αµiτYi

](5.40)

E[Xi] = αµiνi (5.41)

andVar[Xi] = E[Var[Xi|Yi]] + Var[E[Xi|Yi]] (5.42)

Var[Xi] = E[α2σ2

i

τ 2Yi + β2

i

]+ Var

[αµiτYi

](5.43)

Var[Xi] =α2σ2

i

τ 2νiτ + β2

i +α2µ2

i

τ 2νiτ (5.44)

Var[Xi] =α2σ2

i

τ 2νiτ + β2

i +α2µ2

i

τ 2νiτ (5.45)

Var[Xi] =α2νiτ

(µ2i + σ2

i

)+ β2

i . (5.46)

This model can be made simpler by setting βi = 0 because by doing sothe joint probability density function is in the exponential family. Howeverfor the case of βi = 0, the joint probability density function much considerthe special case of Yi = 0 because such a case will drive the variance of Xi|Yito zero.

pXi,Yi (xi, yi) =

e−νiτδ(x) for y = 0

e−νiτ (νiτ)yiτ

yi!√

2πyiασiexp

[−1

2

(xi − yiµiα/τ)2

α2yiσ2i /τ

2

]for y = 1, 2, ···

(5.47)For the y = 1, 2, ··· case, the joint probability density function can be rewrit-ten as

pXi,Yi (xi, yi) =e−νiτ (νiτ)yiτ

yi!√

2πyiασiexp

[−1

2


α2yiσ2i /τ

2

](5.48)

47

pXi,Yi (xi, yi) =e−νiττ√2πασi

exp

[−1

2


α2yiσ2i /τ

2

−yi ln(νiτ)− 1

2ln yi − ln yi!

](5.49)


exp

[− τ 2

2α2σ2i

x2i − 2αµixiyi/τ + α2µ2i y

2i /τ

2

yi

−yi ln(νiτ)− 1

2ln yi − ln yi!

](5.50)


exp

[− τ 2

2α2σ2i

(x2iyi− 2αµixi

τ+α2µ2

i yiτ 2

)−yi ln(νiτ)− 1

2ln yi − ln yi!

](5.51)


exp

[− τ 2

2α2σ2i

x2iyi

+τµiασ2

i

xi

−(µ2i

2σ2+ ln(νiτ)

)yi −

1

2ln yi − ln yi!

]. (5.52)

This shows that the joint probability density function is in the exponentialfamily where the sufficient statistics are X2

i /Yi, Xi and Yi.For the E step, Yi and 1/Yi are estimated given all the parameters using

the conditional expectation yi = E [Yi|Xi = xi] and ζi = E [1/Yi|Xi = xi]respectively. The conditional probability mass function can be obtained by

P (Yi = yi|Xi = xi) =pXi,Yi (xi, yi)

pXi(xi)

(5.53)

where pXi(xi) is the normalization constant. For the special cases

P (Yi = 0|Xi = 0) ∝ δ(0) (5.54)

P (Yi = y|Xi = 0) ∝ (νiτ)yi

yi!√yi

exp

[−1

2


α2yiσ2i /τ

2

]for y = 1, 2, ··· .

(5.55)

48

But because δ(0) is infinite,

P (Yi = y|Xi = 0) =

{1 for y = 0

0 for y = 1, 2, ···(5.56)

andE[Yi|Xi = 0] = 0 . (5.57)

E[1/Yi|Xi = 0] does not exist.Another special case is

P (Yi = 0|Xi = x) ∝ δ(x) for x 6= 0 (5.58)

thenP (Yi = 0|Xi = x) = 0 for x 6= 0 (5.59)

For the general case yi = 1, 2, ··· and xi 6= 0

P (Yi = yi|Xi = xi) =1

pXi(xi)


yi!√

2πyiασiexp

[−1

2


α2yiσ2i /τ

2

](5.60)

where

pXi(xi) =

∞∑y=0

pXi,Yi(xi, yi) (5.61)

pXi(xi) =

∞∑yi=1


yi!√

2πyiασiexp

[−1

2


α2yiσ2i /τ

2

]. (5.62)

The expectation of the sufficient statistics are then

E [Yi|Xi = xi] =1

pXi(xi)

∞∑yi=1


yi!√

2πασiy1/2i exp

[−1

2


α2yiσ2i /τ

2

](5.63)

and

E [1/Yi|Xi = xi] =1

pXi(xi)

∞∑yi=1


yi!√

2πασiy−3/2i exp

[−1

2


α2yiσ2i /τ

2

](5.64)

Unfortunately such a sum cannot be simplified. The fastest (approximate)way to get a value for these expectations is to truncate the sum. Another

49

way to obtain the conditional expectation is to use the sample mean fromsamples of the conditional.

Because the conditional probability mass function is known up to a con-stant, rejection sampling can be used to draw conditional samples. Rejectionsampling samples exactly but a proposal is needed and this influences the re-jection rate. Approximate Bayesian computation (ABC) can be used insteadbecause Yi and Xi|Yi can be simulated. ABC draws approximate sampleswithout a proposal but can suffer from high rejection rates.

In the M step the parameters νi, µi, σ2i are estimated given the latent

variables. This is done by maximising the conditional expectation of the loglikelihood function, that is

Hi =n∑j=1

EYi|Xi

[ln pXi,Yi

(x(j)i , Y

(j)i

)](5.65)

Hi =n∑j=1

EYi|Xi

[−νiτ + Y

(j)i ln(νiτ)− 1

2ln

(α2σ2

i

τ 2Y

(j)i

)

−1

2

(x(j)i − Y

(j)i µiα/τ

)2α2Y

(j)i σ2

i /τ2

+ c1 . (5.66)

Hi = −nνiτ + ln(νiτ)n∑j=1

EYi|Xi

[Y

(j)i

]− n

2ln

(α2σ2

i

τ 2

)

− 1

2

n∑j=1

EYi|Xi

(x(j)i − Y

(j)i µiα/τ

)2α2Y

(j)i σ2

i /τ2

+ c2 . (5.67)


EYi|Xi

[Y

(j)i

]− n

2ln

(α2σ2

i

τ 2

)

− 1

2

n∑j=1

EYi|Xi

(x(j)i

)2− 2x

(j)i Y

(j)i µiα/τ +

(Y

(j)i µiα/τ

)2α2Y

(j)i σ2

i /τ2

+ c2 . (5.68)

50


EYi|Xi

[Y

(j)i

]− n

2ln

(α2σ2

i

τ 2

)

− τ 2

2α2σ2i

n∑j=1

(x(j)i

)2EYi|Xi

[1/Y

(j)i

]+τµiασ2

i

n∑j=1

x(j)i −

µ2i

2σ2i

n∑j=1

EYi|Xi

[Y

(j)i

].

(5.69)

Let EYi|Xi

[Y

(j)i

]= y

(j)i and EYi|Xi

[1/Y

(j)i

]= ζ

(j)i which was obtained

from from the E step. Then


y(j)i −

n

2ln

(α2σ2

i

τ 2

)

− τ 2

2α2σ2i

n∑j=1

(x(j)i

)2ζ(j)i +

τµiασ2

i

n∑j=1

x(j)i −

µ2i

2σ2i

n∑j=1

y(j)i . (5.70)

By setting the partial differentials to zero with respect to µi, σ2i and νi to

zero, then the following M step estimators can be obtained:

µi =τ∑n

j=1 x(j)i

α∑n

j=1 y(j)i

(5.71)

σ2i =

1

n

[τ 2

α2

n∑j=1

(x(j)i

)2ζ(j)i −

2τµiα

n∑j=1

x(j)i + µ2

i

n∑j=1

y(j)i

](5.72)

νi =

∑nj=1 y

(j)i

nτ(5.73)

where(x(j)i

)2ζ(j)i = 0 if

(x(j)i

)2= 0.

The constant α only sets the scale of µi and σ2i but can be calibrated so

that it has units of keV. The following properties of the X-ray tube follows(NO UNCERTAINTY CALCULATIONS SO FAR):

• Voltage = V = 100 kV

• Power = P = 33.0 W

• Exposure time = τ = 500 ms

51

• Target material is tungsten.

The efficiency η of photon production is given as

η = aV Z (5.74)

where a = 1.1×10−9 V−1 and Z is the atomic number of the target material.For tungsten, Z = 74. Thus η = 8.1× 10−3.

Assume the X-ray spreads out and targets the detector with 19962 pixelsonly and nothing else, then the total energy detected in one pixel is

U =ηPτ

19962= 2.1× 108 keV · pixel−1 . (5.75)

Assuming the air has attenuation coefficient of zero, then α can be crudelyestimated by

α =60 000τ

U= 1.4× 10−4 s · keV−1 · pixel . (5.76)

Other parameters for air such as the rate and energy of photons can becalculated.

ν = ηdN

dQ

dQ

dt=ηP

V e= 1.7× 1013 s−1 (5.77)

dE

dN=

dE

dQ

dQ

dN= V e = 1.0× 102 keV (5.78)

5.4.1 Small example

A dataset was simulated, with n = 100 samples, using the follow parameterswith arbitrary units: α = 1, τ = 1, ν = 5, µ = 3 and σ2 = 0.1. TheEM algorithm was initialized by assigning each latent variable such thatY (j) ∼ Poisson(φ) for i.i.d. j = 1, 2, ··· , n. The sum was truncated to include100 terms.

The figures show that the EM algorithm works because for different initialvalues, the parameters converge towards the true value and the log likelihoodalways increases.

For high ντ , problems arises as more and more terms are needed in thetruncated sum. Too many terms in the truncated sum slow down the programa lot.

52

Number of EM Steps

0 50 100 150 200 250 300

Me

an

estim

ate

(a

rb.

un

it)

0

5

10

15

20

25

Number of EM Steps

0 50 100 150 200 250 300

Va

ria

nce

estim

ate

(a

rb.

un

it)

10 -2

10 -1

10 0

10 1

10 2

10 3

Number of EM Steps

0 50 100 150 200 250 300

Ra

te e

stim

ate

(a

rb.

un

it)

0

1

2

3

4

5

6

7

8

9

10

Number of EM Steps

0 50 100 150 200 250 300

Lo

g lik

elih

oo

d (

na

ts)

-480

-460

-440

-420

-400

-380

-360

-340

-320

-300

Figure 5.2: The parameters and log likelihood at each step of EM. The EMalgorithm was used to estimate the parameters of a single simulated n = 100dataset. The latent variables were initialized Poisson(φ) 50 times. Blue:φ = 1, Red: φ = 4, Green: φ = 9. Dotted line: true value

53

Number of EM Steps

0 50 100 150 200 250 300

Me

an

estim

ate

(a

rb.

un

it)

2.8

3

3.2

3.4

3.6

3.8

4

4.2

4.4

Number of EM Steps

0 50 100 150 200 250 300

Va

ria

nce

estim

ate

(a

rb.

un

it)

10 -2

10 -1

10 0

10 1

10 2

Number of EM Steps

0 50 100 150 200 250 300

Ra

te e

stim

ate

(a

rb.

un

it)

3.4

3.6

3.8

4

4.2

4.4

4.6

4.8

5

5.2

5.4

Number of EM Steps

0 50 100 150 200 250 300

Lo

g lik

elih

oo

d (

na

ts)

-400

-380

-360

-340

-320

-300

-280

Figure 5.3: The parameters and log likelihood at each step of EM. TheEM algorithm was used to estimate the parameters of 50 different simulatedn = 100 dataset. The latent variables were initialized Poisson(4). Dottedline: true value

54

Bibliography

[1] Angela Cantatore and Pavel Muller. Introduction to computed tomog-raphy. Technical report, DTU Mechanical Engineering, 2011.

[2] Godfrey N Hounsfield. Computed medical imaging. Journal of computerassisted tomography, 4(5):665–674, 1980.

[3] Greg Michael. X-ray computed tomography. Physics Education,36(6):442, 2001.

[4] Frank Welkenhuyzen, Kim Kiekens, Mieke Pierlet, Wim Dewulf, PhilipBleys, Jean-Pierre Kruth, and Andre Voet. Industrial computer to-mography for dimensional metrology: overview of influence factors andimprovement strategies. In Proceedings of the 4th International Confer-ence on Optical Measurement Techniques for Structures and Systems:Optimess2009, pages 401–410, 2009.

[5] Robert Andrews Millikan. A direct photoelectric determination ofPlanck’s “h”. Physical Review, 7(3):355, 1916.

[6] JH Hubbell. Electron–positron pair production by photons: A historicaloverview. Radiation Physics and Chemistry, 75(6):614–623, 2006.

[7] Arthur H Compton. A quantum theory of the scattering of x-rays bylight elements. Physical review, 21(5):483, 1923.

[8] M Bartscher, U Hilpert, J Goebbels, and G Weidemann. Enhancementand proof of accuracy of industrial computed tomography (CT) measure-ments. CIRP Annals-Manufacturing Technology, 56(1):495–498, 2007.

[9] Rodney A Brooks and Giovanni Di Chiro. Principles of computer as-sisted tomography (CAT) in radiographic and radioisotopic imaging.Physics in medicine and biology, 21(5):689, 1976.

55

[10] Eric W Weisstein. Bonferroni correction. http://mathworld.wolfram.com/BonferroniCorrection.html, 2004. From MathWorld–A WolframWeb Resource., Accessed: 18/04/2016.

[11] Kenneth Franklin Riley, Michael Paul Hobson, and Stephen John Bence.Mathematical methods for physics and engineering. Cambridge Univer-sity Press, 3rd edition, 2006.

56

inside-out: characterisation of computed tomography noise ...sip/miniproject1/report_4.pdf ·...

Documents