uc sf va medical imaging informatics 2009, nschuff course # 170.03 slide 1/31 department of...

UCSF VAMedical Imaging Informatics 2009, NschuffCourse # 170.03Slide 1/31

Department of Radiology & Biomedical Imaging

MEDICAL IMAGING INFORMATICS:Lecture # 1

Basics of Medical Imaging Informatics:Estimation Theory

Norbert SchuffProfessor of Radiology

VA Medical Center and [email protected]



What Is Medical Imaging Informatics?• Picture Archiving and Communication System (PACS) • Imaging Informatics for the Enterprise • Image-Enabled Electronic Medical Records • Radiology Information Systems (RIS) and Hospital Information Systems (HIS) • Digital Image Acquisition • Image Processing and Enhancement • Image Data Compression • 3D, Visualization and Multi-media • Speech Recognition • Computer-Aided Detection and Diagnosis (CAD). • Imaging Facilities Design • Imaging Vocabularies and Ontologies • Data-mining from medical image databases • Transforming the Radiological Interpretation Process (TRIP)[2] • DICOM, HL7 and other Standards • Workflow and Process Modeling and Simulation • Quality Assurance • Archive Integrity and Security • Teleradiology • Radiology Informatics Education • Etc.



What Is Our Focus?Learn using computation tools to maximize information and

gain knowledge

Imaging

Measurements

Model

knowledge

Extract information Compare

withmodelRe-active

ImproveData

collectionRefine Model

Pro-active



Challenge:Extract Maximum Information

1. Q: How can we estimate quantities of interest from a given set of uncertain (noise) measurements?

A: Apply estimation theory (1st lecture by Norbert)

2. Q: How can we code the quantities?

A: Apply information theory (2nd lecture by Wang)



Estimation Theory: Motivation Example I

Gray/White Matter Segmentation

Intensity

0.0

0.2

0.4

0.6

0.8

1.0

Hypothetical Histogram

GM/WM overlap 50:50;Can we do better than flipping a coin?



Estimation Theory: Motivation Example II

Courtesy of Dr. D. Feinberg Advanced MRI Technologies, Sebastopol, CA

Goal:Capture dynamic signal on a static background



Basic Concepts

1 , 2 ,...T

x x x N NxSuppose we have N scalar measurements :

We want to determine M quantities (parameters): 1 2, ,...T

M Mθ

Definition: An estimator is: j jθ̂ h ;N j M x

Error estimator: ˆθ θ θ; for N >M



Examples of Estimators

100 300 500 700 900

Measurements (x)

-50

0

50

100

150

Inte

ns

itie

s (

Y)

Mean Value: 1

1ˆN

j

x jN

Variance 2

2

1

1ˆ ˆ ˆ1

N

j

x jN

100 300 500 700 900

Measurements

-100

0

100

200

Inte

nsi

ty

Amplitude:1̂

Frequency:2̂

Phase:3̂

Decay:4̂



Some Desirable Properties of Estimators I:

ˆ ˆθ - θ = E θ 0 θ θE E E

Unbiased: Mean value of the error should be zero

2

θ̂ - θ 0 for large NMSE E

Consistent: Error estimator should decrease asymptotically as number of measurements increase. (Mean Square Error (MSE))

2 2θ̂ - θ - b bMSE E E

If estimator is biased

variance bias



Some Desirable Properties of Estimators II:

1max

ˆ ˆθ-θ θ-θT

E J θC

Efficient: Co-variance matrix of error should decrease asymptotically to itsminimal value, i.e. inverse of Fisher’s information matrix) for large N



Example:Properties Of Estimators Mean and Variance

1

1 1ˆ

N

j

E E x j NN N

Mean:

The sample mean is an unbiased estimator of the true mean

2

22 22 2

1

1 1ˆ

N

j

E E x j NN N N

Variance:

The variance is a consistent estimator becauseIt approaches zero for large number of measurements.



Least-Squares Estimation

Generally: 0Nv

Linear Model: N M Nx Hθ v where N > M

0 50 100 150 200 250

Y1

40

60

80

100

Y2

1 11 1 12 2 1

2 21 1 22 2 2

1 1 2 2

... ....

N N N N

x h h v

x h h v

x h h v



Least-Squares Estimation

LSEˆ 0 T T

NH x H H θ

Minimizing ELSE with regard to leads to

1

LSE

T T

nθ H H H x

The best what we can do:

21 1arg min

2 2T

LSE N N M N ME v x H x H

•LSE is popular choice for model fitting•Useful for obtaining a descriptive measure

But •Makes no assumptions about distributions of data or parameters•Has no basis for statistics



Maximum Likelihood (ML) Estimator

1 , 2 ,...T

x x x T NxWe have:

Xn is random sample from a pool with certain probability distribution.

ˆ arg max |ML p Nx θ

Goal: Find that gives the most likely probability distribution underlying xN.

Max likelihood function

ln | 0ML

dyp

d

Nθ θ

x θθ

ML can be found by



Example I: ML of Normal Distribution

ML function of a normal distribution

/ 2 22 22

1

1| , 2 exp

2

NN

j

p x j

Nx

log ML function 22 22

1

1ln | , ln 2

2 2

N

j

Np x j

Nx

1st log ML equation 22

1

1ˆ ˆ ˆln | , 0

ˆ

N

ML ML MLjML

dp x j

d

Nx

MLE of the mean 1

1ˆ

N

MLj

x jN

2nd log ML equation 22 2 4

1

1ˆ ˆ ˆln | , 0

ˆ ˆ ˆ2 4

N

ML ML MLjML ML

d Np x j

d

Nx

MLE of the variance 2

1

1ˆ ˆ

N

ML MLj

x jN



Example II: Binominal Distribution(Coin Toss)

!| , 1

! !n yyn

f y n w w wy n y

Probability density function:n= number of tossesw= probability of success

1 2 3 4 5 6 7 8 9 10N

0.0

0.1

0.2

0.0

0.1

0.2

y 0.3

0.7

y

f(y|n=10,w)



Likelihood Function Of Coin Tosses

( | 7, 10) | 0.7, 10L w y n f y w n

Given the observed data f (y|w=0.7, n=10) (and model), find the parameter w that most likely produced the data.

0.1 0.3 0.5 0.7 0.9W

0.00

0.05

0.10

0.15

0.20

0.25

Like

lihoo

d



ML Estimation Of Coin Toss

ln |0

1

d L w y n yy

dw w w

0(1 ) MLE

y nw yw

w w n

!ln | ln ln ln 1

! !

nL w y y w n y w

y n y

Compute log likelihood function

Evaluate ML equation

According to the MLE principle, the PDF f(y|w=y/n) for a given n is the distribution that is most likely to have generated the observed dataof y.



Connection between ML and LSE

are independent of vN

ML and vN have the same distribution

vN is zero mean and gaussian

Assume:

| |p p θ N v Nx θ x Hθ θ

ML function of a normal distribution

2

2 2

1 1| exp exp

2 2T

p

N N N Nx θ x Hθ x Hθ x Hθ

Clearly, p(x|) is maximized when LSE is minimized



Properties Of The ML Estimator

• is consistent: the MLE recovers asymptotically the true parameter values that generated the data for N inf;

• Is efficient: The MLE achieves asymptotically the minimum error (= max. information)



Maximum A-Posteriori (MAP) Estimator

1 , 2 ,...T

x x x T NxWe have random sample:

ˆ arg max |MAP p p Nx θ θ

Goal: Find the most likely (max. posterior density of ) given xN.

Maximize joint density

ln | ln | ln 0MAP

dyp p p p

d

N Nθ θ

x θ θ x θ θθ θ θ

MAO can be found by

pθ θ We also have random parameters:



MAP Of Normal Distribution

We have random sample:

2 22 2

1 1

1 1ˆ ˆ ˆ ˆ ˆ ˆln | , , 0

ˆ ˆ ˆ

N N

MLj j

p p x j pu

N xx μ

x

The sample mean of MAP is:

2

2 21

ˆN

jx

x jT

MAPμ

If we do not have prior information on , inf or T inf

MAP MLˆ ˆ ˆ, LSEμ μ μ



Posterior Density and Estimators

X

p(|x)

MAPMSE



Summary

• LSE is a descriptive method to accurately fit data to a model.

• MLE is a method to seek the probability distribution that makes the observed data most likely.

• MAP is a method to seek the most probably parameter value given prior information about the parameters and the observed data.

• If the influence of prior information decreases, i.e. many any measurements, MAP approaches MLE



Some Priors in Imaging

• Smoothness of the brain• Anatomical boundaries • Intensity distributions• Anatomical shapes• Physical models

– Point spread function– Bandwidth limits

• Etc.



Imaging Software Using MLE And MAP

Packages Applications Languages VoxBo fMRI C/C++/IDL MEDx sMRI, fMRI C/C++/Tcl/Tk SPM fMRI, sMRI matlab/C iBrain IDL FSL fMRI, sMRI, DTI C/C++

fmristat fMRI matlab BrainVoyager sMRI C/C++

BrainTools C/C++ AFNI fMRI, DTI C/C++

Freesurfer sMRI C/C++ NiPy Python



Segmentation Using MLE

A: Raw MRIB: SPM2C: EMSD: HBSA

fromHabib Zaidi, et al, NeuroImage 32 (2006) 1591 – 1607



Brain Parcellation Using MLE

Manual EM-Affine EM-NonRigid EM-Simultaneous

Kilian Maria Pohl, Disseration 1999Prior information for Brain parcellation



MAP Estimation In Image Reconstruction

Human brain MRI. (a) The original LR data. (b) Zero-padding interpolation. (c) SR with box-PSF. (d) SR with Gaussian-PSF.

From: A. Greenspan in The Computer Journal Advance Access published February 19, 2008



Literature

Mathematical• H. Sorenson. Parameter Estimation – Principles and Problems.

Marcel Dekker (pub)1980.

Signal Processing• S. Kay. Fundamentals of Signal Processing – Estimation Theory.

Prentice Hall 1993.• L. Scharf. Statistical Signal Processing: Detection, Estimation, and

Time Series Analysis. Addison-Wesley 1991.

Statistics:• A. Hyvarinen. Independent Component Analysis. John Wileys &

Sons. 2001.• New Directions in Statistical Signal Processing. From Systems to

Brain. Ed. S. Haykin. MIT Press 2007.

uc sf va medical imaging informatics 2009, nschuff course # 170.03 slide 1/31 department of...

Documents