imaging system design and analysisjao/talks/othertalks/darpatalk2.pdf · chrysanthe preza david g....

Imaging System Design and Imaging System Design and Analysis:Analysis:

from optics through information from optics through information extractionextraction

Joseph A. OJoseph A. O’’SullivanSullivan

Electronic Systems and Signals Research LaboratoryDepartment of Electrical Engineering

Washington [email protected]

http://essrl.wustl.edu/~jao

Supported by: ONR, ARO, NIH, DARPAONR, ARO, NIH, DARPA

J. A. O’Sullivan. DARPA ICIS 06/20/2002Imaging System Design

2

CollaboratorsCollaborators

Michael D. DeVoreNatalia A. SchmidMetin OzRyan MurphyJasenka BenacAdam CataldoLee MontagninoAndrew Li

Donald L. SnyderWilliam H. SmithDaniel R. FuhrmannJeffrey F. WilliamsonBruce R. WhitingRichard E. BlahutJohn C. SchotlandMichael I. MillerChrysanthe PrezaDavid G. PolitteJames G. Blaine

Faculty Students and Post-Docs


3

Outline:Outline:

•• Systems View of ImagingSystems View of Imaging•• Opportunities for Joint Design Opportunities for Joint Design •• Roles of Information TheoryRoles of Information Theory•• Examples (as desired)Examples (as desired)

-- Alternating Minimization AlgorithmsAlternating Minimization AlgorithmsApplications in CT, HSIApplications in CT, HSI

-- PerformancePerformance--Complexity TradeoffsComplexity TradeoffsApplications in SAR ATRApplications in SAR ATR

•• ConclusionsConclusions


4

Imaging System DesignImaging System Design•• System ConsiderationsSystem Considerations

– Focus on end-to-end performancerequires clear system goal(s)– Component versus system optimization– Imaging System Components as Processors

•• Irreversible OperationsIrreversible Operations– Role of Sufficient Statistics

•• Adaptability, Flexibility, FeedbackAdaptability, Flexibility, Feedback•• Hardware, Software, ProcessingHardware, Software, Processing•• Roles of Information TheoryRoles of Information Theory


5

System ConsiderationsSystem Considerations

optics FPA Signal Proc.

RemoteProc.

Comm.Link

Information FlowElectromagnetic waves Sampling DSP Comm. DSP Display

Design and Analysis: Forward and BackwardMotivates Adaptability and Feedback

Information flow forward and backward


6

Some Roles of Information TheorySome Roles of Information TheoryIn Imaging ProblemsIn Imaging Problems

Some Important Ideas:Some Important Ideas:•• Roles depend on problem studiedRoles depend on problem studied•• Key problems are in detection, Key problems are in detection,

estimation, and classificationestimation, and classification•• Information is quantifiableInformation is quantifiable•• SNR as a measure has limitationsSNR as a measure has limitations•• Information theory often providesInformation theory often provides

performance boundsperformance bounds


7

How to Measure InformationHow to Measure Information

Information for what?Information for what?Information relative to what?Information relative to what?

IllIll--defined questions defined questions clearly defined problemclearly defined problem

Can we measure the information in an image?Can we measure the information in an image?Does one sensor provide more information than another?Does one sensor provide more information than another?Does resolution measure information?Does resolution measure information?Do more pixels in an image give more information?Do more pixels in an image give more information?

QUESTIONS:QUESTIONS:

ANSWERANSWER::

MAYBEMAYBE


8

Measuring InformationMeasuring InformationInformation for what?Information for what?•• DetectionDetection•• EstimationEstimation•• ClassificationClassification•• Image formationImage formationInformation relative to what?Information relative to what?•• Noise onlyNoise only•• Clutter plus noiseClutter plus noise•• Natural SceneryNatural Scenery


9

Information Theory and ImagingInformation Theory and Imaging

•• Center for Imaging Science established 1995Center for Imaging Science established 1995•• Brown MURI on Performance Metrics established 1997Brown MURI on Performance Metrics established 1997•• Invited Paper 1998Invited Paper 1998

J. A. OJ. A. O’’Sullivan, R. E. Sullivan, R. E. BlahutBlahut, and D. L. Snyder,, and D. L. Snyder,““InformationInformation--Theoretic Image Formation,Theoretic Image Formation,”” IEEEIEEETransactions on Information TheoryTransactions on Information Theory, Oct. 1998., Oct. 1998.

•• IEEE 1998 Information Theory Workshop on Detection,IEEE 1998 Information Theory Workshop on Detection,Estimation, Classification, and ImagingEstimation, Classification, and Imaging

•• IEEE Transactions on Information TheoryIEEE Transactions on Information Theory SpecialSpecialIssue on InformationIssue on Information--Theoretic Imaging, Aug. 2000Theoretic Imaging, Aug. 2000

•• Complexity Regularization, Large Deviations, Complexity Regularization, Large Deviations, ……


10

IEEE Transactions on Information TheoryIEEE Transactions on Information Theory, October 1998, , October 1998, Special Issue to commemorate the 50th anniversary of Special Issue to commemorate the 50th anniversary of

Claude E. Shannon's Claude E. Shannon's A Mathematical Theory of CommunicationA Mathematical Theory of Communication

Problem DefinitionProblem DefinitionOptimality CriterionOptimality Criterion

Algorithm DevelopmentAlgorithm DevelopmentPerformance QuantificationPerformance Quantification


11

HyperspectralHyperspectral ImagingImagingat Washington Universityat Washington University

Donald L. SnyderDonald L. SnyderWilliam H. SmithWilliam H. SmithDaniel R. Daniel R. FuhrmannFuhrmannJoseph A. OJoseph A. O’’SullivanSullivanChrysanthe PrezaChrysanthe Preza

WU TeamWU Team

SnyderSnyder SmithSmith FuhrmannFuhrmann OO’’SullivanSullivanDigital Array Scanning Digital Array Scanning Interferometer Interferometer (DASI)(DASI)


12

Hyperspectral Hyperspectral ImagingImaging• Scene Cube Data Cube• “Drink from a fire hose”• Filter wheel, interferometer,

tunable FPAs• Modeling and processing:

- data models- optimal algorithms- efficient algorithms


13

HyperspectralHyperspectral Imaging Likelihood ModelsImaging Likelihood Models

ideal data : r y( ) = µ y( )µ y( ) = h y − x : λ( )∫∫ s x : λ( )dxdλ

h y − x : λ( )= ha y − x − ∆ : λ( )+ ha y − x + ∆ : λ( )2

∆ shear vector for Wollaston prism h y − x : λ( ) wavelength dependent amplitude PSF of DASI s x : λ( ) scene intensity for incoherent radiation at x, λnonideal (more realistic) data : r y( ) = Poisson µ y( )+ µ 0 y( )[ ]+ Gaussian y( )data likelihood :

E r | scene( ) = −log 1n y( )!

µ y( )+ µ 0 y( )[ ]n y( ) e− µ y( )+ µ0 y( )[ ]e− r y( )− n y( )[ ]2 2σ 2

n y( )=1

∞

∑y =1

Y

∑⎧ ⎨ ⎩

⎫ ⎬ ⎭


14

Idealized Data ModelIdealized Data Model•• Data spectrum for each pixel Data spectrum for each pixel ssjj

•• Linear combination of constituent spectraLinear combination of constituent spectra

•• Problem: Estimate constituents and proportions Problem: Estimate constituents and proportions subject to subject to nonnegativitynonnegativity; ; positivity positivity of of SS assumedassumed

•• Ambiguity if Ambiguity if α > 0, φα > 0, φ11 −− α φα φ22 > 0 , > 0 , −− α φα φ11 + φ+ φ22 > 0 > 0

•• Comments: Radiometric Calibration; Comments: Radiometric Calibration; Constraints FundamentalConstraints Fundamental

∑=

=K

kkjkj as

1φ AS Φ=


15

Idealized Problem Statement:Idealized Problem Statement:MaximumMaximum--Likelihood Likelihood Minimum IMinimum I--divergencedivergence

•• Poisson distributed data Poisson distributed data loglikelihood loglikelihood functionfunction

•• Maximization over Maximization over ΦΦ and A equivalent to and A equivalent to minimization of Iminimization of I--divergencedivergence

AS Φ=

∑∑ ∑∑= = == ⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

−⎥⎥⎦

⎤

⎢⎢⎣

⎡=Φ

I

i

J

j

K

kkjik

K

kkjikij aasASl

1 1 11ln)|( φφ

∑∑ ∑∑= =

== ⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

+−⎥⎥

⎦

⎤

⎢⎢

⎣

⎡=Φ

I

i

J

j

Kk kjikijK

k kjik

ijij as

a

ssASI

1 11

1

ln)||( φφ

•• Information Value Decomposition ProblemInformation Value Decomposition Problem


16

Markov ApproximationsMarkov Approximations•• XX and and YY RVRV’’s on finite sets, s on finite sets, p(x,y) p(x,y) unknownunknown•• Data: Data: NN i.i.d. pairs {i.i.d. pairs {XXii,Y,Yii}}•• Unconstrained ML Estimate of Unconstrained ML Estimate of p(x,y)p(x,y)

•• Lower rank Markov approximation Lower rank Markov approximation X X M M YYMM in a set of cardinality in a set of cardinality KK

•• Factor analysis, contingency tables, economicsFactor analysis, contingency tables, economics•• Problem: Approximation of one matrix by another of Problem: Approximation of one matrix by another of

lower ranklower rank•• C. C. Eckart Eckart and G. Young, and G. Young, PsychometrikaPsychometrika, vol. 1, pp. 211, vol. 1, pp. 211--

218 1936.218 1936.•• SVD SVD IVDIVD

NyxnS ),(

=

AS Φ=ˆ

)||(min ASIA ΦΦ


17

CT Imaging in Presence of High CT Imaging in Presence of High Density AttenuatorsDensity Attenuators

BrachytherapyBrachytherapy applicators applicators AfterAfter--loading loading colpostatscolpostats

for radiation oncologyfor radiation oncology

Cervical cancer: 50% survival rateCervical cancer: 50% survival rateDose prediction importantDose prediction important

ObjectObject--Constrained Computed Constrained Computed TomographyTomography (OCCT)(OCCT)


18

Filtered Back ProjectionFiltered Back Projection

Truth FBP

FBP: inverse Radon transform


19

Transmission Transmission TomographyTomography•• SourceSource--detector pairs indexed by detector pairs indexed by yy; pixels indexed by ; pixels indexed by xx•• Data Data d(y) d(y) Poisson, means Poisson, means g(y:g(y:µµ), ), loglikelihood loglikelihood functionfunction

•• Mean Mean unattenuatedunattenuated counts counts II00, mean background , mean background ββ•• Attenuation function Attenuation function µµ(x,E)(x,E), , EE indexes energiesindexes energies

•• Maximize over Maximize over µµ or or ccii; ; equivalently minimize Iequivalently minimize I--divergencedivergence•• Comment: pose search Comment: pose search c(x) = cc(x) = caa(x:(x:θθ) + ) + ccbb(x)(x)

l (d : ¹ ) =X

y2 Y[d(y) ln g(y : ¹ ) ¡ g(y : ¹ )]

¹ (x; E ) =P I

i = 1 ¹ i (E )ci (x)

g(y : ¹ ) =P

E I 0(y; E) exp [¡P

x h(yjx)¹ (x; E )] + ¯(y)


20

Alternating Minimization AlgorithmsAlternating Minimization Algorithms

•• Define problem as Define problem as minminqq φφ(q)(q)•• Derive Derive Variational Variational Representation: Representation: φφ(q) = (q) = minminpp J(p,q)J(p,q)•• JJ is an auxiliary function is an auxiliary function pp is in auxiliary set is in auxiliary set PP•• Result: double minimization Result: double minimization minminqq minminpp J(p,q)J(p,q)•• Alternating minimization algorithmAlternating minimization algorithm

),(

),(

)1()1(

)()1(

minarg

minarg

qpJq

qpJp

l

Qq

l

l

Pp

l

+

∈

+

∈

+

=

=

Comments: Guaranteed Monotonicity; J selected carefully


21

Alternating Minimization Algorithms:Alternating Minimization Algorithms:II--Divergence, Linear, Exponential FamiliesDivergence, Linear, Exponential Families

•• Special Case of Interest: Special Case of Interest: JJ is Iis I--divergencedivergence•• Families of Interest:Families of Interest:

Linear Family Linear Family L(A,b) = {p: L(A,b) = {p: Ap Ap = b}= b}Exponential Family Exponential Family E(E(ππ,B) = {q: ,B) = {q: qqii = = ππii exp[exp[ΣΣjj bbijij ννjj]}]}

)||(

)||(

)1()1(

)()1(

minarg

minarg

qpIq

qpIp

l

Eq

l

l

Lp

l

+

∈

+

∈

+

=

=

CsiszCsiszáárr and and TusnTusnáádydy; ; DempsterDempster, Laird, Rubin; , Laird, Rubin; BlahutBlahut; ; Richardson; Lucy; Richardson; Lucy; VardiVardi, , SheppShepp, and Kaufman; Cover;, and Kaufman; Cover;Miller and Snyder; O'SullivanMiller and Snyder; O'Sullivan


22

Alternating Minimization ExampleAlternating Minimization Example•• Linear family: pLinear family: p11 + 2 p+ 2 p22 = 2= 2•• Exponential family: qExponential family: q11 = exp (= exp (vv), q), q22 = exp (= exp (--vv))

0 1 2 3 4 50

1

2

3

4

5

p1, q1

p 2, q2

0 1 2 3 4 50

1

2

3

4

5

p1, q1

p 2, q2

0.6 0.8 1 1.2 1.40.2

0.4

0.6

0.8

1

1.2

p1, q1

p 2, q2

0.6 0.8 1 1.2 1.40.2

0.4

0.6

0.8

1

1.2

p1, q1

p 2, q2

)||(minmin qpILpEq ∈∈


23

Information GeometryInformation Geometry

•• II--divergence is nonnegative, convex in pair divergence is nonnegative, convex in pair (p,q)(p,q)•• Generalization of relative entropy, Generalization of relative entropy,

example of fexample of f--divergencedivergence•• First triangle equality: First triangle equality: pp in in LL

•• Second triangle equality: Second triangle equality: qq in in EE

)||()||()||()||( *** minarg qpIppIqpIqpIpLp

+=⇒=∈

)||()||()||()||( *** minarg qqIqpIqpIqpIqEq

+=⇒=∈


24

Variational Variational RepresentationsRepresentations•• Convex decomposition lemma. Let Convex decomposition lemma. Let ff be convex. Thenbe convex. Then

•• Special Case: Special Case: ff is is lnln

•• Basis for EM; see also De Basis for EM; see also De PierroPierro, Lange, , Lange, FesslerFessler

∑

∑ ∑≥=

≤

iii

i ii

irii

rr

xfrxf

0,1

)()( 1

⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

==

−=⎟⎟⎠

⎞⎜⎜⎝

⎛

∑

∑∑∈

ii

i i

ii

Ppii

ppP

qppq

1:

lnminln


25

ShrinkShrink--Wrap Algorithm for Wrap Algorithm for EndmembersEndmembers

FuhrmannFuhrmann, WU, WU

S S = = ΦΦAAΦΦ = = EndmembersEndmembers, K Columns, K ColumnsAA = Pixel Mixture Proportions= Pixel Mixture ProportionsSVD, Then Simplex Volume MinimizationSVD, Then Simplex Volume Minimization

SS==ΦΦAAGiven Given ΦΦ and and SS, estimate , estimate AA..Uses IUses I--Divergence Discrepancy Divergence Discrepancy

Measure.Measure.

Alternating Minimization AlgorithmsAlternating Minimization Algorithmsforfor HyperspectralHyperspectral ImagingImaging

αα11

αα22

αα33

++ PoissonPoissonProcessProcess

CC

AlternatingAlternatingMinimizationMinimization

AlgorithmAlgorithm

InitialInitialEstimatedEstimated

CoefficientsCoefficients

EstimatedEstimatedCoefficientsCoefficients

Snyder, WUSnyder, WU OO’’Sullivan, WUSullivan, WU


27

Information Value DecompositionInformation Value DecompositionApplied to Applied to HyperspectralHyperspectral DataData

• Downloaded spectra from USGS website• 470 Spectral components• Randomly generated A with 2000 columns• Ran IVD on result


28

Information Value DecompositionInformation Value DecompositionApplied to Applied to Hyperspectral Hyperspectral DataData


29

Information Theoretic Imaging:Information Theoretic Imaging:Application toApplication to HyperspectralHyperspectral ImagingImaging

Problem DefinitionProblem DefinitionOptimality CriterionOptimality Criterion

Algorithm DevelopmentAlgorithm DevelopmentPerformance QuantificationPerformance Quantification

•• Likelihood Models for SensorsLikelihood Models for Sensors•• Likelihood Models for ScenesLikelihood Models for Scenes•• Spectrum Estimation andSpectrum Estimation and

DecompositionDecomposition•• Performance QuantificationPerformance Quantification•• Applications to Available SensorsApplications to Available Sensors


30

HyperspectralHyperspectral ImagingImagingOngoing EffortsOngoing Efforts

•• Scene Models Including Spatial and Scene Models Including Spatial and Wavelength CharacteristicsWavelength Characteristics

•• Sensor Models Including Sensor Models Including ApodizationApodization•• Orientation Dependence of Orientation Dependence of HyperspectralHyperspectral

SignaturesSignatures•• Expanded Complementary EffortsExpanded Complementary Efforts•• Atmospheric Propagation ModelsAtmospheric Propagation Models•• Performance BoundsPerformance Bounds•• Measures of Added InformationMeasures of Added Information

imaging system design and analysisjao/talks/othertalks/darpatalk2.pdf · chrysanthe preza david g....

Documents