Multilevel Optimization for Multi-Modal X-rayData Analysis

Zichao (Wendy) Di

Mathematics & Computer Science DivisionArgonne National Laboratory

August 8, 2017

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

2 / 37

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

3 / 37

More is Better

CT:hard-tissue; T2(MRI):soft tissue; PET: functional characteristics 1

1Boss, A. and et al. (2010). Hybrid PET/MRI of Intracranial Masses: InitialExperiences and Comparison to PET/CT, JNM

4 / 37

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based Acceleration

Summary and Outlook

5 / 37

Tomographic Image Reconstruction

I Non-invasive imaging technique to visualize internalstructures of object.

I Tomography applications: physics, chemistry, astronomy,geophysics, medicine, etc.

I Tomographic imaging modalities: X-ray transmission,ultrasound, magnetic resonance, X-ray fluorescence, etc.

I Task: estimate distribution of physical quantities in samplefrom measurements.

I Limited angle tomography reconstruction is naturallyill-conditioned.

6 / 37

Schematic Experimental Setup with Two Modalities

7 / 37

Reconstruction Approaches

I Traditional: filtered backprojectionI Restricted to simple tomographic modelI Requires a large number of projections

I Alternatively, iterative reconstruction from single datamodality

I Requires much less data acquisition, results in higheraccuracy

Our Goal:Formulate a joint inversion integrating XRF and XRT data toimprove the reconstruciton quality of elemental map.

8 / 37

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based Acceleration

Summary and Outlook

9 / 37

X-ray Transmission (XRT)

I Traditionally, the XRT projection of theobject from beam line (θ, τ) is modeled as

FTθ,τ (µE ) = I0 exp

{−∑

v

Lv µEv

}.

to directly solve the linear attenuationcoefficient µE

v for each voxel v ,

I In our approach,notice µEv =

∑e

Wv ,eµEe ,

FTθ,τ (WWW) = I0 exp

{−∑v ,e

LvµEeWv ,e

} I0: incident photon flux

µEe : mass attenuation coefficient of element

e ∈ E at beam incident energy E

WWW =Wv,e : tensor denoting how muchof element e is in voxel v

L = [Lv ]: tensor of intersection lengthof beam line (θ, τ) with the voxel v

10 / 37

X-ray Fluorescence (XRF)

11 / 37

Mathematical Model of XRF

I First, we obtain the unit fluorescencespectrum

Me = F−1(

F (Ie) ∗ F(

1σ√

2πexp

{−x2

2σ2

}))

I Then, the XRF spectrum, FRθ,τ

=∑v ,e,d

LvWv ,eMe

ndexp

−∑v ′,e′Wv ′,e′

(µE

e′Lv ′Iv ′∈Uv + µEee′ Pv ,v ′,d

)P = [Pv,v′,d ]: tensor of intersection length of fluorescence detectorlet path d with the voxel v′

12 / 37

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based Acceleration

Summary and Outlook

13 / 37

Resulting Optimization: Joint Reconstruction (JRT)Goal:FindWWW so that FR

θ,τ (WWW) = DRθ,τ and FT

θ,τ (WWW) = DTθ,τ

minWWW≥0

φ(WWW) =∑θ,τ

(12

∥∥∥FRθ,τ (WWW)− DR

θ,τ

∥∥∥2+β1

2

∥∥∥FTθ,τ (WWW)− β2DT

θ,τ

∥∥∥2)

where

I DRθ,τ ∈ RnE : measurement data of XRF signal detected at angle

θ from light beam τ

I DTθ,τ ∈ R: the measurement data of XRT signal detected at angle

θ from light beam τ

I β1, β2 > 0 are scaling parameters to balance the ability of eachmodality to fit the data, and detect the relative variability betweenthe data sources.

Note: Optimization differs on how FRθ,τ and FT

θ,τ are combined14 / 37

How Multimodality Can HelpConsider an overdetermined (more equations than thenumber of unknowns) but rank deficient system which has aninfinitude of solutions: 1 2

2 40.5 1

[x1x2

]=

12

0.5

Another such rank-deficient (not full-rank) system:[

2 34 6

] [x1x2

]=

[1.53

]However, combine these two can form a full rank andconsistent system with a unique solution x

1 22 4

0.5 12 34 6

[x1x2

]=

12

0.51.53

15 / 37

Optimization Solver

In experiments, we use truncated-Newton (TN) method withpreconditioned conjugate gradient (PCG) to provide a searchdirection:

I To satisfy the bound constraints, the projected PCG isapplied to the reduced Newton system

I One TN iteration typically requires κ(O(10)) PCGiterations

Main expense of each outer iteration is κ+ 2 function-gradientevaluations.

16 / 37

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based Acceleration

Summary and Outlook

17 / 37

JRT versus Single XRF Reconstruction (Synthetic)1

Element Unit: g/µm2

1Di, Leyffer, and Wild (2016). An Optimization-Based Approach forTomographic Inversion from Multiple Data Modalities, SIAMIS

18 / 37

Convergence

19 / 37

Addressing Self-Absorption Effect 1

Sample: Solidglassrod (Silicon)

with 2 wires(Tungsten, Gold).

1Di and et al. (2017). Joint reconstruction of x-ray fluorescence andtransmission tomography, Optics Express

20 / 37

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

21 / 37

Motivation

N = 92, flops = 2×107

N = 52, flops = 6×106

N = 32, flops = 3×106

N = 22, flops = 8×105

22 / 37

Outline

Multi-Modality Imaging

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

23 / 37

Multilevel Acceleration of Image Registration

24 / 37

Outline

Multi-Modality Imaging

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

25 / 37

Introduction of MG/OPT

I Multigrid optimization algorithm (MG/OPT) is a generalframework to accelerate a traditional optimizationalgorithm1.

I Recursively use coarse problems to generate searchdirections for fine problems.

I MG/OPT can deal with more general problems in anoptimization perspective, in particular, it is able to handleinequality constraints in a natural way.

I Multiple options to design the hierarchy of the problem:I through image space.I through data space.

1Nash, S. G. (2000). A Multigrid Approach to Discretized OptimizationProblems, OMS

26 / 37

MG/OPTGiven:

I High-resolution model fh(zh), easier-to-solve low-resolution modelfH(zH)

I z+ ← OPT(f (z), z, k)

I A restriction operator IHh and an interpolation operator Ih

H

I An initial estimate z0h of the solution z∗h on the fine level

I Integers k1 and k2 satisfying k1 + k2 > 0

27 / 37

Outline

Multi-Modality Imaging

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

28 / 37

Interpolation/Restriction Operators

I Restriction on 2D parameterWWWh ∈ R2 to produce WWWHusing full weighted matrix:

I Restriction on gradient IHh = C IH

h where C balances theorder difference between φh(WWWh) and φH(IH

h WWWh)

I Interpolation operator: IhH = 4(IH

h )T .

29 / 37

Coarse Grid Surrogate Model

I Shift experimental data for coarse level:

DDDθ,τ = Dθ,τ − (F,hθ,τ (WWWh)− F,Hθ,τ (IHh WWWh))

The surrogate model:

φH(WWWH) =∑θ,τ

(12

∥∥∥F,Hθ,τ (WWWH)− DDDθ,τ

∥∥∥2)

−(∇φH(IH

h WWWh)− IHh ∇φh(WWWh)

)TWWWH

30 / 37

Outline

Multi-Modality Imaging

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

31 / 37

MG/OPT Search Directions versus Error

Problem Size: [33 17]

32 / 37

2-level MG/OPT Reconstruction

33 / 37

Outline

Multi-Modality ImagingExample: X-ray TomographyForward ModelOptimization AlgorithmsNumerical Results

Multilevel-Based AccelerationOther Applications of Multilevel MethodsIntroduction of MG/OPTApplication of Multilevel in Tomographic ReconstructionNumerical Results

Summary and Outlook

34 / 37

Summary

I Established a link between X-ray transmission and X-rayfluorescence datasets by reformulating their correspondingphysical models.

I Developed a simultaneous optimization approach for thejoint inversion, and achieved a dramatic improvement ofreconstruction quality with no extra computational cost.

I Proposed a multigrid-based optimization framework tofurther reduce the computational cost of the reconstructionproblem.

I Preliminary results show that coarsening in voxel spaceimproves accuracy/speed.

35 / 37

Making More of More

I Extend our multimodal analysis tool to data from differentinstruments, involving varying spatial resolution andcontrast mechanisms.

I Guided by the hierarchical nature of our multilevelalgorithm, we will investigate new data acquisitionstrategies and allow for flexible and adaptive samplingapproaches.

I Enable true real-time feedback.

36 / 37

Acknowledgement

This work was funded under the project ”The Tao of Fusion:Pathways for Big-data Analysis of Energy Materials at Work”and ”Base optimization (DOE-ASCR DE-AC02-06CH11357)”.

Thanks toSven Leyffer, Stefan Wild, andcollaborators at APS:Francesco De Carlo, Si Chen, Doga Gursoy, Young Pyo Hong,Chris Jacobsen, Stefan Vogt

37 / 37