an entropy model for diffusion mri - freedjafari.free.fr/maxent2014/slides/23_slides.pdfmaxent 2014...

MaxEnt 2014

An entropy model fordiffusion MRI

Pierre Marechal

UNIVERSITY DE TOULOUSE

Septembre 24, 2014

– p. 1/30

Outline

• Diffusion MRI• Building a general entropy model• Partially infinite convex programming• Application to Diffusion MRI

– p. 2/30

What is dMRI ?A non-invasive imaging technique of the diffusion of

water in biological tissues, for the study of theconnectivity in the brain

– p. 3/30



ProbabilityP of particle displacements in each voxelof the volume to be imaged within a given time

interval

– p. 3/30



ProbabilityP of particle displacements in each voxelof the volume to be imaged within a given time

interval

dP (x) = p(x) dx

– p. 3/30

Overview of the processStep 1 From MRI Fourier data

zj =

∫

R3

p(x)e−2iπ〈qj ,x〉 dx, j = 1, . . . ,m

reconstruct

• eitherdiffusion tensors, under the assumptionthatP is gaussian[DTI]

• or Orientation Diffusion Functions (ODF) viaprobability distributions

ψ(s) :=

∫

R

p(rs)r2 dr, s ∈ S2

– p. 4/30

Overview of the processStep 1 From MRI Fourier data

zj =

∫

R3

p(x)e−2iπ〈qj ,x〉 dx, j = 1, . . . ,m

reconstruct

• eitherdiffusion tensors, under the assumptionthatP is gaussian[DTI]

• or Orientation Diffusion Functions (ODF) viaprobability distributions

ψ(s) :=

∫

R

p(rs)r2 dr, s ∈ S2

Step 2 Image fibers[fiber tracking]– p. 4/30

Fiber tracking

Source: Human Connectome Project

http://www.humanconnectomeproject.org/gallery/– p. 5/30

Purpose of this workD.C. ALEXANDER, Maximum entropy sphericaldeconvolution for diffusion MRI, InformationProcessing in Medical Imaging, 19:76-87, 2005

– p. 6/30


P is assumed to be supported on a sphere

– p. 6/30



Our aim is to write a flexible model:

– p. 6/30




→ probabilityP supported inR3

– p. 6/30





→ possibility to account for moment constraints

– p. 6/30





→ possibility to account for moment constraints

→ use of more general entropies

– p. 6/30

Outline


– p. 7/30

Fourier data

zj =

∫

R3

e−2iπ〈qj ,x〉 dP (x), j = 1, . . . ,m

– p. 8/30

Fourier data

zj =

∫

R3

e−2iπ〈qj ,x〉 dP (x), j = 1, . . . ,m

yj =

∫

R3

γj(x) dP (x), j = 1, . . . , 2m

with

γj(x) =

cos(2π〈q[(j+1)/2],x〉

)if j is even,

sin(2π〈q[(j+1)/2],x〉

)if j is odd

– p. 8/30

Fourier data

zj =

∫

R3

e−2iπ〈qj ,x〉 dP (x), j = 1, . . . ,m

yj =

∫

R3

γj(x) dP (x), j = 1, . . . , 2m

with

γj(x) =

cos(2π〈q[(j+1)/2],x〉

)if j is even,

sin(2π〈q[(j+1)/2],x〉

)if j is odd

y = EP [γ] with γ(x) :=(γ1(x), . . . , γ2m(x)

)⊤

– p. 8/30

Fourier data

zj =

∫

R3

e−2iπ〈qj ,x〉 dP (x), j = 1, . . . ,m

yj =

∫

R3

γj(x) dP (x), j = 1, . . . , 2m

with

γj(x) =

cos(2π〈q[(j+1)/2],x〉

)if j is even,

sin(2π〈q[(j+1)/2],x〉

)if j is odd

y = EP [γ] with γ(x) :=(γ1(x), . . . , γ2m(x)

)⊤

Normalization:1 = EP [1] =

∫

R3

dP (x)

– p. 8/30

Optional moment constraints

From a physical viewpoint, it seems reasonable toassume in addition that the random variablex is

centered or almost centered

– p. 9/30

Optional moment constraints

From a physical viewpoint, it seems reasonable toassume in addition that the random variablex is

centered or almost centered

EP [x] =

∫

R3

x dP (x) = 0

– p. 9/30

Entropy model∣∣∣∣∣

Minimize K (P‖ν)

s.t. (1,y) = EP [(1,γ)]

– p. 10/30


Minimize K (P‖ν)

s.t. (1,y) = EP [(1,γ)]

K (P‖ν) :=

∫

u(x) lnu(x) dν(x) if P ≺≺ ν

∞ otherwise

– p. 10/30


Minimize K (P‖ν)

s.t. (1,y) = EP [(1,γ)]

K (P‖ν) :=

∫

u(x) lnu(x) dν(x) if P ≺≺ ν

∞ otherwise

u :=dP

dν(Radon-Nikodym derivative)

– p. 10/30

Rewriting Fourier data

(1,y) = EP [(1,γ)]

– p. 11/30


(1,y) = EP [(1,γ)] = A◦u

– p. 11/30


(1,y) = EP [(1,γ)] = A◦u

Notation:

Au = EP [γ] =

∫

R3

γ(x)u(x) dν(x)

Iu = EP [1] =

∫

R3

u(x) dν(x)

Mu = EP [x] =

∫

R3

xu(x) dν(x)

– p. 11/30


(1,y) = EP [(1,γ)] = A◦u

Notation:

Au = EP [γ] =

∫

R3

γ(x)u(x) dν(x)

Iu = EP [1] =

∫

R3

u(x) dν(x)

Mu = EP [x] =

∫

R3

xu(x) dν(x)

A◦u = (Iu,Au)

– p. 11/30

An equivalent formulation

(P)

∣∣∣∣∣∣∣∣

Minimize Hν(u) :=

∫

h(u(x)

)dν(x)

s.t. u ∈ L1ν(R

3)

1 = Iu, y = Au

– p. 12/30


(P)

∣∣∣∣∣∣∣∣

Minimize Hν(u) :=

∫

h(u(x)

)dν(x)

s.t. u ∈ L1ν(R

3)

1 = Iu, y = Au

h(t) :=

t ln t si t > 0

0 si t = 0

∞ si t < 0

– p. 12/30


(P)

∣∣∣∣∣∣∣∣

Minimize Hν(u) :=

∫

h(u(x)

)dν(x)

s.t. u ∈ L1ν(R

3)

1 = Iu, y = Au

h(t) :=

t ln t si t > 0

0 si t = 0

∞ si t < 0

Minimizing Hν(u) corresponds to the desire tointroduceas little prior information as possible. The

reference measure may be chosen as anisotropicgaussian measure, the one we would have in an

isotropic medium with no fiber– p. 12/30

Relaxation

(P )

∣∣∣∣∣∣∣∣∣

Minimize Hν(u)

s.t. u ∈ L1ν(R

3)

1 = Iu, y = Au

– p. 13/30

Relaxation

(Pα)

∣∣∣∣∣∣∣∣∣

Minimize Hν(u)+1

2α‖y −Au‖2

s.t. u ∈ L1ν(R

3)

1 = Iu

– p. 13/30

Relaxation

(Pα)

∣∣∣∣∣∣∣∣∣

Minimize Hν(u)+1

2α‖y −Au‖2

s.t. u ∈ L1ν(R

3)

1 = Iu

(Pα)

∣∣∣∣∣∣∣∣

Minimize Hν(u) − g(Au)

s.t. u ∈ L1ν(R

3)

1 = Iu

– p. 13/30

Relaxation

(Pα)

∣∣∣∣∣∣∣∣∣

Minimize Hν(u)+1

2α‖y −Au‖2

s.t. u ∈ L1ν(R

3)

1 = Iu

(Pα)

∣∣∣∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

1 = Iu

g(η) := −1

2α‖y − η‖2

– p. 13/30

Relaxation

(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3), 1 = Iu

– p. 14/30

Relaxation

(Pα)

∣∣∣∣∣

Minimize Hν(u) − g(Au)+δ(Iu|{1})

s.t. u ∈ L1ν(R

3)

– p. 14/30

Relaxation

(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

δ(x|S) =

{0 if x ∈ S

∞ otherwise

– p. 14/30

Relaxation

(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

δ(x|S) =

{0 if x ∈ S

∞ otherwise

(Pα)

∣∣∣∣∣

Minimize Hν(u) − g◦(A◦u)

s.t. u ∈ L1ν(R

3)

– p. 14/30

Relaxation

(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

δ(x|S) =

{0 if x ∈ S

∞ otherwise

(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

g◦(η◦,η) = g(η) − δ(η◦|{1})

– p. 14/30

Outline


– p. 15/30

Duality (main issues)

(1) Write the dual problem of(Pα)

(2) Study the constraint qualification conditions

(3) Establish theprimal-dualrelationship

– p. 16/30

Fenchel dualityTheorem A

– p. 17/30

Fenchel dualityTheorem A• L,L⋆ vector spaces, paired by〈·, ·〉

– p. 17/30

Fenchel dualityTheorem A• L,L⋆ vector spaces, paired by〈·, ·〉• A : L→ R

d linear

– p. 17/30


d linear• A⋆ : Rd → L⋆ its (formal) adjoint

– p. 17/30


d linear• A⋆ : Rd → L⋆ its (formal) adjoint• H : L→ (−∞,∞] proper convex

– p. 17/30


d linear• A⋆ : Rd → L⋆ its (formal) adjoint• H : L→ (−∞,∞] proper convex• H⋆ : L⋆ → (−∞,∞] its convex conjugate

– p. 17/30


d linear• A⋆ : Rd → L⋆ its (formal) adjoint• H : L→ (−∞,∞] proper convex• H⋆ : L⋆ → (−∞,∞] its convex conjugate• g : Rd → [−∞,∞) proper concave

– p. 17/30


d linear• A⋆ : Rd → L⋆ its (formal) adjoint• H : L→ (−∞,∞] proper convex• H⋆ : L⋆ → (−∞,∞] its convex conjugate• g : Rd → [−∞,∞) proper concave• g⋆ : Rd → [−∞,∞) its concave conjugate

– p. 17/30



Assumeri(A domH) ∩ ri(dom g) 6= ∅. Then

η := infp∈L

{H(p)− g(Ap)

}= max

λ∈Rd

{g⋆(λ)−H⋆(A⋆λ)

}

– p. 17/30



Assumeri(A domH) ∩ ri(dom g) 6= ∅. Then

η := infp∈L

{H(p)−g(Ap)

}= max

λ∈Rd

{g⋆(λ) −H⋆(A⋆λ)︸︷︷︸

D(λ)

}

– p. 17/30

Primal dual relationshipTheorem B

– p. 18/30

Primal dual relationshipTheorem B With the notation and assumptions ofthe previous theorem, assume that

(⋆) ri dom g⋆ ∩ ri dom(H⋆ ◦ A⋆) 6= ∅

– p. 18/30


(⋆) ri dom g⋆ ∩ ri dom(H⋆ ◦ A⋆) 6= ∅

and that

(a) H⋆⋆ = H andg⋆⋆ = g

– p. 18/30


(⋆) ri dom g⋆ ∩ ri dom(H⋆ ◦ A⋆) 6= ∅

and that

(a) H⋆⋆ = H andg⋆⋆ = g

(b) there existλ, a dual solution, andu in ∂H⋆(A⋆λ)such thatH⋆ ◦ A⋆ hasAu as gradient atλ

– p. 18/30


(⋆) ri dom g⋆ ∩ ri dom(H⋆ ◦ A⋆) 6= ∅

and that

(a) H⋆⋆ = H andg⋆⋆ = g

(b) there existλ, a dual solution, andu in ∂H⋆(A⋆λ)such thatH⋆ ◦ A⋆ hasAu as gradient atλ

Thenu is a primal solution

– p. 18/30

Back to our entropy problem

(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

g◦(η◦,η) = g(η) − δ(η◦|{1})

– p. 19/30


(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

g◦(η◦,η) = g(η) − δ(η◦|{1})

The previous framework may be a powerful toolprovided it is possible to compute the conjugate

functionsH⋆ν and(g◦)⋆

– p. 19/30


(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

g◦(η◦,η) = g(η) − δ(η◦|{1})



(g◦)⋆(λ◦,λ) = λ◦ + g⋆(λ) = λ◦ + 〈λ,y〉 −α

2‖λ‖2

– p. 19/30


(Pα)

∣∣∣∣∣


s.t. u ∈ L1ν(R

3)

g◦(η◦,η) = g(η) − δ(η◦|{1})



(g◦)⋆(λ◦,λ) = λ◦ + g⋆(λ) = λ◦ + 〈λ,y〉 −α

2‖λ‖2

The computation ofH⋆ν is more tricky: it involves

conjugacy through the integral sign

– p. 19/30

Conjugacy through the integralPaired spaces

– p. 20/30


• (X,A , ν) complete measure space

– p. 20/30


• (X,A , ν) complete measure space• ν positive andσ-finite

– p. 20/30


• (X,A , ν) complete measure space• ν positive andσ-finite• L,Λ are 2 spaces of measurable functions

– p. 20/30


• (X,A , ν) complete measure space• ν positive andσ-finite• L,Λ are 2 spaces of measurable functions• Assume:∀f ∈ L, ∀ϕ ∈ Λ, fϕ ∈ L1(X)

– p. 20/30



(f, ϕ) 7→ 〈f, ϕ〉 :=

∫

X

f(x)ϕ(x) dν(x)

– p. 20/30



(f, ϕ) 7→ 〈f, ϕ〉 :=

∫

X

f(x)ϕ(x) dν(x)

Example The case whereL = L1ν andΛ = L∞

ν is aclassical example for which the above pairing iswell-defined

– p. 20/30

Conjugacy through the integral

Definition A spaceL of A -measurable functions issaid to bedecomposableif it contains all functions ofthe form

1Af0 + 1ACf

whereA ∈ A is such thatν(A) <∞, f0 is ameasurable function such thatf0(A) is bounded andfis any function inL

– p. 21/30


Definition A spaceL of A -measurable functions issaid to bedecomposableif it contains all functions ofthe form

1Af0 + 1ACf

whereA ∈ A is such thatν(A) <∞, f0 is ameasurable function such thatf0(A) is bounded andfis any function inL

Example TheLp-spaces are decomposable (foreveryp ∈ [1,∞])

– p. 21/30

Conjugacy through the integralTheorem 1 [Rockafellar]

– p. 22/30


• (X,A , ν) a complete measure space

– p. 22/30


• (X,A , ν) a complete measure space• ν positive andσ-finite

– p. 22/30


• (X,A , ν) a complete measure space• ν positive andσ-finite• h : R×X → (−∞,∞] measurable, withh(·, x)

l.s.c. for everyx

– p. 22/30



l.s.c. for everyx

Then, the conjugate integrandh⋆, defined byh⋆(·, x) = [h(·, x)]⋆, is a measurable integrand

– p. 22/30



l.s.c. for everyx

Then, the conjugate integrandh⋆, defined byh⋆(·, x) = [h(·, x)]⋆, is a measurable integrand

Corollary For every measurable functionϕ, thefunctionx 7→ h⋆(ϕ(x), x) is measurable and theintegral ∫

h⋆(ϕ(x), x) dν(x)

is well definedwith the convention∞−∞ = ∞– p. 22/30


Theorem 2 [Rockafellar]

– p. 23/30


Theorem 2 [Rockafellar]LetL,Λ be spaces of measurable functions, paired by

(f, ϕ) 7→ 〈f, ϕ〉 :=

∫

X

f(x)ϕ(x) dν(x)

– p. 23/30


Theorem 2 [Rockafellar]LetL,Λ be spaces of measurable functions, paired by

(f, ϕ) 7→ 〈f, ϕ〉 :=

∫

X

f(x)ϕ(x) dν(x)

With the notation and assumptions of the previoustheorem, assume thatL is decomposable, and that{f ∈ L|H(f) ∈ R} 6= ∅. ThenH⋆ is given onΛ by

H⋆(ϕ) =

∫

h⋆(ϕ(x), x) dν(x)

– p. 23/30

Primal solution∣∣∣∣∣∣


s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

– p. 24/30



s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

Theorem

– p. 24/30



s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

Theorem Assume:

• riA domHν ∩ ri dom g 6= ∅

– p. 24/30



s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

Theorem Assume:

• riA domHν ∩ ri dom g 6= ∅• g⋆⋆ = g

– p. 24/30



s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

Theorem Assume:


• h(·, x) convex l.s.c.

– p. 24/30



s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

Theorem Assume:


• h(·, x) convex l.s.c.• h⋆(·, x) ∈ C 1(R) for almost allx

– p. 24/30



s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

Theorem Assume:


• h(·, x) convex l.s.c.• h⋆(·, x) ∈ C 1(R) for almost allx• ∃λ ∈ int domD, dual optimal, such that

u(x) := (h⋆)′(A⋆λ(x), x

)∈ L1

ν(X)

– p. 24/30



s.t. u ∈ L1ν(X)

withHν(u) =

∫

h(u(x), x

)dν(x)

Theorem Assume:


• h(·, x) convex l.s.c.• h⋆(·, x) ∈ C 1(R) for almost allx• ∃λ ∈ int domD, dual optimal, such that

u(x) := (h⋆)′(A⋆λ(x), x

)∈ L1

ν(X)

Thenu is a primal solution– p. 24/30


Hν(u) =

∫

h(u(x)

)dν(x)

with

h(t) :=

t ln t si t > 0

0 si t = 0

∞ si t < 0

– p. 25/30


Hν(u) =

∫

h(u(x)

)dν(x)

with

h(t) :=

t ln t si t > 0

0 si t = 0

∞ si t < 0

h⋆(τ) = exp(τ − 1)

– p. 25/30


Hν(u) =

∫

h(u(x)

)dν(x)

with

h(t) :=

t ln t si t > 0

0 si t = 0

∞ si t < 0

h⋆(τ) = exp(τ − 1)

SinceL1ν(R

3) is decomposable,

H⋆ν (ϕ) =

∫

R3

exp(ϕ(x) − 1) dν(x), ϕ ∈ L∞ν (R3)

– p. 25/30

The dual problem

(Dα)

∣∣∣∣∣

Maximize D(λ◦,λ)

s.t. (λ◦,λ) ∈ R1+2m

– p. 26/30

The dual problem

(Dα)

∣∣∣∣∣


s.t. (λ◦,λ) ∈ R1+2m

D(λ◦,λ) = (g◦)⋆(λ◦,λ) − (H⋆ν ◦A

⋆◦)(λ◦,λ)

– p. 26/30

The dual problem

(Dα)

∣∣∣∣∣


s.t. (λ◦,λ) ∈ R1+2m

D(λ◦,λ) = (g◦)⋆(λ◦,λ) − (H⋆ν ◦A

⋆◦)(λ◦,λ)

= λ◦ + 〈λ,y〉 −α

2‖λ‖2

−

∫

exp(A

⋆◦(λ◦,λ)(x)

︸︷︷︸

λ◦+〈λ,γ(x)〉

− 1)dν(x)

– p. 26/30

The dual problem

(Dα)

∣∣∣∣∣


s.t. (λ◦,λ) ∈ R1+2m

D(λ◦,λ) = (g◦)⋆(λ◦,λ) − (H⋆ν ◦A

⋆◦)(λ◦,λ)

= λ◦ + 〈λ,y〉 −α

2‖λ‖2

−

∫

exp(A

⋆◦(λ◦,λ)(x)

︸︷︷︸

λ◦+〈λ,γ(x)〉

− 1)dν(x)

= λ◦ + 〈λ,y〉 −α

2‖λ‖2

−exp(λ◦ − 1)

∫

exp〈λ,γ(x)〉 dν(x)

– p. 26/30

Optimality system

• D is concave, with effective domainR1+2m

– p. 27/30

Optimality system


• D ∈ C 1R

1+2m

– p. 27/30

Optimality system


• D ∈ C 1R

1+2m

Dual optimality reads

0 = 1 − exp(λ◦ − 1)

∫

R3


0 = y − αλ − exp(λ◦ − 1)

∫

R3

γ(x) exp〈λ,γ(x)〉 dν(x)

– p. 27/30

Optimality system


• D ∈ C 1R

1+2m

Dual optimality reads

0 = 1 − exp(λ◦ − 1)

∫

R3


0 = y − αλ − exp(λ◦ − 1)

∫

R3

γ(x) exp〈λ,γ(x)〉 dν(x)

which reduces to

0 = y − αλ −

∫

R3

γ(x) exp〈λ,γ(x)〉 dν(x)∫

R3


– p. 27/30

Optimality systemObserve that

0 = y − αλ −

∫

R3


R3


– p. 28/30


0 = y − αλ −

∫

R3


R3


is also the optimality system of

(Dα)

∣∣∣∣∣∣

Maximize 〈λ,y〉 −α

2‖λ‖2 − ln

∫


s.t. λ ∈ R2m

– p. 28/30


0 = y − αλ −

∫

R3


R3


is also the optimality system of

(Dα)

∣∣∣∣∣∣

Maximize 〈λ,y〉 −α

2‖λ‖2 − ln

∫


s.t. λ ∈ R2m

Proposition The function

D(λ) := 〈λ,y〉 − α2‖λ‖2 − ln

∫exp〈λ,γ(x)〉 dν(x) is concave

and smooth (onR2m)

– p. 28/30

Algorithm• MaximzeD(λ) → λ

– p. 29/30


• Compute

exp(λ◦ − 1) =

(∫


)−1

– p. 29/30


• Compute

exp(λ◦ − 1) =

(∫


)−1

• Compute ODF from

u(x) = exp(λ◦ − 1) exp〈λ,γ(x)〉

– p. 29/30


• Compute

exp(λ◦ − 1) =

(∫


)−1

• Compute ODF from

u(x) = exp(λ◦ − 1) exp〈λ,γ(x)〉

The optimalu is searched for in a smooth manifold ofdimension2m in L1

ν(R3)

– p. 29/30

Thank you for your attention !

– p. 30/30

an entropy model for diffusion mri - freedjafari.free.fr/maxent2014/slides/23_slides.pdfmaxent 2014...

Documents