Automated Tracking for Alignment of Electron Microscope Images

Fernando Amat1, Farshid Moussavi1, Luis R. Comolli3, Gal Elidan2, Kenneth H. Downing3, Mark Horowitz1

1-Department of Electrical Engineering, Stanford University2-Department of Computer Science, Stanford University3-Life Sciences Division, Lawrence Berkeley National Laboratories

SIAM Annual Meeting 07/11/2008

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Motivation

One image is worth a thousand words…

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Outline

Correspondence between two images Global correspondence Results Future work Conclusions

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Previous work

Pre-align images

Seed model

Build projection model sequentially

Find new candidates

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Previous work

Advantages Runs very fast Works for high SNRDisadvantages Depends on projection model Fails at low SNR Requires manual intervention to fix

errors

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Outline

Correspondence between two images Global correspondence Results Future work Conclusions

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Correspondence between two images Key building block

Image A Image BImages by Luis R. Comolli

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Correspondence between two images Find point candidates in each image

Image A Image BImages by Luis R. Comolli

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Correspondence between two images Find correspondence between points

Image A Image BImages by Luis R. Comolli

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

a1

a2

a3

aN

Correspondence between two images

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

a1

a2

a3

aN

bk

Correspondence between two images

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Probabilistic framework

Random variables a1,…, aN

Possible assignments ai={b1,…, bK}

Find joint distribution p(a1,…, aN) Exponential complexity KN is intractable

Exploit conditional independence

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Graphical models

a2

aN

a1

a3

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Graphical models

a1

a2

a3

aN

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Graphical models

pa1 , . .. ,an∝∏i=1

n

φ i ai ∏i , j∈E

φ ij ai ,a j

φ i : singleton factorsφ ij : pairwise factors

Solve combinatorial optimization problem

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Graphical models

Complexity reduces to O(K2) Spatial proximity determines the graph Image similarity determines singleton factors

Geometric similarity determines pairwise factors

φ iai=br =NCC patchai , patchbr

φ ijai=br ,a j=bs =exp −∥vec ai a j −vec br bs∥22

σ2

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing

Build table with initial beliefs

Calculate marginal distribution efficiently00.20.70.1bK

0.40…

0.100.3b2

0.20.80b1

bK…b2b1aiaj

pak ∝ ∑i , j≠k

∏i=1

n

φ i ai ∏i , j∈E

φij ai ,a j

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing: inference

Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph

Markov Chain example

A B C D

P(A)= ∑P(A|B)P(B|C)P(C|D)P(D)

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing: inference

Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph

Markov Chain example

A B C D

P(A)= ∑P(A|B) ∑ P(B|C) ∑ P(C|D)P(D)

1

1

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing: inference

Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph

Markov Chain example

A B C D

P(A)= ∑P(A|B) ∑ P(B|C) Ψ(C)

1

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing: inference

Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph

Markov Chain example

A B C D

P(A)= ∑P(A|B) ∑ P(B|C) Ψ(C)

12

2

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing: inference

Basic Idea Calculate local marginals (beliefs) from “root” Propagate up through graph

Markov Chain example

A B C D

P(A)= ∑P(A|B) Ψ(B)

12

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Outline

Previous work Correspondence between two images Global correspondence Results Future work Conclusions

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Global correspondence

Combine pairwise correspondence

Robust matrix decomposition to find outliers1

Image 1 Image 2 Image 3 Image T…

[1] Q. Ke and T. Kanade. Robust L1 Norm Factorization in the Presence of Outliers and Missing Data by Alternative Convex Programming. CVPR, 2005.

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Summary

Local/global trade-off approach

Based on small angle change between images

Independent of projection model

Robust against errors

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Outline

Correspondence between two images Global correspondence Results Future work Conclusions

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Results: tracked trajectories

Amat, F. et al., Markov random field based automatic image alignment for electron tomography, J. Struct. Biol. (2007)

Before alignment After alignment

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Results: 3-D resolutionRAPTOR

Manual

Amat, F. et al. Markov random field based automatic image alignment for electron tomography. J. Struct. Biol. (2007)

Cardone, G. et al. A resolution criteria for electron tomography based on cross­validation. J. Struct. Biol. 2005.

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Outline

Correspondence between two images Global correspondence Results Future work Conclusions

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Future Work

Track multiple axis tomography

Track images with hundreds of markers

Improve local to global correspondence

Parallelization

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Outline

Correspondence between two images Global correspondence Results Future work Conclusions

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Conclusions Graphical models: local/global compromise

Probabilistic framework robust against noise

Algorithms help high throughput data analysis

Software available at http://www-

vlsi.stanford.edu/TEM/software.htm

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

THANKS!

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing

Efficient algorithm to calculate marginals (ai, aj) share an edge in the graph

¿¿

ψ ij ai ,N i j = ∏k∈N i j

φ ik ai ,ak δ ak ai

¿

¿

δaia j =φi ai ∑N i j

ψ ij ai ,N i j

Automated Tracking for Alignment of Electron Microscope Images Fernando Amat et alt.

Message passing

Ground Truth