The Conditional Lucas & Kanade AlgorithmChen-Hsuan Lin, Rui Zhu, and Simon Lucey{chenhsul, rz1}@andrew.cmu.edu, slucey@cs.cmu.edu

Contributions• Establish theoretical connections: The Lucas & Kanade Algorithm (LK) Supervised Descent Method (SDM)

• The Conditional LK Algorithm: efficient aligner which achieves comparable performance with little training data

LK• Aligns a source image against a template image by

minimizing their appearance error SDM

(inverse-compositional)

1st-order Taylor approximation

solve

Similarity

• Learns the appearance-geometry linear relationship from

synthetically generated data

• Linear regressors are trained from independently

sampled data per iteration

• Prediction of the geometric displacement is learned

conditioned on the appearance

Training:

Evaluation:

• Both assume a linear relationship between appearance and geometry

• Solve for geometric updates iteratively until convergence is reached

Difference

N pixels, 2 gradient directions

2N degrees of freedom in R

LK SDM

N pixels, P warp parameters

PN degrees of freedom in R

Full dependency across pixels Pixel independence assumption

Generative appearance synthesis

2N

N P

fits by trying to synthesize

Conditional learning objective

NP P

N

predicts conditioned on

pixelintensity

imagegradients

y

x

N 2N

P

predefined

(regularization)

Conditional LK

NP

2N

N 2N

P• Learn the image gradients conditioned on the appearance

• Final regressor:

Pixel independence assumption

Conditional learning objective

2N degrees of freedom in R

• Warp swapping property:

Geometric warp functions can be swapped and combined with the

conditional image gradients to form another series of linear regressors

• Non-linear least squares problem

Experiments

classically taken via

finite differencing

ITERATIONS 1 TO 5 ITERATIONS 1 TO 5

Visualization of Convergence Analysis

Warp Swapping

Frequency of convergence Convergence rate

Applications

Low frame-rate tracking

Similar results for different warps, feature images, examples/iteration, and test perturbations

BLUE: IC-LK YELLOW: Conditional LK