Image Deformation Using Moving Least Squares

Scott Schaefer, Travis McPhail, Joe Warren

SIGGRAPH 2006

Presented by Nirup Reddy

Input Affine Similarity Rigid

Outline

Introduction Previous work MLS with points MLS with line segments Conclusions

Introduction

Image deformation Animation, morphing, medical imaging… Controlled by handles: points, lines… Function f

Interpolation: f(pi)=qi (handles)

Smoothness ~ smooth deformations Identity: qi=pif(x)=x

Similar to scattered data interpolation

Previous Work Grid/Polygon handles

Free-form deformations [Sederberg and Parry 1986; Lee et al. 1995]

Require aligning grid lines with image features Line handles

[Beier and Neely 1992] Undesirable folding

Point handles RBF

Thin-plate splines [Bookstein 1989] Local non-uniform scaling and shearing

As-rigid-as-possible deformations [Igarashi et al. 2005]

Solve on the whole triangulation Discontinuities

Previous Work

As-Rigid-As-Possible Shape Manipulation

Thin-plate spline As-rigid-as-possible

Previous Work Grid/Polygon handles

Free-form deformations [Sederberg and Parry 1986; Lee et al. 1995]

Require aligning grid lines with image features Line handles

[Beier and Neely 1992] Undesirable folding

Point handles RBF

Thin-plate splines [Bookstein 1989] Local non-uniform scaling and shearing

As-rigid-as-possible deformations [Igarashi et al. 2005]

Solve on the whole triangulation Discontinuities

i

ii ppF 2|ˆ)(|

i

iii ppFw 2|ˆ)(|

Moving Least Squares Deformation Build image deformations based on handles

(pi,qi) Interpolation Smoothness Identity

Given a point v, solve for the best lv(x):

Deformation function f(v)=lv(v) Locally defined --- MOVING Least Squares Satisfy the three properties

i

iivi qplw 2|)(| 2||

1

vpw

ii

Affine Transform Affine transform: lv(x)=xM+T Translation can be removed: T=q*-p*M

So, lv(x)=(x-p*)M+q*

The new cost function:

M could be different class of transformations

i i

i ii

w

pwp*

i i

i ii

w

qwq*

2|ˆˆ|

iiii qMpw

*ˆ ppp ii *ˆ qqq ii

Affine Deformations Solution

The deformation function

Aj can be precomputed

Contains non-uniform scaling and shear

j

jTjj

iii

Ti qpwpwpM ˆˆ)ˆˆ( 1

*ˆ)( qqAvfj

jja Ti

iii

Tij ppwppvA ˆ)ˆˆ)(( 1

*

Similarity Deformations

Only include translation, rotation, and uniform scaling

Remove shear from deformation

Similarity Deformations

Only include translation, rotation, and uniform scaling

Remove shear from deformation

Similarity Deformations (Cont.) A special subset of affine transforms

Translation Rotation Constraints: Uniform-scaling

Requirement: MTM=λ2I Define , where M2=M1

┴ Cost function (Least squares problem) still

quadratic in M1

where (x,y)┴=(-y,x)

Similarity Deformations (Cont.)

Solution for matrix M

where Solution for deformation function

where Property

Preserves angles better than affine deformations

Local scaling can hurt realism

Rigid Deformations Deformations should not

include scaling and shear [Alexa 2000; Igarashi et al.

2005] Best rigid transformation

can be found from best similarity transformation Remove local uniform

scaling MTM=I

Rigid Deformations Deformations should not

include scaling and shear [Alexa 2000; Igarashi et al.

2005] Best rigid transformation

can be found from best similarity transformation Remove local uniform

scaling MTM=I

Rigid Deformations (Cont.) Solution for matrix M

Solution for deformation function

where , and Ai is as in similarity deformations.

Limited precomputation

Example: Actual Image

Deformation’s

Affine Similarity Rigid

Comparison:

Affine Similarity RigidNon-uniform

scaling and shearLocal scaling can

often lead to undesirable

Deformations But Preserves

angles

Deformation is quiterealistic

Computation time very low abt 1.5ms

Highest was computed as 3.4ms

Highest was computed 3.8ms

Some Rigid Deformation Results

Deformation with Curves

Handles are control curves instead of control points

Cost function

T still can be removed

Limitations:

May be difficult to evaluate for arbitrary functions

We restrict these functions to be line segments and derive closed-form solutions for the deformations in terms of the end-points of these segments

Affine Lines

Represent line segments as matrix products

Cost function

Minimizer

Affine Lines (Cont.)

Deformation

Similarity Lines

Cost function

Minimizer

Similarity Lines (Cont.)

Deformation

Rigid Lines

Derive from similarity lines

Deformation

Implementation

Pixel by pixel --- expensive Deformation on a downsampled

grid Fill quads using bilinear

interpolation

Timing

Grid size: 100x100

Limitation: Foldbacks

Conclusions

Moving Least Squares 2x2 linear system at each grid point

Real-time

Similarity and rigid transformations More realistic results Extension: line segments Closed-form expressions

Applications Of MLS: Very powerful tool in

computer graphics Animation Medical

Imaging Image Warping Image Morphing

Conclusions (Cont.)

Future Work Adding topological information Generalizing to 3D to deform surfaces Handles can be any curves