example-driven deformations based on discrete shells

DOI: 10.1111/j.1467-8659.2011.01974.x COMPUTER GRAPHICS forumVolume 30 (2011), number 8 pp. 2246–2257

Example-Driven Deformations Based on Discrete Shells

Stefan Frohlich and Mario Botsch

Computer Graphics Group, Bielefeld University, Germany{sfroehli,botsch}@techfak.uni-bielefeld.de

AbstractDespite the huge progress made in interactive physics-based mesh deformation, manipulating a geometricallycomplex mesh or posing a detailed character is still a tedious and time-consuming task. Example-driven methodssignificantly simplify the modelling process by incorporating structural or anatomical knowledge learned fromexample poses. However, these approaches yield counter-intuitive, non-physical results as soon as the shape spacespanned by the example poses is left. In this paper, we propose a modelling framework that is both example-drivenand physics-based and thereby overcomes the limitations of both approaches. Based on an extension of the discreteshell energy we derive mesh deformation and mesh interpolation techniques that can be seamlessly combined intoa simple and flexible mesh-based inverse kinematics system.

Keywords: mesh deformation, physics-based deformation, discrete shells, inverse kinematics

ACM CCS: I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling—Physically basedmodeling.

1. Introduction

Mesh deformation is a fundamental and challenging topicin digital geometry processing, with various applications inas diverse fields as shape design, engineering, and computeranimation. As a consequence, a huge number of differentmodelling approaches have been proposed over the years,most of which can be classified as physics-based or physics-inspired. They employ elastic energies that approximate (orat least qualitatively mimic) the behaviour of elastic objectsknown from continuum mechanics. Despite the impressiveprogress made in this field, manipulating geometrically com-plex shapes, for instance a detailed character model, can stillbe a time-consuming and tedious process even with state-of-the-art modelling tools. This is mainly because no knowledgeabout the semantic or anatomical structure of the model isincorporated into the physics-based deformation energy.

In contrast, example-driven deformations, as pioneered inthe mesh-based inverse kinematics (MeshIK) system of Sum-ner et al. [SZGP05], achieve natural shape manipulations bya suitable blending of carefully designed example poses. Al-though the input poses now provide the previously missing

knowledge about the model’s deformation behaviour, exist-ing example-driven methods are not physics-based. When themodelling constraints cannot be represented within the shapespace spanned by the example poses (the example space), thedeformation might yield unnatural results.

In this paper, we propose a novel mesh deformation ap-proach that combines the strengths of example-driven andphysics-based techniques and thereby avoids their respectivelimitations. Our method incorporates given example poses,but gracefully falls back to physics-inspired deformationswhen leaving the example space.

In a general example-driven deformation framework, thedesigner imposes modeling constraints and the system com-putes the shape closest to the example space meeting theseconstraints. Therefore, such a framework conceptually con-sists of two main components:

• An interpolation operator that blends between the giveninput poses to find the mesh that best matches the de-signer’s constraints within the example space.

c© 2011 The AuthorsComputer Graphics Forum c© 2011 The EurographicsAssociation and Blackwell Publishing Ltd. Published byBlackwell Publishing, 9600 Garsington Road, Oxford OX42DQ, UK and 350 Main Street, Malden, MA 02148, USA. 2246

S. Frohlich and M. Botsch / Example-Driven Deformations Based on Discrete Shells 2247

• A deformation operator that deforms the interpolatedshape to exactly satisfy the modelling constraints,thereby potentially deviating from the example space.

For instance, the interpolation and deformation opera-tors of the original MeshIK [SZGP05] are both based ondeformation gradients and linear Poisson systems. Lineardeformation gradients, however, were recently shown to yieldartefacts in the context of mesh deformation [BS08] and meshinterpolation [WDAH10], which consequently also lead toproblems for the MeshIK system they are employed for.

Inspired by MeshIK [SZGP05], our goal is to overcome thelimitations inherent to deformation gradients. To this end, webase our modeling system on an extension of the non-lineardiscrete shell energy [GHDS03], which measures stretchingand bending as deviations of edge lengths and dihedral an-gles, respectively. This allows us to develop deformation andinterpolation operators that are free of the above-mentionedlinearization artefacts.

Our deformation and interpolation techniques on its ownare on a par with (but not necessarily superior to) state-of-the-art methods for these problems. The major strength ofour approach is the seamless integration of these two com-ponents into a simple and elegant formulation for example-based and physics-based deformations. We incorporate thecomputation of the optimal interpolation weights, the inter-polation itself, and the constrained mesh deformation intoa simple non-linear energy, which can easily be minimizedthrough a straightforward Gauss–Newton method. We fur-ther present a multiresolution optimization that enables theediting of complex models at interactive rates. Our proposedmodelling framework overcomes several limitations of cur-rent example-based approaches. It can interpolate large rota-tions between example poses, is fully rotation invariant, andfalls back to physically plausible deformations when leavingthe example space.

2. Related Work

In this section, we discuss existing example-driven defor-mation approaches as well as mesh deformation and meshinterpolation methods-because these two techniques are in-tegral components of our modelling framework.

2.1. Mesh deformation

For the following discussion we focus on surface-based de-formation techniques—in contrast to space deformations-toenable the combination with a mesh interpolation operator.Following [BS08], we further classify these approaches into(i) shell-based techniques, which approximate the elastic en-ergy of thin shells [TPBF87], and (ii) methods based ondifferential coordinates, which manipulate the shape throughdeformation gradients, Laplacians or local frames [Sor06].

Most of the earlier techniques simplify the underlying en-ergies to yield a linear variational optimization. The inherentdrawbacks of this linearization have been analysed in [BS08].Shell-based methods typically have problems with large ro-tational deformations (e.g. [KCVS98, BK04]), and methodsbased on differential coordinates or local frames often aretranslation-insensitive (e.g. [YZX*04, ZRKS05, LSLC05,KG08]), which basically requires the user to prescribe posi-tion and orientation of handle points and thereby prevents asimple click-and-drag interaction.

Recent approaches employ non-linear elastic energiesto overcome these limitations, again being shell-based[BPGK06, BMWG07] or based on differential coordinates orframes [SK04, HSL*06, SZT*07, AFTCO07, SA07]. Com-pared to linear methods, the non-linear optimization is con-siderably more involved. Non-linear approaches thereforetypically perform the optimization in a hierarchical, adap-tive, or subspace manner.

In this paper we adapt the discrete shell energy [GHDS03]for our mesh deformation operator. Thanks to this non-linear,physics-based energy our deformations are free of lineariza-tion artefacts, and we can control the local surface stiffness(similar to [PJS06, HZS*06, KG08]) as well as the relationof stretching and bending resistance. Note that while PriMo[BPGK06] performs equally well for pure deformation, itcannot be easily combined with mesh interpolation.

2.2. Mesh interpolation

To avoid the artefacts from linear interpolation of vertexpositions, several approaches interpolate per-triangle affinetransformations instead, that is per-face deformation gradi-ents [ACOL00, XZWB05, SZGP05]. In this case, per-facerotations have to be interpolated in a non-linear manner, typ-ically through spherical linear interpolation. This interpola-tion of deformation gradients can fail for per-face rotationsof more than 180◦, since then the shorter spherical inter-polation path is taken, which might not be the correct one.Baran et al. [BVGP09] avoid large rotations by splitting themesh into patches and interpolating per-face rotations rel-ative to the patch rotation. An alternative is to replace theper-face rotations between example poses by relative rota-tions between neighbouring frames [LSLC05, KG08]. Thethree cited methods successfully interpolate between mesheseven in the presence of large rotations. However, they haveto solve for rotations and vertex positions in sequence us-ing two linear systems, which still is an efficient linear pro-cess, but cannot be easily used as a component of a MeshIKsystem.

Kilian et al. [KMP07] and Chao et al. [CPSS10] interpo-late models along geodesics in suitable shape spaces, but theirmethods are either too complex [KMP07] or work for solidmodels only [CPSS10]. Recently, Winkler et al. [WDAH10]proposed to represent and interpolate meshes in terms of

c© 2011 The AuthorsComputer Graphics Forum c© 2011 The Eurographics Association and Blackwell Publishing Ltd.

2248 S. Frohlich and M. Botsch / Example-Driven Deformations Based on Discrete Shells

edge lengths and dihedral angles. To find the mesh that bestmatches the interpolated edge lengths and angles they employa rather complicated hierarchical shape matching technique,which prevents their method from being directly used in aMeshIK system.

However, Winkler’s approach [WDAH10] uses exactly thesame local mesh properties (edge lengths and dihedral an-gles) for mesh interpolation as discrete shells [GHDS03] usefor mesh deformation. Therefore, these methods conceptu-ally fit nicely together, and we develop a relatively simpleand efficient energy minimization that replaces the complexhierarchical shape matching of [WDAH10].

2.3. Example-based deformation

Early approaches for example-based deformations were pro-posed in the context of skeleton-controlled articulated mod-els, first by pose-dependent corrections of vertex positionsfor linear interpolation skinning [LCF00]. Later approaches[WSLG07, WPP07] yield superior results by employing de-formation gradients for skinning, example-based correctionsand rotational regression. However, these methods require askeleton to control the mesh.

Sumner et al. [SZGP05] were the first to propose a mesh-based (in contrast to skeleton-based) inverse kinematics sys-tem. Given a rest pose and several example poses, theyextract per-face deformation gradients, which are used tointerpolate between examples. The interpolation weights aredetermined automatically from the modelling constraints bya non-linear optimization. However, the reconstruction of amesh from the interpolated deformation gradients, which atthe same time acts as deformation operator for incorporat-ing the user’s constraints, was shown in [BSPG06] to beequivalent to linear Poisson-based interpolation [XZWB05]and deformation [YZX*04]. As a consequence, MeshIK in-herits their linearization problems. It cannot handle per-facerotations larger than 180◦ and yields unnatural deformationresults when leaving the example space.

The highly efficient data-driven deformation method ofFeng et al. [FKY08] fits a set of abstract skeleton bonesto an animation sequence, from which it learns the localbone transformations using canonical correlation analysis.At run-time they derive the bone transformations from themodelling constraints and solve a Poisson system for the finalvertex positions. It is this final linear step that again yieldsunnatural results when leaving the example space.

We are mostly inspired by the original MeshIK [SZGP05],and improve it by replacing the interpolation and deformationoperators by a non-linear, rotation-invariant technique basedon the discrete shell energy. This allows us to handle arbitrarylarge rotations between example poses and to fall back tophysically plausible deformations when leaving the examplespace.

In the following we first explain our deformation and inter-polation operators (Sections 3 and 4), before combining theminto an example-driven deformation framework (Section 5).The numerical minimization of the employed non-linear en-ergy using a Gauss–Newton method is described in Section 6.For increased performance we propose a multiresolution op-timization in Section 7.

3. Mesh Deformation Operator

The elastic energy for our mesh deformation operator extendsthe discrete shell energy by a volume preservation term. Inthe following we assume a mesh M with vertex positionsxi ∈ V , edges eij ∈ E and faces fijk ∈ F .

Analogous to discrete shells [GHDS03] we measurestretching and bending terms E+s and Eb as (weighted)squared deviations of current (lower-case) edge lengths le

and dihedral angles θ e from their original (upper-case) val-ues Le and �e, respectively. To be able to extend the energylater, we denote by l∗e and θ∗

e the desired target edge lengthsand dihedral angles, which for now are just the initial valueswe want to preserve (l∗e = Le and θ∗

e = �e):

Es = 1

2

∑e∈E

(le − l∗e

)2 1

L2e

, (1)

Eb = 1

2

∑e∈E

(θe − θ∗

e

)2 L2e

Ae. (2)

Ae denotes the sum of the areas of the two triangles sharingthe edge e. Note that we omit the deviations of triangle areasfrom the original discrete shell energy, because that term didnot have a significant impact in our examples.

The stretching and bending energies model two-dimensional surfaces (thin shells). Hence, input meshes aretreated as hollow objects, even if they are intended to repre-sent the boundary surface of a solid object. In that case, i.e.for closed surfaces without boundaries, we optionally add aglobal volume preservation term (similar to [HSL*06]):

Ev = 1

2(v − v∗)2 1

V 2. (3)

Again, v, V , and v∗ denote current, initial and target volume,respectively (with v∗ = V). As in [HSL*06] we compute theglobal volume v as the sum of per-face tetrahedral volumes

v = 1

6

∑fi,j,k∈F

(xi × xj ) · xk. (4)

We note that our volume preservation is just a heuristic: Theglobal volume preservation can compensate local volumeschanges of one part of the model in a completely differentsurface region, thereby inducing strain. For a physically ac-curate solution local volumes (e.g. of a tetrahedral mesh)have to be preserved instead (see e.g. [CPSS10]).



Figure 1: Benchmark deformations from [BS08]. Translating the right border of a bumpy plane; twisting a bar by 135◦; bendinga cactus by 70◦; bending a cylinder by 120◦. The grey surface regions are fixed, the yellow regions are handle constraints, theblue regions are unconstrained and determined by minimizing the energy (5).

The total elastic energy is the sum of stretching, bendingand volume preservation terms, weighted by stiffness param-eters λ, μ and ν, respectively:

E = λEs + μEb + νEv. (5)

The weighting coefficients in (1), (2) and (3) are chosen toaccount for irregular tessellations as well as to yield scale-invariant energy terms. The stiffness parameters in (5) do nothave to be adjusted on a per model basis, but are typically setto λ = 100, μ = 1 and ν = 1000 in our examples.

Mesh editing now amounts to a minimization of the en-ergy (5), while satisfying the user’s modeling constraints(prescribed positions for a selected set of vertices). We for-mulate the energy minimization as a non-linear least-squaresproblem and employ a Gauss–Newton method, since thistechnique requires first-order partial derivatives only and stillprovides super-linear convergence (Section 6).

Figure 1 provides a qualitative evaluation of our shell-based deformation on the benchmark models of [BS08].Comparing our results to Figure 10 of [BS08] reveals that ourapproach is qualitatively equivalent to PriMo [BPGK06], astate-of-the-art non-linear approach. A quantitative compar-ison (in terms of performance) is given in Section 6.

A further advantage of our physics-based energy is theexplicit control of stretching and bending stiffness (λ andμ), which allows to vary the surface deformation behavior(Figure 2). Most approaches based on differential coordi-nates can (be extended to) locally adjust the surface stiff-ness, but cannot change the global deformation character-istic. Note that these stiffness parameters, although chosenconstant in (5), can also be controlled on a per-edge basis.In Figure 3, we derive per-edge stretching and bending val-ues from the example poses shown in Figure 9 (as describedin Appendix A). A pure deformation using these learnedstiffness values (without the MeshIK of Section 5) alreadyyields convincing results compared to uniform stiffnessparameters.

Figure 4 shows the effect of volume preservation ona stretched cylinder, where the designer can choose (or

Figure 2: Global stiffness control. Bending a cylinder (left)with either dominating bending stiffness (centre) or stretch-ing stiffness (right).

Figure 3: Local stiffness control. Deriving per-edge stiffnessfrom the example poses of Figure 9 leads to more naturaldeformations (left) than uniform stiffness weights (right).

Figure 4: Stretching a cylinder (left) to 150% of its originallength without (top) and with volume preservation (bottom).The volume error is 42% and 0.007%, respectively.



continuously blend) between a hollow or an (approximately)solid deformation behaviour.

4. Mesh Interpolation Operator

The second core component of our example-based deforma-tion framework is the interpolation operator. Given a rest-pose mesh M and a set of k compatibly tessellated exampleposes M1, . . . ,Mk , we want to explore the shape spacespanned by these meshes through a geometrically meaning-ful interpolation (and extrapolation) technique.

Following the idea of Winkler et al. [WDAH10] we inter-polate meshes in terms of edge lengths and dihedral angles.However, we derive a simpler method for computing themesh that best matches the interpolated edge lengths and an-gles, such that our interpolation operator can be combinedwith the deformation technique of the previous section.

Finding the best matching mesh (in a least squares manner)requires to minimize an energy that penalizes the squareddeviations from prescribed target edge lengths l∗e and dihedralangles θ∗

e . Having the deformation operator of the previoussection at hand, all we need to do is to replace the rest-poselengths Le and angles �e by the desired interpolated values.Similarly, we can also interpolate the global volume V if thevolume preservation is activated.

Given the rest-pose M and example poses M1, . . . ,Mk ,we denote by L(i)

e , �(i)e and V (i) the edge lengths, dihedral

angles and volume of mesh Mi , respectively. The targetvalues for (1), (2) and (3) are then determined by linearlyinterpolating the differences of Mi’s values to those of therest pose M, controlled by interpolation weights α1, . . . , αk:

l∗e = Le +k∑

i=1

αi

(L(i)

e − Le

),

θ∗e = �e +

k∑i=1

αi

(�(i)

e − �e

),

v∗ = V +k∑

i=1

αi(V(i) − V ).

(6)

The same non-linear minimization as used in Section 3 anddescribed in Section 6 then computes the interpolated mesh.

Although in general a mesh with prescribed edge lengthsand dihedral angles does not exist, our least squares ap-proximation leads to physically meaningful interpolatedshapes. In Figure 5 we show the same elephant interpola-tion/extrapolation sequence as used in Figure 1 of [KMP07]and Figure 11 of [WDAH10]. Moreover, Figure 6 basicallyreproduces Figure 6 of [WDAH10] and demonstrates thatlarge rotations and varying mesh resolution are naturally han-dled by our approach.

These results show our interpolation technique to be qual-itatively on a par with the recent state-of-the-art approaches[KMP07, WDAH10]. Because we use the same targetobjective as Winkler et al. [WDAH10], we basically repro-duce their results. However, while they compute the inter-polated mesh by a hierarchical shape matching procedure,our method requires a comparatively straightforward energyminimization only, which is the key to combining the defor-mation and interpolation operators into the example-drivendeformation framework described in the next section.

5. Example-Driven Deformation Framework

After introducing the deformation and interpolation opera-tors, we are now ready to derive our example-driven frame-work. Like in the case of mesh interpolation, the input is a setof example posesM,M1, . . . ,Mk . However, instead of justinterpolating between the examples, the system should now‘learn’ the deformation behaviour of the rest pose M fromthe examples, such that a simple click-and-drag modellingmetaphor leads to natural and meaningful results.

Instead of preserving original edge lengths and dihedralangles (as for mesh deformation), or prescribing interpolatedlengths and angles (as for mesh interpolation), we now haveto optimize for the interpolation weights αi of (6), such thatthe interpolated mesh best fits the modelling constraints.

One particular strength of our approach is that this opti-mization of the interpolation weights, the interpolation op-erator and the deformation operator can all seamlessly beintegrated into one non-linear energy formulation. We sim-ply use the interpolated target values (6) in the elastic energy(5) (as for mesh interpolation) and furthermore extend theset of unknowns to include both the free vertex positionsx1, . . . , xn and the interpolation weights α1, . . . , αk.

Because the number k of examples typically is very smallcompared to the number n of free vertices, and because thederivatives of the target objectives w.r.t. the weights αi aretrivial to compute, incorporating examples poses does notlead to a noticeable performance overhead compared to puremesh deformation or interpolation (Section 6).

If we compare the results of our example-driven deforma-tion to that of existing example-based approaches [SZGP05,FKY08], we can identify three important advantages. First,our deformation and interpolation operators employ thephysics-based discrete shell energy. As a consequence, evenwhen leaving the space of example poses our method stillyields physically meaningful results, as shown in Figure 8.This is in contrast to purely example-driven techniques,which lead to undesired shears when deviating too muchfrom the examples (compare to Figure 15 of [FKY08] andFigure 5 of [SZGP05]).

Secondly, because our interpolation method can deal withlarge rotations between example poses, our MeshIK system



Figure 5: Interpolation and extrapolation of the yellow example poses. The blending weights are 0, 0.35, 0.65, 1.0 and 1.25.

Figure 6: Interpolation of an adaptively meshed and strongly twisted helix with blending weights 0, 0.25, 0.5, 0.75 and 1.0.

[SZGP05]ours

Figure 7: Given a set of input poses (left) the user can drag handle points (yellow spheres) and the example-based deformationproduces a meaningful result by finding the optimal interpolation weights for the input examples. While our rotation-invariantmethod makes use of the bottom example pose in a rotated fashion, the original MeshIK approach leads to artefacts.

can do as well. The original MeshIK [SZGP05] is basedon linear deformation gradients and therefore might yieldartefacts for rotations of more than 180◦ (compare Figure 5.2of [Sum05] to Figure 6 and our accompanying video).

Thirdly, our method is rotation-invariant, because ex-amples are represented by (changes of) edge lengths anddihedral angles—instead of by per-face affine world-spacetransformations from the rest pose to the example poses(deformation gradients). As an immediate consequence, ourmethod can use example poses in arbitrary orientations, forexample, to simplify rotation modelling of example poses(see Figure 7 and the accompanying video). In contrast, theoriginal MeshIK cannot deal with rotated (i.e. mis-aligned)poses.

Although this particular problem can potentially be fixedby including per-example rotations in MeshIK’s optimiza-tion process, a similar problem can occur when combiningrotations of example poses (Figure 9). Thereason for the failure of the originalMeshIK is depicted on the right. In theircase, the combined deformation ‘shoulder+ elbow’ (i.e. α1 = α2 = 1) first rotates theshoulder around the global y-axis, followedby a rotation of the forearms around theglobal x-axis, leading to a forearm twist.In contrast, our rotation-invariant frame-work yields the expected combined defor-mation, as reproduced by our IK system inFigure 9.



Figure 8: The original MeshIK yields unwanted shearswhen leaving the example space (left), whereas our tech-nique still provides physically plausible results (right).

6. Numerical Minimization

We have shown how to formulate the mesh deformationand mesh interpolation operators, as well as the completeexample-driven deformation technique as a nonlinear en-ergy minimization. The simplicity of our energy (5) en-ables the analytic computation of its gradients, such thatwe can employ an efficient Gauss–Newton minimization[MNT04].

6.1. Gauss–Newton minimization

We start by formulating the energy minimization as a non-linear least-squares problem. For a system with n uncon-strained vertices x1, . . . , xn, m unconstrained edges, andk example poses, we set up a residual function f thatmaps the vector x = (x1, y1, z1, . . . , xn, yn, zn, α1, . . . , αk)T

of unknown vertex positions and interpolation weights tothe residuals of edge lengths, dihedral angles and globalvolume:

f :

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

x1

y1

z1

...

xn

yn

zn

α1

...

αk

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

�→

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

ws,1

(l1 − l∗1

)...

ws,m

(lm − l∗m

)wb,1

(θ1 − θ∗

1

)...

wb,m

(θm − θ∗

m

)wv (v − v∗)

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

. (7)

The target values l∗i , θ∗i and v∗ are again determined by in-

terpolating the respective values of the example poses usingthe weights αi as in (6). The weighting factors are chosen as

ws,e =√

λ1

L2e

, wb,e =√

μL2

e

Ae, wv =

√ν

1

V 2, (8)

such that the energy (5) becomes E(x) = 12 f(x)Tf(x).

In each Gauss–Newton iteration we compute new vertexpositions and interpolation weights by solving a linear leastsquares system for an update direction δ, which is then scaledby a step-size h and used to update x:

J(x)T J(x)δ = −J(x)T f(x), (9)

x ← x + hδ. (10)

In the above equation J denotes the (2m + 1) × (3n +k) Jacobian matrix of f. It is built from first-order partialderivatives of the components of f with respect to the ver-tex positions xi or interpolation weights αi. For all deriva-tives simple analytical expressions exist, which are given inAppendix B.

[SZGP05]ours

Figure 9: Using three input poses (left) we fix the legs and drag the handle on the right wrist. Our MeshIK system success-fully combines both example deformations, whereas the world-space deformation gradients of the original MeshIK lead toartefacts.



The step-size h is determined by simple bisection search.Starting with h = 1 we successively halve h until the scaledstep decreases the energy, that is E(x + hδ) < E(x). We stopthe iteration if even a minimum step-size of hmin = 10−10 doesnot lead to a further decrease of the energy.

Note that this minimization procedure can be used for puremesh deformation (no examples, no interpolation weights),pure mesh interpolation (k examples, interpolation weightsknown), as well as the full example-based framework (k ex-amples, interpolation weights unknown).

6.2. Volume preservation

A closer look at the linear system (9) reveals a challengingproblem caused by the volume preservation. In each row theJacobian J contains the partial derivatives of the correspond-ing residual with respect to the components of x. For an edgelength constraint this leads to 2 × 3 = 6 non-zeros per row,for a dihedral angle with four involved vertices we get 12non-zeros. The volume v, however, depends on all vertex po-sitions, such that the bottom row of J is completely filled. Asa consequence, the system matrix JTJ is dense, such that effi-cient sparse solvers cannot be applied. In [HSL*06], volumeconstraints were treated as hard constraints using Lagrangemultipliers. Our soft volume constraint with a controllablestiffness parameter requires a different approach.

We propose to exploit the Sherman–Morrison formula[GL89] to efficiently solve the system (9). To this end, wedenote by u the (dense) bottom row of J, which yields

J =[

J

uT

]⇒ JTJ = JTJ + uuT =: A + uuT.

Hence, the system matrix JTJ can be written as a rank-oneupdate of A = JTJ. The Sherman–Morrison formula stateshow to update a matrix inverse after such an update:

(A + uuT

)−1 = A−1 − A−1uuTA−1

1 + uTA−1u.

The solution of (A + uuT)δ = b with b = −JTf becomes

δ = A−1b − A−1uuTA−1b1 + uTA−1u

.

We can therefore compute the solution of (9) by an efficientthree step procedure:

(i) Compute y by solving Ay = b.(ii) Compute z by solving Az = u.

(iii) Compute δ = y − z(uTy)/(1 + uTz).

Note that A = JTJ is sparse, since J does not contain thevolume constraint. Furthermore, because J includes 2m ≈ 6n

constraints, J has full rank and A is symmetric positive defi-nite. We can therefore use a sparse Cholesky solver [DH05]for steps (i) and (ii). Because the rather expensive matrixfactorization of (i) can be re-used in step (ii), our three-stepprocedure comes at a negligible additional cost compared toan optimization without volume constraints.

Note that this technique cannot only be used for the ef-ficient computation of volume constraints. It can also beemployed for other constraints that depend on a large num-ber of mesh vertices, such as, for instance, constraining thearea of a large surface patch in the framework of [EP09]. Weremark that there are several ways to compute a factoriza-tion of a matrix with dense rows through matrix updates. Forinstance, a general method based on updating a QR factoriza-tion is presented in [Sun95]. Our proposed solution has theadvantage that it is straightforward to implement with anyof the commonly employed sparse Cholesky solvers, such asfor instance CHOLMOD [DH05].

6.3. Performance and robustness

Compared to linear deformation methods, non-linear ap-proaches are computationally more expensive, might getstuck in local minima and are in general numerically moresensitive to ‘bad’ input data. Extremely short edges, largedihedral angles (fold-overs) or other kinds of degenerate tri-angles can cause the minimization to stall or even to fail.However, in this respect our method is not more or less sen-sitive than most other nonlinear deformation techniques.

For instance, the synthetic examples shown in Section 3converged in just a few iterations, starting from the initialpose. For a more demanding test we compare to the dragonexample of PriMo [BPGK06]. As shown in Figure 10, evenfor this complex model (50k vertices) and a bad startingconfiguration our minimization converges in just nine itera-tions, which took 52s in total on a MacBookPro 2.53 GHzIntel Core 2 Duo. In comparison, PriMo requires 6 iterationsand 45s on the same machine (on the original mesh res-olution, i.e. without hierarchical optimization). Because our

Figure 10: Even from this bad initial configuration our non-linear deformation method converges in nine iterations.



interpolation and MeshIK techniques are basically equivalentin terms of computations, their robustness and performanceare comparable to those of the mesh deformation.

Input meshes that contain local fold-overs in joint regionsare problematic for our method, such as, for example theLion and Goblin poses of Figures 7 and 9, which have beenproduced by low quality skinning methods. The ‘hinge-like’fold-overs constitute local minima where the optimizationmight get stuck. However, these configurations are easy todetect and surprisingly easy to handle. As soon as the dihedralangle of an edge e changes by more than 180◦ between therest pose M and some example pose Mi , we deactivate thatedge by setting its bending stiffness μe to zero. This simpleheuristic solved the fold-over problem for all our examples.

7. Multiresolution Optimization

The above results demonstrate our non-linear optimizationto work reliably, but its performance is clearly too slow forinteractive applications. Because this is a standard problemfor non-linear editing or interpolation approaches, severaltechniques have been proposed to increase performance (andfurther improve robustness): performing the optimization ona reduced model [DSP06, FKY08] or in a suitable sub-space [HSL*06], or computing it in a hierarchical manner[BPGK06, SZT*07, KMP07, WDAH10].

The cage-based subspace technique [HSL*06], while be-ing an elegant formulation, is not directly applicable to oursetting, because building compatible cages for all exampleposes is a challenging problem by itself. We instead fol-low the approach of Kilian et al. [KMP07] and concurrentlydecimate all input meshes to a complexity of about 1000vertices, where we use the sum of per-mesh quadric errormetrics [GH97] to prioritize halfedge collapses.

The non-linear optimization is then performed on the re-duced meshes exactly as described in the previous sections.One Gauss–Newton iteration on the coarse mesh takes about30 ms only, which allows for real-time mesh editing (becausewe initialize the Gauss–Newton optimization with the pre-vious frame’s result it typically converges in less than fiveiterations). However, an obvious limitation is that modellingconstraints can only be placed on coarse mesh vertices.

To transfer the coarse solution to the original, high-resolution mesh, we adapt the multiresolution modellingapproach proposed in [BSPG06]. The main idea is to in-terpret the coarse mesh’s vertices as modelling constraints(anchors) on the high-resolution mesh. After decimating theinput meshes, we pre-compute a high-resolution but low-frequency base surface B by minimizing the thin-plate en-ergy of the original mesh M with the anchors as constraints.This amounts to solving a bi-Laplacian linear system. Afterdeforming the coarse mesh C into C ′, we compute the de-formed base surface B′ by updating the anchor positions and

Figure 11: The different meshes involved in our multireso-lution optimization: original resolution (top), smooth basesurface (middle), coarse simulation mesh (bottom), all ininitial (left) and deformed configuration (right).

again minimizing the thin-plate energy. The resulting basesurface deformation B �→ B′ is finally applied to the originalmesh M by deformation transfer [SP04], resulting in the de-formed high-resolution and detailed mesh M′. This processis depicted in Figure 11.

It is important to note that this multiresolution reconstruc-tion is also rotation-invariant. If a rotation R is applied tothe coarse mesh, i.e. C ′ = R(C), then the mesh fairing withrotated anchors will reproduce this rotation, B′ = R(B), aswill the deformation transfer, such that M′ = R(M).

In our interactive modeling system, the nonlinear defor-mation can be performed in real-time on the coarse mesh C,and as soon as the mouse is released, the multiresolution re-construction computes the deformed mesh M′. Table 1 givesperformance statistics and compares the non-linear optimiza-tion on the fine mesh (Section 6) to the proposed multireso-lution optimization.

An important question is how much the results of themultiresolution optimization deviate from the results of thefull high-resolution optimization. We analysed this differencefor interpolation sequences of the Elephant model (Figure 5)and the Dragon model (Figure 10). Table 2 shows that theHausdorff distance between the two optimizations is always



Table 1: Performance statistics, with times given in milliseconds,measured on a MacPro 2.66GHz Intel Xeon. From left to right:number of vertices; time for one Gauss–Newton iteration on theoriginal resolution, for multiresolution pre-processing (computing Cand B) and for reconstructing B′ and M′ from C′.

Model |V| G.-N. B & C B′ &M′

Helix 612 18Lion 5k 250 377 23Elephant 40k 3100 4110 271Dragon 50k 3700 5300 323Armadillo 166k 21000 1278

Note: On the coarse mesh, one Gauss–Newton step takes about30 ms. On the original Armadillo model, the Gauss–Newton stepfailed due to memory restrictions.

Table 2: Hausdorff distance (in percent of the bounding box diag-onal) between the result of the multiresolution optimization and thefull high-resolution optimization.

Blending weight Elephant Dragon

0.25 0.23% 0.49%0.35 0.29%0.50 0.34% 0.77%0.65 0.39%0.75 0.40% 0.69%

below 1% of the bounding box diagonal, which proves ourmultiresolution optimization to be a good approximation. Ifneeded, the more accurate high-resolution result can easilybe obtained in two to three Gauss–Newton iterations startingfrom the fine mesh M′.

In addition to the figures shown in the paper, the accompa-nying video also demonstrates the proposed example-baseddeformation system for a few modeling sessions.

8. Conclusion

We presented a framework for combined physics-based andexample-driven mesh deformations. Thanks to our fully non-linear formulation, we can perform large-scale shape manip-ulations in a high-quality, rotation-invariant manner.

The interpolation and deformation operators by them-selves deliver comparable results to established methods interms of both quality and speed. Compared to purely geomet-ric or purely physics-based approaches, incorporating exam-ple poses into the deformation process leads to more naturalresults for models with non-trivial deformation behaviour.In contrast to purely example-driven approaches, our sys-tem provides physically meaningful deformations even whenleaving the space spanned by the input poses. In addition to

mesh deformations, our system also features nonlinear meshinterpolation and deformation transfer in a rotation-invariantmanner.

Investigating subspace optimizations will be an interestingdirection for future work, since this would lead to an even bet-ter preservation of target values, and hence to physically moreaccurate results. In terms of performance a parallelization orGPU-implementation of the multiresolution reconstructionwould allow to interact with the high-resolution model, whilestill computing on the coarse simulation mesh.

Appendix A: Deriving Stiffness Values from Examples

Similar to [PJS06], we derive per-edge stretching stiffness λe

and bending stiffness μe by linearly mapping the stretchingand bending between the k example poses to stiffness values:

λe = maxi=1,...,k

∣∣L(i)e − Le

∣∣ , λe = 1 − λe

maxe λe + ε,

μe = maxi=1,...,k

∣∣�(i)e − �e

∣∣ , μe = 1 − μe

maxe μe + ε.

Appendix B: Partial Derivatives for Constructing J

x1

x2

x3

x4

n1

n2

e

Here we give the partial deriva-tives of the individual compo-nents of the residual function ffrom (7) required to build the Ja-cobian J. The local mesh config-uration is shown to the right.

We define the following vectors

e = x2 − x1,

n1 = (x3 − x2) × (x3 − x1),

n2 = (x4 − x1) × (x4 − x2).

The derivative of the length constraint for edge (x1, x2) issimply the normalized edge direction

∂

∂x1w(‖e‖ − l∗) = w

−e‖e‖ .

The partial derivatives for constraining the dihedral angleθ = ∠(n1, n2) are surprisingly simple (see [BMF03]):

∂

∂x1w(θ − θ∗) = w

[(x3 − x2)Te‖e‖‖n1‖2

n1 + (x4 − x2)Te‖e‖‖n2‖2

n2

],

∂

∂x3w(θ − θ∗) = w

‖e‖‖n1‖2

n1.

The derivative of the volume constraint with respect to avertex position xk is computed by summing over xk’s incident



faces fijk ∈ N (xk) [see (4)]:

∂

∂xk

w(v − v∗) = w∑

fijk∈N (xk )

(xi × xj ).

Finally, the derivative of a constraint (e.g. edge length)with respect to an interpolation weight is [see (6)]

∂

∂αi

w(l − l∗) = −w(L(i) − L).

Acknowledgements

The authors are grateful to Martin Kilian for the elephantmodel of Figure 5, to Tim Winkler for the helix of Fig-ure 6, and to Bob Sumner for the lion model of Figure 7and an executable of the original MeshIK [SZGP05]. TheArmadillo and Dragon models are courtesy of the Stanford3D Scanning Repository. The authors thank Peter Schroderand Eitan Grinspun for helpful discussions about discreteshells, and Migual Otaduy for pointing us to the Sherman-Morrison update. Both authors are supported by the DeutscheForschungsgemeinschaft (Center of Excellence in ‘CognitiveInteraction Technology’, CITEC).

References

[ACOL00] ALEXA M., COHEN-OR D., LEVIN D.: As-rigid-as-possible shape interpolation. In Proceedings ofACM SIGGRAPH (New York, NY, USA, 2000), ACMPress/Addison-Wesley, pp. 157–164.

[AFTCO07] AU O. K.-C., FU H., TAI C.-L., COHEN-OR

D.: Handle-aware isolines for scalable shape editing.ACM Transactions on Graphics, 26, 3 (2007), doi:10.1145/1275808.1276481.

[BK04] BOTSCH M., KOBBELT L.: An intuitive frameworkfor real-time freeform modeling. ACM Transactions onGraphics, 23, 3 (2004), 630–634.

[BMF03] BRIDSON R., MARINO S., FEDKIW R.: Simulation ofclothing with folds and wrinkles. In Proceedings of ACMSIGGRAPH/Eurographics symposium on Computer ani-mation (New York, NY, USA, 2003), Eurographics Asso-ciation, pp. 28–36.

[BMWG07] BERGOU M., MATHUR S., WARDETZKY M.,GRINSPUN E.: TRACKS: toward directable thin shells.ACM Transactions on Graphics, 26, 3 (2007), doi:10.1145/1275808.1276439.

[BPGK06] BOTSCH M., PAULY M., GROSS M., KOBBELT L.:PriMo: Coupled prisms for intuitive surface modeling.In Proceedings of Eurographics Symposium on GeometryProcessing (Aire-la-Ville, Switzerland, 2006), Eurograph-ics Association, pp. 11–20.

[BS08] BOTSCH M., SORKINE O.: On linear variational surfacedeformation methods. IEEE Transactions on Visualizationand Computer Graphics, 14, 1 (2008), 213–230.

[BSPG06] BOTSCH M., SUMNER R., PAULY M., GROSS M.:Deformation transfer for detail-preserving surface edit-ing. In Proceedings of Vision, Modeling, and Visualization(Amsterdam, Netherlands, 2006), Akademische Verlags-gesellschaft, pp. 357–364.

[BVGP09] BARAN I., VLASIC D., GRINSPUN E., POPOVIC J.:Semantic deformation transfer. ACM Transactions onGraphics, 28, 3 (2009), 36:1–36:6.

[CPSS10] CHAO I., PINKALL U., SANAN P., SCHRODER P.: Asimple geometric model for elastic deformations. ACMTransactions on Graphics, 29, 4 (2010), 38:1–38:6.

[DH05] DAVIS T. A., HAGER W.: CHOLMOD: supern-odal sparse cholesky factorization and update/downdate.http://www.cise.ufl.edu/research/sparse/cholmod, 2005.

[DSP06] DER K. G., SUMNER R. W., POPOVIC J.: Inverse kine-matics for reduced deformable models. ACM Transactionson Graphics, 25, 3 (2006), 1174–1179.

[EP09] EIGENSATZ M., PAULY M.: Positional, metric, and cur-vature control for constraint-based surface deformation.Computer Graphics Forum, 28, 2 (2009), 551–558.

[FKY08] FENG W.-W., KIM B.-U., YU Y.: Real-time data-driven deformation using kernel canonical correlationanalysis. ACM Transactions on Graphics, 27, 3 (2008),91:1–91:9.

[GH97] GARLAND M., HECKBERT P.: Surface simplifica-tion using quadric error metrics. In Proceedings ofACM SIGGRAPH (New York, NY, USA, 1997), ACMPress/Addison-Wesley, pp. 209–216.

[GHDS03] GRINSPUN E., HIRANI A. N., DESBRUN M.,SCHRODER P.: Discrete shells. In Proceedings of ACM SIG-GRAPH/Eurographics Symposium on Computer Anima-tion (Aire-la-Ville, Switzerland, 2003), Eurographics As-sociation, pp. 62–67.

[GL89] GOLUB G. H., LOAN C. F. V.: Matrix Computations.Johns Hopkins University Press, Baltimore, 1989.

[HSL*06] HUANG J., SHI X., LIU X., ZHOU K., WEI L.-Y.,TENG S., BAO H., GUO B., SHUM H.-Y.: Subspace gradientdomain mesh deformation. ACM Transactions on Graph-ics, 25, 3 (2006), 1126–1134.

[HZS*06] HUANG J., ZHANG H., SHI X., LIU X., BAO H.: In-teractive mesh deformation with pseudo material effects.Computer Animation and Virtual Worlds, 17, 3–4 (2006),383–392.



[KCVS98] KOBBELT L., CAMPAGNA S., VORSATZ J., SEIDEL

H.-P.: Interactive multi-resolution modeling on arbitrarymeshes. In Proceedings of ACM SIGGRAPH (1998),ACM Press, New York, NY, pp. 105–114.

[KG08] KIRCHER S., GARLAND M.: Free-form motion pro-cessing. ACM Transactions on Graphics, 27, 2 (2008),12:1–12:13.

[KMP07] KILIAN M., MITRA N. J., POTTMANN H.: Geometricmodeling in shape space. ACM Transactions on Graphics,26, 3 (2007), doi: 10.1145/1275808.1276457.

[LCF00] LEWIS J. P., CORDNER M., FONG N.: Pose spacedeformation: a unified approach to shape interpola-tion and skeleton-driven deformation. In Proceedings ofACM SIGGRAPH (New York, NY, USA, 2000), ACMPress/Addison-Wesley, pp. 165–172.

[LSLC05] LIPMAN Y., SORKINE O., LEVIN D., COHEN-OR D.:Linear rotation-invariant coordinates for meshes. ACMTransactions on Graphics, 24, 3 (2005), 479–487.

[MNT04] MADSEN K., NIELSON H. B., TINGLEFF O.: Methodsfor Non-Linear Least Squares Problems (2nd ed.). Tech.rep., Informatics and Mathematical Modelling, TechnicalUniversity of Denmark, DTU, 2004.

[PJS06] POPA T., JULIUS D., SHEFFER A.: Material-aware meshdeformations. In Proceedings of IEEE International Con-ference on Shape Modeling and Applications (2006), IEEEComputer Society, pp. 141–152.

[SA07] SORKINE O., ALEXA M.: As-rigid-as-possible surfacemodeling. In Proceedings of Eurographics Symposium onGeometry Processing (Aire-la-Ville, Switzerland, 2007),Eurographics Association, pp. 109–116.

[SK04] SHEFFER A., KRAEVOY V.: Pyramid coordinates formorphing and deformation. In Proceedings of Symposiumon 3D Data Processing, Visualization and Transmission(2004), IEEE Computer Society, pp. 68–75.

[Sor06] SORKINE O.: Differential representations for meshprocessing. Computer Graphics Forum, 25, 4 (2006),789–807.

[SP04] SUMNER R. W., POPOVIC J.: Deformation transfer fortriangle meshes. ACM Transactions on Graphics, 23, 3(2004), 399–405.

[Sum05] SUMNER R. W.: Mesh Modification Using Defor-mation Gradients. PhD thesis, Massachusetts Institute ofTechnology, 2005.

[Sun95] SUN C.: Dealing with Dense Rows in the Solutionof Sparse Linear Least Squares Problems. Tech. Rep.,Cornell University, 1995.

[SZGP05] SUMNER R. W., ZWICKER M., GOTSMAN C., PopovicJ.: Mesh-based inverse kinematics. ACM Transactions onGraphics, 24, 3 (2005), 488–495.

[SZT*07] SHI X., ZHOU K., TONG Y., DESBRUN M.,BAO H., GUO B.: Mesh puppetry: cascading optimiza-tion of mesh deformation with inverse kinematics.ACM Transactions on Graphics, 26, 3 (2007), doi:10.1145/1275808.1276479.

[TPBF87] TERZOPOULOS D., PLATT J., BARR A., FLEISCHER

K.: Elastically deformable models. In Proceedings ofACM SIGGRAPH (1987), ACM Press, New York, NY,pp. 205–214.

[WDAH10] WINKLER T., DRIESEBERG J., ALEXA M., HORMANN

K.: Multi-scale geometry interpolation. Computer Graph-ics Forum, 29, 2 (2010), 309–318.

[WPP07] WANG R. Y., PULLI K., POPOVIC J.: Real-time en-veloping with rotational regression. ACM Transactions onGraphics, 26, 3 (2007), doi: 10.1145/1275808.1276468.

[WSLG07] WEBER O., SORKINE O., LIPMAN Y., GOTSMAN

C.: Context-aware skeletal shape deformation.Computer Graphics Forum, 26, 3 (2007), 265–273.

[XZWB05] XU D., ZHANG H., WANG Q., BAO H.: Poissonshape interpolation. In Proceedings of ACM Symposiumon Solid and Physical Modeling (2005), ACM Press, NewYork, NY, pp. 267–274.

[YZX*04] YU Y., ZHOU K., XU D., SHI X., BAO H., GUO B.,SHUM H.-Y.: Mesh editing with Poisson-based gradientfield manipulation. ACM Transactions on Graphics, 23, 3(2004), 644–651.

[ZRKS05] ZAYER R., ROSSL C., KARNI Z., SEIDEL

H.-P.: Harmonic guidance for surface deformation.Computer Graphics Forum, 24, 3 (2005), 601–609.


example-driven deformations based on discrete shells

Documents