deformable models in medical image analysis- a survey
DESCRIPTION
Survey paperTRANSCRIPT
-
Deform
able
Modelsin
MedicalIm
ageAnalysis:
ASurvey
Tim
McInerney
1an
dDem
etriTerzopou
los2
,3
1DepartmentofComputerScience,RyersonUniversity,Toronto,ON,CanadaM5B2K3
2ComputerScience
Department,University
ofCalifornia,LosAngeles,CA
90095,USA
3DepartmentofComputerScience,University
ofToronto,Toronto,ON,CanadaM5S3H5
Abstract
This
article
surveysdeform
able
models,
apromisingandvigorouslyresearched
computer-
assistedmedicalim
ageanalysistechnique.
Amongmodel-basedtechniques,deform
able
models
oer
auniqueandpow
erfulapproach
toim
ageanalysisthatcombines
geometry,physics,and
approxim
ationtheory.They
haveprovento
beeective
insegmenting,matching,andtracking
anatomicstructuresbyexploiting(bottom-up)constraintsderived
from
theim
agedata
together
with(top-dow
n)know
ledgeaboutthelocation,size,andshapeofthesestructures.
Deform
able
modelsare
capableofaccommodatingthesignicantvariabilityofbiologicalstructuresover
time
andacross
dierentindividuals.Furthermore,they
support
highly
intuitiveinteractionmecha-
nismsthat,when
necessary,allowmedicalscientistsandpractitionersto
bringtheirexpertise
tobearonthemodel-basedim
ageinterpretationtask.Thisarticle
reviewstherapidly
expanding
bodyofworkonthedevelopmentandapplicationofdeform
able
modelsto
problemsoffun-
damentalim
portance
inmedicalim
ageanalysis,includingsegmentation,shaperepresentation,
matching,andmotiontracking.
KeyWords:
Deform
ableModels,ShapeModeling,Segmentation,Matching,MotionTracking.
PublicationHistory
Thissurvey
isanupdatedversionofthearticle
rstpublished
as
Deform
ablemodelsin
medicalim
ageanalysis:
Asurvey,
T.McInerney,D.Terzopoulos,Med-
icalIm
age
Analysis,1(2),1996,91108,
whichwasreprintedin
thevolume
Deform
ableModelsin
MedicalIm
age
Analysis,
A.Singh,D.Goldgof,D.Terzopoulos(eds.),
IEEEComputerSocietyPress,LosAlamitos,CA,1998,219,
andasthechapter
Deform
able
Models,T.McInerney,D.Terzopoulos,
inHandbookofMedicalIm
aging:
Pro-
cessingandAnalysis,I.Bankman(ed.),Academ
icPress,SanDiego,2000,Ch.8,127145.
Thesecondeditionofthechapter,
Deform
ableModels,T.McInerney,D.Terzopoulos,in
HandbookofMedicalIm
age
Processing
andAnalysis(2ndEdition),I.Bankman(ed.),Academ
icPress,SanDiego,2008,Ch.8,145166,
isclose
tothepresentform
.
1
Contents
Abstract
1
Contents
2
1Introduction
3
2MathematicalFoundationsofDeform
able
Models
42.1
Energy-M
inim
izingDeform
able
Models..........................
42.2
Dynamic
Deform
able
Models
...............................
62.3
DiscretizationandNumericalSim
ulation.........................
62.4
ProbabilisticDeform
able
Models
.............................
7
3MedicalIm
ageAnalysiswithDeform
able
Models
83.1
ImageSegmentationwithDeform
able
Curves
......................
83.2
VolumeIm
ageSegmentationwithDeform
able
Surfaces.................
11
3.3
IncorporatingKnow
ledge
.................................
12
3.4
Matching
..........................................
14
3.5
MotionTrackingandAnalysis...............................
16
4Discussion
20
4.1
AutonomyvsControl
...................................
20
4.2
Generality
vsSpecicity
..................................
21
4.3
CompactnessvsGeometricCoveragevsTopologicalFlexibility
............
21
4.4
CurvevsSurface
vsSolidModels.............................
21
4.5
Accuracy
andQuantitative
Pow
er.............................
22
4.6
Robustness
.........................................
22
4.7
LagrangianvsEulerianDeform
able
Models
.......................
23
5Conclusion
23
Acknowledgements
24
References
35
Index
35
2
-
1Introduction
Therapid
developmentan
dproliferationofmedicalimagingtechnologiesisrevolutionizingmedicine.
Medicalim
agingallow
sscientistsandphysiciansto
gleanpotentiallylife-savinginform
ationby
peeringnoninvasively
into
thehumanbody.
Therole
ofmedicalim
aginghasexpanded
beyond
thesimple
visualizationandinspectionofanatomic
structures.
Ithasbecomeatool
forsurgical
planningandsimulation,intra-operative
navigation,radiotherapyplanning,andfortrackingthe
progress
ofdisease.Forexample,ascertainingthedetailed
shapeandorganizationofanatomic
structuresenablesasurgeonpreoperatively
toplananoptimalapproach
tosometarget
structure.
Inradiotherapy,medicalim
agingallow
sthedeliveryofanecroticdose
ofradiationto
atumorwith
minim
alcollateraldamageto
healthytissue.
Withmedicalim
agingplayinganincreasingly
prominentrolein
thediagnosisandtreatm
entof
disease,themedicalim
ageanalysiscommunityhasbecomepreoccupiedwiththechallengingprob-
lem
ofextracting,withtheassistance
ofcomputers,clinicallyusefulinform
ationaboutanatomic
structuresim
aged
throughCT,MR,PET,andother
modalities
[Stytz
etal.,1991;Robb,1994;
Hohne&
Kikinis,1996;Ayache,
1995a;
Bizaiset
al.,1995;Duncan&
Gindi,1997;Ayache,
1995b;
Troccazet
al.,1997;Wellset
al.,1998;Duncan&Ayache,2000].Althoughmodernim
agingdevices
provideexceptionalviewsofinternalanatomy,
theuse
ofcomputers
toquantify
andanalyze
the
embedded
structureswithaccuracy
ande
ciency
islimited.Accurate,repeatable,quantitative
data
must
bee
cientlyextracted
inorder
tosupport
thespectrum
ofbiomedicalinvestigations
andclinicalactivitiesfrom
diagnosis,to
radiotherapy,
tosurgery.
For
example,segmentingstructuresfrom
medicalim
ages
andreconstructingacompact
geo-
metricrepresentationofthesestructuresis
di
cultdueto
thesheersize
ofthedatasets
andthe
complexityan
dvariabilityof
theanatomicshapes
ofinterest.Furthermore,theshortcomingstypi-
calofsampleddata,such
assamplingartifacts,spatialaliasing,andnoise,may
cause
theboundaries
ofstructuresto
beindistinct
anddisconnected.Thechallengeisto
extract
bou
ndary
elem
ents
be-
longingto
thesamestructure
andintegrate
theseelem
ents
into
acoherentan
dconsistentmodel
ofthestructure.Traditionallow-level
imageprocessingtechniques
whichconsider
only
localin-
form
ationcanmake
incorrectassumptionsduringthisintegrationprocess
andgenerate
infeasible
object
bou
ndaries.Asaresult,thesemodel-freetechniques
usuallyrequireconsiderable
amounts
ofexpertintervention.Furthermore,thesubsequentanalysisandinterpretationofthesegmented
objectsishindered
bythepixel-orvoxel-level
structure
representationsgeneratedbymost
image
processingoperations.
Thischaptersurveysdeform
ablemodels,oneofthemostintensively
researched
model-basedap-
proaches
tocomputer-assistedmedicalim
ageanalysis.
Thewidelyrecognized
potency
ofdeform
able
modelsstem
sfrom
theirabilityto
segment,
match,andtrack
images
ofanatomic
structuresby
exploiting(bottom-up)constraints
derived
from
theim
agedata
together
with(top-dow
n)know
l-edgeaboutthelocation,size,andshapeofthesestructures.
Deform
able
modelsare
capable
of
accommodatingtheoften
signicantvariabilityof
biologicalstructuresover
timeandacross
dier-
entindividuals.Furthermore,deform
able
modelssupport
highly
intuitiveinteractionmechanisms
thatallow
medicalscientistsandpractitionersto
bringtheirexpertise
tobearonthemodel-based
imageinterpretationtask
when
necessary.
Wewillreview
thebasicform
ulationofdeform
able
modelsandsurvey
theirapplicationto
fundamentalmedicalim
ageanalysisproblems,
including
segmentation,shaperepresentation,matching,andmotiontracking.Thechapteris
anupdated
versionof
[McInerney
&Terzopoulos,1996](see
alsothecompilation[Singhet
al.,1998]).
3
2MathematicalFoundationsofDeform
able
Models
Theclassicalmathem
aticalfoundationsofdeform
ablemodelsrepresenttheconuence
ofgeometry,
physics,andapproxim
ationtheory.Geometry
serves
torepresentobject
shape,
physics
imposes
constraints
onhow
theshapemay
vary
over
space
andtime,
andoptimalapproxim
ationtheory
provides
theform
alunderpinningsofmechanismsforttingthemodelsto
measureddata.
Deform
ablemodelgeometry
usuallypermitsbroadshapecoveragebyem
ployinggeometricrep-
resentationsthatinvolvemanydegrees
offreedom,such
assplines
.Themodelremainsmanageable,
how
ever,because
thedegrees
offreedom
are
generallynotpermittedto
evolveindependently,
but
are
governed
byphysicalprinciplesthatbestowintuitivelymeaningfulbehavioruponthegeometric
substrate.Thenamedeform
ablemodels
stem
sprimarily
from
theuse
ofelasticitytheory
atthe
physicallevel,generallywithin
aLagrangiandynamicssetting.Thephysicalinterpretationviews
deform
able
modelsaselastic
bodieswhichrespondnaturallyto
applied
forces
andconstraints.
Typically,
deform
ationenergyfunctionsdened
interm
softhegeometricdegrees
offreedom
are
associated
withthedeformab
lemodel.Theenergygrowsmonotonicallyasthemodeldeform
saw
ayfrom
aspecied
naturalorrest
shape
andoften
includes
term
sthatconstrain
thesm
oothness
orsymmetry
ofthemodel.In
theLagrangiansetting,thedeform
ationenergygives
rise
toelastic
forces
internalto
themodel.Takingaphysics-basedview
ofclassicaloptimalapproxim
ation,ex-
ternalpotentialenergyfunctionsare
dened
interm
softhedata
ofinterest
towhichthemodelis
tobetted.Thesepotentialenergiesgiverise
toexternalforces
whichdeform
themodelsuch
that
itts
thedata.
Deform
ablecurve,surface,andsolidmodelsgained
popularity
after
they
wereproposedforuse
incomputervision[Terzopouloset
al.,1988]andcomputergraphics[Terzopoulos&Fleischer,1988]
inthemid
1980s.Terzopoulosintroducedthetheory
ofcontinuous(m
ultidim
ensional)deform
able
modelsin
aLagrangian
dynamicssetting[Terzopoulos,
1986a],based
on
deform
ation
energies
intheform
of(controlled-continuity)generalizedsplines
[Terzopoulos,
1986b].
Ancestors
ofthe
deform
able
modelsnow
incommonuse
includeFischlerandElshlagers
spring-loaded
templates
[1973]
andWidrowsrubber
mask
technique[1973].
Thedeform
able
model
thathasattracted
themost
attentionto
date
ispopularlyknow
nas
snakes[K
ass
etal.,1988].
Snakesoractivecontourmodels
representaspecialcase
ofthe
generalmultidim
ensionaldeform
able
model
theory
[Terzopoulos,
1986a].
Wewillreview
their
simple
form
ulationin
theremainder
ofthissectionin
order
toillustrate
withaconcreteexample
thebasicmathem
aticalmachinerythatispresentin
manydeform
able
models.
Snakesare
planardeform
ablecontours
thatare
usefulin
severalim
ageanalysistasks.
They
are
often
usedto
approxim
ate
thelocationsandshapes
ofobject
boundaries
inim
ages
basedonthe
reasonable
assumptionthatbou
ndaries
are
piecewisecontinuousorsm
ooth
(Fig.1).
Initsbasic
form
,themathem
aticalform
ulationofsnakes
drawsfrom
thetheory
ofoptimalapproxim
ation
involvingfunctionals.
2.1
Energy-M
inim
izingDeform
able
Models
Geometrically,
asnake
isaparametriccontourem
bedded
intheim
ageplane(x,y)
2.The
contouris
representedasv(s)=
(x(s),y(s)),wherexandyare
thecoordinate
functionsand
s
[0,1]is
theparametricdomain.
Theshapeofthecontoursubject
toanim
ageI(x,y)is
dictatedbythefunctional
E(v)=
S(v)+P(
v).
(1)
4
-
Figure
1:Snake
(white)
attracted
tocellmem
branein
anEM
photomicrograph(from
[Carlbom
etal.,1994]).
Thefunctionalcanbeviewed
asarepresentationoftheenergyofthecontourandthenalshape
ofthecontourcorrespondsto
theminim
um
ofthisenergy.
Therstterm
ofthefunctional,
S(v)=
1 0w1(s)
v s 2 +
w2(s)
2v
s2
2 ds,
(2)
istheinternaldeform
ationenergy.
Itcharacterizes
thedeform
ationofastretchy,
exible
contour.
Twophysicalparameter
functionsdictate
thesimulatedphysicalcharacteristics
ofthecontour:
w1(s)controlsthetensionofthecontourwhilew2(s)controlsitsrigidity.1
Thesecondterm
in(1)couplesthesnake
totheim
age.
Traditionally,
P(v)=
1 0P(v(s))ds,
(3)
whereP(x,y)denotesascalarpotentialfunctiondened
ontheim
ageplane.
Toapply
snakes
toim
ages,externalpotentials
are
designed
whose
localminim
acoincidewithintensity
extrem
a,
edges,andother
imagefeaturesofinterest.Forexample,thecontourwillbeattracted
tointensity
edges
inanim
ageI(x,y)bychoosingapotentialP(x,y)=
c|
[GI
(x,y)]|,whereccontrols
themagnitudeofthepotential,
isthegradientop
erator,andG
I
denotestheim
ageconvolved
witha(G
aussian)sm
oothinglter
whose
characteristicwidth
controlsthespatialextentofthe
localminim
aofP.
Inaccordance
withthecalculusofvariations,thecontourv(s)whichminim
izes
theenergyE(v)
must
satisfytheEuler-Lagrangeequation
s
( w1v
s
) +2
s2
( w22v
s2
) +P
(v(s,t))
=0.
(4)
Thisvector-valued
partialdierentialequationexpresses
thebalance
ofinternalandexternalforces
when
thecontourrestsatequilibrium.Thersttw
oterm
srepresenttheinternalstretchingand
bendingforces,respectively,whilethethirdterm
represents
theexternalforces
thatcouple
the
snake
totheim
agedata.Theusualapproach
tosolving(4)isthroughtheapplicationofnumerical
algorithms(Sec.2.3).
1Theva
lues
ofthenon-negativefunctionsw1(s)andw2(s)determinetheex
tentto
whichthesnakecanstretch
orben
datanypointsonthesnake.
Forex
ample,increasingthemagnitudeofw1(s)increasesthetensionand
tendsto
elim
inate
extraneousloopsandripplesbyreducingthelength
ofthesnake.
Increasingw2(s)increasesthe
ben
dingrigidityofthesnakeandtendsto
makethesnakesm
oother
andless
ex
ible.Settingtheva
lueofoneor
both
ofthesefunctionsto
zero
atapointspermitsdiscontinuitiesin
thecontourats.
5
Figure
2:Snake
deform
ingtowardshighgradientsin
aprocessed
cardiacim
age,inuencedbypin
points
andaninteractivespringwhichpullsthecontourtowardsanedge(from
[McInerney
&Terzopoulos,1995a]).
2.2
Dynamic
Deform
able
Models
Whileitisnaturalto
viewenergyminim
izationasastaticproblem,apotentapproach
tocomputing
thelocalminim
aofafunctionalsuch
as(1)is
toconstruct
adynamicalsystem
thatisgoverned
bythefunctionalandallow
thesystem
toevolveto
equilibrium.Thesystem
may
beconstructed
byapplyingtheprinciplesofLagrangianmechanics.
This
leadsto
dynamic
deform
able
models
thatunifythedescriptionofshapeandmotion,makingit
possible
toquantify
notjust
static
shape,
butalsoshapeevolutionthroughtime.
Dynamic
modelsare
valuable
formedicalim
age
analysis,since
most
anatomicalstructuresare
deform
ableandcontinuallyundergononrigid
motion
invivo.Moreover,dynamicmodelsexhibitintuitivelymeaningfulphysicalbehaviors,makingtheir
evolutionamenable
tointeractiveguidance
from
auser(Fig.2).
Asimple
example
isadynamic
snakewhichcanberepresentedbyintroducingatime-varying
contourv(s,t)=
(x(s,t),y(s,t))
withamass
density
(s)andadampingdensity
(s).
The
Lagrangeequationsofmotionforasnake
withtheinternalenergygiven
by(2)andexternalenergy
given
by(3)is
2v
t2
+v t
s
( w1v
s
) +2
s2
( w22v
s2
) =
P(v(s,t)).
(5)
Thersttw
oterm
sonthelefthandsideofthispartialdierentialequationrepresentinertialand
dampingforces.Referringto
(4),theremainingterm
srepresenttheinternalstretchingandbending
forces,whiletherighthandsiderepresents
theexternalforces.Equilibrium
isachievedwhen
the
internalandexternalforces
balance
andthecontourcomes
torest
(i.e.,v/t=
2v/t2
=0),
whichyieldstheequilibrium
condition(4).
2.3
DiscretizationandNumericalSim
ulation
Inorder
tonumericallycompute
aminim
um
energysolution,itisnecessary
todiscretizetheen-
ergyE(v).
Theusualapproach
isto
representthecontinuousgeometricmodelvin
term
soflinear
combinationsoflocal-support
orglobal-support
basisfunctions.
Finiteelem
ents
[Zienkiewicz&
6
-
Taylor,
1989],nitedierences[Press
etal.,1992],andgeometricsplines
[Farin,1993]arelocal
representationmethods,whereasFourier
bases[Ballard
&Brown,1982]areglobalrepresentation
methods.
Thecontinuousmodelv(s)isrepresentedin
discreteform
byavectoruofshapeparam-
etersassociatedwiththebasisfunctions.
Thediscreteform
ofenergiessuch
asE(v)forthesnake
may
bewritten
as
E(u)=
1 2uKu+P(u)
(6)
whereK
iscalled
thestinessmatrix,an
dP(u)is
thediscreteversionoftheexternalpotential.
Theminim
um
energysolutionresultsfrom
settingthegradientof
(6)to
0,whichisequivalentto
solvingthesetofalgebraic
equations
Ku=
P=f
(7)
wherefisthegeneralizedexternalforcevector.
Thediscretized
versionoftheLagrangiandynamicsequation(5)may
bewritten
asasetof
secondorder
ordinary
dierentialequationsforu(t):
Mu+Cu+Ku=f,
(8)
whereM
isthemass
matrix
andC
isadampingmatrix.Thetimederivativesin
(5)areapproxi-
matedbynitedierencesandexplicitorim
plicitnumericaltimeintegrationmethodsare
applied
tosimulate
theresultingsystem
ofordinary
dierentialequationsin
theshapeparametersu.
2.4
ProbabilisticDeform
able
Models
Analternative
view
ofdeform
able
modelsem
erges
from
castingthemodel
ttingprocess
ina
probabilisticframew
ork,often
takingaBayesianapproach.Thispermitstheincorporationofprior
model
andsensormodel
characteristics
interm
sofprobabilitydistributions.
Theprobabilistic
framew
ork
alsoprovides
ameasure
oftheuncertainty
oftheestimatedshapeparametersafter
the
model
istted
totheim
agedata
[Szeliski,1990].
Let
urepresentthedeform
able
model
shapeparameterswithapriorprobabilityp(u)on
the
parameters.
Let
p(I|u)betheim
aging(sensor)
modeltheprobabilityof
producinganim
ageI
given
amodelu.Bayestheorem
p(u
|I)=
p(I|u)p(u)
p(I)
(9)
expresses
theposteriorprobabilityp(u
|I)of
amodelgiven
theim
age,in
term
softheim
agingmodel
andthepriorprobabilitiesofmodel
andim
age.
Itis
easy
toconvert
theinternalenergymeasure
(2)of
thedeform
able
model
into
aprior
distributionover
expectedshapes,withlower
energyshapes
beingthemore
likely.Thisisachieved
usingaBoltzm
an(orGibbs)
distributionoftheform
p(u)=
1 Zsexp(
S(u)),
(10)
whereS(u)is
thediscretized
versionofS(v)in
(2)an
dZsis
anorm
alizingconstant(called
the
partitionfunction).
Thispriormodelisthen
combined
withasimplesensormodelbasedonlinear
measurements
withGaussiannoise
p(I|u)=
1 ZIexp(
P(u)),
(11)
whereP(u)is
adiscreteversionofthepotentialP(
v)in
(3),
whichis
afunctionoftheim
age
I(x,y).
7
Modelsmay
betted
byndinguwhichlocallymaxim
izep(u
|I)in
(9).
Thisisknow
nasthe
maxim
um
aposteriori(M
AP)solution.Withtheaboveconstruction,ityieldsthesameresultas
minim
izing(1),theenergycongurationofthedeform
able
model
given
theim
age.
Theprobabilisticframew
ork
canbeextended
byassumingatime-varyingpriormodel,orsystem
model,in
conjunctionwiththesensormodel,resultingin
aKalm
anlter.
Thesystem
model
describes
theexpectedevolutionoftheshapeparametersuover
time.
Iftheequationsofmotion
ofthephysicalsnakesmodel
(8)areem
ployed
asthesystem
model,theresult
isasequential
estimationalgorithm
know
nasKalm
ansnakes[Terzopoulos&
Szeliski,1992].
3MedicalIm
ageAnalysiswithDeform
able
Models
Althoughoriginallydeveloped
forapplicationto
problemsin
computervisionandcomputergraph-
ics,thepotentialofdeform
ablemodelsforuse
inmedicalim
ageanalysishasbeenquickly
realized.
They
havebeenapplied
toim
ages
generatedbyim
agingmodalities
asvaried
asX-ray,computed
tomography(C
T),angiography,
magnetic
resonance
(MR),andultrasound.Twodim
ensionaland
threedim
ensionaldeform
able
modelshavebeenusedto
segment,visualize,track,andquantify
avarietyof
anatomic
structuresrangingin
scale
from
themacroscopic
tothemicroscopic.These
includethebrain,heart,face,cerebral,coronary
andretinalarteries,kidney,lungs,stomach,liver,
skull,vertebra,objectssuch
asbrain
tumors,afetus,andeven
cellularstructuressuch
asneurons
andchromosomes.Deform
ablemodelshavebeenusedto
track
thenonrigid
motionoftheheart,the
growingtipofaneurite,andthemotionoferythrocytes.
They
havebeenusedto
locate
structures
inthebrain,andto
registerim
ages
oftheretina,vertebra
andneuronaltissue.
Inthefollow
ingsections,wereviewanddiscuss
theapplicationofdeform
ablemodelsto
medical
imageinterpretationtasksincludingsegmentation,matching,andmotionanalysis.
3.1
ImageSegmentationwithDeform
able
Curves
Thesegmentationofanatomic
structures
thepartitioningoftheoriginalsetofim
agepoints
into
subsets
correspondingto
thestructures
isanessentialrststageofmost
medicalim
age
analysistasks,
such
asregistration,labeling,andmotiontracking.Thesetasksrequireanatomic
structuresin
theoriginalim
ageto
bereducedto
acompact,analyticrepresentationoftheirshapes.
Perform
ingthissegmentationmanuallyisextrem
elylaborintensive
andtime-consuming.Aprimary
example
isthesegmentationoftheheart,especiallytheleftventricle
(LV),from
cardiacim
agery.
Segmentationoftheleft
ventricle
isaprerequisiteforcomputingdiagnostic
inform
ationsuch
as
ejection-fractionratio,ventricularvolumeratio,heart
output,andforwallmotionanalysiswhich
provides
inform
ationonwallthickening,etc.
[Singhet
al.,1993].
Most
clinicalsegmentationiscurrentlyperform
edusingmanualsliceediting.In
thisscenario,
askilledoperator,usingacomputermouse
ortrackball,manuallytracestheregionofinterest
on
each
sliceofanim
agevolume.
Manualsliceeditingsuersfrom
severaldrawbacks.
Theseinclude
thedi
cultyin
achievingreproducible
results,
operatorbias,
forcingtheoperatorto
view
each
2D
sliceseparately
todeduce
andmeasure
theshapeandvolumeof3D
structures,
andoperator
fatigue.
Segmentationusingtraditionallow-level
imageprocessingtechniques,such
asthresholding,
regiongrowing,edgedetection,andmathem
aticalmorphologyoperations,alsorequires
considerable
amounts
ofexpertinteractiveguidance.Furthermore,automatingthesemodel-freeapproaches
isdi
cultbecause
oftheshapecomplexityandvariabilitywithin
andacross
individuals.In
general,
theunderconstrained
nature
ofthesegmentationproblem
limitsthee
cacy
ofapproaches
that
consider
localinform
ationonly.
Noiseandother
imageartifactscancause
incorrectregionsor
bou
ndary
discontinuitiesin
objectsrecoveredbythesemethods.
8
-
(a)
(b)
(c)
(d)
(e)
(f)
Figure
3:(a)Intensity
CT
imagesliceofcanineLV.(b)Edgedetectedim
age.
(c)Initialsnake.
(d)-(f)Snake
deform
ingtowardsLV
bou
ndary,drivenbyinationforce.
(From
[McInerney
&Terzopoulos,1995a].)
Adeform
able
model
basedsegmentationschem
e,usedin
concert
withim
agepre-processing,
canovercomemanyofthelimitationsofmanualsliceeditingandtraditionalim
ageprocessing
techniques.Theseconnectedandcontinuousgeometricmodelsconsider
anobject
bou
ndary
asa
wholeandcanmake
use
ofpriorknow
ledgeofobject
shapeto
constrain
thesegmentationproblem.
Theinherentcontinuityandsm
oothnessofthemodelscancompensate
fornoise,
gapsandother
irregularities
inobject
bou
ndaries.
Furthermore,theparametricrepresentationsofthemodels
provideacompact,analyticaldescriptionofobject
shape.
Theseproperties
leadto
ane
cient,
robust,accurate
andreproducible
techniqueforlinkingsparseornoisylocalim
agefeaturesinto
acomplete
andconsistentmodel
oftheobject.
Amongtherstandprimary
usesofdeform
able
modelsin
medicalim
ageanalysiswas
the
applicationofdeform
ablecontourmodels,such
assnakes[K
ass
etal.,1988],to
segmentstructures
in2D
images
[Berger,1990;Cohen,1991;Ueda&
Mase,1992;Rou
gon&
Preteux,1993;Cohen
&Coh
en,1993;Leitner
&Cinquin,1993;Carlbom
etal.,1994;Gupta
etal.,1994;Lobregt&
Viergever,1995;Davatzikos&
Prince,1995;Pau
luset
al.,1999].
Typicallyusers
initializeda
deform
ablemodelneartheobject
ofinterest(Fig.3)
andallow
editto
deform
into
place.Userscould
then
exploittheinteractivecapabilitiesofthesemodelsandmanuallyne-tunethem
.Furthermore,
once
theuseris
satised
withtheresult
onaninitialim
ageslice,
thetted
contourmodel
may
then
beusedastheinitialbou
ndary
approxim
ationforneighboringslices.Thesemodelsare
then
deform
edinto
place
andagain
propagateduntilallslices
havebeenprocessed.
Theresulting
sequence
of2D
contours
canthen
beconnectedto
form
acontinuous3D
surface
model
[Lin
&Chen,1989;Changet
al.,1991;Cohen,1991;Cohen
&Cohen,1993].
Theapplicationofsnakesandother
similardeform
able
contourmodelsto
extract
regionsof
interest
is,how
ever,notwithoutlimitations.
Forexample,snakesweredesigned
asinteractive
models.
Innon-interactiveapplications,they
must
beinitializedclose
tothestructure
ofinterest
toguaranteegoodperform
ance.Theinternalenergyconstraints
ofsnakescanlimittheirgeometric
exibilityan
dpreventasnake
from
representinglongtube-like
shapes
andshapes
withsignicant
protrusionsorbifurcations.
Furthermore,thetopologyofthestructure
ofinterestmustbeknow
nin
advance
since
classicaldeform
able
contourmodelsare
parametricandare
incapable
oftopological
transform
ationswithoutadditionalmachinery(such
asthatin
T-snakes[M
cInerney
&Terzopoulos,
2000]).
Variousmethodshavebeenproposedto
improve
andfurther
automate
thedeform
ablecontour
segmentationprocess.Cohen
andCohen
[1993]usedaninternalinationforceto
expandasnakes
model
past
spuriousedges
towardstherealedges
ofthestructure,makingthesnake
less
sensitive
toinitialconditions(inationforces
werealsoem
ployedin
[Terzopouloset
al.,1988]).Aminiet
al.
[1990]
use
dynamic
programmingto
carryoutamore
extensive
searchforglobalminim
a.
9
Figure
4:Im
agesequence
ofclipped
angiogram
ofretinashow
inganautomaticallysubdividing
snake
ow
ingandbranchingalongavessel(from
[McInerney
&Terzopoulos,1995b]).
(a)
(b)
(c)
(d)
Figure
5:Segmentationofacrosssectionalim
ageofahumanvertebra
phantom
withatopologically
adaptablesnake(from
[McInerney
&Terzopoulos,1995b]).Thesnake
beginsasasingleclosedcurve
andbecomes
threeclosedcurves.
Poon
etal.[1994]
andGrzeszczukandLevin
[1997]
minim
izetheenergyofactivecontourmodels
usingsimulatedannealingwhichisknow
nto
giveglobalsolutionsandallow
stheincorporationof
non-dierentiable
constraints.
Other
researchers[Rou
gon&
Preteux,1991;
Ivins&
Porrill,1994;
Chakraborty&
Duncan,
1999;Herlinet
al.,1992;Gauch
etal.,1994;
Mangin
etal.,1995;
Gunn&
Nixon,1997;
Jones
&Metaxas,
1997]haveintegratedregion-basedinform
ationinto
deform
able
contourmodelsor
usedother
techniques
inanattem
ptto
decrease
sensitivityto
insignicantedges
andinitialmodel
placement.
For
example,Poonet
al.[1994]
use
adiscrim
inantfunctionto
incorporate
regionbased
imagefeaturesinto
theim
ageforces
oftheiractivecontourmodel.Thediscrim
inantfunctionallow
stheinclusionofadditionalim
agefeaturesin
thesegmentationandserves
asaconstraintforglobal
segmentationconsistency
(i.e.everyim
agepixel
contributesto
thediscrim
inantfunction).
Severalresearchers[Leitner
&Cinquin,1991;
Malladiet
al.,1995;
Whitaker,
1994;
Caselles
etal.,1995;Sapiroet
al.,1995;Malladiet
al.,1996;
Caselles
etal.,1997;
Yezziet
al.,1997;
Niessen
etal.,1998;Lachaud&
Montanvert,1999;
McInerney
&Terzopoulos,2000]
havebeendeveloping
topologicallyadaptive
shapemodelingschem
esthatare
notonly
less
sensitive
toinitialconditions,
butalsoallow
adeform
able
contourorsurface
model
torepresentlongtube-likeshapes
orshapes
withbifurcations(Fig.4),andto
dynamicallysense
andchangeitstopology(Fig.5).
Finally,an
other
developmentisasnake-liketechniqueknow
naslive-w
ire
[Falcaoet
al.,1998;
Barrett&
Mortensen,1996-7].
This
semiautomaticboundary
tracingtechniquecomputesand
selectsoptimalbou
ndaries
atinteractiveratesastheusermoves
amouse,startingfrom
auser-
specied
seed
point.
When
themouse
ismoved
close
toanobject
edge,alive-w
ireboundary
snaps
to,andwrapsaroundtheobject
ofinterest.Thelive-w
iremethodhasalsobeencombined
with
snakes,
yieldingasegmentationtoolthatexploitsthebestproperties
ofboth
techniques
[Liang
etal.,2006,1999].
10
-
(a)
(b)
Figure
6:(a)Deform
ableballoonmodelem
bedded
involumeim
agedeform
ingtowardsLVedges
ofcanineheart
(b)ReconstructionofLV.(From
[McInerney
&Terzopoulos,1995a].)
3.2
VolumeIm
ageSegmentationwithDeform
able
Surfaces
Segmenting3D
imagevolumes
slicebysliceusingmanualsliceediting(orim
ageprocessingtech-
niques)is
alaboriousprocess
andrequires
apost-processingstep
toconnectthesequence
of2D
contours
into
acontinuoussurface.Furthermore,theresultingsurface
reconstructioncancontain
inconsistencies
orshow
ringsorbands.
Asdescribed
intheprevioussection,theapplicationofa2D
activecontourmodelto
aninitialsliceandthepropagationofthemodelto
neighboringslices
can
signicantlyim
prove
thevolumesegmentationprocess.How
ever,theuse
ofatrue3D
deform
able
surface
modelcanpotentiallyresultin
even
greaterim
provements
ine
ciency
androbustnessand
italsoensuresthatagloballysm
ooth
andcoherentsurface
isproducedbetweenim
ageslices.
Deform
able
surface
modelsin
3D
wererstusedin
computervision[Terzopouloset
al.,1988].
Manyresearchershavesince
exploredtheuse
ofdeform
ablesurface
modelsforsegmentingstructures
inmedicalim
agevolumes.Miller[1991]
constructsapolygonalapproxim
ationto
asphereand
geometricallydeform
sthisballoonmodeluntiltheballoonsurface
conform
sto
theobject
surface
in3DCTdata.Thesegmentationprocessisform
ulatedastheminim
izationofacostfunctionwhere
thedesired
behavioroftheballoonmodel
isdetermined
byalocalcost
functionassociatedwith
each
modelvertex.Thecostfunctionisaweightedsum
ofthreeterm
s:adeform
ationpotentialthat
expands
themodel
vertices
towardstheobject
bou
ndary,an
imageterm
thatidenties
features
such
asedges
andopposestheballoonexpansion,andaterm
thatmaintainsthetopologyofthe
model
byconstrainingeach
vertex
toremain
close
tothecentroid
ofitsneighbors.
Cohen
andCohen
[1992b
;1993]an
dMcInerney
andTerzopoulos[1995a]use
niteelem
entan
dphysics-basedtechniques
toim
plementan
elasticallydeform
able
cylinder
andsphere,
respectively.
Themodelsare
usedto
segmenttheinner
walloftheleft
ventricle
oftheheart
from
MR
orCT
imagevolumes
(Fig.6).Thesedeform
ablesurfacesare
basedonathin-plate
under
tensionsurface
spline,
thehigher
dim
ensionalgeneralizationofequation(2),
whichcontrols
andconstrainsthe
stretchingandbendingofthesurface.Themodelsare
tted
todata
dynamicallybyintegrating
Lagrangian
equationsofmotion
through
timein
order
toadjust
thedeform
ationaldegrees
of
freedom.Furthermore,theniteelem
entmethodisusedto
representthemodelsasacontinuous
surface
intheform
ofweightedsumsoflocalpolynomialbasisfunctions.
Unlike
Millers[1991]
polygonalmodel,theniteelem
entmethodprovides
ananalyticsurface
representationandtheuse
11
ofhigh-order
polynomialsmeansthatfewer
elem
entsare
required
toaccurately
representanobject.
PentlandandSclaro[1991]
andNastarandAyache[1996]
alsodevelopphysics-basedmodelsbut
use
areducedmodalbasisfortheniteelem
ents
(see
Section3.5).
Staib
andDuncan[1992b
]describea3D
surface
model
usedforgeometricsurface
matchingto
3D
medicalim
agedata.ThemodelusesaFourier
parameterizationwhichdecomposesthesurface
into
aweightedsum
ofsinusoidalbasisfunctions.
Severaldierentsurface
types
are
developed
such
astori,open
surfaces,closedsurfacesandtubes.Surface
ndingisform
ulatedasanoptimization
problem
usinggradientascentwhichattractsthesurface
tostrongim
agegradientsin
thevicinityof
themodel.AnadvantageoftheFourier
parameterizationisthatitallow
sawidevarietyofsm
ooth
surfacesto
bedescribed
withasm
allnumber
ofparameters.
Thatis,aFourier
representation
expresses
afunctionin
term
sofanorthonorm
albasisandhigher
indexed
basisfunctionsin
the
sum
representhigher
spatialvariation.Therefore,theseries
canbetruncatedandstillrepresent
relatively
smooth
objectsaccurately.
Inadierentap
proach,Szeliskiet
al.
[1993]
use
adynamic,self-organizingorientedparticle
system
tomodel
surfacesofobjects.
Theorientedparticles,whichcanbevisualizedassm
all,at
disks,
evolveaccordingto
New
tonianmechanicsandinteract
throughexternalandinterparticle
forces.
Theexternalforces
attract
theparticlesto
thedata
whileinterparticle
forces
attem
pt
togrouptheparticlesinto
acoherentsurface.Theparticlescanreconstruct
objectswithcomplex
shapes
andtopologiesbyow
ingover
thedata,extractingandconform
ingto
meaningfulsurfaces.
Atriangulationisthen
perform
edwhichconnects
theparticlesinto
acontinuousglobalmodelthat
isconsistentwiththeinferred
object
surface.
Wehavegeneralizedthetopologicallyadaptivesnakes
(T-snakes)[M
cInerney
&Terzopoulos,
2000]thatwerecitedearlier(Fig.5),to
higher-dim
ensionalsurfaces.
Know
nasT-surfaces[M
cIn-
erney
&Terzopoulos,1999,1997],thesedeform
able
surface
modelsare
form
ulatedin
term
sofan
AneCellIm
ageDecomposition(A
CID
),whichsignicantlyextendsstandard
deform
ablesurfaces
whileretainingtheirinteractivityan
dother
desirable
properties.In
particular,theACID
induces
ane
cientreparameterizationmechanism
thatenablesparametricdeform
able
surfacesto
evolve
into
complexgeometries
andeven
modifytheirtopologyasnecessary
inorder
tosegmentcomplex
anatomic
structuresfrom
medicalvolumeim
ages.
Other
notable
workinvolving3D
deform
able
surface
modelsandmedicalim
ageapplications
canbefoundin
[Delingette
etal.,1992;
Whitaker,1994;
Tek
&Kim
ia,1997;
Davatzikos&
Bryan,
1995;Neuenschwander
etal.,1997;Caselles
etal.,1997;
Delibasiset
al.,1997;
Xu&
Prince,1998;
Zenget
al.,1998]as
wellasseveralmodelsdescribed
inthefollow
ingsections.
3.3
IncorporatingKnowledge
Inmedicalim
ages,thegeneralshape,locationandorientationofananatomicalstructure
isknow
nandthisknow
ledgemay
beincorporatedinto
thedeform
ablemodelin
theform
ofinitialconditions,
data
constraints,constraints
onthemodel
shapeparameters,orinto
themodel
ttingprocedure.
Theuse
ofim
plicitorexplicitanatomicalknow
ledgeto
guideshaperecovery
isespeciallyim
portant
forrobust
automaticinterpretationofmedicalim
ages.Forautomaticinterpretation,itisessential
tohaveamodelthatnotonly
describes
thesize,shape,locationandorientationofthetarget
object
butthatalsopermitsexpectedvariationsin
thesecharacteristics.
Automaticinterpretationof
medicalim
ages
canrelievecliniciansfrom
thelaborintensive
aspects
oftheirwork
whileincreasing
theaccuracy,consistency,an
dreproducibilityoftheinterpretations.
Inthis
section,andin
the
follow
ingsectionsonmatchingandmotiontracking,wewilldescribeseveraldeform
able
model
techniques
thatincorporate
prioranatomicalknow
ledgein
dierentways.
Anumber
ofresearchershaveincorporatedknow
ledgeofobject
shapeinto
deform
able
models
byusingdeform
able
shapetemplates.
Thesemodelsusuallyuse
hand-craftedglobalshapepa-
12
-
rametersto
embodyaprioriknow
ledgeofexpectedshapean
dshapevariationofthestructuresand
havebeenusedsuccessfullyformanyapplicationsofautomaticim
ageinterpretation.Theidea
of
deform
abletemplatescanbetracedback
totheearlyworkonspringloaded
templatesbyFischler
andElshlager
[1973].Anexcellentexample
incomputervisionistheworkofYuille
etal.
[1992]
whoconstruct
deform
abletemplatesfordetectinganddescribingfeaturesoffaces,such
astheeye.
Inanearlyexamplefrom
medicalim
ageanalysis,Lipsonet
al.[1990]note
thataxialcross
sectional
images
ofthespineyield
approxim
ately
ellipticalvertebralcontours
andconsequentlyextract
the
contours
usingadeform
able
ellipsoidaltemplate.Subsequently,
MontagnatandDelingette
[1997]
usedadeform
able
surface
template
oftheliverto
segmentit
from
abdominalCT
scans,
anda
ventricle
template
tosegmenttheventriclesofthebrain
from
MR
images.
Deform
ablemodelsbasedonsuperquadrics
are
another
exampleofdeform
ableshapetemplates
thatare
gainingin
pop
ularity
inmedicalim
ageresearch.Superquadrics
contain
asm
allnumber
of
intuitiveglobalshapeparametersthatcanbetailoredto
theaverageshapeof
atarget
anatomic
structure.Furthermore,theglobalparameterscanoften
becoupledwithlocalshapeparameters
such
assplines
resultingin
apow
erfulhybridshaperepresentationschem
e.For
example,Metaxas
andTerzopoulos[1993]em
ployadynamicdeform
ablesuperquadricmodel[Terzopoulos&Metaxas,
1991]to
reconstruct
andtrack
humanlimbsfrom
3Dbiokineticdata.Theirpriormodelscandeform
bothlocallyandgloballybyincorporatingtheglobalshapeparametersofasuperellipsoid
withthe
localdegrees
offreedom
ofamem
branesplinein
aLagrangiandynamicsform
ulation.Theglobal
parameterse
cientlycapture
thegross
shapefeaturesofthedata,whilethelocaldeform
ation
parametersreconstruct
thenedetailsofcomplexshapes.UsingKalm
anlteringtheory,they
developanddem
onstrate
abiokineticmotiontracker
basedontheirdeform
ablesuperquadricmodel.
Vem
uri
andRadisavljevic
[1993;
1994]construct
adeform
able
superquadricmodel
inanor-
thonorm
alwavelet
basis.
Thismulti-resolutionbasisprovides
themodelwiththeabilityto
contin-
uouslytransform
from
localto
globalshapedeform
ationstherebyallow
ingacontinuum
ofshape
modelsto
becreatedandto
berepresentedwithrelatively
fewparameters.
They
apply
themodel
tosegmentan
dreconstruct
anatomicalstructuresin
thehumanbrain
from
MRIdata.
Asanalexample,Bardinet
etal.[1996]tadeform
ablesuperquadricto
segmented3Dcardiac
images
andthen
renethesuperquadrictusingavolumetricdeform
ationtechniqueknow
nasfree
form
deform
ations(FFDs).
FFDsare
dened
bytensorproduct
trivariate
splines
andcanbe
visualizedasarubber-likebox
inwhichtheobject
tobedeform
ed(inthiscase
thesuperquadric)
isem
bedded.Deform
ationsofthebox
are
automaticallytransm
ittedto
embedded
objects.
This
volumetricaspectofFFDsallow
stw
osuperquadricsurface
modelsto
besimultaneouslydeform
edin
order
toreconstruct
theinner
andoutersurfacesoftheleftventricleoftheheart
andthevolume
inbetweenthesesurfaces.
Further
examplesofdeform
ablesuperquadrics
canbefoundin
[Pentlan
d&
Horowitz,
1991;Chen
etal.,1994](see
Section3.5).
Further
examplesofFFD-based(orFFD-
like)deform
able
modelsformedicalim
agesegmentationcanbefoundin
[McInerney
&Kikinis,
1998;Lotjonen
etal.,1999].
Severalresearcherscast
thedeform
able
model
ttingprocess
inaprobabilisticframew
ork
(see
Section2.4)an
dincludepriorknow
ledgeofobject
shapebyincorporatingpriorprobabilitydistri-
butionsontheshapevariablesto
beestimated[Vem
uri
&Radisavljevic,1994;Staib
&Duncan,
1992a;
Worringet
al.,1996;Gee,1999].For
example,Staib
andDuncan[1992a]use
adeform
able
contourmodelon2DechocardiogramsandMRim
ages
toextract
theLVoftheheartandthecorpus
callosum
ofthebrain,respectively.This
closedcontourmodel
isparameterized
usinganelliptic
Fou
rier
decompositionandapriorishapeinform
ationisincluded
asaspatialprobabilityexpressed
throughthelikelihoodofeach
model
parameter.Themodel
parameter
probabilitydistributions
are
derived
from
asetofexample
object
bou
ndaries
andserveto
biasthecontourmodel
towards
expectedormore
likely
shapes.
Szekely
etal.[1996]havealso
developed
Fou
rier
parameterized
models.
Furthermore,they
have
13
added
elasticityto
theirmodelsto
create
Fourier
snakes
in2Dandelasticallydeform
ableFourier
surface
modelsin
3D.ByusingtheFourier
parameterizationfollow
edbyastatisticalanalysisof
atrainingset,they
denemeanorganmodelsandtheireigen-deform
ations.
Anelastic
tofthe
meanmodel
inthesubspace
ofeigenmodes
restrictspossible
deform
ationsandndsanoptimal
matchbetweenthemodel
surface
andboundary
candidates.
Cooteset
al.
[1994]
andHillet
al.
[1993]
presentastatisticallybasedtechniqueforbuilding
deform
able
shapetemplatesanduse
thesemodelsto
segmentvariousorgansfrom
2D
and3D
medicalim
ages.Thestatisticalparameterizationprovides
globalshapeconstraints
andallow
sthe
model
todeform
only
inwaysim
plied
bythetrainingset.
Theshapemodelsrepresentobjects
bysets
oflandmark
points
whichare
placedin
thesameway
onanobject
boundary
ineach
inputim
age.
Forexample,to
extract
theLV
from
echocardiograms,
they
choose
points
around
theventricle
bou
ndary,thenearbyedgeoftherightventricle,andthetopoftheleftatrium.The
points
canbeconnectedto
form
adeform
ablecontour.
Byexaminingthestatisticsoftrainingsets
ofhand-labeled
medicalim
ages,andusingprincipalcomponents
analysis(P
CA),
ashapemodel
isderived
thatdescribes
theaveragepositionsandthemajormodes
ofvariationoftheobject
points.New
shapes
are
generatedusingthemeanshapeandaweightedsum
ofthemajormodes
ofvariation.Object
bou
ndaries
are
then
segmentedusingthis
activeshape
model
byexamining
aregionaroundeach
model
pointto
calculate
thedisplacementrequired
tomoveit
towardsthe
bou
ndary.Thesedisplacements
are
then
usedto
update
theshapeparameter
weights.Anexample
oftheuse
ofthistechniqueforsegmentingMRbrain
images
canbefoundin
[Duta
&Sonka,1998].
Anextrem
eexample
ofincorporatingpriorknow
ledge,
aspiringtoward
fullyautomatedmedi-
calim
agesegmentation,isdeform
ableorganisms[M
cInerney
etal.,2002;
Hamarneh
etal.,2001].
Thisrecentparadigm
forautomaticim
ageanalysiscombines
deform
ablemodelsandconcepts
from
articiallife
modeling.Thegoalis
toincorporate
andexploit
alltheavailable
priorknow
ledge
andglobalcontextualinform
ationin
anyspecicmedicalim
ageanalysistask.Analogousto
nat-
uralorganismscapable
ofvoluntary
movem
ent,
deform
able
organismspossessdeform
able
bodies
withdistributedsensors,aswellas(rudim
entary)brainswithmotor,
perception,behavior,
and
cognitioncenters.
Deform
able
organismsare
perceptuallyaw
are
oftheim
ageanalysisprocess.
Theirbehaviors,whichmanifestthem
selves
involuntary
movem
entandbodyshapealteration,
are
baseduponsensedim
agefeatures,
storedstructuralknow
ledge,
andacognitiveplan.
The
organism
framew
ork
separatesglobaltop-dow
n,model-ttingcontrolfunctionality
from
thelocal,
bottom-up,feature
integrationfunctionality.Thisseparationenablesthedenitionofmodel-tting
controllersorbrainsin
term
softhehigh-levelstructuralfeaturesofobjectsofinterest,rather
than
thelow-level
imagefeatures.
Theresultisanintelligentagentthatiscontinuouslyawareofthe
progress
ofthesegmentationprocess,allow
ingitto
apply
priorknow
ledgeabouttarget
objectsin
adeliberative
manner
(Fig.7).3Dphysics-baseddeform
ableorganismshaverecentlybeendeveloped
[McIntosh
&Hamarneh,2006b]an
dsoftware
isavailable
[McIntosh
&Hamarneh,2006a].
3.4
Matching
Matchingofregionsin
images
canbeperform
edbetweentherepresentationofaregionandamodel
(labeling)orbetweentherepresentationoftw
odistinct
regions(registration).
Nonrigid
registration
of2D
and3D
medicalim
ages
isnecessary
inorder
tostudytheevolutionofapathologyin
an
individual,orto
take
fulladvantageofthecomplementary
inform
ationcomingfrom
multim
odality
imagery[M
aintz
&Viergever,1998;Goshtasbyet
al.,2003].Examplesoftheuse
ofdeform
able
modelsto
perform
medicalim
ageregistrationare
foundin
[Moshfeghi,1991;
Moshfeghiet
al.,
1994;Gueziec&
Ayache,
1994;Feldmar&
Ayache,
1994;
Bookstein,1989;
Hamadeh
etal.,1995;
Lavallee&
Szeliski,1995;Thirion,1994].Thesetechniques
primarily
consist
ofconstructinghighly
structureddescriptionsformatching.This
operationis
usuallycarriedoutbyextractingregions
14
-
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
Figure
7:Automaticbrain
MRim
agesegmentationbymultipledeform
ableorganisms(from
[McIn-
erney
etal.,2002]).Thesequence
ofim
ages
illustratesthetemporalprogressionofthesegmentation
process.Deform
able
lateralventricle
(17),caudate
nucleus(810),andputamen
(1116)organ-
ismsare
spaw
ned
insuccessionandprogress
throughaseries
ofbehaviors
todetect,localize,and
segmentthecorrespondingstructuresin
theMR
image.
15
ofinterest
withanedgedetectionalgorithm,follow
edbytheextractionoflandmark
points
or
characteristiccontours
(orcurves
onextracted
boundary
surfacesin
thecase
of3D
data).
In3D,
thesecurves
usuallydescribedierentialstructuressuch
asridges,ortopologicalsingularities.An
elastic
matchingalgorithm
canthen
beapplied
betweencorrespondingpairsofcurves
orcontours
wherethestartcontourisiteratively
deform
edto
thegoal
contourusingforces
derived
from
localpatternmatches
withthegoalcontour[M
oshfeghi,1991].
Anexample
ofmatchingwheretheuse
ofexplicitpriorknow
ledgehasbeenem
bedded
into
deform
able
modelsis
theautomaticextractionandlabelingofanatomic
structuresin
thebrain
from
MR
images,ortheregistrationofmultim
odality
brain
images.Theanatomicalknow
ledge
ismadeexplicitin
theform
ofa3D
brain
atlas.
Theatlasis
modeled
asaphysicalobject
and
isgiven
elastic
properties.After
aninitialglobalalignment,theatlasdeform
sandmatches
itself
onto
correspondingregionsin
thebrain
imagevolumein
response
toforces
derived
from
image
features.
Theassumptionunderlyingthisapproach
isthatatsomerepresentationallevel,norm
al
brainshavethesametopologicalstructure
anddier
only
inshapedetails.
Theidea
ofmodeling
theatlasasanelasticobject
was
originatedbyBroit[1981],whoform
ulatedthematchingprocess
asaminim
izationofacost
function.
Subsequently,
Bajcsy
andKovacic[1989]
implementeda
multiresolutionversionofBroitssystem
wherethedeform
ationoftheatlasproceedsstep-by-step
inacoarseto
nestrategy,increasingthelocalsimilarity
andglobalcoherence.Anearlierwork
with
similarobjectives,albeitnotapplied
tomedicalim
ageanalysis,wasthatbyWitkin
etal.[W
itkin
etal.,1987],
whoform
ulatedthegeneralnonrigid
registrationproblemi.e.,thatofmatching
anynumber
ofarbitrary-dim
ensionalim
ages
orsignalsasoneofminim
izinganonconvex
energy
functionalcombiningacontrolled-continuitygeneralizedsplinedeform
ationenergywithnorm
alized
cross-correlationande
cientlysolved
itusinganinnovative
scale-space
continuationmethod.The
elasticallydeform
ableatlastechniqueisvery
promisingandconsequentlyhasbecomeavery
active
areaofresearchthatisbeingexploredbyseveralgroups[Evanset
al.,1991;
Gee,1999;
Sandor&
Leahy,1995;Christensenet
al.,1995;Bookstein,1991;
Bozm
a&
Duncan,1992;
Declercket
al.,
1995;McD
onald
etal.,1994;Delibasis&
Undrill,1994;
Subsolet
al.,1995;
Davatzikoset
al.,1996;
Snellet
al.,1995;Thompson&
Toga,
1996-7;Vaillant&
Davatzikos,1997;
Wang&
Staib,1998].
Theautomaticbrain
imagematchingproblem
isextrem
elychallengingandthereare
many
hurdlesthatmust
beovercomebefore
thedeform
able
atlastechniquecanbeadoptedforclinical
use.For
example,thetechniqueis
sensitive
totheinitialpositioningoftheatlas
iftheinitial
rigid
alignmentis
obytoomuch,then
theelastic
matchmay
perform
poorly.
Thepresence
ofneighboringfeaturesmay
alsocause
matchingproblems
theatlasmay
warp
toanincorrect
bou
ndary.Finally,withoutuserinteraction,theatlascanhavedi
cultyconvergingto
complicated
object
bou
ndaries.A
proposedsolutionto
theseproblemsis
touse
imagepreprocessingin
con-
junctionwiththedeform
able
atlas.
SandorandLeahy[1995]
use
thisapproach
toautomatically
label
regionsofthecorticalsurface
thatappearin
3D
MR
images
ofhumanbrains(Fig.8).
They
automaticallymatchalabeled
deform
able
atlasmodel
topreprocessed
brain
images,wherepre-
processingconsistsof3D
edgedetectionandmorphologicaloperations.
Theselteringoperations
automaticallyextract
thebrain
andsulci(deepgrooves
inthecorticalsurface)from
anMR
image
andprovideasm
oothed
representationofthebrain
surface
towhichtheir3D
B-splinedeform
able
surface
model
canrapidly
converge.
3.5
MotionTrackingandAnalysis
Theidea
oftrackingobjectsin
time-varyingim
ages
usingdeform
ablemodelswasoriginallyproposed
inthecontextofcomputervision[K
ass
etal.,1988;
Terzopouloset
al.,1988].Deform
able
models
havebeenusedto
track
nonrigid
microscopicandmacroscopicstructuresin
motion,such
asblood
cells[Leymarie&
Levine,1993]an
dneurite
growth
cones
[Gwydiret
al.,1994]
incine-microscopy,
16
-
Figure
8:Theresultofmatchingalabeled
deform
ableatlasto
amorphologicallypreprocessed
MR
imageofthebrain
(from
[Sandor&
Leahy,1995]).
aswellascoronary
arteriesin
cine-angiography[Lengyelet
al.,1995].
How
ever,theprimary
use
ofdeform
ablemodelsfortrackingin
medicalim
ageanalysisisto
measure
thedynamicbehaviorof
thehumanheart,especiallytheleftventricle.Regionalcharacterizationoftheheart
wallmotionis
necessary
toisolate
theseverity
andextentof
diseasessuch
asischem
ia.Magnetic
resonance
and
other
imagingtechnologiescannow
providetimevaryingthreedim
ensionalim
ages
oftheheart
withexcellentspatialresolutionandreasonable
temporalresolutions.
Deform
able
modelsare
well
suited
forthisim
ageanalysistask.
Inthesimplest
approach,a2D
deform
ablecontourmodelisusedto
segmenttheLV
boundary
ineach
sliceofaninitialim
agevolume.
Thesecontours
are
then
usedastheinitialapproxim
ation
oftheLV
boundaries
incorrespondingslices
oftheim
agevolumeatthenexttimeinstantandare
then
deform
edto
extract
thenew
setofLV
boundaries
[Singhet
al.,1993;Ueda&
Mase,1992;
Ayacheet
al.,1992;Herlin&
Ayache,1992;Geiger
etal.,1995].Thistemporalpropagationofthe
deform
able
contours
dramaticallydecreasesthetimetakento
segmenttheLV
from
asequence
of
imagevolumes
over
acardiaccycle.
Singhet
al.[1993]
report
atimeof15minutesto
perform
the
segmentation,considerably
less
thanthe1.5-2
hours
thatahumanexperttakesformanualseg-
mentation.Deform
ablecontourmodelshavealsobeensuccessfullyusedto
track
theLV
bou
ndary
innoisyechocardiographic
imagesequences[Jacobet
al.,1999;Chalanaet
al.,1996].
McInerney
andTerzopoulos[1995a]haveap
plied
thetemporalpropagationapproach
in3D
using
a3D
dynamic
deform
able
balloonmodel
totrack
thecontractilemotionoftheLV
(Fig.9,10).
Inamore
involved
approach,AminiandDuncan[1992]use
bendingenergyandsurface
curvature
totrack
andanalyze
LV
motion.For
each
timeinstant,
twosparsesubsets
ofsurface
points
are
createdbychoosinggeometricallysignicantlandmark
points,onefortheendocardialsurface,and
theother
fortheepicardialsurface
oftheLV.Surface
patches
surroundingthesepoints
are
then
modeled
asthin,exible
plates.
Makingtheassumptionthateach
surface
patchdeform
sonly
slightlyandlocallywithin
asm
alltimeinterval,foreach
sampledpointontherstsurface
they
construct
asearchareaontheLV
surface
intheim
agevolumeatthenexttimeinstant.
Thebest
matched
(i.e.minim
um
bendingenergy)pointwithin
thesearchwindow
onthesecondsurface
istakento
correspondto
thepointontherstsurface.Thismatchingprocess
yieldsasetofinitial
17
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
Figure
9:Sagittalsliceofsuccessive
CT
volumes
over
onecardiaccycle(116)show
ingmotionof
canineLV
(from
[McInerney
&Terzopoulos,1995a]).
motionvectors
forpairsofLV
surfacesderived
from
a3D
imagesequence.A
smoothingprocedure
isthen
perform
edusingtheinitialmotionvectors
togenerate
adense
motionvectoreldover
the
LV
surfaces.
Cohen
etal.[1992a]also
employabendingenergytechniquein
2D
andattem
ptto
improve
on
thismethodbyaddingaterm
tothebendingenergyfunctionthattendsto
preservethematching
ofhighcurvature
points.Goldgofet
al.
[Goldgofet
al.,1988;
Kambhamettu
&Goldgof,1994;
Huang&
Goldgof,1993;Mishra
etal.,1991]
havealsobeenpursuingsurface
shapematchingideas
primarily
basedonchanges
inGaussiancurvature
andassumeaconform
almotionmodel
(i.e.
motionwhichpreserves
anglesbetweencurves
onasurface
butnotdistances).
Analternative
approach
isthatofChen
etal.
[1994],whouse
ahierarchicalmotionmodel
oftheLV
constructed
bycombiningagloballydeform
able
superquadricwithalocallydeform
able
surface
usingsphericalharm
onic
shapemodelingprimitives.Usingthismodel,they
estimate
the
LVmotionfrom
angiographicdata
andproduce
ahierarchicaldecompositionthatcharacterizes
the
LV
motionin
acoarse-to-nefashion.
PentlandandHorowitz[1991]an
dNastarandAyache[1996]are
alsoableto
produce
acoarse-to-
18
-
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
Figure
10:TrackingoftheLV
motionofcanineheart
duringonecardiaccycle(116)usingde-
form
able
balloonmodel
(from
[McInerney
&Terzopoulos,1995a]).
19
necharacterizationoftheLV
motion.They
use
dynamicdeform
ablemodelsto
track
andrecover
theLV
motionandmake
use
ofmodalanalysis,
awell-know
nmechanicalengineeringtechnique,
toparameterizetheirmodels.
Thisparameterizationisobtained
from
theeigenvectors
ofanite
elem
entform
ulationofthemodels.
Theseeigenvectors
are
often
referred
toasthefree
vibration
modes
andvariable
detailofLV
motionrepresentationresultsfrom
varyingthenumber
ofmodes
used. Theheart
isarelatively
smooth
organandconsequentlythereare
fewreliablelandmark
points.
Theheart
alsoundergoes
complexnonrigid
motionthatincludes
atw
isting(tangential)component
aswellasthenorm
alcomponentofmotion.Themotionrecoverymethodsdescribed
aboveare,
ingeneral,notable
tocapture
this
tangentialmotionwithoutadditionalinform
ation.Magnetic
resonance
techniques,basedonmagnetictagging[Axel&
Dougherty,1989]
havebeendeveloped
totrack
materialpoints
onthemyocardium
inanon-invasiveway.Thetemporalcorrespondence
of
materialpoints
thatthesetechniques
provideallow
forquantitative
measurementoftissuemotion
anddeform
ationincludingthetw
istingcomponentoftheLV
motion.
Severalresearchershave
applied
deform
able
modelsto
imagesequencesofMR
tagged
data
[Younget
al.,1993,
1995;
Park
etal.,1996;Duncanet
al.,1994;Kumar&
Goldgof,1994;
Kraitchmanet
al.,1995;
Aminiet
al.,
1998].
For
example,Aminiet.al[1998]
andKumarandGoldgof[1994]
use
a2D
deformab
legrid
tolocalize
andtrack
SPAMM
(SpatialModulationofMagnetization)tagpoints
ontheLV
tissue.
Parket
al.[1995;
1996]tavolumetricphysics-baseddeform
able
model
toMRI-SPAMM
data
of
theLV.Theparametersofthemodelare
functionswhichcancapture
regionalshapevariationsof
theLVsuch
asbending,tw
isting,andcontraction.Basedonthismodel,theauthors
quantitatively
compare
norm
alheartsandheartswithhypertrophic
cardiomyopathy.
Another
problem
withmost
ofthemethodsdescribed
aboveisthatthey
modeltheendocardial
andepicardialsurfacesoftheLVseparately.In
reality
theheartisathick-walled
structure.Duncan
etal.[1994]an
dParket
al.[1995;1996]developmodelswhichconsider
thevolumetricnature
ofthe
heart
wall.Thesemodelsuse
theshapeproperties
oftheendocardialandepicardialsurfacesand
incorporate
mid-walldisplacementinform
ationoftagged
MR
images.Byconstructing3D
nite
elem
entmodelsoftheLV
withnodes
inthemid-wallregionaswellasnodes
ontheendocardial
andepicardialsurfaces,
more
accurate
measurements
oftheLV
motioncanbeobtained.Young
andAxel[1992;
1995],an
dCresw
ell[1992]
havealsoconstructed
3D
niteelem
entmodelsfrom
the
bou
ndary
representationsoftheendocardialandepicardialsurfaces.
4Discussion
Intheprevioussectionswehavesurveyed
theconsiderableandstillrapidly
expandingbodyofwork
ondeform
able
modelsin
medicalim
ageanalysis.
Thesurvey
hasrevealedseveralissues
thatare
relevantto
thecontinued
developmentofthedeform
ablemodelapproach.Thissectionsummarizes
keyissues
andindicatessomepromisingresearchdirections.
4.1
AutonomyvsControl
Interactive(sem
i-automatic)
algorithmsandfullyautomaticalgorithmsrepresenttw
oalternative
approaches
tocomputerizedmedicalim
ageanalysis.
Certainly
automaticinterpretationofmedi-
calim
ages
isadesirable,albeitvery
di
cult,long-term
goal,since
itcanpotentiallyincrease
the
speed,accuracy,consistency,an
dreproducibilityoftheanalysis.
How
ever,theinteractiveorsemi-
automaticmethodologyislikely
toremain
dominantin
practiceforsometimeto
come,
especially
inapplicationswhereerroneousinterpretationsare
unacceptable.Consequently,themost
immedi-
ately
successfuldeform
ablemodelbasedtechniques
willlikely
bethose
thatdrasticallydecrease
the
laborintensivenessofmedicalim
ageprocessingtasksthroughpartialautomationandsignicantly
20
-
increase
theirreproducibility,whilestillallow
ingforinteractiveguidance
oreditingbythemedical
expert.
Althoughfullyautomatictechniques
basedondeform
ablemodelswilllikely
notreach
their
fullpotentialforsometimeto
come,they
canbeofim
mediate
valuein
specicapplicationdomains
such
asthesegmentationofhealthytissuesurroundingapathologyforenhancedvisualization.
4.2
Generality
vsSpecicity
Ideallyadeform
ablemodelshould
becapableofrepresentingabroadrangeofshapes
andbeuseful
inawidearray
ofmedicalapplications.
Generality
isthebasisofdeform
able
model
form
ulations
withlocalshapeparameterssuch
assnakes.
Alternatively,highly
specic,
hand-craftedorcon-
strained
deform
ablemodelsappearto
beusefulin
applicationssuch
astrackingthenonrigid
motion
oftheheart(Section3.5),automaticallymatchingandlabelingstructuresin
thebrain
from
3DMR
images
(Section3.4),orsegmentingvery
noisyim
ages
such
asechocardiograms.
Certainly
attem
pts
tocompletely
automate
theprocessingofmedicalim
ages
would
requireahighdegreeofapplication
andmodelspecicity.A
promisingdirectionforfuture
studyappears
tobetechniques
forlearning
tailoredmodelsfrom
simple
generalpurpose
models.
Thework
ofCooteset
al.
[1994]
may
be
viewed
asanexample
ofsuch
astrategy.
4.3
Compactness
vsGeometric
CoveragevsTopologicalFlexibility
Ageometricmodel
ofshapemay
beevaluatedbasedontheparsim
onyof
itsform
ulation,its
representationalpow
er,anditstopologicalexibility.
Generally,
parameterized
modelsoer
the
greatest
parsim
ony,
free-form
(spline)
modelsfeature
thebroadestcoverage,
andim
plicitmodels
havethegreatest
topologicalexibility.
Deform
able
modelshavebeendeveloped
basedoneach
ofthesegeometricclasses.
Increasingly,researchersare
turningto
thedevelopmentofhybrid
deform
ablemodelsthatcombinecomplementary
features.
Forobjectswithasimple,xed
topology
andwithoutsignicantprotrusions,parameterized
modelscoupledwithlocal(spline)
and/orglobal
deform
ationsschem
esappearto
provideagoodcompactness-descriptivenesstradeo[Terzopoulos
&Metaxas,
1991;Pentland&
Horowitz,
1991;Vem
uri
&Radisavljevic,1994;Chen
etal.,1994].
Ontheother
hand,thesegmentationandmodelingofcomplex,multipart
objectssuch
asarterial
orbronchialtreestructures,ortopologicallycomplexstructuressuch
asvertebrae,
isadi
cult
task
withthesetypes
ofmodels[M
cInerney
&Terzopoulos,1999].Polygonbasedorparticlebased
deform
able
modelingschem
esseem
promisingin
segmentingandreconstructingsuch
structures.
Polygonbasedmodelsmay
becompacted
byremovingandretiling[Turk,1992;Gourdon,1995;
Delingette,1997]polygonsin
regionsoflow
shapevariation,orbyreplacingaregionofpolygons
withasingle,high-order
niteelem
entor
splinepatch[Q
inet
al.,1998].
Apossible
research
directionis
todevelopalternative
modelsthatblendorcombinedescriptive
primitiveelem
ents
(rather
thansimple
particles),such
asexible
cylinders,into
aglobalstructure.
4.4
CurvevsSurface
vsSolidModels
Theearliest
deform
ablemodelswerecurves
andsurfaces.
Anatomicstructuresin
thehumanbody,
how
ever,are
either
solidorthick-walled.Tosupport
theexpandingrole
ofmedicalim
ages
into
taskssuch
assurgicalplanningandsimulation,andthefunctionalmodelingofstructuressuch
asbon
es,muscles,
skin,orarterialbloodow
,may
requirevolumetricorsoliddeform
able
models
rather
thansurface
models.
For
example,theplanningoffacialreconstructivesurgeryrequires
the
extractionandreconstructionoftheskin,muscles,andbones
from
3D
images
usingaccurate
solid
models.
Italsorequires
theabilityto
simulate
themovem
entandinteractionsofthesestructuresin
response
toforces,theabilityto
move,cutandfuse
piecesofthemodelin
arealisticfashion,andthe
abilityto
stim
ulate
thesimulatedmusclesofthemodelto
predicttheeectofthesurgery.
Several
21
researchershavebegunto
explore
theuse
ofvolumetricorsoliddeform
able
modelsofthehuman
face
andheadforcomputergraphicsapplications[Lee
etal.,1995;Essaet
al.,1993]andformedical
applications[W
aters,1992;Delingette
etal.,1994;
Pieper
etal.,1992;
Geiger,1992;
Keeve
etal.,
1998;Gibsonet
al.,1998;Wells
etal.,1998],particularlyreconstructivesurgerysimulation,and
thereismuch
room
forfurther
research.Researchershavealsobegunto
use
volumetricdeform
able
modelsto
more
accurately
track
andanalyze
LVmotion[Young&
Axel,1992;
Cresw
ellet
al.,1992;
Duncanet
al.,1994;Parket
al.,1996].
4.5
Accuracy
andQuantitativePower
Ideallyitshould
bepossibleto
measure
andcontroltheaccuracy
ofadeform
ablemodel.Themost
commonaccuracy
controlmechanismsare
theglobalorlocalsubdivisionofmodel
basisfunctions
[Milleret
al.,1991],ortherepositioningofmodel
points
toincrease
theirdensity
inregionsofthe
data
exhibitingrapid
shapevariations[Vasilescu&
Terzopoulos,
1992].Other
mechanismsthat
warrantfurther
researcharethelocalcontrolandadaptationofmodelcontinuity,parameter
evolu-
tion(includingtherate
andschedulingoftheevolution),andtheautomationofallaccuracy
control
mechanisms.
Theparametricform
ulationofadeform
ablemodelshould
notonly
yield
anaccurate
descriptionoftheobject,butitshould
alsoprovidequantitative
inform
ationabouttheobject
inanintuitive,
convenientform
.Thatis,themodel
parametersshould
beusefulforoperationssuch
asmeasuring,matching,modication,rendering,andhigher-level
analysisorgeometricreasoning.
Thisparameter
descriptiveness
criterionmay
beachievedin
apostprocessingstep
byadaptingor
optimizingtheparameterizationto
more
ecientlyormore
descriptively
matchthedata.How
ever,
itispreferableto
incorporate
thedescriptiveparameterizationdirectlyinto
themodelform
ulation.
Anexample
ofthisstrategyisthedeform
able
model
ofPark
etal.[1996].
4.6
Robustness
Ideally,
adeform
able
model
should
beinsensitive
toinitialconditionsandnoisydata.Deform
able
modelsare
ableto
exploitmultipleim
ageattributesandhighlevelorglobalinform
ationto
increase
therobustnessofshaperecovery.Forexample,manysnakes
modelsnow
incorporate
regionbased
imagefeaturesaswellasthetraditionaledgebasedfeatures(Section3.1).
Strategiesworthyof
further
researchincludetheincorporationofshapeconstraints
into
thedeform
ablemodelthatare
derived
from
lowlevelim
ageprocessingoperationssuch
asthinning,medialaxistransform
s[Fritsch
etal.,1997],ormathem
aticalmorphology.
Aclassicalapproach
toim
provetherobustnessofmodel
ttingistheuse
ofmultiscaleim
agepreprocessingtechniques
[Kass
etal.,1988;
Terzopouloset
al.,
1988],
perhapscoupledwithamultiresolutiondeform
able
model
[Bajcsy
&Kovacic,
1989].
Amultiresolutiontechniquethatmeritsfurther
researchin
thecontextofdeform
able
models,isthe
use
ofwavelet
bases[Strang&
Nguyen,1996]
fordeform
ations[Vem
uri
etal.,1993;
Vem
uri
&Radisavljevic,1994].
Adeform
able
model
should
beable
toeasily
incorporate
added
constraints
andanyother
apriori
anatomic
know
ledgeofobject
shapeandmotion.
Section3.3
reviewed
severalofthemost
promisingtechniques
toincorporate
apriori
know
ledge.
Forexample,forLV
motiontracking,apromisingresearchdirectionistheincorporationofbiomechanicalproperties
of
theheart
andtheinclusionofthetemporalperiodic
characteristics
oftheheart
motion.Future
directionsincludemodelingschem
esthatincorporate
reasoningandrecognitionmechanismsusing
techniques
from
articialintelligence,such
asrule-basedsystem
sorneuralnetworks,andtechniques
from
articiallife
[McInerney
etal.,2002].
22
-
4.7
LagrangianvsEulerianDeform
able
Models
Analternative
totheLagrangianform
ulationofdeform
able
modelsuponwhichthis
survey
has
focused,is
theEulerianform
ulation.Thelatter
leadsto
socalled
levelsetmethods,
whichhave
attracted
much
attentionin
themedicalim
ageanalysisliterature.Coveringthenow
voluminous
literature
isbeyondthescopeof
thissurvey,butwepointthereader
totherecentvolumebyOsher
andParagios[2003]
andrecentsurvey
articles[Suriet
al.,2002;Tsai&
Osher,2003;Cremerset
al.,
2007]forrelevantbackgroundmaterialandreferences.
Theim
portantpointis
thatthetw
oap-
proaches
are
complementary
inprecisely
thesamesense
thatLagrangiansolidmodelscomplement
Eulerianuid
modelsin
continuum
mechanicsandthatparametricandim
plicitgeometricmodels
complementoneanother
incomputer-aided
geometricdesign.Thesubstantialmedicalim
ageanal-
ysisliterature
onLagrangianandEuleriandeform
able
modelsisatestamentto
thefact
thateach
approach
isusefulin
particularapplications.
TheinitialmotivationfortheEulerianform
ulationofdeform
ablemodelswasto
introduce
topo-
logicalexibilityinto
themodel-basedim
agesegmentationproblem
throughtheadaptationofthe
Osher-Sethianlevelsetevolutiontechnique[Caselles
etal.,1993;Malladiet
al.,1995;Whitaker,
1994;Caselles
etal.,1995;Sapiroet
al.,1995].
Formulatedasevolvingcontours
(surfacesin
3D)
orfrontswhichdenethelevelsetofsomehigher-dim
ensional(hyper-)
surface
over
theim
-agedomain,themain
feature
ofthis
approach
isthattopologicalchanges
are
handlednaturally,
since
thelevelsetneednotbesimply
connected;thehigher-dim
ensionalsurface
remainsasimple
functioneven
asthelevelsetchanges
topology.
Whilethelevelsettechniqueisanattractivemath-
ematicalframew
ork,partialdierentialequationsgoverningcurvature-dependentfrontevolution,
implicitform
ulationsare
generallynotasconvenientas
theexplicit,parametricform
ulationswhen
itcomes
toincorporatingadditionalcontrolmechanismsincludinginternaldeform
ationenergies
andexternalinteractiveguidance
byexpertusers.Furthermore,thehigher-dim
ensionalim
plicit
surface
form
ulationmakesitdi
cultto
impose
arbitrary
geometricortopologicalconstraints
on
thelevelsetindirectlythroughthehigher
dim
ensionalrepresentation.Therefore,theim
plicitfor-
mulationmay
potentiallylimittheease
ofuse,e
ciency
anddegreeofautomationachievable
inthesegmentationtask.
Amongnew
erapproaches
thataddress
thesedi
cultiesare
T-snakesandT-surfaces[M
cInerney
&Terzopoulos,
2000,1999],whichcanbeviewed
ashybridmodelsthatcombineaspects
ofthe
LagrangianandEulerianapproaches
(withtheACID
introducingaspects
ofthelatter
toinduce
topologicalexibility).
Arecent,
purely
Lagrangianapproach
thatmaintainstheadvantages
of
levelsetmethodswhileavoidingtheirdrawbacksis
theDelaunay
Deform
able
Models[Pon
s&
Boissonnat,
2007].
This
reference
provides
additionaldiscussionoftheshortcomingsofthelevel
setmethodan
dcitesseveralother
recentalternativesto
theT-snakesapproach.
5Conclusion
Theincreasingly
importantrole
ofmedicalim
agingin
thediagnosisandtreatm
entofdisease
has
opened
anarray
ofchallengingproblemscenteredonthecomputationofaccurate
geometricmod-
elsofanatomic
structuresfrom
medicalim
ages.Deform
able
modelsoer
anattractiveapproach
totacklingsuch
problems,
because
thesemodelsare
able
torepresentthecomplexshapes
and
broadshapevariabilityof
anatomicalstructures.
Deform
able
modelsovercomemanyofthelimi-
tationsoftraditionallow-level
imageprocessingtechniques,byprovidingcompact
andanalytical
representationsofobject
shape,
byincorporatinganatomic
know
ledge,
andbyprovidinginterac-
tive
capabilities.
Thecontinued
developmentan
drenem
entofthesemodelsshould
remain
an
importantarea
ofresearchinto
thefores