animated solid model of the lung constructed from unsynchronized

This article appeared in a journal published by Elsevier. The attachedcopy is furnished to the author for internal non-commercial researchand education use, including for instruction at the authors institution

and sharing with colleagues.

Other uses, including reproduction and distribution, or selling orlicensing copies, or posting to personal, institutional or third party

websites are prohibited.

In most cases authors are permitted to post their version of thearticle (e.g. in Word or Tex form) to their personal website orinstitutional repository. Authors requiring further information

regarding Elsevier’s archiving and manuscript policies areencouraged to visit:

http://www.elsevier.com/copyright

http://www.elsevier.com/copyright

Author's personal copy

Computer-Aided Design 41 (2009) 573–585www.elsevier.com/locate/cad

Animated solid model of the lung constructed from unsynchronized MRsequential images

M.S.G. Tsuzukia,∗,1, F.K. Takasea, T. Gotohb, S. Kageib, A. Asakurab, T. Iwasawad, T. Inouec

a University of Sao Paulo, Sao Paulo, Brazilb Yokohama National University, Yokohama, Japan

c Yokohama City University, Yokohama, Japand Kanagawa Cardiovascular and Respiratory Centre, Yokohama, Japan

Received 30 January 2006; accepted 1 October 2007

Abstract

This work discusses a 4D lung reconstruction method from unsynchronized MR sequential images. The lung, differently from the heart, doesnot have its own muscles, turning impossible to see its real movements. The visualization of the lung in motion is an actual topic of research inmedicine. CT (Computerized Tomography) can obtain spatio-temporal images of the heart by synchronizing with electrocardiographic waves. TheFOV of the heart is small when compared to the lung’s FOV. The lung’s movement is not periodic and is susceptible to variations in the degree ofrespiration. Compared to CT, MR (Magnetic Resonance) imaging involves longer acquisition times and it is not possible to obtain instantaneous3D images of the lung. For each slice, only one temporal sequence of 2D images can be obtained. However, methods using MR are preferablebecause they do not involve radiation. In this paper, based on unsynchronized MR images of the lung an animated B-Rep solid model of the lung iscreated. The 3D animation represents the lung’s motion associated to one selected sequence of MR images. The proposed method can be dividedin two parts. First, the lung’s silhouettes moving in time are extracted by detecting the presence of a respiratory pattern on 2D spatio-temporal MRimages. This approach enables us to determine the lung’s silhouette for every frame, even on frames with obscure edges. The sequence of extractedlung’s silhouettes are unsynchronized sagittal and coronal silhouettes. Using our algorithm it is possible to reconstruct a 3D lung starting from asilhouette of any type (coronal or sagittal) selected from any instant in time. A wire–frame model of the lung is created by composing coronal andsagittal planar silhouettes representing cross-sections. The silhouette composition is severely underconstrained. Many wire–frame models can becreated from the observed sequences of silhouettes in time. Finally, a B-Rep solid model is created using a meshing algorithm. Using the B-Repsolid model the volume in time for the right and left lungs were calculated. It was possible to recognize several characteristics of the 3D real rightand left lungs in the shaded model.c© 2007 Elsevier Ltd. All rights reserved.

Keywords: 4D MRI; Animated lung visualization; Lung’s volume in time

1. Introduction

MR imaging of the lung is limited by the low signal-to-noiseratio (SNR) due to the low proton density of the air [9]. Thelow SNR demands longer acquisition time for each image andshortening the acquisition time yields lower spatial resolution.

∗ Corresponding address: Escola Politecnica da Universidade de SaoPaulo, Department of Mechatronics and Mechanical Systems Engineering,Laboratory of Computational Geometry, Av. Prof. Mello Moraes, 2231, CidadeUniversitaria, Sao Paulo, SP, Brazil.

E-mail address: [email protected] (M.S.G. Tsuzuki).1 Partially supported by CNPq.

Thus, recent lung MR imaging sequence developments havefocused on the functional aspects of the lung, such as ventilationand perfusion, and on lesion characterization, all of which canbe evaluated predominantly at MR imaging with low spatialresolution [14]. The knowledge of the lung volume is ofspecial interest and approaches to obtain this knowledge wereproposed. One approach to evaluate the lung volume is relatedto the observation of the diaphragm. Cluzel et al. [3] managedto obtain a 3D MRI of a breath holding lung shortening theexposition time for each image. The image sequence was usedto measure the diaphragmatic area as well as the relationshipbetween the diaphragmatic area and the lung volume. Gauthieret al. [6] evaluated not only the area of the diaphragm but

0010-4485/$ - see front matter c© 2007 Elsevier Ltd. All rights reserved.doi:10.1016/j.cad.2007.10.001


574 M.S.G. Tsuzuki et al. / Computer-Aided Design 41 (2009) 573–585

(a) Sagittal image. (b) Coronal image.

(c) Coronal image. (d) Coronal image.

Fig. 1. Samples of MR images. Figure (b) shows a coronal image that intersects the ascending aorta. Figures (c) and (d) show different instants in time of the sameslice.

also the shape of the diaphragm at different lung volumesand Gierada et al. [7] considered the diaphragmatic motionusing sequential MR images taken during quiet breathing.Iwasawa et al. [8] quantitatively evaluated the diaphragmaticmotion using MR imaging from the displacement area and thetotal diaphragmatic movement in a respiratory cycle. Anotherapproach tried to relate the lung volume to the chest wall motionretrieved from MRI sequences [3,6,7,19].

Lung deformations have been studied for the verification ofmedical imaging equipment and for medical training purposes.The initial methods to model the lung deformation were basedon physiology and clinical measurements. The deformationof the lung model as a linear model was proposed byPromayon [15]. A nonphysically-based method to describe lungdeformations was proposed using NURBS surfaces based onimaging data from CT scans of actual patients [20]. Zordanet al. created an anatomical inspired, physically-based modelof human torso for the visual simulation of respiration [22].To obtain 3D information about the lung from MRI images theboundaries of the lung in each image should be retrieved. Rayet al. [16] proposed the segmentation of the image by mergingparametric contours within homogeneous image regions. Inthis work a 4D lung model is retrieved from a sequence of

MR images taken from a breathing lung. This is accomplishedthrough the analysis of the respiratory motion of the lungboundaries. DeCarlo et al. [5] examined the Finite ElementMethod implementation of a two-dimensional idealized lung.

2. Background

Fig. 1(a) shows an example of MR sagittal image. It ispossible to process the image shown in Fig. 1(a) using themethods present in the literature, the lung’s silhouette is veryclear and the heart is not present. Fig. 1(b) shows a coronalimage that intersects the ascending aorta, where it is possible toextract the silhouette of the lung automatically using a specificknowledge. Fig. 1(c) shows an example of a blurred image ofthe chest, where bronchi and large vessels are present. It is verydifficult to determine the silhouette of the lung in this case. Theknowledge of a medical doctor is necessary in the definition ofthe lung’s silhouette. Fig. 1(c) and (d) show different instantsin time of the same slice, the number of vessels inside the lungcan vary very much in time, even for the same slice [12].

The lung motion is caused by the following mechanism. Theintercostal muscles around the rib-cage move forward, and thediaphragm, the muscle beneath the lungs, moves downward


M.S.G. Tsuzuki et al. / Computer-Aided Design 41 (2009) 573–585 575

(a) Chest’s MR image and a vertical 2DSTI. (b) A horizontal 2DSTI.

(c) An oblique 2DSTI. (d) Region showed by an arrow in (a).

Fig. 2. A sample of a chest MR image and its 2D spatio-temporal images (2DSTI) (from [12]).

simultaneously. These movements cause the pressure changesin the pleural cavity surrounding the lungs and the alveoli thusallowing the air from the atmosphere to flow inside the lungs.The packing and unpacking of the alveoli during inspiration andexpiration causes surface tension to decrease and increase in amanner that produces the characteristic hysteresis of the lungs.Hysteresis means that inflation of the lung follows a differentpressure/volume relationship from deflation [17]. Then, therespiration has two phases: inspiratory and expiratory.

2.1. Respiratory function

The sequence of MR images corresponding to slice s definesthe spatio-temporal volume (STV) Is(x, y, t)where x and y arethe coordinates of the image and t is the time.

A plane Qs(xs, ys, θs) that pass on point (xs, ys), makes anangle of θs with the x axis and is parallel to the time axis isdefined. Fig. 2(a) shows a chest MR image with three differentplanes Qs(xs, ys, θs) represented (θs = 90◦, θs = 0◦ andθs = −45◦). Plane Qs(xs, ys, θs) defines a 2D spatio-temporal(2DST) image Fs(χ, t). Fig. 2(a)–(c) show vertical, horizontaland oblique 2DST images, respectively.

A standard respiratory function is extracted from planeQs(xs, ys, 90◦) with 2DST image Fs(y, t). Fig. 15 shows anexample of standard respiratory function that was extractedfrom the region indicated by an arrow in Fig. 2(a). The standardrespiratory function is represented as χ = fs(t) over the 2DSTimage. The value of Fs(y, t) with lowest y is defined as ξs andthe correspondent instant in time is given by τs . This special

point (ξs, τs) defines the origin of the 2DST image. This way, itis possible to map the coordinates of the 2DST image Fs(χ, t)with the original STV Is(x, y, t)

x = χ cos(θs)+ ξsx

y = χ sin(θs)+ ξsy .

A pixel Fs(χ, t) in the 2DST image is associated to pixel

Is(χ cos(θs)+ ξsx , χ sin(θs)+ ξsy, t)

in the original STV. The smallest value of time for min( fs(t)),named τs , defines the standard frame. Fig. 2(d) shows anampliation of Fig. 2(a). The curve shown in Fig. 2(d) is dotted,showing the influence of the heart beating in the imaging.

As shown by Matsushita et al. [12], the Hough transform canbe used to determine the presence of scaled respiratory patternsin 2DSTIs. The respiration movement associated to point ψs isrepresented by (ψsx (t), ψsy(t)). It will be considered only thecase where θs = 90◦.2 The following expression can be written:

(ψsx (t), ψsy(t), t) = (x(t), a · fs(t)+ b, t) (1)

where a is a scale factor and b is a bias in the y axis.Considering that x(t) is constant, (a · fs(t)+b, t) is representedin the 2DST image Fs(y, t) as a scaled and biased respiratoryfunction (see Fig. 3). However, in the Hough space, therespiratory function represented by (a · fs(t)+ b, t) is mapped

2 The cases for θs = 0◦, θs = 45◦ and θs = −45◦ can be derived by asimilar procedure.



Fig. 3. Hough transformation.

to point (a, b). A point (y, t) in the 2DST image is mapped toline y = a · fs(t)+ b in the Hough space.

Fig. 4(a) shows the result of the Sobel operator applied to2DST image shown in Fig. 2(a), every pixel from Fig. 4(a) isplotted in the Hough space as a line using its intensity. Afterprocessing all pixels from Fig. 4(a), Fig. 4(b) is created. Theintensity of the pixels in the Hough space shows the confidenceto exist the respiratory function in the 2DST image. The pixelwith highest intensity determines with most confidence theexistence of a respiratory function. Then, it is necessary tocontinue the search for other respiratory functions present inthe 2DST image. The pixels in the 2DST image that belong tothe just found respiratory function, are resampled with negativevalues and a new respiratory function can be discovered.The process continues until no new respiratory function canbe distinguished. Fig. 4(c) shows the discovered respiratoryfunctions. Then, the convenient values of a and b to completeEq. (1) are found for θs = 90◦.

This process happens to all four directions. Fig. 5(a) showsthe standard frame with the original image. The result of allfour directions (θs = 0◦, θs = 45◦, θs = 90◦ and θs =

−45◦) combined is shown in Fig. 5(b), it is possible to identifythat some points have enough confidence to have a respiratoryfunction starting on it. However, it is not possible to determinethe best θs associated to the most confident respiratory function.

Fig. 4. Search for respiratory function using Hough transformation. (a) 2DSTimage. (b) Hough space. (c) Respiratory functions (from [12]).

In the next section it will be presented an algorithm to determinesuch θs .

2.2. Lung silhouette representation

The lung’s silhouette is represented as an interpolated curvedefined by Bezier segments [21]. The points to be interpolateddefine a closed polygon represented by the following sequenceof points:

Ps = {p0,p1, . . . ,pn} (2)

where the number of points is n + 1. The control points ofthe Bezier curve associated to segment pi pi+1 are calculated

(a) Original image. (b) All directions.

Fig. 5. Figure (b) shows the confidence summation in the four directions (θs = 0◦, θs = 45◦, θs = 90◦ and θs = −45◦) (from [12]).



(a) The analysed points. (b) Average.

(c) Standard deviation. (d) Estimation.

Fig. 6. Three samples of functions used in the tracking of the respiration pattern. The arrow in Figure (a) shows the analysed points and the arrow shows the pointassociated to Figures (b), (c) and (d). The arrow in Figure (d) shows the position (ζ, ϕ) where expression (4) has the maximum value (from [1]).

based on points pi−1, pi , pi+1 and pi+2. Angles αi =6 (pi−1,pi ,pi+1) and αi+1 = 6 (pi ,pi+1,pi+2) are defined.−−→ni−1, −→ni and −−→ni+1 are unit vectors in the directions of −−−→pi−1pi ,−−−→pi pi+1 and −−−−−→pi+1pi+2, respectively. −−→mi−1 and −→mi are unitvectors in the direction of−−→ni−1+

−→ni and−→ni +−−→ni+1, respectively.

Thus, a Bezier curve of third degree with four control points isdefined by the following control points:

q0 = pi

q1 =

{pi +

|pi−1 − pi |

3−→mi if αi > 90◦

pi if αi ≤ 90◦

q2 =

{pi+1 −

|pi+1 − pi+2|

3−−→mi+1 if αi+1 > 90◦

pi if αi ≤ 90◦

q3 = pi+1.

3. Proposed algorithm

The algorithm is divided in two steps: 2D respiratory motionanalysis and 3D lung’s reconstruction. In this work it is

proposed a method that uses a sample function estimation todetermine the presence of respiratory functions. One can checkthe result, and it is possible to correct the position of anypoint whenever necessary. The algorithm is reapplied if anycorrection where done. The second step is the creation of a B-Rep solid model of the lung, that is constructed in three steps:silhouette composition, mesh creation and corner creation. Onecan select one sequence of MR images, the B-Rep solid modelof the lung associated to each silhouette in the sequence isconstructed, defining a 4D B-Rep solid model of the lung forthe chosen STV.

3.1. Tracking the respiratory pattern [1]

All images in the sequence are processed and the edgesare determined using the Sobel operator that calculates thegradient of the image intensity at each point (mathematically,the gradient of a two-variable function is at each image pointa 2D vector with the components given by the derivatives inthe horizontal and vertical directions). Every image has its ownset of edges. After the application of the Sobel operator theintensity of the edges is high.



The algorithm seeks the respiratory movement representedby (ψsx (t), ψsy(t)), associated with an arbitrary pointψs shown in Fig. 6(a). However, as most organs movesynchronously with breathing, the following expression can bewritten:

(ψsx (t), ψsy(t), t) = ( fs(t)ζ cosϕ + ψsx (τs),

fs(t)ζ sinϕ + ψsy(τs), t) (3)

where ζ and ϕ represents the scale and the orientation of therespiratory function associated to point ψs . The most suitablerespiratory movement is searched, considering that the pointbelongs to the set of edges and the movement’s coherence withfs(t). The search is done by varying ζ and ϕ (amplitude andangle).(ψsx (t), ψsy(t), t) defines a sequence of pixels for a given ζ

and ϕ. It is considered the pixel’s intensity after the applicationof the Sobel operator. The average of the pixel’s intensitymust be high and the standard deviation must be low. Thedomain space is discrete, as the algorithm works on images,all possible values are determined and calculated. Fig. 6(b) and(c) shows the result of such calculations, respectively averageand standard deviation, for the point represented in Fig. 6(a).Finally, they are used in the following expression empiricallydetermined:

E(ψ, ϕ, ζ ) = 0.9|ζ |(

1−S(ψ, ϕ, ζ )

3A(ψ, ϕ, ζ )

)(4)

where A(ψ, ϕ, ζ ) and S(ψ, ϕ, ζ ) represents the average and thestandard deviation previously calculated. When the respiratorymovement of ψs(t) is in coherence with fs(t), the average ofthe edges’s intensity is high and the standard deviation is low.

Curves with shape similar to the diaphragm can haveseveral candidates of respiratory movement with differentangles and amplitudes. The algorithm seeks the candidatewith smaller amplitude by penalizing greater values of ζ . Therespiration movement associated to point ψs(t) is given bymax(E(ψ, ϕ, ζ )). Fig. 6(d) shows the result of such calculationfor the point represented in Fig. 6(a). Then it is possible todetermine, for a given point, the angle and scale that bestrepresent the respiratory function associated to it.

3.2. Silhouette points as scaled functions

With the angle and scale that best represent the respiratoryfunction fs(t) associated with each point psi (τs) that lies onthe lung’s boundary in a given reference frame it is possibleto estimate their position psi (t) in other slices. The orientationand scale may be represented by a vector Evsi in the image space,according to the following expression:

psi (t) = psi (τs)+ Evsi · fs(t) (5)

where psi (τs) is a point in slice s from the standard frame, Evsi isa vector in the image space defined by the orientation and scalethat best represent the respiratory function for this point, fs(t)is the respiratory function for slice s (see Fig. 7).

Fig. 7. The scaled lung model.

For a given slice the respiratory function is working asa scale where α = fs(t). Then it is possible to simplifyexpression (5) by not considering the time, as

psi (α) = psi (τs)+ Evsi · α. (6)

Using expression (6), one can determine a lung’s silhouettein a slice s for a given scale α. The lung’s silhouette is definedas a set of points

Ps(α) = {ps0(α),ps1(α), . . . ,psn(α)}. (7)

3.3. Reconstruction of the 3D lung

The result from the 2D analysis is a set of silhouettes movingin time Ps(t), that are represented as scalable silhouettes.Although belonging to the same lung, each silhouette hasits own time axis due to the long image acquisition time ofMR devices, resulting in image sequences without coherence.The reconstruction of the 3D Lung is accomplished throughthe interpolation between coronal and sagittal silhouettesconsidering the geometrical information present in the DICOMimages and the variation in the degree of breathing, in this work,the scale α given in expression (6).

3.3.1. Interpolation criteriaThe matrix that converts the coordinates from the 2D images

into the 3D space is defined asrsi [0]rsi [1]rsi [2]

1.0

=

xx ·∆i yx ·∆ j 0.0 sxxy ·∆i yy ·∆ j 0.0 syxz ·∆i yz ·∆ j 0.0 sz

0.0 0.0 0.0 1.0

·

psi [0]psi [1]

0.01.0

= [M] ·

psi [0]psi [1]

0.01.0

(8)

where psi [0] and psi [1] are the column and row to the imageplane. xx , xy and xz are the row x direction cosine of the imageorientation. yx , yy and yz are the column y direction cosine ofthe image orientation. ∆i is the column pixel resolution and ∆ jis the row pixel resolution. sx , sy and sz are the start position for



Fig. 8. Mapping between edge length and scale for silhouette Ps (α) and givencoordinates x and z.

the first voxel. Usually, coronal images have a bigger value for∆i and ∆ j when compared to sagittal images from the samepatient. In order to interpolate and match coronal and sagittalsilhouettes the following function is defined:

gs(x, z, α) =∥∥∥([M] · Ps(α))

⋂line(x, z)

∥∥∥ (9)

where line(x, z) is a line parallel to the y axis and passingthrough coordinates x and z. Function gs(x, z, α) returns thelength of the edge defined by the intersection between thepolygonal silhouette in the 3D space defined by [M] · Ps(α)

and the straight line line(x, z). An example of the mappingbetween edge length and scale for given coordinates x andz is shown in Fig. 8. According to the MR device used inthis research, the sagittal images have x coordinates constantand the coronal images have z coordinates constant. As aconsequence, line(x, z) represents an intersection line betweentwo slices, one sagittal and another coronal.

3.3.2. Silhouette compositionThis section explains how to reconstruct the 3D lung

associated to a given silhouette, using the acquired setof sagittal and coronal sequences of silhouettes. Forsimplification, it is assumed that a coronal silhouette isgiven; for a sagittal silhouette the method applies similarly.The sagittal images have a better quality when comparedto the coronal images. However, the coronal image containsthe silhouettes for the right and left lungs, allowing thereconstruction of both lungs simultaneously.

For a given coronal silhouette, a sagittal silhouette nearthe middle of the lung that fits the selected coronal silhouetteaccording to the interpolation criteria described in the previoussection, is determined. This sagittal silhouette is used asguide, and coronal silhouettes that fits the determined sagittalsilhouette are computed. This way, the right and left lungs arereconstructed simultaneously. This fit is given by the absoluteposition of the silhouette present in DICOM images (see Eq.(8)) and the length of the silhouette intersection as function ofthe scale (see Eq. (9)). Fig. 9 shows an example of coronal

Fig. 9. Right and left lungs with its coronal silhouettes coherently composed.

Fig. 10. Right and left lungs with its sagittal silhouettes coherently composed.

Fig. 11. Right and left lungs with its sagittal and coronal silhouettes coherentlycomposed.

silhouettes that fits to a determined sagittal silhouette. Next,sagittal silhouettes that fits the first and last coronal silhouettesare computed and clipped (Fig. 10). The 3D lung skeleton isobtained (Fig. 11).



3.3.3. Mesh creationNow, it is necessary to create a B-Rep solid model of the

lung, given a collection of planar silhouettes representing cross-sections. It is necessary to define the edges connecting thepoints from one silhouette to a point from a neighbouringsilhouette. This is known as tiling problem and is severelyunderconstrained [13]. Many surfaces could give rise to theobserved cross-sections. In order to choose the “correct”surface, an optimization with respect to some objective functionis realized. The chosen objective function should capturesome notion of what a “good” surface is and should be easyto compute. Meyers et al. [13] analysed several objectivefunctions. State of the art methods for shape reconstructionconsider the possibilities of complex geometrical shapes andquick shape changes [4,10], however this is not the case in thisresearch.

When the cross-sections are convex polygons a “good”surface can be easily created. However, the lung has concavepolygons as cross- sections. As a first step, neighbouringsilhouettes are registered by searching vertices with highercurvature and if two of them are near enough they are connectedby the creation of edges named spans. Fig. 12 shows thecross-sections connected at registered vertices, the majority ofthose spans define the diaphragmatic contour. Each edge inB-Rep solid model has two halfedges orientated in opposeddirections [11]. Both halfedges from the spans connectingregistered vertices are pushed into a stack and are used as initialhalfedges in the meshing algorithm.

The B-Rep solid model is composed by vertices, edgesand faces. Each vertex in the B-Rep solid model originatedfrom a distinct silhouette. The intersection between sagittal andcoronal silhouettes is an exception. We associated an attributeto each vertex representing from which silhouette such vertexoriginated from. A span can only connect vertices from distinctsilhouettes, i.e. vertices with different attributes.

Fig. 12 shows an instant in the mesh creation process. Thealgorithm is creating a mesh between silhouettes P and Q.Initially, the registered vertices p1 and q1 are popped from thestack; then, the algorithm follows the sequence of vertices inthe corresponding silhouettes, creating the edges with smallersize, as described below:

1: i ← 1, j ← 12: while < not finished the mesh > do3: d1 = length(pi q j+1 ), d2 = length (pi+1q j )

4: if d1 > d2 then5: < Create edge pi+1q j >

6: i ← i + 17: else8: < Create edge pi q j+1 >

9: j ← j + 110: end if11: end while

Fig. 13 shows the meshed right and left lungs. Thealgorithm created a surface interpolating sagittal and coronalneighbouring silhouettes. Remains to define the four corners ofthe lung defined by sagittal and coronal silhouettes having onlyone neighbouring silhouette.

Fig. 12. Right and left lungs with registered vertices connected by spans areshown. The mesh is created by connecting two neighbouring silhouettes. Thedetail in the right shows two spans defining a polygon that will be meshed bythe algorithm.

Fig. 13. Right and left lungs with their mesh. Definition of a blend curve inthe lung’s corner. Vertex p1 has three adjacent faces and vertex p2 has fouradjacent faces. The average of the normals of the adjacent faces define thenormal associated to the vertices. q1 and q2 define the internal control pointsof the Bezier curve.

3.3.4. Corner creationA blend surface is created to model the corner of the lung.

The blend surface is created in two steps: firstly a blendingBezier curve [21] is defined, and then the resulted B-Rep solidmodel is meshed using an algorithm very similar to the onepreviously defined.

A Bezier curve is created by connecting two vertices fromdistinct silhouettes (one from a sagittal silhouette and anotherfrom a coronal silhouette). Considering that p1 originated froma sagittal silhouette and that p2 originated from a coronalsilhouette. Fig. 13 shows the definition of the attributes ofthe Bezier curve associated to edge p1p2. The normal ofvertices p1 and p2 are calculated as the normal average of theirneighbouring faces (ignoring the face associated to the lung’scorner), such information is explicitly stored in B-Rep solidmodellers. The normal of vertices p1 and p2 are representedas −→n1 and −→n2 , respectively. Then a Bezier curve is defined bythe following control points:

q0 = p1

q1 = p1 +−→n1 ∧ (

−→n1 ∧ n(−−→p1p2))‖−−→p1p2‖

3

q2 = p2 +−→n2 ∧ (

−→n2 ∧ n(−−→p2p1))‖−−→p1p2‖

3



Fig. 14. Right and left lungs and their corners meshed.

Table 1Characterizations of the three sequences of coronal and sagittal MR imagesused in the experiments

Case A B C

fov - cor (mm) 420× 420 420× 420 420× 420fov - sag (mm) 380× 380 380× 380 380× 380total time (s) 17 11 11age 31 31 24

Fig. 15. Standard respiratory pattern extracted from a 2DST image.

q3 = p2

where n(−−→p1p2) and n(−−→p2p1) represents the normalized vectorsin the direction of −−→p1p2 and −−→p2p1, respectively. Fig. 14 showsthe resulting corner already meshed for the right and left lungs.

4. Results

The sequence of MR images used in the experiment wereobtained by Symphony (1.5 T) made by Siemens, using themethod true FISP (Fast Imaging with Steady State Precession).The slice’s thickness is 10 mm. It was used 22 coronal slices and10 sagittal slices, with 256 × 256 pixels and 12 bits per pixel.The MR images were taken from three healthy nonsmokingpersons. Table 1 shows some characterizations of the threesequences of MR images.

Fig. 16. The circles represent the points initially defined at the standard frame.

Fig. 17. The triangles represent the corrected points.

Fig. 18. The squares represent the result from the proposed algorithm.

4.1. Tracking the respiratory pattern

The following operation was done to every sequence ofMR images. The silhouette of the lung and its respectivelyinterlobular pleurisy were defined at the standard frame (seeFig. 16). The standard respiration pattern was defined. Fig. 15shows an example of a standard respiration pattern. Next, thesilhouette of the lung is tracked over the sequence of images.Fig. 17 shows the result after some corrections. The squared



(a) Left lung characteristics. (b) Left lung characteristics.

(c) Left lung characteristics. (d) Right lung characteristics.

Fig. 19. (a) and (b) Mediastinal surface of the left lung and its characteristics. (c) Lateral view of the left lung and its characteristics. (d) and (e) Mediastinal surfaceof the right lung and its characteristics. (f) Lateral view of the right lung and its characteristics.

points represent the algorithm’s output and the triangular pointsrepresent the corrections.

After the corrections, the silhouette of the lung is trackedagain with two fixed frames. Fig. 18 shows the result of anotherframe after the first correction. After correcting a single frame,the algorithm succeed in tracking correctly all the silhouettes ofthe lung in time.

It is possible to observe from Fig. 18 that the three lobesfrom the right lung and the two lobes from the left lung werealso tracked. They are moving synchronously to the boundaryof the lung. The ground–truth comparison was done elsewhere[1].

4.2. Reconstruction of the 3D lung

The three dimensional right and left lungs were createdbased on the selected sequence of MR images, the result can

be seen in Fig. 19(a)–(f). It is possible to recognize severallung characteristics in the three-dimensional model. Fig. 19(a)shows the mediastinal surface of the left lung, and the followingcharacteristics can be observed: apex, inferior border, cardiacimpression, groove for descending aorta, anterior border anddiaphragmatic surface. Fig. 19(b) shows the same mediastinalsurface of the left lung with a clear view of the diaphragmaticsurface. Fig. 19(c) shows a lateral view of the left lung.The following characteristics can be observed: apex, superiorlobe and inferior border. Fig. 19(d) shows the mediastinalsurface of the right lung. The following characteristics can beobserved: apex, inferior border, cardiac impression, groove foroesophagus and diaphragmatic surface. Fig. 19(e) shows thesame mediastinal surface of the right lung with a clear viewof the diaphragmatic surface. Fig. 19(f) shows a lateral view of



(e) Right lung characteristics. (f) Right lung characteristics.

Fig. 19. (continued)

Fig. 20. Lateral view of the right and left lungs shaded and meshed.

the right lung The following characteristics can be observed:apex, superior lobe and inferior border.

Fig. 20 shows the shaded model of the left and right lungswith their mesh.

4.3. Lung’s volume

The result is a B-Rep solid model, ensuring that there are noholes or empty spaces. As a consequence the lung’s volume canbe calculated [2]. Two sequences of MR images were chosenand the volume of the right and left lungs for each imagewas calculated (see Figs. 21 and 22). The volume of the leftlung is always smaller when compared to the right lung in thesame instant. The patient executed normal breathing when thesequence of MR images were acquired.

Fig. 21. Volume of right and left lungs calculated for a sequence of MR images— I.

Figs. 21 and 22 show the volume of the right lung in therange 1.2–1.9 L and the volume of the left lung in the range0.9–1.5 L. The residual volume for the right and left lungs arenear 1.2 litres and 0.9 litres, respectively. The inspired volumefor the right and left lungs are near 0.7 litres and 0.6 litres,respectively.

It is possible to observe that the volume of right andleft lungs are synchronized during inspiratory and expiratoryphases.

4.4. Discussion and future works

Physiologically, very little is understood about respiratorymovement. Troyer et al. [18] exposed the ribs of a dog and fixedspindles to measure the respiratory movement. It is not possibleto apply this technique to humans. The research presented herewalks in the direction of a safer and more accurate measurementmethod.



Fig. 22. Volume of right and left lungs calculated for a sequence of MR images— II.

It is possible to examine the respiratory functioning andlung capacity using extremely low price and simple devices;however, those devices demand some breathing effort thatpatients with bad health conditions are not able to accomplish.A safer and less intrusive evaluation of the respiratorymovement will help in the selection of the appropriate medicineand dose, the determination of the effectiveness of a treatment,the reduction of the number of clinical trials, the observation ofthe progress of rehabilitation treatments, among other possibleapplications.

A confirmation of the results obtained here can be maderegistering the quantity of air inspired/expired by the lung intime, simultaneously to the MR image acquisition. This kind ofexperiment is not so easy to implement, and its implications arebeing considered at the moment. This experiment will providethe knowledge needed to define the number of coronal andsagittal slices necessary to correctly estimate the lung’s volumeaccording to this criteria.

Given a coronal or sagittal silhouette, this methodreconstruct the 3D lung using the acquired set of coronal andsagittal unsynchronized sequential images. The silhouette caneven be obtained from other devices such as PET or CT. It ispossible to position the image associated to a given silhouettein the 3D lung. For example, CT images are acquired duringradiation therapy and the 3D position of the surface of themarker can be estimated.

Given a coronal or sagittal silhouette, this methodreconstruct the 3D lung using the acquired set of coronal andsagittal unsynchronized sequential images. The silhouette caneven be obtained from other devices such as PET or CT. It ispossible to position the image associated to a given silhouettein the 3D lung. For example, CT images are acquired duringradiation therapy and the 3D position of the surface of themarker can be estimated.

Each acquired sequence of MR images, coronal or sagittal,represents the intersection of the lung with the observedplane and therefore, the error associated to the contour curveapproximation is smaller as the lungs surface normal becomesparallel to the observation plane. As a consequence this error

is larger as the observation plane gets far from the centralslices and could be verified in the respiratory pattern trackingphase. In this work, the slices where the silhouettes couldnot be tracked were not used in the 3D reconstruction. Evenconsidering this fact, a large number of silhouettes was used,resulting in the pictures shown. However, it is possible to geta better estimate of regions far from the central slice applyingpoint projection algorithms [10]. The reconstructed 3D modelof the lung in each instant is a surface- based estimate ofthe lung for each instant and may be used to perform pointregistration in time through the projection of this point on thesubsequent 3D model. With this point movement estimationit is possible to build a model with an unique topologicaldescription, the basis to the application of Finite ElementMethods and study the lung’s dynamics.

5. Conclusions

Our work involves the simulation, modelling and visualiza-tion of the lung. This work discussed a 4D lung reconstructionmethod from unsynchronized MR sequential images. It waspossible to recognize several characteristics of the lung in thethree-dimensional model as shown in Fig. 19(a) to (f).

This preliminary work defined a research direction. Thetracking of the respiratory pattern succeed in determining theboundary of the lung in time, as well the boundary of thethree lobes from the right lung and the boundary of the twolobes from the left lung. We are working on an algorithm tocreate the B-Rep representation of the five lobes. The silhouettecomposition must have a different algorithm. The amount ofmanual work, in this research, is quite high, as it is necessary todefine the boundary of the lung once for every sequence of MRimages. We are researching an alternative proposal where theuser defines only the boundary for one sagittal and one coronalsequence of MR images. The silhouette composition beginsusing just these two sequences. Then the silhouette compositionmodule will supply information for an automatic 2D track ofrespiratory patterns.

References

[1] Asakura A, Gotoh T, Kagei S, Iwasawa T, Inoue T. Computeraided system for respiratory motion analysis of the lung region bysequential MR images. Medical Imaging Technology 2005;23(1):39–46[in Japanese].

[2] Cattani C, Paoluzzi A. Boundary integration over linear polyhedra.Computer Aided Design 1990;22(2):130–5.

[3] Cluzel P, Similowski T, Chartrand-Lefebvre C, Zelter M, Derenne JP,Grenier PA. Diaphragm and chest wall: Assessment of the inspiratorypump with MR imaging-preliminary observations. Radiology 2000;215(2):574–83.

[4] Diebel JR, Thrun S, Brunig M. A Bayesian method for probable surfacereconstruction and decimation. ACM Transactions on Graphics 2006;25(1):39–59.

[5] DeCarlo D, Kaye JM, Metaxas D, Clarke JR, Webber B, Badler NI.Integrating anatomy and physiology for behavior modeling. In: Morgan K,Satava RM, Sieburg HB, Mattheus R, Christensen RP, editors. Interactivetechnology and the new paradigm for healthcare. IOS Press and Ohmsha;1995. p. 81–7.



[6] Gauthier AP, Verbatick S, Estenn M, Segebartg C, Macklem PT, Paiva M.Three dimensional reconstruction of the in vivo human diaphragm shapeat different lung volumes. Journal of Applied Physiology 1994;76(2):495–506.

[7] Gierada DS, Curtin JJ, Erickson SJ, Prost RW, Strandt JA, Goodman LR.Diaphragmatic motion: Fast gradient-recalled-echo MR imaging inhealthy subjects. Radiology 1995;194(3):879–84.

[8] Iwasawa T, Kagei S, Gotoh T, Yoshiike Y, Matsushita K, Kurihara H,et al. Magnetic resonance analysis of abonormal diaphragmatic motionin patients with emphysema. European Respiratory Journal 2002;19(2):225–331.

[9] Ko JP, Naidich DP. Computer aided diagnosis and the evaluation of lungdisease. Thoracic Imaging 2004;19(3):136–55.

[10] Liu YS, Paul JC, Yong JH, Yu PQ, Zhang H, Sun JG, et al. Automaticleast-squares projection of points onto point clouds with applications inreverse engineering. Computer Aided Design 2006;38(12):1251–63.

[11] Mantyla M. An introduction to solid modeling. Rockville (MD):Computer Science Press; 1988.

[12] Matsushita K, Asakura A, Kagei S, Gotoh T, Iwasawa T, Inoue T. Shapetracking on chest MR sequence images using respiratory patterns. TheJournal of the Institute of Image Electronics Engineers of Japan 2004;33(6):1115–22 [in Japanese].

[13] Meyers D, Skinner S, Sloan K. Surfaces from countours. ACMTransactions on Graphics 1992;11(3):228–58.

[14] Mills GH, Wild JM, Eberle B, van Beek EJR. Functional magneticresonance imaging of the lung. British Journal of Anaesthesia 2003;91(1):16–30.

[15] Promayon E, Baconnier P, Puech C. Physically-based model for simulat-ing the human trunk respiration movements. In: CVRMeD–MRCAS’97:

Proceedings of first joint conference on computer vision, virtual real-ity and robotics in medicine and medial robotics and computer-assistedsurgery. London (UK): Springer–Verlag; 1997. p. 379–88.

[16] Ray N, Acton ST, Altes T, de Lange EE, Brookeman JR. Mergingparametric active contours within homogeneous image regions for MRI-based lung segmentation. IEEE Transactions on Medical Imaging 2003;22(2):189–99.

[17] Santhanam A, Pattanaik S, Fidopiastis C, Rolland JP, Davenport P.Physically-based deformation of high-resolution 3D lung models foraugmented reality based medical visualization. In: MICCAI–AMI–ARCSinternational workshop on augmented environments for medical imagingand computer-aided surgery 2004. Rennes (St. Malo France): SpringerVerlag; 2004.

[18] Troyer AD, Leduc D. Effects of inflation on the coupling between the ribsand the lung in dogs. Journal of Physiology 2003;555(2):481–8.

[19] Suga K, Tsukuda T, Awaya H, Takano K, Koike S, Matsunaga N, et al.Impaired respiratory mechanics in pulmonary emphysema: Evaluationwith dynamic breathing MRI. Journal of Magnetic Resonance Imaging1999;10(4):510–20.

[20] Tsui BMW, Segars WP, Lalush DS. Effects of upward creep andrespiratory motion in myocardial SPECT. IEEE Transactions on NuclearScience 2000;47(3):1192–5.

[21] Yamaguchi F. Curves and surfaces in computer aided geometric design.Springer Verlag; 1988.

[22] Zordan VB, Celly B, Chiu B, DiLorenzo PC. Breathe easy: Modeland control of simulated respiration for animation. In: Proceedingsof the 2004 ACM SIGGRAPH/eurographics symposium on computeranimation. 2004.

animated solid model of the lung constructed from unsynchronized

Documents