1

Volume rendering in the presence of partial

volume effects

Andre Souza, Jayaram K. Udupa and Punam K. Saha

Medical Image Processing Group, Department of Radiology, University of Pennsylvania

Address for correspondence:

Jayaram K. Udupa

Medical Image Processing Group

Department of Radiology

University of Pennsylvania

Fourth Floor, Blockley Hall

423 Guardian Drive, Philadelphia, PA 19104-6021

Ph: (215) 662-6780

Fax: (215) 898-9145

E-mail: jay@mipg.upenn.edu

2

ABSTRACT

In tomographic imagery, partial volume effects (PVE) cause several artifacts in volume renditions.

In x-ray CT, for example, soft-tissue-like pseudo structures appear in bone-to-air and bone-to-fat

interfaces. Further, skin, which is identical to soft tissue in terms of CT number, obscures the

rendition of the latter. The purpose of this paper is to demonstrate these phenomena and to provide

effective solutions that yield significantly improved renditions. We introduce two methods that

detect and classify voxels with PVE in x-ray CT. Further, a method is described to automatically

peel skin so that PVE-resolved renditions of bone and soft tissue reveal considerably more detail. In

the first method to address PVE, called the fraction measure (FM) method, the fraction of each tissue

material in each voxel v is estimated by taking into account the intensities of the voxels neighboring

v. The second method, called uncertainty principle (UP) method, is based on the following postulate

(IEEE PAMI, vol. 23 pp. 689- 706, 2001): In any acquired image, voxels with the highest

uncertainty occur in the vicinity of object boundaries. The removal of skin is achieved by means of

mathematical morphology.

Volume renditions have been created before and after applying the methods for several patient CT

datasets. A mathematical phantom experiment involving different levels of PVE has been conducted

by adding different degrees of noise and blurring. A quantitative evaluation is done utilizing the

mathematical phantom and clinical CT data wherein an operator carefully masked out voxels with

PVE in the segmented images. All results have demonstrated the enhanced quality of display of bone

and soft tissue after applying the proposed methods. The quantitative evaluations indicate that more

than 98% of the voxels with PVE are removed by the two methods and the second method performs

slightly better than the first. Further, skin peeling vividly reveals fine details in the soft tissue

structures.

Keywords: volume rendering, partial volume effect, computed tomography (CT), 3D

visualization.

3

1. INTRODUCTION

Compared to the size of the details of the structures we wish to visualize via medical imaging, the

size of the voxels defined by the imaging devices is often large. Consequently, an aggregate value of

the imaged property of multiple tissues that fall into a voxel is assigned to the voxel, especially at

tissue interfaces. This phenomenon is generally referred to as partial volume effect (PVE). During

the past ten years, PVE has been studied largely in the context of image segmentation [1-5]. It has

not received much attention as related to object rendering [2, 6]. The quality of the result of

segmentation and of the three-dimensional (3D) depiction of anatomical structures in CT (and other

modalities) is usually impaired by PVE. In volume rendering (VR) and in surface rendering (SR),

particularly in medical applications, the challenge is to clearly portray specific tissues in relationship

to the neighboring structures [7-9]. Both techniques have advantages and shortcomings, and both are

not immune to PVE. In SR, pseudo structures may be created that may obscure the structure of

interest if PVE is not properly handled. In VR, utilizing color, pseudo structures may appear in the

same color as the color of the structures of interest. This may mislead the observers and make them

reach wrong conclusions. This phenomenon is illustrated in Figure 1 wherein a slice of a CT image

of the head of a patient is displayed in Figure 1(a), and Figure 1(b) shows the voxels with PVE that

resemble voxels containing soft tissue. Figure 2(a) shows a volume rendition, obtained by using a

simple trapezoidal classification function [10], that portrays both bone and soft tissue. (Please view

the original electronic versions of all images in order to relate to the subtle points made in this

manuscript.) The colored stripes that appear in the region of the skull are due to soft-tissue like

voxels in the interface between bone and fat. Similar pseudo structures occur in the interior of the

skull in bone-to-air and bone-to-fat interfaces. The skin, which has CT numbers identical to those of

soft tissue, also reduces the rendition quality of the soft tissue structures. In Figure 2(b), muscles are

4

highlighted via a simple trapezoidal classification function on coronal image slices at planes selected

via the craniofacial rendition that precisely cut through the stripes. This is for illustrating the fact that

the stripes are due to partial volume effects and not due to the artifacts that may have been

introduced by the approximation of the volume rendering integral, as discussed by Boer et al. and

Novins and Arvo [11, 12]. The proposed methods have the objective to overcome such difficulties,

leading to final renditions that are superior and that show fine details that are otherwise hidden due

to PVE.

(a) (b)

Figure 1. Illustration of the regions affected by PVE in a 2D slice of a 3D CT head scene. (a) Original image, (b) regions with significant PVE (white arrows).

5

(a)

(b)

Figure 2. (a) Volume rendition of the data set of Figure 1 shows facial musculature and the skull. The stripes on the skull are the false occurrences of muscle due to PVE. (b) Coronal slices (right) corresponding to the planes passing through the stripes in the rendition (left). Muscles are highlighted by the use of a trapezoidal classification function. The arrows point to the PVE regions that contribute to the stripes.

6

Sato et al. [1] presented a method of tissue classification that employs a multi-dimensional opacity

function. These filter responses and the original intensity values are used to define a

multidimensional feature space in which tissue classification strategies are designed. Although this

approach uses information from 3D local neighborhoods that have explicit geometry, such as edge,

sheet, line, and blob, which are related to specific tissue structures, the authors report improvement

in the overall quality of the 3D renditions after applying the method. No quantitative evaluation for

the effectiveness of removing voxels with PVE was presented. Kindlmann and Durkin [13], [14] also

addressed the problem of selecting opacity functions in order to produce informative volume

renditions. They focused on using higher dimensional opacity functions (3D function based on

original intensity values, first and second derivative values). Although the authors did not directly

address PVE issues, their approach seems to have some effect on suppressing PVE. Again no

evaluation was presented. Beier et al. [2] proposed another method in which morphological filters

are used to reduce PVE. Höhne and Hanson [15] also used morphological operations and connected-

component-analysis to mask out PVE regions. Tiede et al. [6] proposed a method for locating

boundaries of adjacent objects at a subvoxel resolution and for subsequently assigning labels to each

tissue type. Their approach reclassifies voxels with PVE into specific tissue type by using

information from a 2x2x2 neighborhood. No quantitative assessment of the accuracy of the method

was provided in these publications. Santago and Gage [3] developed a method based on a statistical

model of PVE that assumed a Gaussian system for noise and intrinsic material variances with

Gaussian or Poisson statistics. Their methods focused mainly on segmentation and quantification by

utilizing finite mixture densities in order to more accurately model the distribution of voxel

intensities. They did not investigate rendering issues. Rusinek et al. [4] and Heuscher and Vembar

[5] evaluated inaccuracies arising in the display and measurement of structures in CT images owing

7

to PVE without any quantitative assessment of improvements in VR. Laidlaw et al. [16] proposed a

histogram-based method that used information from neighboring voxels for the classification of

material mixtures. Bayesian probability theory is used to estimate the highest-probability

combination of materials within each voxel. An early method, named eigenimage filtering [17],

provided an optimal linear transformation which was used for PVE estimation. This method did not

take into account neighboring structure information. Recently, a method for improving this approach

has been proposed by Siadat and Soltanian-Zadeh [18] which considers neighboring information to

yield a continuous, smooth, and accurate estimation of PVE at the object boundaries that is suitable

for visualization purposes. However, it did not address the rendering process.

We present two methods in Section 2 for overcoming PVE. Their main departure from the methods

reported in the literature is that they consider both intensity and structure information in identifying

and overcoming PVE. Toward this goal, both methods utilize a fundamental concept known as scale

[19]. Scale indicates the local structure size which is determined at every voxel in a given image

based only on an intensity homogeneity criterion. The basic premise is that regions of small scale

represent regions of potential PVE. Algorithms are presented under the two methods to recognize

voxels with PVE. A method to remove skin is also presented subsequently. In Section 3, we show

the results of the application of our method on clinical images. An evaluation of the method,

consisting of both qualitative and quantitative assessment, is presented wherein we use clinical CT

images and mathematical phantoms. Finally, we state our conclusions in Section 4. A preliminary

version of this paper was presented at the SPIE Symposium on Medical Imaging 2002 [20].

8

2. THEORY AND ALGORITHMS

2.1 Overview

For brevity, we refer to an acquired digital volume image as a scene and represent it by a pair

),( fC=C where =C { jjj bcbc ≤≤−| for some 3+∈Zb }, 3

+Z is the set of 3-tuples of positive

integers, called voxels, f is a function whose domain is C , called the scene domain, and whose

range is a set of integers [L, H], and for any Cc∈ , )(cf is referred to as the intensity of c . We call

C a binary scene if the range of f is { }1,0 . A digital ball (or simply a ball) of radius r centered at

any voxel c in C is the set }|{ rdcCd(c)Br ≤−∈= . For any set X , we use the notation X to

denote its cardinality.

We present two methods in this paper to identify and suppress voxels with PVE. The first method,

called the fraction measure (FM) method, examines certain regions of the scene domain where the

scale value is small. In these regions, at every voxel c , by examining a neighborhood whose size is

again determined by the scale at c , a fraction is estimated for each tissue type within c . In the

second method, an information theoretic principle called uncertainty principle (UP) [21], also guided

by scale and the associated criteria, is used to estimate the fraction of each tissue in every voxel in

regions of high uncertainty. Since the concept of scale forms the basis of these methods, we first

briefly describe in Section 2.2 its principles and an algorithm to estimate scale. Please refer to [19]

for further details. This will be followed by a description of the FM and UP methods in sections 2.3

and 2.4, respectively. Both methods accept as input a scene and output a modified scene wherein the

voxels affected by PVE are identified and suppressed. Any SR or VR method can be subsequently

applied to the resulting scene.

9

The overall conceptual placement of these methods and of the skin peeling method described in

Section 2.5 in a SR or a VR process is as follows. It will later become clear that Step 2 can be

integrated into Step 4. For clarity and for keeping these methods independent of any segmentation

and classification strategies, we have kept them separate. Here we have assumed the given scene to

be C .

1. For C , determine the scale scene SC .

2. Utilizing SC and C and the FM or UP method, determine a new scene 1C in which the PVE

has been suppressed.

3. Peel skin in 1C to output a new scene 2C .

4. Segment or classify 2C to output a data structure D appropriate for SR or VR.

5. Do SR or VR utilizing D.

These individual steps are described in more detail in the following sections.

2.2 Scale

In this section, we describe the algorithm that produces the object scale )(cfS for any voxel c in a

given scene ),( fC=C . For any ball )(cBr of radius r centered at c , we define a fraction )(cFOr ,

that indicates the fraction of the set of the voxels in the ball boundary whose intensities are

sufficiently uniform with that of c , by

( ))()(

)()()(

1

)()( 1

cBcB

dfcfWcFO

rr

cBcBd

rrr

−

−∈

−

−=∑

−ψ

, (2.1)

10

where )(xWψ is a fuzzy membership function corresponding to the predicate “ x is small”. In this

paper, a zero mean unnormalized Gaussian function with standard deviation ψσ is used for ψW . ψσ

is a homogeneity parameter that is estimated from the given scene as follows [19]. Over the entire

scene domain C , local intensity differences |)()(| dfcf − are computed for all possible pairs ),( dc

of voxels such that c and d are 6-adjacent. The upper 10 percentile values of these differences are

discarded to account for inter object boundaries. The mean hM and the standard deviation hσ of

these differences are computed over the lower 90 percentile values. These estimates are then used in

setting up the value of ψσ as follows:

hhM σσψ 3+= . (2.2)

The rationale for this choice is that, in a normal distribution, three standard deviations on both sides

of the mean cover 99.7 percent of the population. The algorithm for object scale estimation (OSE) is

summarized below.

Algorithm OSE

Input: C , ,ψW a fixed threshold st .

Output: A scale scene ),( SS fC=C of the scene C .

begin

for each Cc∈ do

set 1=r ;

while sr tcFO ≥)( do

set r to 1+r ;

endwhile;

set )(cfS to r ;

endfor;

end

11

The algorithm iteratively increases the ball radius r by 1, starting from 1=r , and checks )(cFOr ,

the fraction of the object containing c that is contained in the ball boundary. The first time when this

fraction falls bellow the tolerance parameter st , we consider that the ball enters an object region

different from that to which c belongs. Following the recommendation in [19], we have used

85.0=st . Roughly speaking, )(cfS is the radius of the largest ball centered at c within which the

voxel intensities are sufficiently homogeneous. Figure 3 shows for a 2D slice of a 3D CT head scene

the corresponding scale scene.

(a) (b)

Figure 3. (a) Original scene and (b) the corresponding scale scene. Brightness at a pixel is proportional to its scale value in the scale scene. Scales of size greater than 8 are all set to 8 in this scene since, for the purposes of this paper, we are not interested in distinguishing among large scale values.

2.3 The Fractional Measure Method

The basic idea of this method is as follows. The interior regions of objects are more-or-less

homogeneous, and, therefore, voxels in these regions have large scale values. Such regions are

neither affected by PVE nor of interest in the rendering process. On the other hand, voxels with

small scale values lie in the region of fuzzy boundaries and tissue interfaces and experience

maximum PVE. Thus, the scale value )(cfr S= associated with a voxel c in a given scene

),( fC=C can guide us in identifying the voxels affected by PVE. Further, the fractions of different

12

types of tissues within such voxels can be estimated by examining the intensities of voxels within the

ball )(cBr centered at c in the following manner. Let τ denote the tissue type. In CT images, we

are interested mainly in soft tissue ( st ) and bone (bn ), and therefore, },{ bnst∈τ . Although the

methods presented here can be readily extended to the case of mixtures of more than two tissues, we

will confine ourselves to the case of two tissues. For a fixed scale value ρ , and for any voxel Cc∈

in a given scene ),( fC=C such that ρ≤)(cfS , we define the fraction of the tissue },{ bnst∈τ

contained in c to be

( ))(

)()(

)(

cB

dfWcF

cBd

ρ

ττρ

ρ∑ ∈

= , (2.3)

where )(xWτ is a fuzzy membership function that expresses the relationship between scene intensity

x and percent (fractional) content for tissue τ . The functional forms used for )(xWτ are as follows,

as depicted in Figure 4.

,))((2

2

2

))((

st

stdf

st edfW σµ−

−

=

>≤=

−−

.)(,1

)(,))((2

2

2

))((

bn

bn

df

bn

df

dfedfW bn

bn

µµσ

µ

Figure 4. Functional forms used for )(xWτ , },{ bnst∈τ .

The fraction scene ),( ststF FC ρ=C corresponding to the scene shown in Figure 3 is displayed in

Figure 5. It may be pointed out that, since stFρ and bnFρ , for bnst ττ ≠ , are computed by using

13

independent membership functions stW and bnW , the fraction of tissues )(cF stρ and )(cF bn

ρ at any

given voxel c may not be related, i.e., ( ) ( )st bnF c F cρ ρ+ is not necessarily constant.

Figure 5. The fraction scene for soft tissue corresponding to the scene shown in Fig. 3(a). Brightness at a voxel is proportional to its fractional value. For clarity and emphasizing PVE at interfaces, the interior voxels of soft tissue regions, where the fraction value is close to 1, are not displayed.

Since the objects of interest in CT are composed of tissue },{ bnst∈τ , we are interested in finding

fractions of the tissue only for bone and soft tissue. And, to recall, our aim here is accurate soft tissue

display. Accordingly, the algorithm presented below determines only these fractions and decides

where pseudo soft tissue regions exist to subsequently suppress them in the scene that is output. A

more general treatment considering all possible interfaces and other particular tissues such as bone

are also plausible, along similar lines. We shall come back to this issue as related to bone in Section

3.

Algorithm FM

Input: A scene ),( fC=C , ρ , τW , ),( SS fC=C , constants stt and bnt .

Output: ),( FMFM fC=C .

begin

set )()( cfcfFM = for all Cc∈ ;

for all voxels Cc∈ such that ρ≤)(cfS do

compute )(cF stρ and )(cF bn

ρ ;

14

if stst tcF <)(ρ and 1)( <cF bn

ρ and bntcf <)( then

set 0)( =cfFM ;

endif;

endfor;

end

In the above algorithm, stt and bnt are fixed threshold values. stt represents a threshold on soft tissue

fraction, and bnt is a lower threshold on intensity for bone. We assume stW to be a Gaussian and bnW

to be a half-Gaussian as depicted in Figure 4. Their mean τµ and standard deviation τσ are

estimated by training, which consists of an operator painting the appropriate pure soft tissue and

bone regions in slice displays and determining the mean and standard deviation of intensities in these

regions. The value of ρ is set to 5. The results deteriorate if 5>>ρ . We use 3.0=stt and

100,1=bnt (on a 0 to 4095 CT gray scale). These values are also determined experimentally. We

assume that all voxels Cc∈ which have )(cF stρ smaller than 30%, i.e., 3.0=stt , are voxels with

potential PVE. In addition, to prevent misclassification of voxels affected by PVE at the interior of

bone and in the vicinity of air and fat regions, we also constrain )(cF bnρ to be smaller than 100% and

their intensities to be less than bnt .

2.4 Method based on Uncertainty Principle

The basis for this method is the following postulate proposed in [21] in connection with finding

optimum thresholds in scene segmentation by thresholding: In any acquired scene, voxels with the

highest class uncertainty occur in the vicinity of object boundaries. The total class uncertainty

)(cU ρ at any voxel c in a given scene ),( fC=C is given by

15

)(ln)()( cPcPcU τρ

τ

τρρ ∑−= , (2.4)

where )(cPτρ is the probability that voxel with intensity )(cf belongs to tissue class τ , and is given

by

∑=

τ

τρ

τρτ

ρ)(

)()(

cF

cFcP . (2.5)

Figure 6 shows for a 2D slice of a CT head scene the regions where uncertainty is highest and

lowest. Clearly, at the fuzzy boundaries, most of the voxels have high uncertainty.

Figure 6. The class uncertainty scene corresponding to the scene shown in Fig. 1 (a). Brightness at a voxel is proportional to its class uncertainty value.

The basic idea in this method is to identify voxels affected by PVE in a given scene by using the

uncertainty principle and to suppress them in the output scene. Thus, all voxels Cc∈ that have

uncertainty )(cU ρ not greater than 85%, i.e., 85.0=stT , and probability )(cPbnρ smaller than 100%

are considered to be voxels with PVE. We also impose constraints on the intensities )(cf as

explained in algorithm FM. The algorithm for the UP method is as follows. We note here that, since

( ) ( )st bnF c F cρ ρ+ is not necessarily constant, there may not exist a one-to-one mapping between any of

)(cF τρ , },{ bnst∈τ , and )(cU ρ . Therefore, the FM- and UP- methods generally yield different

16

results for any combination of thresholds chosen for the two methods. Further, the uncertainty

principle provides a classical approach for formulating tissue mixtures and subsequently for

estimating the fractional occupancy information for the case of multiple (≥ 2) tissue types.

Algorithm UP

Input: A scene ),( fC=C , ρ , τW , ),( SS fC=C , constants stT and bnt .

Output: ),( UPUP fC=C .

begin

set )()( cfcfUP = for all Cc∈ ;

for all voxels Cc∈ such that ρ≤)(cfS do

compute )(cPstρ , )(cPbn

ρ , and )(cU ρ ;

if stTcU <)(ρ and 1)( <cPbnρ and bntcf <)( then

set 0)( =cfUP ;

endif;

endfor;

end

In this algorithm, all parameters are determined as in Algorithm FM except stT which is set to 0.85.

stT is quite insensitive within the range [0.6, 0.85] as revealed in our experiments.

2.5 Skin Peeling

The method utilizes the probability map )(cPstρ corresponding to soft tissue to determine a thin

external layer with high soft tissue probability. The soft tissue probability map is used because the

voxels in the region of the skin have CT intensity values similar to those of soft tissue. We obtain

from each slice of the soft-tissue probability scene ),( ststρ fC ρ=C , where for any Cc∈ ,

)()( cPcf ststρρ = , an outer contour that surrounds the skin. These contours are tracked by scanning

stρC starting from the border of its scene domain and proceeding inward and by utilizing a threshold

17

of 0.5. Let ηC be the binary scene representing just the contour voxels in all slices of stρC . We then

apply n successive dilatation operations [22] to ηC by using a 5x5 structuring element. This would

create an annular shell region in the resulting binary scene nηC which would enclose all voxels that

are likely to constitute skin. A masking operation is then applied to the scene output by algorithms

FM and UP to yield a new scene in which the voxels affected by PVE as well as voxels in the skin

region (as indicated by nηC ) are removed. The algorithm, labelled SP, for skin peeling is outlined

below.

Algorithm SP

Input: A scene ),( xx fC=C , ),( ststρ fC ρ=C , a constant n .

Output: ),( SPSP fC=C .

Auxiliary Data Structure: Binary scenes ),( ηfCη =C and ),( iiη fC η=C

begin

set 1=i ;

set )()( cfcf xSP = for all Cc∈ ;

by scanning in each slice of stρC determine the skin contours and produce a binary scene ηC ;

while ni ≤ do

dilate ηC to produce a new binary scene iηC ;

set ηC to iηC and i to 1+i ;

endwhile;

for all voxels Cc∈ such that 1)( =cfη do

set 0)( =cf SP ;

endfor;

end

18

In our experiments, we have utilized

×

=pixsize

n2

5, were . represents the closest upper

integer. The rationale here is that, with n dilations, a thickness of ( pixsizen ×2 ) in mm is identified

for removal which is usually adequate for removing skin without affecting muscle and other deeper

soft tissue structures.

3. EVALUATION AND RESULTS

In this section, we present volume renditions created from scenes before and after applying the PVE

and skin removal methods for a craniofacial and knee CT data set, and the visible woman head CT

data [23]. An experiment involving a mathematical phantom created with three levels of PVE (low,

medium, and high) has also been conducted. The phantom data are created by using geometric

objects, and by assigning intensities to these objects that are in the relationship as inferred from CT

scenes, and by applying different levels of blurring (using a 2D Gaussian kernel) and a zero-mean

Gaussian correlated noise component to the geometric objects. A quantitative evaluation is carried

out by utilizing five clinical CT data sets and three mathematical phantoms. Table 1 summarizes

pertinent details related to the data sets used in this paper. We have used fixed parameter settings (as

described in the paper) for all CT scene data analyzed in this paper. For the 8-bit knee scene, the

intensity dependent parameters were proportionately scaled. Per scene adjustment of the parameters

was neither required nor allowed.

19

Table 1. A summary of the data sets used in this paper.

Dataset N Size Voxel size Number of bits

Craniofacial 5 307 x 267 x 58 0.49 x 0.49 x 1.50 mm3 16 bits

Knee 1 256 x 256 x 69 0.68 x 0.68 x 1.00 mm3 8 bits

Visible woman head 1 512 x 512 x 209 0.49 x 0.49 x 1.0 mm3 16 bits

Mathematical phantom 3 512 x 512 x 58 0.49 x 0.49 x 1.50 mm3 16 bits

3.1 Qualitative Evaluation

In Figures 8-11, we show volume renditions of one craniofacial, knee, visible woman, and a

phantom data set, respectively, for different situations. All renditions are created via shell-rendering

[24] by utilizing 3DVIEWNIX [25] (can be downloaded from www.mipg.upenn.edu) and a

trapezoidal opacity function [10]. The figures show renditions created from the original scene

without applying any of the three methods (FM, UP, SP) and after applying the methods in different

combinations. A cross section of the phantom scene with a medium level of blur and noise is shown

in Figure 7. All cross sections are similar as far as the geometric objects in them are concerned in the

phantom scene. For comparison, a surface rendition of one of the data sets – the visible woman – is

also included in Figure 12.

20

Figure 7. A slice of the 3D Phantom scene at medium level of blur and noise.

21

Figure 8. Volume renditions of the craniofacial CT data set. Top-left: from the original scene. Top-right: after peeling skin. Middle-left: after applying the FM method only. Middle-right: after applying the UP method only. Bottom left: after peeling skin and applying the FM method. Bottom-right: after peeling skin and applying the UP method.

22

Figure 9. Volume renditions of the knee CT data set. The arrangement of methods is as in Figure 8.

23

Figure 10. Volume renditions of the visible woman CT data set. The arrangement of methods is as in Figure 8.

24

Figure 11. Volume renditions of the phantom data set. Left column: after peeling skin and applying the FM method. Middle column: after peeling skin and applying the UP method. Right column: before peeling skin and PVE removal. From top to bottom, the three rows represent low, medium, and high level of blur and noise.

25

Figure 12. Surface renditions of the visible woman CT data set. The arrangement of methods is as in Figure 8.

Application of the FM and UP methods suppresses voxels with PVE (appearing as red stripes in the

forehead area in Figure 8). Figures 9 and 10 show an example of how the FM and UP methods

enhance the contrast among soft tissue structures. It is clear from Figure 11 that fine structures which

are lost in renditions created before PVE removal are portrayed remarkably well after PVE

suppression. Skin removal additionally brings out the details more vividly and we can start

identifying specific neuro-vascular and muscular substructures (Figures 8, 9, 10).

26

We have demonstrated so far that high quality renditions free of PVE can be created by using a

simple trapezoidal opacity function after applying the proposed methods. One may surmise,

however, that if more sophisticated opacity functions are used, perhaps similar effects may be

obtained. We argue that this is unlikely to be true, by observing the highly non linear and shift-

variant nature of the PVE suppression (and skin peeling) process and by noting that most opacity

functions do not satisfy these attributes. Consider, for example, an opacity function which considers

scene intensity and the magnitude of its gradient as independent variables; such functions are often

used in VR [26], and prior to the advent of VR, as implemented in 3DVIEWNIX [25], even in

segmentation [27]. The application of one form of this function, on the phantom data set utilized in

Figure 11 with a medium level of blur and noise, is demonstrated in Figure 13. In this

implementation, the opacity function is specified by two nested rectangles on a scatter plot of )(cf

and |)(| cf∇ as illustrated in Figure 13 (a). The opacity function is such that, for a point

|))(|),(( cfcf ∇ in the scatter plot falling inside the inner rectangle, its value is 1. For a point outside

the outer rectangle, the opacity value is 0. For any point falling between the two rectangles, the

opacity value varies bilinearly with )(cf and |)(| cf∇ . In the example demonstrated, skin peeling is

done first by using Algorithm SP. Figure 13 (b) and (c) show closeup renditions resulting from two

different selections of the rectangles – one (b) in which we tried to suppress as much of the voxels

subjected to PVE as possible, and the other (c) wherein we have attempted to retain as much of the

voxels containing soft-tissue as possible. Figure 13 (d) and (e) show the matching rendition for the

FM and UP methods, respectively. Comparing Figures 13 (b) and (c) to (d) and (e), it is clear that it

is very difficult to suppress PVE to the same extent by using a bivariate opacity function as is

possible by using the FM and UP methods. Perhaps by using more sophisticated functional forms,

the gradient methods could be marginally improved, but, we believe, not significantly. There are two

27

main reasons as to why such a strategy cannot be effective and which led us to the proposed methods

to begin with. First, there are voxels affected by PVE in the middle of the fuzzy boundary where

|)(| cf∇ is lower, as well as at sites adjacent to objects where |)(| cf∇ is much higher. Second, an

estimate of the amount of tissue in each voxel guided by some structural information (in our case,

scale) to determine a neighborhood would become necessary for the effectiveness of PVE removal,

and a uniform shift-invariant criterion based only on |)(| cf∇ and )(cf that does not depend on c is

unlikely to be effective. Finally, we note that something more than opacity assignment is needed in

order to get explicit hard or fuzzy segmentation so that the segmented objects can be used in further

analysis beyond simply rendering them.

28

(a)

(b) (c)

(d) (e)

Figure 13. A comparison of the FM and UP methods with a method of opacity assignment that uses )(cf and |)(| cf∇ . (a) Illustration of the specification of opacity function on a scatter plot of )(cf

(feature 1) and |)(| cf∇ (feature 2). (b), (c) Renditions (closeup views) resulting from an opacity assignment that attempted to suppress as much of the voxel affected by PVE as possible (b), and to retain as much of the soft tissue as possible (c). (d), (e) Corresponding renditions for the FM and UP methods, respectively.

29

3.2 Quantitative Evaluation

We have conducted an experiment to evaluate the performance of the FM and UP methods in

removing voxels with PVE. Since the voxels with PVE that introduce the artifacts in the phantoms

are known, the extent to which they have been removed can be quantified. A similar quantitative

evaluation is carried out by utilizing the five clinical CT data sets wherein an operator carefully

masked out voxels with PVE in the segmented images. Our method of evaluation is described below.

For any scene ),( fC=C , let ),( ,, hlhl tttt fC=C denote the binary scene resulting from thresholding

C with the threshold interval [ hl tt , ]. That is, for any Cc∈

≤≤

=otherwise.,0

,)(if,1)(, hltt tcft

cf hl (3.1)

In this paper, our main focus is on soft tissue, and, hence, in the following analysis, [ hl tt , ] pertains

to soft tissues. We define a figure of merit xFOM to describe the accuracy of removing voxels with

PVE for the FM and UP methods as follows. For ∈x {FM, UP}, and a fixed [ hl tt , ],

hlhl

hlhlhlhl

hl

,ttgs

,tt

,ttx

,tt,ttgs

,tt

ttx

EORFOM

CC

CCCCC

−

−−−=

)()(1)(, , (3.2)

where hl ttgs

,C is our “gold-standard” binary scene resulting from masking out voxels with PVE

manually in hl tt ,C . hl ttx

,C is a binary scene resulting after applying method x to C and subsequently

thresholding the resulting scene xC with threshold interval [ hl tt , ]. hlh ,ttgs

t CC lt −, denotes the number

of 1-valued voxels in )( hlhl ,ttgs

,tt CC − , which in turn represents the true set of voxels in C that are

affected by PVE. )( ,, hlh ttx

t CC lt − represents the PVE voxels that were identified by method x .

EOR denotes the exclusive OR operation between the two binary scenes, and

30

)()( hlhlhlhl ,ttx

,tt,ttgs

,tt EOR CCCC −− therefore indicates the number of voxels with PVE that were

missed by method x at threshold interval [ hl tt , ]. In order to avoid the dependence on [ hl tt , ], we

vary lt and ht within reasonable limits (as determined by visual inspection of the resulting binary

scenes) and obtain the best xFOM for each method x for each scene:

)]([max)( ,

,CC hl

hl

ttx

ttx FOMFOM = . (3.3)

For each method x , )(CxFOM thus represents the best possible degree of match between the voxels

with PVE captured in )( ,, hlh ttgs

t CC lt − and the voxels with PVE classified in )( ,, hlh ttx

t CC lt − over all

possible threshold intervals [ hl tt , ] in C . The mean, standard deviation, and minimum of )(CxFOM

over the eight scene data sets are listed in Table 2 for methods FM and UP. The results indicate that,

on the average, a minimum of 98.2% and a maximum of 99.7% of voxels affected by PVE are

removed by the two methods. A paired student’s t -test [28] of the 8 pairs of mean )(CxFOM data

was conducted under the null hypothesis that there is no statistical difference between the two

methods. The hypothesis was rejected at a p-value of 0.302. Thus, although both qualitatively and

quantitatively the UP method seems to be slightly better, the difference between the two methods is

not statistically significant as revealed by our quantitative evaluation.

31

Table 2. Mean, Standard Deviation (SD), and Minimum of the xFOM values for FM and UP methods.

FM

Mean SD Minimum

UP

Mean SD Minimum

Mathematical phantom (N=3) 0.982 0.019 0.956 0.990 0.010 0.979

Craniofacial (N=5) 0.992 0.011 0.972 0.997 0.004 0.988

Although we have concentrated mainly on PVE and soft tissue display, PVE also affects bone

display, mainly in the portrayal of thin bones. Further, skin, which in terms of CT numbers, appears

exactly like thin bone, obscures the display of the latter. These phenomena are illustrated in Figure

14 for the knee scene data set. Soft tissue and skin, which now manifest themselves as pseudo-bone

in the renditions, can be suppressed by using a variant of the UP method described previously for

soft tissues but modified for bone. As seen in Figure 14 (c), these effects can be overcome to produce

clean volume renditions of bone, and at the same time, not losing thinner aspects of the bone, in this

case in the vicinity of the medial epicondyle of femur and patella. In Figure 14 (b), skin was

removed but the soft-tissue obscuration produces a volume rendition of the bone with unrealistically

smooth appearance. For comparison, we have included in Figure 14 (d) the straightforward bone

rendition (analogous to (a)) obtained by choosing the opacity function so as to remove pseudo-bone.

This results in the loss of thin bones. Comparing Figures (a) and (c) with (d) as a reference, it is clear

that more detailed rendition devoid of pseudo-bone is created by a combination of skin peeling and

PVE suppression.

32

(a) (b) (c)

(d)

Figure 14. (a) Straightforward bone rendition of the knee data set. The portrayal of thin bones in the region of medial epicondyle of femur and patella is obscured by pseudo-bone (skin and soft tissues). (b) Bone display after peeling skin. The obscuration due to soft tissue structures still remains, and hence the unrealistic smooth appearance of bone on the lateral and posterior aspects of the femur. (c) Bone display after peeling skin and suppressing PVE due to soft tissue by utilizing the UP method. (d) Straightforward bone rendition from an opacity assignment that attempted to remove pseudo-bone.

For the CT data sets used in this paper (craniofacial and knee), the average processing time for the

proposed methods on a Pentium III (450MHz and 256MB RAM) PC was as follows: 141 s for scale

computation, 168 s for the FM method and 273 s for the UP method. The total processing time is

under 1 minute on modern 3GHz PCs. Shell rendering subsequently proceeds at interactive speeds (a

fraction of a second per frame). Please refer to [24], [29] for detailed information about shell-

rendering performance.

33

4. CONCLUSIONS

We have presented two methods called the FM (fractional measure) and the UP (uncertainty

principle) method for correcting for partial volume effects in CT images, mainly for the purpose of

3D visualization, although the results can be utilized in conjunction with further segmentation and

analysis of the image. Both methods utilize the scale information at every voxel which is derived

from the given scene by using a simple homogeneity criterion. The FM method estimates tissue

fractions in every voxel and conditionally suppresses them. The UP method uses an information-

theoretic uncertainty principle to detect voxels with high uncertainty and PVE to suppress them.

Once the scale scene is computed, both methods are quite simple and easy to implement within any

volume rendering software, and operate completely automatically, taking a total processing time of

under 1 minute on modern 3 GHz PCs. We have also presented a method to peel skin in CT images.

Skin causes a considerable degree of obscuration of soft tissue display and even of bone where they

are thin or fine. The combined effect of the three methods is a significant improvement in the quality

of renditions and in the details they portray, particularly for soft-tissue display and for the display of

bone with fine details.

In CT images, since we are interested mainly in portraying soft tissue and bone, renditions become

defiled due to PVE at bone-to-air and bone-to-fat interface where soft-tissue-like pseudo structures

appear. These may tarnish the display of both bone and soft-tissue structures, as we have

demonstrated in this paper. We have confined ourselves to the case of mixtures of two tissues in the

treatment of voxels with PVE. Nevertheless, the FM and UP methods can be both extended to the

case of mixtures of more than two tissues, e.g., the intersection regions of bone, fat and air, although

such regions are very sparse in the CT scene. Although we have removed voxels with PVE in our

34

experiments in a hard fashion, linear or non-linear fraction combinations of the different types of

tissues may be devised in order to smoothly attenuate the voxels with PVE in a fuzzy manner.

Perhaps such strategies could be built into the classification functions for opacity assignment in

volume rendering. That is, the opacity assigned to any voxel c by an opacity function should be

further modulated by the evidence gathered by the scale scene (as utilized in the FM or the UP

method) as to the degree of presence of PVE at c . To accomplish this, information pertaining to

PVE should be first obtained, perhaps in the form of a scene. As illustrated by the gradient method

(Figure 13), simple shift-invariant strategies employed in opacity functions themselves are unlikely

to be able to suppress PVE without jeopardizing the tissues of interest. Note that, in the proposed

method, all voxels in the scale region of c are consulted in estimating PVE at c . Perhaps opacity

functions could be devised that utilize the scale scene SC computed before hand for a given scene

C in the spirit of the FM and UP methods.

3D renditions should be used with care in radiology in view of artifacts demonstrated in this work

coming from PVE. These artifacts can be effectively overcome by the proposed methods, the UP

method performing slightly better. Further, skin peeling combined with PVE suppression

significantly enhances the quality of 3D renditions particularly for portraying fine details. The degree

of improvement achieved is higher for images of lower resolution. The proposed methods removed

more than 98% of the voxels with PVE in clinical CT data and mathematical phantoms. Clearly, it is

worth studying the PVE vis-a-vis visualization phenomenon in MR imagery, since the latter has

become ubiquitous in Radiology. In the current method, the voxels with PVE are removed altogether

for volume rendering. Our future research along this line will be to obtain the fuzzy occupancy of

each tissue type at partially occupied voxels and to use them in volume and surface rendering.

Another alternative for the method will be to generate a higher resolution scene by utilizing the

35

knowledge of fuzzy occupancy of different tissue types at partially occupied voxels and the scene

intensity distributions.

ACKNOWLEDGMENTS

The research reported here is supported by the Sao Paulo Research Foundation-FAPESP (proc.

97/09202-0) and a US DHHS grant NS37172.

REFERENCES

1. Y. Sato, C. Westin, A. Bhalerao, S. Nakajima, N. Shiraga, T. Tamura, R. Kikinis, “Tissue

classification based on 3D local intensity structures for volume rendering”, IEEE Trans. Visual.

Comput. Graph., vol. 6, no. 2, pp. 160-179, 2000.

2. J. Beier, R. C. Bittner, N. Hosten, J. Troger, R. Felix, “Morphological image processing

operators: reduction of partial volume effects to improve 3D visualization based on CT data”,

Radiologe, vol. 38, no. 10, pp. 860-866, 1998.

3. P. Santiago, H. D. Gage, “Statistical-models of partial volume effect”, IEEE Trans. Image

Processing, vol. 4, no. 11, pp. 1531-1540, 1995.

4. H. Rusinek, M. E. Noz, G. Q. Maguire, A. Kalvin, B. Haddad, D. Dean, C. Cutting,

“Quantitative and qualitative comparison of volumetric and surface rendering techniques”, IEEE

Trans. Nucl. Sci., vol. 38, no. 2, pp. 659-662, 1991.

5. D. J. Heuscher, M. Vembar, “Reduced partial volume artifacts using computed tomography and

an integrating interpolator”, Med. Phys., vol. 26, no. 2, pp. 276-286, 1999.

6. U. Tiede, T. Schiemann, K. H. Höhne, “High quality rendering of attributed volume data”, Proc.

IEEE Visualization 1998, pp. 255-262, 1998.

36

7. J. K. Udupa, H. M. Hung, K. S. Chuang, “Surface and volume rendering in 3D imaging: a

comparison”, J. Digital Imaging, vol. 4, no. 4, pp. 159-68, 1991.

8. A. Kaufman, “Introduction to volume visualization. In: Kaufman A. (ed.)”, Volume

Visualization, pp. 1-18, 1991.

9. J. K. Udupa, G. Herman (Eds.), 3D Imaging in Medicine, CRC Press, Boca Raton, FL, 1991.

10. R. A Drebin, L. Carpenter, P. Hanrahan, “Volume rendering”, Comput. Graph., vol. 22, no. 4,

pp. 65-74, 1988.

11. M. de Boer, A. Gröpl, J. Hesser, R. Männer, “Reducing artifacts in volume rendering by higher

order integration”, Proc. IEEE Visualization 1997, pp. 1-4, 1997.

12. K. Novins, J. Arvo, “Controlled precision volume Integration”, 1992 Workshop on Volume

Visualization, pp. 83-89, 1992.

13. G. Kindlmann, J. W. Durkin, “Semi-automatic generation of transfer functions for direct volume

rendering”, Proc. IEEE Symp. Volume Visualization, pp. 79-86, 1998.

14. J. Kniss, G. Kindlmann, C. Hansen, “Multi-Dimensional Transfer Functions for Interactive

Volume Rendering”, IEEE Trans. Visual. Comput. Graph., vol. 8, no. 4, pp. 270-285, 2002.

15. K. H. Höhne, W. A. Hanson, “Interactive 3D-segmentation of MRI and CT volume using

morphological operations”, J. Comput. Assist. Tomogr., vol. 16, no. 2, pp. 285-294, 1992.

16. D. H. Laidlaw, K. W. Fleischer, A. H. Barr, “Partial-volume bayesian classification of material

mixture in MR volume data using voxel histograms”, IEEE Trans. Med. Imag., vol. 17, no. 1, pp.

74-86, 1998.

17. J. P. Windham, M. A. Abd-Allah, D. A. Reimann, J. W. Froelich, A. M. Haggar, “Eigenimage

filtering in MRI imaging”, J. Comput. Assist. Tomogr., vol. 12, no. 1, pp. 1-9, 1988.

37

18. M. R. Siadat, H. Soltanian-Zadeh, “Partial volume estimation using continuous representations”,

Proc. SPIE, vol. 4322, pp. 1025-1034, 2001.

19. P. K. Saha, J. K. Udupa, D. Odhner, “Scale-based fuzzy connected image segmentation: theory,

algorithms and validation”, Comput. Vis. Image Understanding, vol. 77, pp. 145-174, 2000.

20. A. Souza, J. K. Udupa, P. K. Saha, “Volume rendering the in presence of partial volume effects”,

Proc. SPIE, vol. 4681, pp. 649-660, 2002.

21. P. K. Saha, J. K. Udupa, “Optimum image thresholding via class uncertainty and region

homogeneity”, IEEE Trans. Pattern Anal. Machine Intell., vol. 23, no. 7, pp. 689-706, 2001.

22. J. Serra, Image Analysis and Mathematical Morphology, London: Academic, 1982.

23. M. J. Ackerman, “The Visible Human Project, National Library of Medicine”, World Wide Web

site, http://www.nlm.nih.gov, 1995.

24. J. K. Udupa, D. Odhner, “Shell rendering”, IEEE Comput. Graph. Applicat., vol. 13, no. 6, pp.

58-67, 1993.

25. J. K. Udupa, D. Odhner, S. Samarasekera, R. Goncalves, K. Iyer, K. Venugopal, S. Furie,

“3DVIEWNIX: An open, transportable, multidimensional, multimodality, multiparametric

imaging software system”, Proc. SPIE, vol. 2164, pp. 58-73, 1994.

26. M. Levoy, “Display of surfaces from volume data”, IEEE Comput. Graph. Applicat., vol. 8, no.

3, pp. 29-37, 1988.

27. S. S. Trivedi, G. T. Herman, J. K. Udupa, “Segmentation into three classes using gradients”,

IEEE Trans. Med. Imag., vol. MI-5, pp. 116-119, 1986.

28. B. Ronser, Fundamentals of Biostatistics. New York: Duxbury Press, 1995.

38

29. G. J. Grevera, J. K. Udupa, D. Odhner, “An order of magnitude faster isosurface rendering in

software on a PC than using dedicated, general purpose rendering hardware”, IEEE Trans.

Visual. Comput. Graph., vol. 6, no. 4, pp. 335-345, 2000.