the effect of median filtering on edge estimation and detection

14
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987 The Effect of Median Filtering on Edge Estimation and Detection ALAN CONRAD BOVIK, MEMBER, IEEE, THOMAS S. HUANG, FELLOW, IEEE, AND DAVID C. MUNSON, JR., SENIOR MEMBER, IEEE Abstract-In this paper we consider the effect of median prefiltering on the subsequent estimation and detection of edges in digital images. Where possible, a quantitative statistical comparison is made for a number of filters defined with two-dimensional geometries; in some cases one-dimensional analyses are required to illustrate certain points. Noise images prefiltered by median filters defined with a variety of win- dowing geometries are used to support the analysis, and it is found that median prefiltering improves the performance of both thresholding and zero-crossing based edge detectors. Index Terms-Computer vision, edge detection, image processing, median filter, order statistics. I. INTRODUCTION M /[EDIAN filtering is a nonlinear filtering technique that has been successfully applied to many signal and image processing tasks. Most notably, the median fil- ter (first described by Tukey [1]) has been observed to be very effective for removing noise, especially impulse noise from one- or two-dimensional signals, while satis- fying the usually conflicting goal of preserving informa- tion-bearing edges [2], [3]. For filtering problems in im- age processing and computer vision these are very important considerations, as digital images are strongly characterized by sustained intensity changes correspond- ing to the boundaries between objects, or between object and background. This is illustrated by the fact that the most fundamental task in most machine vision systems involves the location of edges for segmentation purposes or other higher level processing. In this context, it is clear that if noise reduction is to be effected prior to detecting edges, then the filtering strategy used must not severely degrade the edge content in the image. Several authors have discussed the effectiveness of the median filter for image processing problems, including Pratt [4], who observed that, while the technique is effec- tive for removing impulse noise (which he interpreted as noise which is not smoothly generated, arising, for ex- Manuscript received May 25, 1984; revised August 27, 1986. Recom- mended for acceptance by S. W. Zucker. A. C. Bovik was with the Coordinated Science Laboratory and the De- partment of Electrical and Computer Engineering, University of Illinois, Urbana, IL 61801. He is now with the Department of Electrical and Com- puter Engineering, University of Texas, Austin, TX 78712. T. S. Huang and D. C. Munson, Jr. are with the Coordinated Science Laboratory and the Department of Electrical Engineering, University of Illinois, Urbana, IL 61801. IEEE Log Number 8612398. ample, from bit errors) from images while maintaining the integrity of edges, it is an "ad hoc" tool of somewhat limited use. Bovik, Huang, and Munson [5] have suc- cessfully applied generalized filters based on linear com- binations of order statistics to the image filtering problem, and demonstrated that among these the median filter is nearly optimal for suppressing noise which is character- ized by a large percentage of outliers, viz. very heavy- tailed or impulsive noise. Yang and Huang [6] found that median filtering does not improve (least-square) estima- tion of edge location, but more recently Haralick [7] has demonstrated a (related) technique to be effective for pre- processing noisy images prior to applying directional de- rivative based edge operators. However, most of the me- dian filtering literature to this point has been concerned with root-signal (signals which are invariant to median filtering) analyses and with nondynamical statistical con- siderations. Our goal in this paper is to cast some light on the nonstatic behavior of the filter, particularly regarding its efficacy as a prefiltering strategy prior to edge esti- mation or detection. First we will review the median fil- ter. Suppose we have a finite set of independent and iden- tically distributed (iid) random variables X = (XI, X2, X2m + I), where m is a positive integer (the median filter is most often defined as operating on an odd number of input values). Associated with X we define a random vector X(.) = ( X1), X( 2) * ** , X(2m +I)) of order statistics, which are the algebraically ordered versions of the elements of X such that X(l) c X(2) cX(2m + ). Thus the me- dian, or middle value of X in algebraic rank is X(m±+ 1) The median has long been recognized as a useful estimate of location (mean) in the statistical literature, as it is ef- ficient when the underlying distribution has very heavy tails, and is in fact the maximum likelihood location es- timator (MLE) in the asymptotic sense (as m T oo) if the elements of X are double-exponentially distributed. We will find it useful to note that it X is linearly related to another random vector N = (N1, N2, * N2,+ I according to Xit=iNit+a;oio= 1, s2mt+ then it also follows that 0162-8828/87/0300-0181$01.00 © 1987 IEEE 181

Upload: sjusd

Post on 26-Nov-2023

0 views

Category:

Documents


0 download

TRANSCRIPT

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987

The Effect of Median Filtering on Edge Estimationand Detection

ALAN CONRAD BOVIK, MEMBER, IEEE, THOMAS S. HUANG, FELLOW, IEEE,

AND DAVID C. MUNSON, JR., SENIOR MEMBER, IEEE

Abstract-In this paper we consider the effect of median prefilteringon the subsequent estimation and detection of edges in digital images.Where possible, a quantitative statistical comparison is made for anumber of filters defined with two-dimensional geometries; in somecases one-dimensional analyses are required to illustrate certain points.Noise images prefiltered by median filters defined with a variety of win-dowing geometries are used to support the analysis, and it is found thatmedian prefiltering improves the performance of both thresholding andzero-crossing based edge detectors.

Index Terms-Computer vision, edge detection, image processing,median filter, order statistics.

I. INTRODUCTIONM/[EDIAN filtering is a nonlinear filtering technique

that has been successfully applied to many signaland image processing tasks. Most notably, the median fil-ter (first described by Tukey [1]) has been observed to bevery effective for removing noise, especially impulsenoise from one- or two-dimensional signals, while satis-fying the usually conflicting goal of preserving informa-tion-bearing edges [2], [3]. For filtering problems in im-age processing and computer vision these are veryimportant considerations, as digital images are stronglycharacterized by sustained intensity changes correspond-ing to the boundaries between objects, or between objectand background. This is illustrated by the fact that themost fundamental task in most machine vision systemsinvolves the location of edges for segmentation purposesor other higher level processing. In this context, it is clearthat if noise reduction is to be effected prior to detectingedges, then the filtering strategy used must not severelydegrade the edge content in the image.

Several authors have discussed the effectiveness of themedian filter for image processing problems, includingPratt [4], who observed that, while the technique is effec-tive for removing impulse noise (which he interpreted asnoise which is not smoothly generated, arising, for ex-

Manuscript received May 25, 1984; revised August 27, 1986. Recom-mended for acceptance by S. W. Zucker.

A. C. Bovik was with the Coordinated Science Laboratory and the De-partment of Electrical and Computer Engineering, University of Illinois,Urbana, IL 61801. He is now with the Department of Electrical and Com-puter Engineering, University of Texas, Austin, TX 78712.

T. S. Huang and D. C. Munson, Jr. are with the Coordinated ScienceLaboratory and the Department of Electrical Engineering, University ofIllinois, Urbana, IL 61801.

IEEE Log Number 8612398.

ample, from bit errors) from images while maintaining theintegrity of edges, it is an "ad hoc" tool of somewhatlimited use. Bovik, Huang, and Munson [5] have suc-cessfully applied generalized filters based on linear com-binations of order statistics to the image filtering problem,and demonstrated that among these the median filter isnearly optimal for suppressing noise which is character-ized by a large percentage of outliers, viz. very heavy-tailed or impulsive noise. Yang and Huang [6] found thatmedian filtering does not improve (least-square) estima-tion of edge location, but more recently Haralick [7] hasdemonstrated a (related) technique to be effective for pre-processing noisy images prior to applying directional de-rivative based edge operators. However, most of the me-dian filtering literature to this point has been concernedwith root-signal (signals which are invariant to medianfiltering) analyses and with nondynamical statistical con-siderations. Our goal in this paper is to cast some light onthe nonstatic behavior of the filter, particularly regardingits efficacy as a prefiltering strategy prior to edge esti-mation or detection. First we will review the median fil-ter.Suppose we have a finite set of independent and iden-

tically distributed (iid) random variables X = (XI, X2,X2m + I), where m is a positive integer (the median filter ismost often defined as operating on an odd number of inputvalues). Associated with X we define a random vector X(.)= ( X1), X( 2) * ** , X(2m +I)) of order statistics, whichare the algebraically ordered versions of the elements ofX such that X(l) c X(2) cX(2m+ ). Thus the me-dian, or middle value of X in algebraic rank is X(m±+ 1)The median has long been recognized as a useful estimateof location (mean) in the statistical literature, as it is ef-ficient when the underlying distribution has very heavytails, and is in fact the maximum likelihood location es-timator (MLE) in the asymptotic sense (as m T oo) if theelements of X are double-exponentially distributed.We will find it useful to note that it X is linearly related

to another random vector N = (N1, N2, * N2,+ I

according to

Xit=iNit+a;oio= 1, s2mt+

then it also follows that

0162-8828/87/0300-0181$01.00 © 1987 IEEE

181

IEEE 'FRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987

(a) (b)

(c)

Fig. 1. Common two-dimensional median filter geometries: (a) square, (b)cross-shaped, and (c) X-shaped.

where a and A are arbitrary constants, and where N(.) =(N(1), N 2 *, N(2m+1)) are the order statistics of N.

In one dimension, the median filter is defined as fol-lows: for a one-dimensional input sequence { Xi } we takethe output of a (2m + 1 )-point median filter to be

'j(m) = median (Xj , X * Xj +m)when the filter is centered at signal coordinate j. Hereafterwe will drop the superscript and denote the median simplyas Xj, tacitly assuming the window to contain 2m + 1values unless otherwise stated. Two-dimensional medianfilters can be defined in many different ways, dependingon both the size and geometry of the filter window; themost common geometries in general use are the square-,cross-, and X-shaped windows (see Fig. 1). A separablemedian filter has also been discussed [8], which is definedby first passing a horizontally oriented one-dimensionalfilter along each image row; the resulting image is thenfiltered by passing the filter (rotated 90°) over each col-umn. This is not a true two-dimensional filter, however,and will not be considered here. The two-dimensionalmedian filtering operation is defined as follows: when a2-D filter window W(i, j ) is centered at image coordi-nates (i, j ), we take the filter output to be

Xij = median {Xi : (i', j' ) E W(i, j)}.

In much of the following we will be concerned with therelative performances of filters defined with different win-dow geometries and sizes. The selection of a filter ge-ometry is important for two reasons: 1) the degree ofoverlap between adjacent or neighboring filter windows(roughly, the percentage of pixels relative to the windowsize being used that the windows have in common) deter-mines the statistical dependence of the output medians [9],and 2) the geometry effects the performance of the filternear edges aligned along various orientations.

II. NOISE SUPPRESSIONAn important measure of the effectiveness of a prefil-

tering strategy is the ability of the technique to suppressnoise in regions of constant image intensity, since noise

can confound the following edge detection process andlead to the detection of false edges having no physicalcorrelate.

A. Variance of Edge Estimates in Level Regions

Suppose that { Xij } is a white-noise contaminated ver-sion of a constant-valued signal:

Xij = h + aNi,j v (i,j)where { Nij } is a two-dimensional sequence of zero-meaniid random variates with continuous distribution F, den-sity f, and variance 1. Let us also define the median fil-tered versions of the observed and noise sequences,{Xj } = med{IX }, {Ni,j } = med{ Ni, }. Although theassumption of independence is clearly not realistic formost applications (we briefly discuss the effect of noisedependence later), it does allow us to obtain certain re-sults describing the accuracy of edge estimates in medianfiltered image data. To further simplify the analysis, weconsider the most primitive type of edge estimate: at im-age coordinates (i, j) define the simple differencing es-timates

A1x(i,j) = -Xi,j+and

Ad(i,j) = Xi,j-Xi+ ,j+I (1)

along the horizontal (for estimating vertical edges) andone diagonal direction, respectively (the analysis is iden-tical for estimates defined along directions orthogonal tothese). Of interest are the normalized variances of edgeestimates

F= Var [(i,j)]/2 = (Ni,jNi +

and

O2= Var [Ad (i, j)]/a2 = E[(Ni j -Ni+ I,j+ 1)2(AV and Ad are clearly zero-mean), whose values dependon three factors: 1) the number of image samples spannedby the filter windows, 2) the number of samples simul-taneously covered by both filter windows when thefilters are positioned at adjacent image coordinates, whichis determined by the size and geometry of the windowbeing used, and 3) the form of the underlying noise distri-bution F. We consider square-, cross-, and X-shaped geom-etries of integer span 2m + 1 (m > 0) (hereafter re-ferred to as SQUARE (2m + 1), CROSS (2m + 1), andXSHAPE(2m + 1)) as shown in Fig. 1(a)-(c), respec-tively, for 2m + 1 = 5. The number of samples spannedby SQUARE (2m + 1) is M = (2m + 1 )2, while for bothCROSS (2m + 1) and XSHAPE (2m + 1) geometries M=4m + 1.For two windows centered at respective image coordi-

nates (i, j) and (i, j + 1) (horizontally adjacent win-dows) the number of image samples covered by both win-

182

BOVIK et al.: EFFECT OF MEDIAN FILTERING ON EDGE ESTIMATION

dows is given by

2m(2m + 1);

K2, = i 2m;

t0;

SQUARE(2m + 1)

CROSS(2m + 1)

XSHAPE(2m + 1)

while for two windows centered at respective image co-ordinates ( i, j ) and ( ' + 1, j + 1 ) the samples shared bythe windows is

Kd :=

4m2; SQUARE(2m + 1)

2; CROSS (2m + 1)

0.6k

G j X - XSHAPE\ + - CROSS

0.4+0 -SQUARE

0.21t\ x

F `111~~~~~~ +

3x3 5x5 span > 1 3x13

Fig. 2. Variance of edge estimates in a level image region along the hor-izontal/vertical tracks, following median prefiltering using XSHAPE,CROSS, and SQUARE filter geometries.

XSHAPE(2m + 1).

The number of shared samples, or overlaps K, and Kdstrongly constrains the dependencies of the median valuestaken from adjacent filter windows; in fact, for anyK1,, Kd > 0 there is a nonzero probability that the medianswill take equal value (such an event will occur when themedian values arising from both windows coincide in asample lying in the region covered by both windows), de-spite the assumption of continuity on the underlying noisedistribution F [3, 10]. The joint probability density fM,Kof median values taken from two windows, each contain-ing M samples, which coincide in K samples can be foundusing formulas in [9] or [10]; however, as the expressionis somewhat involved we will not reproduce it here.The variances in (6) are then given by

tK (x - Y) fMK,I (X, y) dx dy

and

d (x - Y) fMK(l(x y) dX dy (2)

where fM,K,, and fJM,K, are the bivariate densities of (Nij,Ni,j+1) and (Nij, Ni+±j+±), respectively. Using (2), wehave computed the variances of edge estimates followingmedian prefiltering over a level region under a Gaussian-q(0, 1) noise assumption, for both the horizontal (A,,)and diagonal (Ad ) directions for each of the filter geom-etries being considered (generally the expressions in-volved require numeric integration; for our purposes a 256Gaussian tensor product formula was sufficient to achievefour decimal digits of precision in all cases). Specifically,u, and2J are shown in Figs. 2 and 3, respectively, forCROSS and XSHAPE filters with spans 2m + 1 = 3, 5,and 13, and for SQUARE filters with spans 2m + 1 = 3and 5 (so that comparison can be made between both fil-ters of similar spans and overall area). The plots indicatethat for filters with either similar spans or areas, theCROSS filter effects a greater degree of noise suppressionthan the XSHAPE along horizontal or vertical imagetracks, while the reverse usually holds along the diagonaltracks. SQUARE filtering gives the best results for filters

3x3 5x5 span - 13x13

Fig. 3. Variance of edge estimates in a level image region along the di-agonal tracks, following median prefiltering using XSHAPE, CROSS,and SQUARE filter geometries.

defined with similar spans, but the degree to which this istrue is largely due to the greater number of samples con-tained on SQUARE windows as compared to the others.However, for windows containing the same number ofsamples (windows with the same overall areas), theSQUARE filter still effects superior noise suppression:compare the variances for CROSS(13) and XSHAPE(13)with that of SQUARE(5), or the variances of CROSS(5)and XSHAPE(5) with that of SQUARE(3). Clearly, theability of a filter to effect the variance of edge estimatesin level regions (away from edges) is enhanced by an in-creased dependency between adjacent filtered image sam-ples, which is constrained by the percentage of sampleswhich is shared between the respective windows. Adja-cent SQUARE filters overlap in a greater percentage ofthe windowed samples, affording superior noise suppres-sion, at least in flat regions, while the relative merits ofCROSS and XSHAPE geometries depends on the direc-tion that the edge estimate is being taken.

B. Detection of Psuedo-EdgesHere we analyze the effectiveness of the various filters

in the context of (false-alarm type) error rates: what is theprobability that a diagonal or vertical edge will be incor-rectly detected at image coordinates (i, j ) following me-dian prefiltering? In the following we define the edge de-tection operation as a thresholding of the edge estimates(1): for a predetermined threshold parameter T, define the

2m;

u.

183

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987

1.0

pd

- XSHAPE(3)--- CROSS(3)- SQUARE(3)

0.8 -

0.6

0.4

0.2

0.5 1.0 1.5 2.0t/ >

(a)1 .0 -

PV0.8-

0.6

0.4 -f

0.2 -

0.0

0.4

I"I." - SC

I~~~~~~~

I §S

-- XSHAPE(3). CROSS(3!- SQUARE(3)

0.5 1.0 1.5>

2.0

(a)

SHAPE(5)ROSS(5)QUARE(5)

i i0. 1.o0 1.5-~--

0.5 1.0 1.5

(b)

2.0

Fig. 4. Probabilities of detecting pseudo-edges along the horizontal/ver-tical tracks in a level image region for SQUARE, CROSS, and XSHAPEfilters: (a) filter span 2m + 1 = 3, (b) filter span 2m + 1 = 5.

XSHAPE(5)

CROSS(5)SQUARE(5)

0 .0v a

0.0 0.5 1.0 1.5 2.0/1 >

(b)Fig. 5. Probabilities of detecting pseudo-edges along the diagonal tracks

in a level image region for SQUARE, CROSS, and XSHAPE filters: (a)filter span 2m + 1 = 3, (b) filter span 2m + 1 = 5.

false detection probabilities

PV = PVM() = Pr {Aj(ij) > T}

= Pr1-Nj Nj+I

= 1- BfM, Kv(x, y) dx dy

and

Pd= Pd,M() Pr { Ad(i, j)I > 7}

Pr Ni-Ni4

= 1 - , , tM Kd (XI y) dx dy

where B = {(x, y): x- yI < T/a}. Thus, Pv andPdrepresent the probabilities that the edge estimates (1) willtake values greater than r (i.e., a psuedo-edge has oc-

curred) along the vertical/horizontal or diagonal tracks as

shown in Fig. 4(a), (b) (Pv) and Fig. 5(a), (b) (Pd), re-

spectively, under the same Gaussian noise assumption,for filter spans 2m + 1 = 3 and 5. The performances herebear out the estimator variances found in the previous sec-

tion: for windows of the same size (span or area), prefil-tering with CROSS median filters yield fewer pseudo-edges along the horizontal or vertical tracks than prefil-tering with XSHAPE filters, while the reverse can be seen

to generally hold along the diagonal tracks (although theCROSS and XSHAPE error rates are the same for diago-

nal pseudo-edges for 2m + 1 = 3, since the overlap be-tween adjacent windows is Kd = 2 for each geometry).SQUARE filtering affords the lowest error rates, for filtersof similar spans (again, largely because SQUARE geom-etries contain a larger number of samples), and also forfilters containing the same number of samples (comparethe results of SQUARE(3) with those of CROSS(5) andXSHAPE(5)). It is interesting to note that Pv < 1 for allvalues of /a for SQUARE and CROSS filters, since theevent { Ni = Ni1+ 1 } takes nonzero probability for any m> 0 (the filters overlap), while PV -- as r -* 0 for

XSHAPE filters, since XSHAPE geometries do not over-

lap along horizontal/vertical tracks. Similarly, Pd < 1 forall of the filters we are considering for m > 0. The ten-dency of the median filter to produce equal-valued outputsat adjacent image coordinates may be regarded as one ofthe powerful features of the technique, since such an oc-

currence will guarantee that no edge will be detected re-

gardless of the threshold used. However, as discussed inSection III, these events can also occur when an edge ispresent, suggesting that the subsequent edge detection willeither miss the edge or displace it from its true coordi-nates; also, when several consecutive median filter out-puts (over a line or small region) take equal values, thevisual effect in the filtered image may be one of "streak-ing" or "blotching," viz. the creation of artifacts havingno physical correlate (see Section III).We may conclude from the preceding analysis that, un-

der our somewhat limited assumptions of noise indepen-dence and idealized image modeling, that SQUARE fil-tering is best for removing unwanted edges in any givendirection, while the choice between CROSS filtering andXSHAPE filtering depends on the relative preponderancesof edges at horizontal and vertical orientations as opposed

PV0.8

0.6

0.4

0.2

184

0

BOVIK et al.: EFFECT OF MEDIAN FIt TERING ON EDGE ESTIMATION

to diagonally placed edges. However, there are manyother filtering strategies that provide better noise suppres-sion than median filtering for most noise possibilities; thepower of the technique lies in the ability of the filter toremove noise while maintaining edge integrity. In the nextsection we investigate this aspect of the median filter,making comparisons between the various filter geometriesin two dimensions when possible, and reverting to a one-dimensional analysis when necessary.

III. EDGE DEGRADATIONThe median filter has been widely recognized as an edge

preserver, but not an edge enhancer, in the sense that itdoes not ordinarily increase the contrast of an edge.Therefore we might suspect that the median filter may tendto introduce a certain degree of edge degradation (partic-ularly in the presence of noise), although clearly the tech-nique is superior to linear methods as a prefiltering strat-egy.

A. Moments of Median Filtered Data Near EdgesHere we consider the behavior of median filtered image

data in the vicinity of an idealized step edge with addednoise. We start by defining the idealized noisy edge im-ages {X"j } and {Xd } given by (h > 0)

oNi j;ij h + aNi, ;

j Oj-1>

and

Xd (aNi,j i +j < 0

i +j > 0

where { Nij } is a zero-mean sequence of iid random vari-ates with continuous distribution F, density f, and unityvariance as before. Thus, {Xj } and {Xj } representnoisy images containing a single edge oriented verticallyand diagonally, respectively, and having constant meanaway from the edge. We will consider the first and secondmoments of median filtered data near the edges in (3) and(4); specifically, define

A2(i,j) Var [X1'/u]At'(i, j ) = E[X^, a/ar];and

Ad (i, j ) = E[XI aa] d i,j) - Var [Xijf/u]where {Xfj } and {Xdi } are the median filtered versionsof the edge images { X''; } and { Xij }, for some (as yetundefined) filter geometry. The normalized median valuesXj/ a and Xij/ a are computed from L"' and Ld valueswith distribution F(x), respectively, and M- L and M- Ld values with distribution F(x -h/a), respectively,where M is the number of image samples spanned by thefilter window (as before), and where L" and Ld depend onthe filter span 2m + 1 and on the distance of the operatorfrom the edge. Fig. 6 illustrates the horizontal and diag-

<./-XN

L:-1I

(b)Fig. 6. Filtering paths used in comparing moments of median filtered im-

age data across an idealized (a) vertical edge, (b) diagonal edge.

onal filter paths we will consider in computing the mo-ments of the filtered image data across the edges in (3)and (4), as a function of the distance of the filter windowfrom the edge. It should be noticed that Ld and M - Ldare not symmetrically defined for CROSS and SQUAREfilter windows which pass over the diagonal edge (4). Inwhat follows we consider the result of filtering along theline i j. The density function of the median value takenfrom L variates with distribution F(x) andM - L variateswith distribution F(x - h/a) can be found in [5], wherean expression is given for the univariate density of anyorder statistic taken from a sample of independent, non-identically distributed random variates; however, theexpression is extremely cumbersome (it is in the form ofa matrix permanent) and will not be reproduced here. Fig.7(a), (b), and Fig. 8(a), (b) show plots of t,, and ,ud, and

2 2

U,, and ad, respectively, for XSHAPE(5), CROSS(5), andSQUARE(3) filters, under a Gaussian q, (0, 1) noise as-sumption with SNR (edge height-to-noise ratio) h/a =

4. For filters covering an equal number of image samples(M = 9), it is apparent that CROSS filtering effects agreater degree of horizontal/vertical edge preservationthan XSHAPE filtering, in the sense that the expected fil-ter output takes values closer to the actual signal withmuch smaller variances. The reason for this is simply that,for filter windows of a given span lying at the same po-sition on one side of a horizontal/vertical edge, theCROSS filter contains a greater percentage of samplesfrom that side of the edge than the XSHAPE filter, whichtends to draw the filter output closer to the actual signalvalue. The situation is basically reversed for diagonaledges, but the difference between the moments is gener-ally less pronounced, as for some filter positions relativeto the edge the XSHAPE geometry overlaps the edgewhere the CROSS does not. It would be tempting to sug-gest that CROSS filtering yields better edge preservation

(a)

185

_s

I III "

L

.14;;I.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987

+Xx

; < X SHAPE 5

+- CROSS(50SQUARE(

0 2 3

(a)

x<21+

X XSHAPE(51+ CROSS(5)El SQUARE(3)

0 1 2

(b)

3

Fig. 7. Normalized expected values of medians taken from XSHAPE(5),CROSS(5), and SQUARE(3) filters passed over an ideal (a) vertical, (b)diagonal two-dimensional edge. SNR = 4.

x XSHAP

x---x + CROSS

.2 + X +

1 0 1 2 3i(a

(a)

Figs. 7 and 8 indicate that SQUARE(3) filtering gives bet-ter results than XSHAPE(5) filtering but worse results thanCROSS(5) filtering; however, it should be noted that sincethe filter span is smaller, the signal estimate is more lo-calized, so that the filter does not overlap the edge untilit is centered closer to it. For diagonal edges, usingSQUARE(3) clearly gives better results than CROSS(5)filtering, but results roughly comparable (overall) to thoseobtained with XSHAPE(5) filtering, except very close tothe edge. It was also found (not shown) that SQUAREfilters of the same span [i.e., SQUARE(5)] give similarexpected outputs values A,,, Ad as SQUARE(3) near theedge, but much smaller variances than any of the filterscovering nine samples, suggesting that increasing the spanof the filter does not necessarily introduce a greater degreeof edge degradation (at least for sustained edges).

In Section IV we present some empirical results to fur-ther compare the filters, but first we will derive some re-sults in one dimension that will allow us to make somegeneral observations about the dependence of median val-ues taken across an edge.

B. Edge Miss Probability in One DimensionThe determination of the general joint distribution of

median values taken from overlapping two-dimensionalwindows lying across an edge is an extremely complexproblem, as there are many possible (algebraic) orderingsof the observed image samples taken from either side ofthe edge in each of the windows. However, the problemsimplifies greatly in the one-dimensional case, since thefilter windows coincide in all but one sample.We begin by considering the idealized one-dimensional

noisy edge sequence { Xi } with values given by (h > 0)X XSHIAPE ':3

++ CROSS 5;

/ ic SQUAREl

xxX-~~~--~~~Cl~ Xj =hNI

h + aNi ;

-2 -1 0 1 2i->

3

(b)

Fig. 8. Normalized variances of medians taken from XSHAPE(5),CROSS(5), and SQUARE(3) filters passed over an ideal (a) vertical, (b)diagonal two-dimensional edge. SNR = 4.

overall than XSHAPE filtering, but in view of the ideal-ized models that we are considering (and the limited filterspans), it is safer to suggest that CROSS filtering affordsbetter results for images containing a preponderance ofhorizontal/vertical edges, while XSHAPE is to be pre-

ferred if the image contains mostly diagonal edges.It is more difficult to make a direct comparison between

the effects obtained with SQUARE filtering with those ob-

tained by median filtering with the other geometries, since

SQUARE filters with the same span do not cover the samenumber of image samples. For horizontal/vertical edges,

where we assume as before that the values { Ni } are zero-

mean iid random variates with continuous distribution F,density f, and variance 1. If we denote the correspondingoutput of a median filter which spans 2m + 1 signal sam-

ples as { Xi }, then we have

X0 median {X ,t, *., X0, , X,,} aNo(h/a)

and

X1 = median {XI m, XI, , X + tn}

-N11(h/a)

where for i 0, 1

Ni (h/) = median{ Ni m N* ,

Ni+I + h/l, - * *, Ni+m + h/u}, (5)

i.e., the scaled versions (by the variance parameter a) of

4,

~' v

3,

2

o01I

4,

~Ld3,

2

1~

-2 -1

2v~

.5

.4

.3

1

0'

2XId

.4

.3

.2

i c 0

i I

n &-- fp -.,-- -

o t_

186

BOVIK et al.: EFFEC I OF MED)IAN FILTFRING ON EDGE ESTIMATION

the sample medians taken at image coordinates 0 and 1(both sides of the edge).Again we define a simple differencing edge estimate by

AA

Ai = Xi+, -Xifor simplicity. Then for a predetermined edge thresholdparameter -, we define the edge miss probability

PM = Pr {\i . r |r < 1}= Pr {NI(h/uor) -NO(h/a) < T/u T < h}= PM (rT/u, h/u),

that is, we only regard an edge to be detected if Ai > T,rather than Ai > T, viz. the edge must be correctlydetected. In what follows, we will first derive an expres-sion for the complementary (correct detection) probabil-ity.

PD(T/u, h/a) = 1 - PM(Tr/u, h/u)

= Pr {Ni(h/a) - No(h/u)

> /ulT < h},

since the derivation turns out to be somewhat shorter. Letus further define the set N(h /a) = NL (h/a) UNR(h/a), where

NL(h/u) = {N-It,, *-- ,Noand

NR(h/u) I{N + h/u, ...* NI,1+1 + h/ur}(the superscripts "L" and "R" denote the scaled obser-vations lying to the left and right of the edge, respec-tively), and defining the order statistics

N(.)(h/u) order {N(h/uf)}

= {N(I), N(), * *N(2n+2)it can be easily seen that the scaled median valuesN1(h / a), NO(h / a) E { N(m+ 1) NfV(m + 2) } according to

No(h /a) = ({Ni(n +2),

and

if N,n+ + h/u > N(m+I)if Nni+i + h/u( < N(m +2)

(6)

using (6) and (7). To further express this we will requirethe following definition:

Definition: A pattern i-r ( i ) of the ordered values aboutan edge is an event of the form

7rn(i) = {exactly i elements of NL(h/u) and m + 1- i elements of NR(h /a) are < N(m +2) }

= {exactly m + 1 - i elements of NL(h/a)and i elements of NR(h/a) are >N(m±+)}

fori = 1, .. ,m+ 1.A pattern 7m (i ) describes the incidence of elements of

NL(h/a) and NR (h /a) falling above or below the me-dian value (N(m + I) + N(m +2)) / 2. The number of possiblepatterns irm (i ) for given values of m and i is simply thenumber of ways that m + 1 samples can be grouped intotwo sets of i and m + 1 - i samples, which is given bythe binomial coefficient

(m + 1) (m + 1)!\ (m + 1 -i)!

Using total probability over all possible 7rm(i ) we maywrite (8) as

m+ I

PD(r/u, h/a) = L Pr {N(,±+ 2) -N(m + l) < T/U,

N-iN < N(m1+2), Nt,1+1 + h/u> N(M+1, 7rw,(i)T7 < h}

m+ I

2 E i2 Pr {exactly i elements(m + 1) i=

of NL (h /a) and m + 1

- i elements of NR(h/u) are

< exactly m + 1-- i elements

of NL(h/a) - T/a and i elements

of NR(h/u) - -/uo}

1 :E i2 (m2 1) Pr {M1(i)

< M2(i) - T/ur} (9)

NI(h/u) = I N(m+1)' if N-in > N(0n 1)Nha

tn+2 ) if N-117 < N(m+ 2)7Clearly, the probability PD (i-/ , h / a) takes nonzero

value only over the event {N1(h /u) = N(n + 2), No(h/)- N(m + 1) }, since i > 0. We may then write

PD(T/u, h/a)

= Pr {N(m+2) -N(m+1) > i-/a, N-

< N(m+2), N12+ I + h/u > N(mr+l) IT < h}(8)

where by "NNL(hl/u) - Tr/ua" we mean the set whoseelements consist of the elements of NL(h/a) with T-/usubtracted from each element (and similarly for"NNR(h/o) - T/a" ), and where

Ml (i ) = max { a given i elements of NL(h /a),

m + 1 -i elements of NR(h /a)} (10)

and

M2(i) = min {a given m + 1 - i elements of NL(h/a),

i elements of NR(h/u)} . (11)

187

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987

Clearly, Ml ( i ) and M2( i ) are independent randomvariables [since they represent the maximum and mini-mum, respectively, of mutually disjoint subsets ofN(h /a) in (9)] with continuous distributions respectivelygiven by

GI(x; i ) = Pr { all of a given i elements of NL(h/ua)

and m + 1 - i elements of NR(h/a)

are <x}

= Fl(x) Fm1 -'(x - h/u) (12)(a)

and

G2(y; i) = Pr { all of a given m + 1 - i elements

of NL(h/u) and i elements of NR(h/u)

are .y}

=1 - [1 -F(y)] [1 -F(y-h/or)](13)

fori = 1, ,m+ 1.Using (9), (12), and (13) we can finally write the edge

miss probability as

(b)PM(T/u, h/l) =

,1.0

P/

1 m+ + 1 )2(m + I)/ i=l i

*XGI(x - -rla; i ) dG2(x; i ) .

(14)

Fig. 9(a)-(c) shows the edge miss probabilitiesPM ( T / , h /u) as a function of the threshold-to-noise ra-tio (TNR) r / a and signal-to-noise ratio (SNR) h / a undera Gaussian 7 (0, 1) noise assumption on the noise se-quence { Ni }, for one-dimensional median filter spans 2m+ 1 = 1, 3, and 9 (note that 2m + 1 = 1 corresponds tonot filtering at all). The most apparent observation thatcan be made is that median filtering does increase the edgemiss probability (14), the effect becoming more markedfor larger values of m. However, we mention in passingthat we performed a similar analysis for averaging prefil-ters and found PM to take much larger values except whenr << h, where neither filter has much effect on subse-quent edge detection. One possible explanation for the ap-parent degradation introduced by median filtering is sug-gested by a study done by Yang and Huang [6], who notedthat while median filtering tends to preserve edges, thefiltering operation does not improve (least-squares) esti-mation of the edge location; part of the reason for this liesin the tendency of the median filter to occasionally shiftthe edge boundary one or more units to the left or right.Such an edge shift in the filtered signal can occur whenthe medians N1 ( h / u) and NAO h / u) take equal values, an

1.0

5.000~~~~0.

(c)

Fig. 9. Edge miss probability in one dimension following filtering withmedian filters of span (a) 2m + 1 = 1 (i.e., no filter), (b) 2m + 1 - 3,and (c) 2m + 1 - 9.

event which has nonzero probability given by

PEQ(h/l) = Pr {Ni(h/o) = Po(h/u)}

= Pr {Nm+I + h/u > N(m+l), N-m

> N(M+I)} + Pr {Nm+j + h/u

< N(m+2), N_m < N(m+2)} (15)

using (6) and (7). Noting that the two probabilities in (15)are equal (by symmetry), and recalling our definition of

188

BOVIK et al.: EFFECT OF MEDIAN FILTERING ON EDGE ESTIMATION

the patterns 7rm(i ), we havem

PEQ(h/lu) = 2 >3 Pr {h/o + Nm+i < N(m+2)

N-m < N(m+2)9 Tm() }

are < m + 1 - i elements of

NL(h/a), i elements of NR(h/o)}

2m +±

- 2 2 ii(m+ I -i)Pr(m + 1) i-l

{ielementsofNL(h/lr),m + 1

- i elements of NR (h/or)

_ 2 m+l2M Ei(m +1)1

Pr {Mt(i) < M2(i)}

where MI(i) and M2(i) are given in (10) and (11), re-spectively. Finally, we need only note that

Pr {M1(i) < M2(i)} = GI(x; i) dG2(x; i)

with G1 and G2 as given in (12) and (13). For Gaussian- (0, 1) noise, Fig. 10 shows the probability of equal me-dians PEQ (h / a) for values of the SNR h / a ranging from0 to 4, for filter spans 2m + 1 = 3, 5, and 9 (the proba-bility is zero for 2m + 1 = 1). Clearly, PEQ takes largevalues for h / a < 3, suggesting that unless the edge mag-nitude is significantly greater than the noise deviation,there is a significant possibility that the edge will beshifted, particularly for large filter spans. This kind ofedge shifting does not necessarily increase the rate ofmissed edges (although it is possible that the edge may beboth degraded and shifted), so that the overall edge missprobabilities shown in Fig 9(a)-(c) reflect a pessimisticview of the appearance of the resultant edge image.

C. Discussion of the Two-Dimensional Edge-MissProblem

Unfortunately, we have been unable to derive a tracta-ble expression for the miss probability PM for two-dimen-sional filters applied to two-dimensional edges degradedby noise. This is largely a consequence of the fact thatexpressions of the form (4) are not possible, since the me-dian values taken from windows which overlap in less thanall but a single sample can take a larger range of valuesthan in (6) and (7). However, as we have seen that themiss probability is strongly affected by by the probabilityof equal medians PEQ, we can draw certain conclusions.Clearly, PEQ is constrained by the dependence of the me-dians taken from the overlapping windows, which in turnis determined by the relative degree of overlap [2], [10].For one-dimensional median filters centered at adjacent

EQ

3h/G -)

Fig. 10. Probabilities that median values taken from filter windows cen-tered at adjacent signal coordinates across an edge will take equal value,as a function of the SNR h / a.

signal coordinates, the ratio of overlap to filter span is2m / (2m + 1) which approaches unity as m T oo. Forhorizontally adjacent two-dimensional windows, the ratioof overlap to filter size is also 2m/(2m + 1) for theSQUARE geometry, but only 2m/(4m + 1) (* 12 as mT oo) and zero for CROSS and XSHAPE geometries, re-spectively. Similar results hold for diagonally adjacent fil-ter windows. Hence, we should expect edge shifting to bemore severe for SQUARE windows than for CROSS orXSHAPE.

D. Dependent NoiseThroughout the previous discussions we have made the

assumption that the signal or image of interest is contam-inated by iid ("white") additive noise samples. While thisassumption serves to simplify the ensuing analyses, formost image processing applications the white noise modeldoes not hold. The question remains, then, what is theeffect of median prefiltering on images contaminated bycolored noise? Unfortunately, there has been no work yetadvanced for analyzing order statistics arising from over-lapping samples of dependent data, or indeed, for singlesamples consisting of dependent variates; the question isfurther complicated by the various dependencies that thenoise data can obey. However, if the noise is stronglycorrelated or bandlimited to lower frequencies, we maygenerally expect median filtering to afford less desirableresults than for independent data, as correlated noise isoften characterized by artifacts which may be interpretedas sustained changes in intensity (such as the "speckling"effect found in coherent imaging applications). Thus, me-dian prefiltering, which tends to preserve edges, may alsobe found to preserve artifacts arising from the dependentnoise, unless the size of the filter is taken to be somewhatlarger than the "average artifact size." However, the useof very large median filters can be deleterious, as finerdetails may be removed and there may be more shiftingand missing of real edges. As a general rule of thumb, wemay say that median filtering prior to edge detection af-fords an advantage only if the image of interest generallycontains lower frequencies than the noise process.

189

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987

Fig. 11. Noisy il(0, 1600) image.

IV. EXAMPLESThe statistical analyses provided in the previous sec-

tions used idealized models of the edges found in digitalimages. Although the analysis provided several indica-tions regarding the relative merits of various two-dimen-sional filtering techniques, real images contain edgeswhich vary over a broad range of spatial scales, orienta-tions, and curvatures. Since it is not practicable to attempta rigorous analysis of the effect of median prefiltering onsubsequent detection of every kind of edge, we insteadmust be satisfied with some emperical observations ob-tained through simulation.

Consider the image (240 x 240, 8 bits/pixel gray-levelresolution) shown in Fig. 11 (University of Illinois quad-rangle and Matthews Avenue, circa 1950). This imagehas been contaminated with independent Gaussian -q (0,1600) additive noise samples. Fig. 12(a)-(d), illustrates,respectively, the result of applying (a) SQUARE(7), (b)XSHAPE(7), (c) CROSS(7), and (d) CROSS(7) (appliedtwice) two-dimensional median filters to the image in Fig.11. The images filtered by CROSS and XSHAPE filtersexhibit an apparent "streaking" effect, which is mani-fested both by streaks and by a jaggedness or blurringalong edges, while the SQUARE filtered image is blotchedand somewhat blurred along the edges. The reason for thestreaking effect is similar to the edge shifting effect ob-served in Section III-B: medians which are taken fromwindows which overlap substantially along some orien-tation have a greater dependency, and therefore tend totake equal value with higher probability. When severalwindows centered at consecutive image coordinates (overa region or line in the image) take equal values (an eventwhich may occur with nonzero probability for regions orlines of some finite extent, even if the noise is continu-ous), the effect may be seen as linear streaks or blotches.Thus CROSS filters, which have a high degree of hori-zontal/vertical dependence will tend to result in streaksoriented along these directions, while XSHAPE filters willtend to produce diagonally oriented streaks. SQUARE fil-ters, which result in data dependencies in every direction,produce blotches. For a more in-depth discussion of theseeffects, see Bovik [I1].

Despite the introduction of artifacts in the form ofstreaks and blotches having no physical meaning, consid-

(a)

(b)

(c)

(d)Fig. 12. Result of applying (a) SQUARE(7), (b) XSHAPE(7), (c)CROSS(7), and (d) CROSS(7) (twice) median filters to image in Fig. I1l.

190

BOVIK et al.: EFFECT OF MEDIAN FILTERING ON EDGE ESTIMATION

(a) (b)

(c) (d)Fig. 13. Edge maps obtained by applying Roberts' edge detector to the

median filtered images in Fig. 12: (a) SQUARE(7), (b) XSHAPE(7), (c)CROSS(7), and (d) CROSS(7) (twice).

erable noise suppression can be seen to follow from themedian prefiltering, while the integrity of the edges, al-though modified somewhat, has been largely retained,particularly in the twice-filtered image.The result of applying Robert's edge detector [12] to

each of the filtered images in Fig. 12 is shown in Fig. 13,using an edge threshold value of T = 25. The use of theRobert's detector without prefiltering produced resultswhich conveyed virtually no edge information, regardlessof threshold selection, and so is not shown. The Roberts'operator is defined very similarly to our primitive estimate(1), except that the edge estimates are taken along the twodiagonals, and are combined in an rms point operationyielding a single statistic. The use of XSHAPE(7) andCROSS(7) filters gave uniformly poor results (raising thethreshold reduced both the incidence of false edges andthe edge information), albeit much better than not filteringat all (not shown); the edge maps are noisy, and there isconsiderable edge displacement. However, although thetwo filtering techniques resulted in a comparable degreeof noise suppression, the CROSS filter clearly afforded agreater degree of edge preservation (fewer missed edges)than the XSHAPE; this clearly supports the analytic re-

sults which suggest that CROSS filters are more effectivethan XSHAPE for images containing an abundance ofhorizontal/vertical edges, as this image is largely com-posed of structures with linear edges in these directions.Similarly, the XSHAPE filter can be observed to haveproduced better results near the relatively sparse diagonaledges. SQUARE(7) filtering produced much more ac-ceptable results (noting that although the filters are of sim-ilar spans, thev are of different coverages), in the sensethat the resultant edge image is more visually appealing;however, there is evident edge breaking (missed edges)and edge displacement. In experiments with SQUARE(3)filters, which have a similar pixel coverage as theCROSS(7) and XSHAPE(7) filters, we found similar vi-sual results as with the nonsquare filters of span seven,although there were slightly more psuedo-edges and fewermissed edges than with the other filters. Most likely thisobserved effect is due to the lower pixel coverage (moremissed edges) but greater localization due to the smallerspan (fewer missed edges) in accordance with the analyticresults.Much of the previous work in median filtering theory

has dealt with the root signal behavior of one-dimensional

191

IEEE TRANSACTIONS ON PAITERN ANAI YSIS ANI) MACHINE IN'FELLIGENCE. VOL PAMI-9, NO. 2, MARCH 1987

filters, viz. the behavior of signals which are repeatedlypassed through a median filter. While no results have yetbeen developed along these lines for truly two-dimen-sional filters, multiple filtering in two dimensions has beenobserved not to degrade edges, while affording additionalnoise suppression. Fig. 13(d) shows the result on Roberts'edge detection of twice prefiltering with CROSS(7); here,the results are very comparable to those obtained with themuch larger SQUARE(7) filter, in that there is very sub-stantial noise suppression and edge preservation. How-ever, it should be noted that several linear artifacts canalso be observed, which can most likely be attributed tothe linear streaks introduced in Fig. 12(d). These artifactsare not present in the SQUARE prefiltered edge image;the less directional dependency of the SQUARE filteredimage data can be seen to produce a smoother edge de-tection.

Enhancement/thresholding based edge detectors such asthe Roberts' device are still widely used, but in many ap-plications more sophisticated edge detectors modeled afterthe narrowband characteristics found in some mammalianvisual systems are being used, such as the Marr and Hil-dreth [13] Laplacian-of-a-Gaussian (V2G ) approximationto the difference-of-Gaussians (DOG) filter implementa-tion in the post-retinal ganglion cells. One such operatorin common usage [14], [15] is the difference-of-boxes,where the sample averages taken from two square win-dows of different sizes are differenced at each image lo-cation; the zero crossings of the resulting narrow-bandimage are then used to mark the location of edges. Theuse of averaging filters is not optimal in the sense thatGaussian-based filters are (greater edge localization andnoise suppression), but it is somewhat simpler to imple-ment and serves to illustrate our points.

Fig. 14(a)-(c) shows the results of applying a differ-ence-of-boxes detector (with boxes of sizes 5 x 5 and 9x 9) to the original (noisy) image, the SQUARE filteredimage, and the twice-CROSS filtered image, respectively.The edge map obtained from the noisy image [Fig. 14(a)]contains many obvious spurious zero-crossings, althoughthe results were much better than those obtained with theRoberts' detector applied directly to the noisy image(again, not shown). Clearly, median preprocessingyielded a cleaner edge map, and seemed, in effect, tolower the center frequency of the subsequent narrowbandprocessing. Again it may be observed that twiceCROSS(7) filtering gave greater edge localization, butmore pseudo-edges, while SQUARE(7) prefiltering re-sulted in smoother edges.

Fig. 15(a)-(c) illustrates the effect of using larger boxes(9 x 9 and 15 x 15); here we again find improvementfollowing median prefiltering, but the effects are much lesspronounced. We suspect that the reason for this lies in thetendency of narrow-band differential operators to producelarge SNR's close to substantial edges, resulting in a kindof "dead zone" about the edge. It is quite likely that me-dian prefiltering would be quite effective for making thedetection of isolated edges easier, even for edge operators

(a)

(b)

(c)Fig. 14. Edge maps obtained using a difference-of-boxes edge detector

(box sizes 5 x 5 and 9 x 9) applied to (a) noisy, unfiltered image, (b)SQUARE(7) filtered image, and (c) twice-CROSS(7) filtered image.

of large spatial extent, particularly in the suppression ofpsuedo-edges edges away from edges having meaning.

V. CONCLUDING REMARKSIn this paper we have endeavored to provide a quanti-

tative justification for using median prefiltering prior tothe application of various edge detection devices to digitalimages corrupted by noise. Through the use of computedmoments, error probabilities, and simulations, we found

192

BOVIK et al.: EFFECT OF MEDIAN FILTERING ON EDGE ESTIMATION

(a)

(b)

(c)

Fig. 15. Edge maps obtained using a difference-of-boxes edge detector(box sizes 9 x 9 and 15 x 15) applied to (a) noisy, unfiltered image,(b) SQUARE(7) filtered image, and (c) twice-CROSS(7) filtered image.

that median prefiltering can improve edge detector per-

formance, as measured by increased noise suppressionaway from edges, and insubstantial loss of edge detail. Inparticular, we found that filters with cross- and X-shapedgeometries generally afford better results when applied toimages containing a majority of horizontal/vertical anddiagonal edges, respectively, while square-shaped me-

dian filters yield edge maps which are generally smoother,

but with larger amounts of edge displacement when allorientations are considered. It was also found that medianprefiltering can improve the performance of zero-crossingtype edge operators, as well as more conventional gra-dient-based edge operators.We have not addressed the effect of median prefiltering

on the performance of other edge operators. It is very dif-ficult to make any strong statements regarding the efficacyof the technique for any of the many other interesting edgedetectors without massive experimentation, as the analy-sis of median filters remains intractable for all but the mostsimple assumptions. However, the improvement gainedfor the simple gradient operators is very suggestive for thegeneral application, as these are very fundamental edgeenhancement/detection devices, and most other localiza-ble edge detection schemes represent an attempt to im-prove on the gradient concept to deal with noise, context,etc. However, there are certain edge operators whichcould be expected to respond to the pixel dependenciesintroduced by the median filtering operation, most nota-bly those based on least-squares estimates, or the so-called"parametric edge operators. Also, operators which arebased on cooperative processing (viz. relaxation opera-tors) could be expected to respond adversely to pixel de-pendencies, particularly the streaking or blotching effectdescribed earlier. If exotic or very specific edge detectionschemes are needed for some application where noise isa problem, then care should be taken before median fil-tering is used.

ACKNOWLEDGMENTThe authors would like to express their appreciation to

the referees for their many valuable suggestions whichgreatly improved the readability and interest of this paper.

REFERENCES[1] J. W. Tukey, "Nonlinear (nonsuperposable) methods for smoothing

data," in Conf. Rec., 1974 EASCON, p. 673.[2] N. C. Gallagher, Jr., and G. L. Wise, "A theoretical analysis of the

properties of median filters," IEEE Trans. Acoust., Speech, SignalProcessing, vol. ASSP-29, pp. 1136- 1141, 1981.

[3] T. S. Huang, Ed., Two-Dimensional Digital Signal Processing IH:Transforms and Median Filters. New York: Springer-Verlag, 1981.

[4] W. K. Pratt, Digital Image Processing. New York: Wiley, 1978.[5] A. C. Bovik, T. S. Huang, and D. C. Munson, Jr., "A generalization

of median filtering using linear combinations of order statistics, " IEEETrans. Acoust., Speech, Signal Processing, vol. ASSP-31, pp. 1342-1350, 1983.

[6] G. J. Yang and T. S. Huang, "The effect of median filtering on edgelocation estimation," Comput. Graphics Image Processing, vol. 15,pp. 224-245, 1981.

[7] R. M. Haralick, "Digital step edges from zero crossings of direc-tional derivatives," IEEE Trans. Pattern Anal. Machine Intell., vol.PAMI-7, pp. 58-68, 1984.

[8] T. A. Nodes and N. C. Gallagher, "Two-dimensional root structuresand convergence properties of the separable median filter," IEEETrans. Acoust., Speech, Signal Processing, vol. ASSP-3 1, pp. 1350-1365, 1983.

[9] F. Kuhlman and G. L. Wise, "On second moment properties of me-dian filtered sequences of independent data," IEEE Trans. Commun.,vol. COM-29, pp. 1374-1379, 1981.

[10] A. C. Bovik, "The joint distribution of order statistics from overlap-ping samples," Comput. and Vision Res. Center, Univ. Texas, Aus-tin, Tech. Rep., 1985.

193

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-9, NO. 2, MARCH 1987

[11] -, "Streaking in median filtered images," IEEE Trans. Acoust.,Speech, Signal Processing, to appear; also Comput. and Vision Res.Center, Univ. Texas, Austin, Tech. Rep., 1985.

1121 L. G. Roberts, "Machine perception of three-dimensional solids,"in Optical and Electro-Optical Information Processing, J. T. Tippettetal., Eds. Cambridge, MA: M.I.T. Press, 1965, pp. 159-197.

[131 D. Marr and E. Hildreth, "Theory of edge detection," Proc. Roy.Soc. London B, vol. 204, pp. 187-217, 1980.

[14] T. E. Dietz, L. S. Davis, K. R. Diller, and J. K. Aggarwal, "Com-puter recognition and analysis of freezing cells in noisy, cluttered im-ages," Cryobiology, vol. 19, pp. 539-549, 1982.

[15] J. Canny, "Finding edges and lines in images," M.I.T. Al Lab.,Tech. Rep. 720, 1983.

Alan Conrad Bovik (S'80-M'84) was born inKirkwood, MO, on June 25, 1958. He receivedthe B.S. degree in computer engineering in 1980,and the M.S. and Ph.D. degrees in electrical and

_ computer engineering in 1982 and 1984, respec-lively, all from the University of Illinois, Urbana-

He is currently an Assistant Professor in theDepartment of Electrical and Computer Engineer-ing at the University of Texas at Austin. His re-search interests include the application of nonlin-

ear statistical methods to problems in digital signal and image processing,biomedical image processing, computer vision, and computational aspectsof human vision.

Dr. Bovik is a member of the IEEE Computer Society, the IEEE Acous-tics, Speech, and Signal Processing Society, the IEEE Information TheoryGroup, the Pattern Recognition Society, and the Honor Society of Phi KappaPhi.

Thomas S. Huang (S'61-M'63-SM'76-F'79) re-ceived the B.S. degree in electrical communica-tion from the National Taiwan University, Taipei,and the M.S. and Sc.D. degrees in electrical en-gineering from the Massachusetts Institute ofTechnology, Cambridge.

From 1963 to 1973, he was on the faculty ofthe Department of Electrical Engineering, M.I.T.During the academic year 1971-1972, he was onsabbatical leave visiting ETH-Zirich, Switzer-land, as a Guggenheim Fellow. During the aca-

demic year 1972-1973, he was again on leave, working in the Optics Groupat the M.I.T. Lincoln Laboratory, Lexington. From 1973 to 1980, he wasProfessor of Electrical Engineering and Director of the Laboratory for In-formation and Signal Processing, Purdue University, West Lafayette, IN.During the summers of 1974, 1979, and 1981, he was a Visiting Professorat INRS-Telecommunications, University of Quebec, Montreal. During theacademic year 1976-1977, he was a recipient of the Humboldt Foundation

U.S. Senior Scientist Award, working on the archaelogical applications ofimage processing at the Rheinisches Landesmuseum, Bonn, West Ger-many. During the summer semester of 1978, he was a Visiting Professorat the Technical University of Hanover, West Germany, and during thesummer semester of 1980, he was a Visiting Professor at the Swiss Instituteof Technology, Lausanne. In August 1980 he joined the University of Il-linois, Urbana-Champaign, where he is currently a Professor of ElectricalEngineering and Research Professor of the Coordinated Science Labora-tory. His professional interest lies in the broad areas of information andcommunication technology, but especially the transmission and processingof multidimensional signals. He has served as a consultant to numerousindustrial firms and government agencies in the U.S. and abroad. He isauthor (with R. R. Parker) of the book Network Theory: An IntroductoryCourse (Reading, MA: Addison-Wesley, 1972) and Editor (with 0. J. Tre-tiak) of the book Picture Bandwidth Compression (New York: Gordon andBreach, 1972) and the books Picture Processing and Digital Filtering(1975, 2nd ed. 1979), Two-Dimensional Digital Signal Processing 1: Lin-ear Filters (1981), Two-Dimensional Digital Signal Processing 11: Trans-forms and Median Filters (1981), and Image Sequence Analysis (1981), allpublished by Springer-Verlag.

Dr. Huang is an Editor of the international journal Computer Graphicsand Image Processing, an Associate Editor of Pattern Recognition, an As-sociate Editor for Signal Processing of the IEEE TRANSACTIONS ON Acous-TICS, SPEECH, AND SIGNAL PROCESSING, and on the Overseas Editorial Boardof Signal Processing, the official journal of the European Association forSignal Processing.

David C. Munson, Jr. (S'75-M'79-SM'84) wasborn in Red Oak, IA, on October 19, 1952. Hereceived the B.S. degree in electrical engineering(with Distinction) from the University of Dela-ware, Newark, in 1975, and the M.S., M.A., andPh.D. degrees in electrical engineering fromPrinceton University, Princeton, NJ, in 1977,1977, and 1979, respectively.

Since August 1979, he has been with the Uni-versity of Illinois, Urbana, where he is currentlyAssociate Professor in the Department of Electri-

cal and Computer Engineering and Associate Research Professor in theCoordinated Science Laboratory. His research interests are in the generalarea of signal and image processing with current work focused on radarimaging, signal reconstruction from incomplete observations, time-varyingsignal processing, and the effects of finite register length in digital signalprocessing. He has held summer positions in data communications andspeech processing, and he has served as a consultant in synthetic apertureradar to the Lockheed Palo Alto Research Laboratory.

Dr. Munson is a member of both the Digital Signal Processing and theMultidimensional Signal Processing Technical Committees of the IEEEAcoustics, Speech, and Signal Processing Society, and he is Associate Ed-itor of the IEEE TRANSACTIONS ON AcouSTICS, SPEECH, AND SIGNAL PRO-CESsING for the subject area of signal and image reconstruction and resto-ration. He is also a member of both the Administrative Committee and theDigital Signal Processing Technical Committee of the IEEE Circuits andSystems Society.

194