optical multiscale morphological processor using a complex-valued kernel

Optical multiscale morphological processor usinga complex-valued kernel

Adam Fedor and Mark 0. Freeman

Morphological transformations are typically performed on binary images by convolution with a binarykernel, which is followed by a threshold. We present an alternate approach that uses a complex-valuedkernel with odd symmetry to perform these morphological operations. The complex-valued kernelincreases the information-processing ability of the processor with no increase in system complexity. Oneadvantage is that the processor operates on all constant regions of a gray-level image in parallel. Ascale-space representation of this processor is obtained by varying the size of the kernel continuouslythrough a range of scales. By using redundant information in the scale representation, this system isfound to be robust in the presence of noise and spatial nonuniformities in the image. An optical systemto perform morphological filtering based on this system is presented.

1. Introduction

Mathematical morphology is a branch of mathemat-ics that has proved its usefulness in many image-processing and computer-vision problems.' Thesenonlinear image-processing operations are based oncomparing the shapes within the image with theshape of a structuring element. By operating di-rectly on the shape information rather than onspatial-frequency content, one can operate on detailsof a certain size without affecting, for example, edgesthat share some of the same spatial frequencies.Almost all optical morphological processors proposedto date rely on a similar approach.23 4 These pro-cesses perform a convolution between the binaryinput and a binary structuring element. The outputof this convolution is thresholded by using a variablethreshold device at a level based on the absolute inputintensities and the filtering operation desired. Wedescribe a new morphological processor that uses acomplex-valued kernel with odd symmetry to imple-ment the structuring element. A correlation is per-formed between this kernel and the image, and a fixedthreshold is used to locate the zeros of the output.Knowledge of the absolute intensity of the inputimage is not required, which permits the processor to

The authors are with the Optoelectronic Computing SystemsCenter, Campus Box 525, University of Colorado, Boulder, Colo-rado 80309-0525.

Received 18 June 1991.0003-6935/92/204042-09$05.00/0.© 1992 Optical Society of America.

perform binary morphological operations on all con-stant regions of a gray-level image in parallel.

The initial idea for this complex-valued kernelcame from the centroid scale-space map, an image-representation scheme first introduced in Ref. 5.This scale-space representation is useful in a numberof different applications such as attention focusing,object tracking, and segmentation. Its applicationtoward morphological filtering is described here.First, a short review of morphology is presented inSection 2. Section 3 contrasts the most commonapproach to optical morphology, which we call theabsolute-intensity approach, with our new approach,which we call the relative-intensity approach. Thedescription of specific morphological operations thatmay be performed by using the new optical processoris presented in Section 4. The performance of thisprocessor on images with multiple gray levels andquasi-constant images is described in Section 5. Theoptical implementation is the subject of Section 6.The system is described and results are shown.Section 7 concludes the paper.

2. Image Morphology

Morphological transformations are nonlinear opera-tions that cannot be expressed as simple frequency-domain filtering ofthe objects. The preferred methodfor discussing morphological operations is with settheory. Operations in mathematical morphology aredefined between the object set X and a structuringelement B. In binary morphology a binary structur-ing element is used to transform a binary image.

4042 APPLIED OPTICS / Vol. 31, No. 20 / 10 July 1992

Although B may be easily changed to suit a particularapplication, here B is assumed to be a disk.

Two basic morphological operations are erosionand dilation. Erosion is defined as

X B = la A C (1)

where e denotes erosion and Ba is the transpose ofthe structuring element with its center shifted to thelocation a. For a disk-shaped structuring element,B = B. To change Eq. (1) into words, erosion is theset of locations a for which the structuring elementfits completely within the objectX. Places where thecurvature of the boundary of the object is greaterthan that of B (i.e., a bump or a small arm of theobject) disappear. In other locations the erodedimage is a shape whose boundary is exactly one radiusof the structuring element from the original objectboundary. An example is shown in Fig. 1. Thecomplementary operation, dilation, is defined as

X lB=alBanX;0, (2)

where ff denotes dilation. It is the set of locations awhere the structuring element and the object have atleast one common point.

From these simple operations, more complicatedoperations may be defined: opening is

XOB = (XEB) B, (3)

where 0 is the opening operator, and closing is

X-B = (XfDB)EB,

withinX; i.e.,

XOB= U Ba.BaCX

(5)

Using these transformations, one can perform avariety of nonlinear filtering operations on a binaryimage, such as edge detection, impulse noise removal,and skeletonization.1,3

3. Optical Approaches to Morphological Filtering

A. Absolute-Intensity Processor

A processor that is commonly used to perform morpho-logical filtering optically23 works by convolving theimage X with a structuring element B and by thresh-olding the output. The threshold level determinesthe type of filtering operation. For instance, if weassume a foreground intensity If, a background inten-sity Ib, and a structuring-element intensity I,, thenthresholding at a value of Td = IIf + (M - l)Ib],where M is the number of pixels in the structuringelement, produces the dilation of Xby B; i.e.,

X EDB = Td(X* B). (6)

This threshold is set to detect where at least one pixelof the structuring element intersects the object [Eq.(2)]. The threshold for an erosion is set at level Te =MIsIf since erosion requires that the entire structur-ing element (composed of M pixels) is contained in theobject:

(4)

where * is the closing operator. An opening opera-tion leaves the foreground object at its original size,but it removes the details that were small comparedto the structuring element. The closing operationdoes the same thing to the background. A moreintuitive way of defining the opening is by the unionof all structuring elements Ba that completely fit

(d) (e) (f)

Fig. 1. Erosion and dilation obtained with the absolute-intensityapproach: (a) a two-dimensional input image, (b) structuringelement B, (c) the corresponding erosion of the image by B, (d) aone-dimensional cross section of the image, (e) a one-dimensionalcross section of B, and (f) the corresponding convolution with thethresholds for erosion Te and dilation Td indicated.

XEB = T(X*B). (7)

Notice that the erosion threshold does not depend onthe background intensity since the structuring ele-ment is entirely contained in the foreground object.This processor uses a global threshold whose leveldepends on the brightness of the image and thestructuring element; thus the inputs must be uni-form and binary. If the input intensities vary spa-tially, then a global threshold will not produce thecorrect output. Figure 1 shows the convolution of abinary image with a binary structuring element.The threshold levels necessary for the two filteringoperations are indicated.

B. Relative Intensity Processor

In this processor the binary kernel is replaced with acomplex-valued kernel [bipolar real for one-dimen-sional (1-D) images] that has odd symmetry on bothits real and imaginary parts. We see that this choicefor a kernel permits us to perform binary morpholog-ical filtering without regard for the absolute intensityof the input image. As a consequence this approachis tolerant of nonuniform contrast in the input andcan be used to perform binary morphological opera-tions on all constant gray levels in parallel.

For an initial study we choose the kernel

c(x,y; r) = (x + iy)w(x,y; r), (8)

10 July 1992 / Vol. 31, No. 20 / APPLIED OPTICS 4043

where the window w(x, y; r) defines the shape ofthe morphological structuring element. Consistentwith our restricting B to be a disk for our purposes,w(x, y; r) is a circular window of radius r; i.e.,

w(x,y; r) = circ[(x 2 +y 2)1/2/r]. (9)

Figure 2 is analogous to Fig. 1, except that therelative-intensity approach is used. Figure 2(a)shows a 1-D cross section of the complex-valuedkernel. The result of cross correlating this kernelwith the 1-D image cross section of Fig. 1(d) is shownin Fig. 2(b). As we see in Section 4, both the erosionand dilation threshold levels are shifted to zero as aresult of the relative-intensity approach. Figure 2(c)shows the result of thresholding the magnitude ofFig. 2(b) at zero, and the full two-dimensional (2-D)output is shown in Fig. 2(d). This represents theparallel erosion of both the foreground and thebackground of the input Australia map. Comparethe interior with Fig. 1(c).

The choice of this particular kernel arises as anatural extension of our previous research on cen-troid scale-space.5 Indeed, if the window functiontotally encloses an object, then we find the firstmoment of the object in the x direction with the realpart of the kernel (structuring element) and we findthe first moment in they direction with the imaginarypart of the kernel. The centroid of the object is thelocation where both of these first moments are equalto zero. Equivalently, this is the location where themagnitude of the cross correlation between the objectand the kernel is equal to zero. In Ref. 5 we appliedthis property to locating and tracking objects. Thediameter of the circular window was changed from aslarge as the full scene down to vanishingly small inorder to permit searching for objects of any size.This yields a scale-space map of the centroids inwhich the evolution of the centroids is viewed as afunction of the window size or scale. At scales wherethe window is larger than the objects in a scene we

(a)

(c)

(b)

(d)

Fig. 2. (a) Centroid kernel, (b) the correlation with a slice of theimage shown in Fig. 1(a), (c) the thresholded output, (d) the 2-Dversion of the output.

find information on the locations of the objects. Asthe scale is reduced, we eventually reach a pointwhere the window fits within the objects. At thisand smaller scales, morphological operations are per-formed. Here, we focus on the behavior of thecentroid scale-space map at this lower range of scales.

In general, the essential property of any kernel tobe used in this relative-intensity approach to morphol-ogy is that it have odd symmetry. That is, within thewindow function that defines the structuring-ele-ment shape, the kernel must integrate to zero. Inaddition, the function must have one and only onezero crossing [in one dimension, c(x; r) • 0 for x • 0and x < r]. If the function contains more than onezero crossing, a ringing effect occurs because severalodd-symmetric functions are contained within themain function.

Figure 3 shows a 1-D centroid scale-space map fora simple square pulse input. The scale-space kernelin one dimension reduces to the function x truncatedto a length that determines the scale. This is shownby the dashed curves through the pulse input at theleft for three different scales. At the right are thezeros of the cross correlation between the input pulseand the scale-space kernel as a function of scale.Two different regions of the scale are clearly seen.At large scales, there is a single zero at the centroidlocation of the pulse. At some intermediate scale thewindow size exactly corresponds to the size of thepulse. This is mapped into the peak of the solidtriangle in the scale-space map. We refer to thispoint as a transition point. (It was called a boilingpoint in Ref. 5. Here we choose the term transitionpoint because the meaning is more obvious.) For2-D images these transition points occur when thewindow is within the object and touches the boundaryof the object at a minimum of two points. As thescale is reduced further below a transition point,regions of zeros occur in the scale-space map.Transition points and zero regions occur only forconstant regions of the input, and the zero regionsalways take the form of a 450 isosceles triangle for a1-D input. For 2-D inputs these regions becomepyramidlike blocks, which depend on the shape of theconstant region in the input. Morphological process-ing takes place at scales smaller than or equal to atransition point.

II

-,I-I_

I - I

I' I

I I l

0a'F

I ,, t_I / Zero Locations

Fig. 3. Simple centroid scale-space map. Zeros occur when thekernel fits completely within a constant region or when the kernelis centered on the centroid of the object.


-

A number of the main morphological filteringoperations that can be obtained by using this scale-space approach are summarized in Fig. 4. We referto this figure as each operation is described. Theoriginal image of Australia is shown in Fig. 4(a), andthe smallest six scales of its centroid scale-space mapappear in Figs. 4(b)-4(g). These were computed bycross correlating the original image with the kernelc(x, y; r), where the window size r decreases by twopixels for each successive scale. This is thresholdedto yield the elements of the scale-space map.Figures 4(h)-4(1) show examples of morphologicaloperations that result from postprocessing the cen-troid scale-space map. These postprocessing opera-tions are the subject of Section 4.

4. Morphological Operations Using theRelative-intensity Scale-Space Approach

The postprocessing of the output zeros needed toobtain specific morphological transformations is pre-sented below. The morphological operation of paral-

(a)@ 15 *11 *9 *7 *5 3

(b) (c) (d) (e) (f) (g)

Erode once: (g) AND (a)

Dilate twice: (f) OR (a)

Edge detect: (g)

Medial Axis Skeleton:

L)i(h)

w(i)

0)3(k)

Open Foreground: [skel(n)*circ(n)] (a) (1)

Open Background: [skel(n)*circ(n) r)° ()

Fig. 4. Morphological transformations with a centroid scale-space map: (a) original image; (b)-(g) smallest six scales of thecentroid scale-space map; (h), (i) erosion and dilation operations;() edge detection; (k) medial-axis skeleton; (1), (m) opening opera-tions.

lel erosion of all constant gray levels is performeddirectly by the optical processor. This is discussed inSubsections 4.A and 4.D. As the scale is changed inthe optical processor, these erosions are produced fora range of structuring-element sizes. Other morpho-logical operations such as those shown in Fig. 4require electronic postprocessing of the optical output.The required postprocessing is discussed in Subsec-tions 4.B and 4.C. We begin the discussion byconsidering only binary images and then generalizethis approach to images that contain many constantgray-level regions.

It has been shown (Ref. 1, p. 56) that the simplemorphological transformations of erosion and dila-tion and the associated opening and closing opera-tions can describe a large class of operations thataccount for almost all useful morphological algorithms.The centroid scale-space representation contains allthe information necessary for one to obtain thesebinary image morphological transformations. Thisinformation is contained in the solid regions of zeros,or zero regions, in the map.

At sufficiently large scales the map does not containzero regions (see, for example, the central line ofcentroids in Fig. 3), which indicates that that size ofthe structuring element at those scales is bigger thanany feature within the scene. Let z(x, y; r) be thecentroid scale-space map; i.e.,

z(x, y; r) = ZEROS[c(x, y; r) *f (x, y)]. (10)

Consider the subset of z(x, y; r), which contains onlythe zeros that are generated when the kernel fitscompletely within a constant region (e.g., the fore-ground or the background in a binary image). Wecall this subset z'(x, y; r). Strictly speaking, morpho-logical operations are extracted from z'(x, y; r). Inmany cases, where relatively small structuring ele-ments are used, one works directly with the zeros ofthe scale-space map. At larger scales it is a simplematter to obtain z'(x, y; r) from z(x, y; r), as we nowdescribe.

The basic idea is to remove any isolated points fromz(x, y; r) with the exception of transition points.Isolated points are found by convolving each scale ofthe scale-space map with an isolated point detectorsuch as

-1 -1 -1-1 8 -1-1 -1 -1.

Isolated points are removed with a threshold bykeeping only those points of the convolution thathave a value of 7 or less. Transition points areretained by performing a logical AND operation be-tween the raw scale-space map at a given scale andthe isolated-point-free map at the next smaller scale.


A. Parallel Erosion of Foreground and Background

When the odd-symmetric kernel is completely con-tained in either the foreground or the background,the correlation of the kernel with the image producesa zero. We can state that

z'(x, y; r) = U a = (x, y) IBa(r) C Xi], (11)

where Ba(r) is the structuring element translated toposition a, Xi is any particular constant region withinthe image, and z'(x, y; r) is the subset of the scale-space map described above. Comparing this withEq. (1), we see that Eq. (11) describes the erosion ofevery region Xi by a structuring element of radius r.This shows the fundamental morphological buildingblock of the relative-intensity approach, namely, par-allel erosion of all constant gray levels. One canextract the erosion of any particular region by per-forming a binary AND operation between the region Xiand the set of zeros z'(x, y; r):

X, e B(r) = Xi n z(x, y; r). (12)

Dilation and erosion are dual operators with re-spect to their complements.1 In other words, thedilation of the foreground is equivalent to the erosionof the background. Thus one can extract the dila-tion of the foreground X by obtaining the erosion ofthe background and by inverting the result; i.e.,

X E B(r) = X n z'(x, y; r). (13)

Using De Morgan's law, one can write this equationmore simply as

X B(r) = X U z'(x, y; r). (14)

Two successive erosions (dilations) of an image by thesame structuring element are equivalent to a singleerosion (dilation) of the image by a structuringelement twice the size. Thus iterated morphologicaloperations are performed with the scale-space ap-proach by moving to a larger scale. Figs. 4(h) and4(i) show how erosion and dilation operations can beobtained from the centroid scale-space map.

B. Edge Detection

In image morphology, edge detection is performed byeroding the image with some structuring element, bydilating the image with the same element, and thenby subtracting the erosion from the dilation.3 Thisproduces an edge that is centered on the true edge ofthe object. This is related to the centroid scale-space map by using Eq. (11):

[X B(r)] - [X B(r)] = [X B(r)] - [X B(r)]

= [Xe B(r)] U [XeB(r)]= z'(x, y; r), (15)

where r is the scale used to obtain the edge map.

One can see that morphological edge detection is alsoa direct operation by using the relative-intensityapproach. The centroid scale-space map tracks themorphological edges of the objects through the scale.Whereas scale-space methods that track edges thatare defined by the zero crossings of the Laplacian ofthe image6 tend to lose the exact location of the edge,the morphological edge is always exactly centeredover the true edge of the image. The edge is alwaysof thickness 2r for images defined over a continuousset or (2r - 1) for discrete (pixelated) images. Obvi-ously, the best-resolved edge map is obtained fromthe smallest scale. An example is shown in Fig. 4(j).

C. Medial-Axis Transform and Morphological Opening

The medial-axis transform 7 (MAT) is a representa-tion that contains much information about the struc-ture of an image. For images with large constantregions it is a compact representation and can be usedfor data compression. The medial-axis skeleton ofthe object is the set of the center points of all themaximum disks of the object. A maximum disk isthe largest disk B(r) contained in X such that there isno larger disk that contains B (r) (not necessarilycentered at the same point) and is itself completely inthe region X. Figure 5 shows how the skeleton isformed from the set of maximum disks. The medial-axis transform consists of the skeleton and the size ofthe maximum disk associated with each point on theskeleton. Using this information, one can com-pletely reconstruct the original region.1 7

A transition point defines the center and the size ofa maximum disk. Thus the union of all transitionpoints in a region is the MAT of the region. If wedefinep(x, y; r) as the set of all the transition points ina centroid scale-space map, then p(x, y; r) is the MATof all the constant regions in the image. In the limitthat the window radius changes continuously, eachfully connected skeletal component corresponds to aconnected region of some particular gray value. Toisolate the MAT of a particular region X, intersectthat region with p(x, y; r):

S(X; r) = X np(xy; r). (16)

One can reconstruct the original region from the

Fig. 5. Method for forming the skeleton of an object. The centerof each maximum disk (which touches the boundary of the object attwo or more points) is kept.


union of all the maximum disks:

X = U MDa(r)la E S(X; r)1,r>O

where MDa(r) is the maximum disk associated withthe point a. An example of a skeleton computedfrom a centroid scale-space map is shown in Fig. 4(k).

The opening is defined in Eq. (5) as the union of allstructuring elements of a particular size that fitwithin the object. Consider a region described bythe union of every maximum disk in region X of sizer 2 k. One can construct this region by using theformula

X= U {MDa(r)laES(X). (18)r~k

We now show that this is equivalent to the opening ofXby a structuring element of size k.

The union of every disk smaller than B(r) that isincluded in B(r) gives us B(r); i.e.,

B(r) = U Ba(k) Ba(k) C B(r)). (19)k<r

Thus the region X can equivalently be formed fromthe union of all the maximum disks of size k plus allthe disks of size k that fit within all the othermaximum disks of size r > k.

X U B(k) C MDa(r)la E S(X)1r~k

- U Ba(k)Ba(k)CX

=X O B(k). (20)

Thus the opening of a region by the structuringelement B(k) can be computed by reconstructing theskeleton of the region for all scales greater than orequal to k as given in Eq. (18).

D. Regions in a Multiple Gray-Level Image

As we have already seen, morphological transforma-tions are most easily applied to binary images. Toapply these transformations to gray-level images, acommon technique is to use threshold decomposition.8' 9

The image is sliced into a number of binary images atdifferent gray levels, and each slice is operated onindependently. The results of these binary opera-tions are then summed to yield a transformation onthe gray-level image. A centroid scale-space proces-sor can operate on simple gray-level images withoutthis decomposition. If, for instance, the image con-tains n gray levels, one can perform morphologicaloperations on all n levels simultaneously. Figure 6shows the skeleton of a clown face generated by thecentroid scale-space method. The clown has 10 graylevels, and all 10 levels were processed simulta-neously by the processor. In such a multiple gray-level image, erosions, dilations, openings, and skele-tons of all of the constant regions Xi in an image arecomputed in parallel simply by performing the opera-

(a) (b)

Fig. 6. Skeleton for a multilevel image computed from a centroidscale-space map: (a) clown image, (b) skeleton of the clown.

tion on every region and by isolating the region ofinterest. It should be emphasized, though, that ourapproach performs binary morphological operationson all constant-intensity regions in parallel, but itdoes not perform gray-level morphology as it isusually defined.

5. Processing Images With Quasi-Constant Regions

A real image rarely contains purely constant intensityregions. Many images, however, contain regionsthat are quasi-constant, i.e., that have small intensityvariations around a constant gray value. Examplesinclude images with regions of a constant texture andforward-looking infrared (FLIR) images. We canmodel these quasi-constant regions as being purelyconstant with additive noise. Rather than finding amethod to quantize these images, we consider process-ing them directly, i.e., by applying the same morpho-logical techniques that are optimum for piecewiseconstant images to images that are made from thesequasi-constant regions.

Owing to the averaging properties of the correla-tion, centroid scale-space maps tend to be robust inthe presence of (uncorrelated) noise.5 Many of theimportant features of the map, such as the transitionpoints and the zero regions, are recognizable evenwhen a large amount of uncorrelated constant-meannoise is added to the input. One significant reasonthat this information is retained is that the map iscomposed of a continuous view of all scales and assuch has a high degree of redundancy between nearbyscales. If one looks at a particular scale of a centroidscale-space map, it may be difficult to decide whichregions are the result of a quasi-constant region of theinput. But, by merging the information that iscontained in several scales of the map, one can obtainobvious trends in the density of the centroids thatindicate that a zero region exists, which is an indica-tion that this region of the map came from a relativelyconstant region of the input. Figure 7(a) shows animage with 100% white noise added (signal-to-noiseratio of 1). We present two methods for estimatingthe erosion of this binary image buried in noise. Thefirst method merges information obtained at differentscales. Figures 7(b)-7(d) show the correlation out-put for three successively increasing scales. Thezeros of these correlations appear in Figs. 7(e)-7(g).The zeros were searched across the three scales bytesting each point for an indication of the characteris-tic 45° slope (in scale) seen at the edge of a zero region(see Fig. 3). This information was used in regionswhere the density of zeros was high to fill in the


(17)

(b)

(d) (g) (i)

Fig. 7. (a) Noisy input image, (b)-(d) correlation output forseveral scales, (e)-(g) output thresholded at zero for several scales,(h) erosion of the clean image obtained by reconstructing the zeroregions in the scale-space map, (i) erosion reconstructed by raisingthe zero-crossing detector threshold applied to (b).

appropriate zero regions. The last step was to ex-tract the erosion of the image from the smallest scaleof the reconstructed scale-space map. The result isshown in Fig. 7(h).

In the optical implementation a threshold is ap-plied to the squared magnitude (detected intensity) ofthe correlation in order to detect the zeros of the crosscorrelation. The second method for estimating theerosion of the cross in Fig. 7(a) is far simpler, bothcomputationally and conceptually, than the methodjust discussed. The level of the threshold applied tothe correlation outputs is raised slightly above zerowith the result that most of the low-level fluctuationsin the correlation are filtered out. In this way we geta good approximation of the erosion of the binarycross, but we lose some accuracy in the location of theedges of the zero regions. Figure 7(i) shows theerosion that was estimated from the noisy imagesimply by raising the threshold in this manner.Notice that this simple technique produces a resultthat is qualitatively similar to (perhaps better than)the previous method. This technique can be usedwhen a fast, approximate answer is required in anoisy environment.

6. Optical System

The optical morphological processor is shown in Fig.8. The processor is based on ajoint-transform corre-lator (JTC) architecture.0- 15 There are many opti-cal systems that could have been chosen to performthe basic cross-correlation operation. The JTC archi-tecture was chosen since both the input image andthe kernel are entered into this processor in theirspace-domain representations. In this way, the scaleof the structuring element can be easily changedsimply by changing the size of an iris that is used towindow the function x + iy. It would be much morecomplicated to change the structuring-element sizewith an optical architecture that requires the Fourier

- f f+ f -+ f -

Image f beam beam Centroid

Optically CorrelationAddressed

SLMFig. 8. Schematic of the scale-space morphological processor.SLM, spatial light modulator.

representation of the kernel function. The complexkernel x + iy is implemented with a computer-generated hologram (CGH). A second advantage ofworking with the space-domain representation of thekernel is that the function x + iy varies more slowlyand has a smaller dynamic range than its Fouriertransform. This permits us to code the computer-generated holographic element with greater accuracy.

The CGH of the x + iy kernel was coded by usingBurckhardt's modification1 6 of the Lee delayed-sampling coding technique.17 18 The CGH wasprinted as a chromium-on-glass mask with e-beamlithography by Abek Scientific of Colorado Springs,Colo. The mask consists of a 32,000 x 32,000 arrayof binary spots, each spot being 0.5 lum in size. Inthe coding scheme, we used 30 x 30 binary spots tocode each complex coefficient, which gave us an arrayof approximately 1090 x 1090 complex transmissioncoefficients. Binary CGH's have an inherent limita-tion of poor diffraction efficiency. To improve thediffraction efficiency, we used the CGH to create theobject beam for a new hologram recorded optically ina high-diffraction-efficiency volume holographic mate-rial. This technique was first introduced by Barteltand Case.1 9

Briefly, the JTC works as follows. The inputs areFourier transformed upon an optically addressedspatial light modulator (OASLM). The OASLM is adevice that detects the intensity of an incoming writesignal, and it modulates the amplitude of an outputlight beam (read beam) according to the intensityvariations of the write beam. The output of theOASLM therefore contains the squared magnitude ofthe sum of the input signals:

P(u, v) = F(u, v)exp(iirud) + G(u, v)exp(-iTrud) 12,

(21)

where F(u, v) is the Fourier transform of the inputimage and G(u, v) is the Fourier transform of x + iy.The phase factor exp(ilrud) results from displacingthe inputs from the optical axis by a distance d/2.Expanding this equation, we find two cross termsFG * exp(i2rrud) and F*G exp(-i2rud), which, whenFourier transformed again by a second lens, producethe desired correlation output (offset from the axis bya distance d). This correlation is detected and thresh-olded at the output plane to produce the parallel


I I

erosion of all constant gray levels. Simple electronicpostprocessing is performed on this output to producevarious other morphological operations as is de-scribed in Section 4.

Previous research with the JTC focused on using itto perform matched filter operations."-' 5 These sys-tems were either optical or hybrid electronic-opticalsystems. A matched filter ideally produces a spike inthe output plane that corresponds to the location ofthe matched object in the input image. Nonlineari-ties in the system have been found to increase theperformance of the matched filter so that the objectcan be located more accurately.1213 A nonlinearmodulator at the Fourier plane has been exploited toproduce sharper correlation peaks.'3 It was foundthat if the power spectrum is binarized before beingreconstructed, the matched-filter peak to sideloberatio can be increased by orders of magnitude. Theeffect is similar to that seen with binary phase-onlymatched-filter systems.

Optical devices tend to have a nonlinear behavior.The ferroelectric liquid-crystal OASLM,20 in particu-lar, is based on a smectic C * ferroelectric liquid-crystal structure that is theoretically bistable. AJTC based on this SLM was shown to provide resultssimilar to those of a binary phase-only matched-filtersystem."

The morphological processor described here, how-ever, requires the accurate detection of zeros of theoutput intensity, not peaks. It can be shown (Ref.21; Ref. 10, p. 235) that this nonlinearity can causeextreme distortions in the amplitude of the outputsignal. Although these distortions can be quanti-fied, it is not obvious how one could use this informa-tion to extract the proper output from the system.A SLM that is linear in input intensity is required.Currently available OASLM's do not offer the neces-sary degree of linearity. By comparing with anelectronic computer simulation of the system, weestimate that an adequate OASLM should provide alinear response over roughly 250 gray levels, i.e., adynamic range of two orders of magnitude. We hopethat the next generation of OASLM's will make thisfeasible.

To get around this problem, we have implementedthe processor on a hybrid optical-electronic system.The first part of the system uses a CCD camera as thejoint-transform power spectrum detector (to replaceOASLM). The second Fourier transform and thethresholding steps are done electronically. Experi-mental results are shown in Figs. 9 and 10. Figure9(a) shows the correlation output from our hybridoptical-digital system when a solid rectangle is usedas the input image. The parallel erosion of theforeground and background obtained from threshold-ing Fig. 9(a) is shown in Fig. 9(b). The ideal resultobtained by digital simulation of the complete systemis shown in Fig. 9(c) for comparison. The digitalsimulation electronically implements exactly the oper-ations that are performed in a JTC. An imagecomposed of two inputs that are separated by a

(a) (b) (c)

Fig. 9. Output from the hybrid processor, with a rectangle asinput: (a) after correlation, (b) the zero regions obtained after athreshold, (c) computer-simulated output.

distance d is fast-Fourier transformed. The squaredmagnitude of this result is then passed once againthrough the fast-Fourier transform. The spatialregion that corresponds to the cross correlation isthresholded to obtain the multiple-gray-level erosionoutout.

One application that highlights this processor'sability to perform morphological operations on quasi-constant gray-level images is in the detection andrecognition of vehicles in FLIR images. The morpho-logical edge operations may be useful, in conjunctionwith other centroid information obtained with thesame x + iy filter,5 for recognition of objects. Figure10 shows the experimental results when a quasi-binary input, a simulated FLIR image of a truck, isused. The input is shown in Fig. 10(a), the correla-tion output from our hybrid system is in Fig. 10(b),and the thresholded morphological output appears inFig. 10(c). Again, the image in Fig. 10(d) is theresult of a totally digital simulation of the system forcomparison. In this case both the hybrid systemresults and the simulated system results are quitesimilar. It is difficult to make a value judgement asto which is better for this quasi-binary case.

Both Figs. 9 and 10 show slight distortions owing tospeckle noise and phase aberrations within the opti-cal system. Still, there is good qualitative agree-ment that may be sufficient for some applications.Our results continue to improve as we make furtherrefinements to the system.

(a) (b)

(c) (d)

Fig. 10. (a) FLIR image of a truck, (b) hybrid output aftercorrelation, (c) the thresholded output, (c) a computer-simulatedoutput.


7. Conclusion

A new scale-space-based morphological processorthat we refer to as the relative-intensity approach wasintroduced. This approach is tolerant of nonunifor-mities in the contrast of an input, it permits morpho-logical edge maps to be obtained in a single-step, andit operates on multiple constant gray levels in paral-lel, not on just binary images. The computationallyintensive correlation operations can be performedwith an optical system. The output of this systemwould then be passed on to an electronic system thatperforms the relatively simple binary operations re-quired to complete the morphological transformations.Once the scale-space map has been computed, anynumber of morphological operations may be per-formed without the need for recomputing the map, orsimple morphological operations can be performed onthe fly, without the need for storing the scale-spacemap. A number of SLM's with integrated electronicbackplanes are being investigated that permit thiskind of simple local processing to occur.22'23 It isreasonable to expect that the necessary postprocess-ing of the output can be performed in parallel byusing these optically addressed hybrid SLM's.

References1. J. Serra, Image Analysis and Mathematical Morphology (Aca-

demic, New York, 1982).2. K. S. O'Neill and W. Rhodes, "Morphological transformations

by hybrid optical-electronic methods," in Hybrid Image Pro-cessing, D. Casasent and A. Tescher, eds., Proc. Soc. Photo-Opt. Instrum. Eng. 638,41-44 (1986).

3. D. Casasent and E. Botha, "Optical symbolic substitution formorphological transformations," Appl. Opt. 27, 3806-3810(1988).

4. Y. Li, A. Kostrzewski, D. Kim, and G. Eichmann, "Compactparallel real-time programmable optical morphological imageprocessor," Opt. Lett. 14, 981-983 (1989).

5. M. Freeman and B. E. A. Saleh, "Centroid scale-space maps,"J. Opt. Soc. Am. A 8, 1474-1487 (1991).

6. A. P. Witkin, "Scalespace filtering," in Proceedings of theEighth International Joint Conference on Artificial Intelli-gence, A. Bundy, ed. (Kaufmann, Los Angeles, Calif., 1983),pp. 1019-1022.

7. A. Rosenfeld and A. Kak, Digital Picture Processing (Aca-demic, New York, 1982).

8. P. Maragos, "Tutorial on advances in morphological imageprocessing and analysis," Opt. Eng. 26, 623-632 (1987).

9. J. Hereford and W. Rhodes, "Nonlinear optical image filteringby time-sequential threshold decomposition," Opt. Eng. 27,274-279 (1988).

10. J. W. Goodman, Introduction to Fourier Optics (McGraw-Hill,New York, 1968).

11. D. A. Jared, K. M. Johnson, and G. Moddel, "Joint transformcorrelator using an amorphous silicon ferroelectric liquidcrystal spatial light modulator," Opt. Commun. 76, 97-102(1990).

12. B. Javidi and C. Kuo, "Joint transform image correlation usinga binary spatial light modulator at the Fourier plane," Appl.Opt. 27, 663-665 (1988).

13. B. Javidi and S. Odeh, "Multiple object identification bybipolar joint transform correlation," Opt. Eng. 27, 295-300(1988).

14. T. Hudson and D. Gregory, "Joint transform correlation usingan optically addressed ferroelectric LC spatial light modulator,"Appl. Opt. 29, 1064-1066 (1990).

15. F. T. S. Yu and X. J. Lu, "A real-time programmable jointtransform correlator," Opt. Commun. 52, 10-16 (1984).

16. C. W. Burckhardt, "A simplification of Lee's method ofgenerating holograms by computer," Appl. Opt. 9, 1949(1970).

17. W. Lee, "Sampled Fourier transform hologram generated bycomputer," Appl. Opt. 9, 639-643 (1970).

18. A. Fedor and M. 0. Freeman, "User's guide to the computer-generated hologram facility at the Optoelectronic ComputingSystems Center-version 2.0," OCS Tech. Rep. 90-20B (Opto-electronic Computing Systems Center, University of Colorado,Boulder, Colo., 1991).

19. H. Bartelt and S. K. Case, "High-efficiency hybrid computer-generated holograms," Appl. Opt. 21, 2886-2890 (1982).

20. G. Moddel, K. M. Johnson, W. Li, R. Rice, and L. Pagano-Stauffer, "High speed binary optically addressing spatial lightmodulators," Appl. Phys. Lett. 55,537-539 (1989).

21. A. Fedor, "Centroid scale-space maps: applications andimplementation," OCS Tech. Rep. 90-23 (Optoelectronic Com-puting Systems Center, University of Colorado, Boulder, Colo.,1990).

22. T. Drabik and M. Handschy, "Silicon VLSI/ferroelectric liquidcrystal technology for micropower optoelectronic computingdevices," Appl. Opt. 29, 5220-5223 (1990).

23. D. A. Jared and K. M. Johnson, "Optically addressed threshold-ing VLSI/liquid crystal spatial light modulators," Opt. Lett.16, 967-969 (1991).


optical multiscale morphological processor using a complex-valued kernel

Documents