self-organizing fusion neural networks

Self-Organizing Fusion Neural Networks

Paper: jc11-6-2962:2007/5/10


Jung-Hua Wang, Chun-Shun Tseng, Sih-Yin Shen, and Ya-Yun Jheng

Electrical Engineering Department, National Taiwan Ocean University

2 Peining Rd. Keelung, Taiwan

E-mail: [email protected]

[Received January 16, 2007; accepted March 20, 2007]

This paper presents a self-organizing fusion neuralnetwork (SOFNN) effective in performing fast clus-tering and segmentation. Based on a counteract-ing learning scheme, SOFNN employs two parame-ters that together control the training in a counteract-ing manner to obviate problems of over-segmentationand under-segmentation. In particular, a simultane-ous region-based updating strategy is adopted to fa-cilitate an interesting fusion effect useful for identify-ing regions comprising an object in a self-organizingway. To achieve reliable merging, a dynamic merg-ing criterion based on both intra-regional and inter-regional local statistics is used. Such extension in ad-jacency not only helps achieve more accurate segmen-tation results, but also improves input noise tolerance.Through iterating the three phases of simultaneousupdating, self-organizing fusion, and extended merg-ing, the training process converges without manualintervention, thereby conveniently obviating the needof pre-specifying the terminating number of objects.Unlike existing methods that sequentially merge re-gions, all regions in SOFNN can be processed in paral-lel fashion, thus providing great potentiality for a fullyparallel hardware implementation.

Keywords: neural networks, image segmentation, clus-tering, counteracting learning, watershed

1. Introduction

The goal of image segmentation is to identify objectsof interest that satisfy certain pre-defined homogeneitycriteria, whereas the primary objective of clustering [1]is to partition a given set of data or objects into clusters(subsets, groups, classes). This partition should have thefollowing properties: 1) Homogeneity within the clusters,i.e., data that belongs to the same cluster should be assimilar as possible and 2) Heterogeneity between clusters,i.e., data that belongs to different clusters should be asdifferent as possible. This concept has been found useful[2–4] in realizing image segmentation wherein a raw im-age is taken as the input data, then clustering operates onthe feature space to label sufficiently similar and adjacentpixels. After the clustering process, the labeled results

are mapped onto the image plane to obtain the final seg-mented objects.

On the other hand, in hybrid-based segmentation meth-ods [5] or the so called split-and-merge techniques, a pre-process is initially invoked to dissect the input image intoa set of small primitive regions referred to as initial imagetessellation. One well known such pre-process is the mor-phological watershed transform [6, 7] which has drawngreat attentions from researches for two main reasons:its easiness in implementation, and more importantly, itgenerates a completely closed contour. However, due tothe keen sensitivity to local variance the technique suffersa major drawback known as over-segmentation problem[6], hence an effective merging process is needed in orderto achieve valid segmentation. To this end, Vincent et al.[8] used an undirected RAG (Region Adjacency Graph)[9] to sequentially merge small regions pre-generated bywatershed analysis. Two regions having the minimumedge are searched globally and merged. Such sequentialoperation is rather time-consuming. Haris et al. [10] pro-posed the nearest neighbor graph (NNG) region mergingmethod. Unlike the work of Vincent et al. [8], not allRAG is kept in the heap; instead only a small portion ofit is needed. But the improvement is still rather limitedas NNG also conducts merging in a sequential fashion.Another shortcoming of sequential merging is the lack ofspatial information needed for estimating the merging cri-terion, resulting in false contours and less accurate seg-mentation. Obviously, this undesired effect is further de-teriorated by the more number of initial primitive regions.

To overcome the aforementioned shortcomings en-countered in conventional split-and-merge techniques,and to further improve merging accuracy while resolv-ing the time consuming problem, we propose a novelapproach called self-organizing fusion neural network(SOFNN) to achieve image segmentation in a parallelfashion. Unlike k-means algorithm [4], SOFNN does notneed to pre-specify the final number of objects. Each in-dividual neuron is associated with a working feature tocharacterize its gray or color features. Another key at-tribute of SOFNN is the association of each target neu-ron with two statistical parameters α i and β i which areused to adjust the learning rate in a counteracting manner,namely one favors region-merging while the other favorsregion-splitting. After counteracting learning, the work-ing features are simultaneously updated, and features of

Vol.11 No.6, 2007 Journal of Advanced Computational Intelligence 1and Intelligent Informatics

Wang, J.-H. et al.

adjacent regions will become more similar, and tend toblend to each other, hence the name fusion. During thetraining process, local statistics pertaining to current adja-cency information and relative similarities are collected toupdate the values of α i and β i. We will show that becauseof the counteracting effect between α i and β i, the learningcan be iterated into a convergence state that represents thefinal segmentation result that is free of over-segmentationand under-segmentation problems. In addition, in orderto further improve segmentation accuracy and the abil-ity of noise tolerance, an extended adjacency strategy isinvolved in updating the connection weight between twoadjacent neurons. The strategy emphasizes that not onlythe immediate regions of the target region but their neigh-boring regions are also accounted for. Therefore, a widerrange of local statistics is collected to compute for a morereliable merging criterion, which not only improves noiseimmunity, but also help avoiding trapping in a local mini-mum.

Thus, the main contributions of this paper are the pro-posal for a novel approach that incorporates the self-organizing fusion neural network, a concept of adjacencyextension, and accurate identification of regions for per-forming image segmentation, characterizing propertiessuch as convergence and data-driven training parame-ters, and contrasting the performance with other mergingmethods and further validation through simulation stud-ies. The rest of the paper is organized as follows. Section2 elaborates the three major phases of SOFNN: counter-acting learning (i.e., simultaneous updating), fusion, andmerging, they perform segmentation in a manner of relayrace. In Section 3, we demonstrate how to apply SOFNNto perform segmentation for fairly complex real images.To further exploit its applicability, SOFNN is also used asa pre-process prior to clustering gene-expression data, re-sults show that the scale variation of gene-expression datacan be reduced without sacrificing essential informationsuch as inherent hierarchical structure of data. In Section4, important properties of SOFNN such as convergenceanalysis and parametric characterizations are addressed.Finally, concluding remarks and discussions are given inSection 5.

2. Self-Organizing Fusion Neural Networks

Network architecture of the proposed SOFNN is shownin Fig. 1. Initially the average intensity of all pixels ineach primitive region dissected by a pre-process, e.g. thewatershed transform, is used as an input working featureto be associated with an individual neuron (indicated by ablack circle in Fig. 1). The similarity between an arbitraryneuron and its adjacent neuron is used as the connectionweight. After applying a counteracting learning strategyto simultaneously update the working features, adjacentregions will become more similar in feature space (thismeans the corresponding neurons will get closer to eachother). Interestingly, the working features of adjacent re-gions with similar features tend to blend to each other.

Fig. 1. Architecture of SOFNN.

This fusion phenomenon is one of many key attributes ofSOFNN; it acts the underlying mechanism that facilitatesthe subsequent self-organizing merging. That is, if someworking features are similar enough, their associated neu-rons will be combined into a neuron, meaning the per-taining regions will be merged. In this context, fusiondesirably serves the role of identifying objects compris-ing a meaningful, resolvable object. Note that after themerging, RAG will be updated in order to track the newadjacency configuration for the next training iteration.

2.1. Fusion in Feature Space

Our idea of segmentation is rooted in the observationthat regions comprising a resolvable object are at leastnear-homogeneous in the feature space, and more impor-tantly, they must be adjacent in the image plane. Thus,it is vital to quickly search adjacent regions for each tar-get region and accurately identify which are qualified tobe merged. In addition, it is necessary to keep track ofadjacency information if there are regions merged duringthe preceding iterations, the statistical parameters used inthis work are closely related to the adjacency information.All these operations require an efficient data structure forrepresenting adjacency, an ideal choice of data structurewould be the undirected Region Adjacency Graph (RAG)[9]. The interpretation of an edge in RAG is this: the moretwo adjacent regions are similar, the more value the edgeis. RAG can iteratively update the adjacent informationaccurately.

After RAG is built, an arbitrary regionRi is character-ized by a working feature I t�Ri� defined as the average in-tensity of all pixels in Ri at iteration t, value of It�Ri� willbe iteratively updated until convergence. Because eachneuron corresponds to a region in SOFNN, the similaritybetween an arbitrary region Ri and its immediate neigh-boring regions R j��Rj� j� 1�2� � � � �n�1� is characterizedby the free parameter of connection weight prescribed as

wti j �

1

eλ �t��It �Ri��It �Rj�� 0 � wt

i j � 1 . . . (1)

2 Journal of Advanced Computational Intelligence Vol.11 No.6, 2007and Intelligent Informatics


where λ (t� is a time-varying function defined as

λ �t� �ninit

nt . . . . . . . . . . . . . . (2)

where ninit denotes the number of initial regions generatedby watershed transform, nt is the number of regions atiteration t. Note that because It�Ri� of all regions are time-varying, the weight defined in Eq. (2) is also time-varying.

Although Eq. (1) quantitatively calculates the weightbetween two adjacent regions, using it alone is not ade-quate in practical implementations. The main difficultyarises from the fact that regions subjectively perceived byhuman eyes as belonging to an object might not possessthe similarity sufficient to be identified as the same objectthrough a thresholding scheme in ordinary computer al-gorithms. In fact, a universal threshold for sufficient sim-ilarity is very difficult, if not impossible, to be specified.Thus, instead of using a pre-set global threshold [11], weemploy a statistical merging criterion based on a homoge-neous measure using local statistics. Furthermore, priorto merging, regions are subjected to a simultaneously up-dating process each iteration and checked if some of themhave become homogeneous. The iterative update equationfor It�Ri� is given as

It�1 �Ri� � It �Ri�� εti

�It �Ri�� It

av �N �Ri��

. . (3)

where Itav �N �Ri��

∑Rj�N�Rj�

It �Rj��wti j �Rj�

n�

N�Ri� denotes the set of adjacent regions surrounding toRi in RAG, including Ri itself. N�Ri� is also referredto as the immediate neighbors of Ri. We will elaboratethe learning rate ε i in the next section. The parame-ter It

av�N�Ri�� represents the average working feature ofN�Ri� weighted by the connection weight.

The beauty of iteratively applying Eq. (3) is that it canfacilitate systematic identification of which adjacent re-gions to be merged. To see this, we first note that afterapplying watershed operation, the primitive small regionsthat compose an object should be quite similar in a lo-cal sense. The update by Eq. (3) tends to force the re-gion and its adjacent regions to become more alike, col-lectively appearing as a fusion phenomenon taking placeamong the adjacent regions. Very soon the features ofadjacent regions become similar enough to be identifiedas candidates for merging. However, after some begin-ning iterations, updating should be made more gradualby increasing λ (t� in order to avoid regions with distinctdissimilarity are erroneously merged into a bigger region(i.e. under-segmentation). This purpose is easily servedby the design of Eq. (2), where nt normally decreases asthe number of primitive regions cannot be increased asiterating continues.

Following the update, adjacent regions are compared toa statistical merging criterion to check if they are qualifiedas the comprising components of an object. If so, they willbe merged into a bigger region. Just like the weights of allregions are simultaneously updated, the region merging inSOFNN can be conducted in parallel fashion. In subsec-

Fig. 2. Similarity flow diagram.

tion 2.3, we will discuss the merging criterion in detail.

2.2. The Counteracting LearningFor the learning rate parameter ε i in Eq. (3), we define

two counteracting parameters. First, a statistical param-eter α i is defined as the number of adjacent regions that“see” Ri as their most similar region. The initial valueof αi � 1 for i � 1� � � � �nt , nt is the number of regionsat iteration t. The rationale of using α i is best explainedwith the directed similarity flow diagram in Fig. 2. Thedirected arrow, say, the one pointing from R3 to R1 repre-sents that among the adjacent regions of R3, namely �R1,R2, R4�, R1 is seen by R3 as the most similar region, i.e.,w3max � Max�w13�w23�w43� � w13. On the other hand,besides R1 itself there are three regions �R3,R5,R6� thatsee R1 as their most similar region, thus α1 � 1� 3 � 4.The learning rate εi of region Ri is prescribed as

εt�1i � εinit � exp

��

�β t

i

βT�

αT

αti

��. . . . (4)

αT �nt

∑i�1

αti and βT �

nt

∑i�1

β ti

where εinit is the initial learning rate. If otherwise spec-ified, εinit � 0�017. Examining Eq. (4) clearly indicatesthat a larger αi�� 1� will increase the learning rate. Thatin turn, according to Eq. (3), encourages fusion (andhence merging) to occur.

Another parameter βi in Eq. (4) provides, however, justthe opposite effect. The rational of βi is explained as fol-lows. Consider the case when R3 is distinctively differ-ent from its surrounding regions �R1, R2, R4�, namelyw13, w23, w43 are rather small, in this case it is de-sired to inhibit the merge of R3 to R1. Nonetheless, asdescribed previously, α 1 will be increased by 1 sinceMax�w13�w23�w43� � w13, no matter how small values ofw13, w23, w43! Therefore, in order to counteract the effectcaused by α i and prevent under-segmentation, anotherstatistical parameter β i defined as the sum squared differ-ences between Ri and its adjacent regions, i.e. Σ[I�Ri��


Wang, J.-H. et al.

(a)

(b)

Fig. 3. Illustration of SOFNN (a) in image space (b) infeature space.

I�Rj�]2 is incorporated into Eq. (4). The larger β i is, thesmaller the learning rate ε i will be, thus inhibit fusion.The use of β i advantageously prevents fusion between re-gions with distinct dissimilarity, thereby avoiding under-segmentation.

Illustrations of how SOFNN performs merging in im-age space and feature space are given in Figs. 3(a) and (b),respectively. We take each circle as an individual neuron,then update features and merge neurons in feature spaceusing Eqs. (3) and (7), which result in parallel fusion andmerging regions in image plane, respectively. The arrowsin Fig. 3(b) indicate the direction to which a neuron movefrom its original position (dotted circle) to a new positionin feature space. The bold circle in Fig. 3(b) represents amerged neuron, corresponding to the two regions markedwith the arrow in Fig. 3(a). The number of remainingneurons stands for the number of regions left when themerging process is terminated.

2.3. The Merging Criterion

As aforementioned, it is the simultaneous updating inthe working feature space that creates the fusion phe-nomenon useful for identifying regions comprising an ob-ject. Following the updating, adjacent regions must bechecked to see if their fused features are sufficiently ho-mogeneous such that they are reliably qualified as thecomprising components of an object. To proceed, we first

define

ξ t�1�Ri� �1n ∑

Rj�N�Ri�

It�1�Rj�� . . . . . . (5)

For an arbitrary region Ri, besides the intra-regional pa-rameter It�Ri� that measures the average intensity of pix-els in Ri, the inter-regional parameter ζ t�Ri� is moreeffective in characterizing overall intensity of area sur-rounding Ri. Thus, the value of � ζ t�Ri��ζ t�Rj� � broadlystands for the dissimilarity (in feature space) between twoareas centered at Ri and R j, respectively. And the meandifference between and those of regions adjacent to Ri iswritten as

dt�1 �Ri� �1

n�1 ∑Rj�N�Ri�

��ζ t�1 �Ri��ζ t�1 �Rj�� (6)

Note that, with ζ t�Ri� prescribed in Eq. (5), the calcula-tion of dt�Ri� in Eq. (6) actually takes into account theadjacent regions of N�Ri� too, namely, neighbors’ neigh-bors, not just immediate neighbors of Ri, N�Ri�, them-selves. We denote the regions participating in evaluatingdt�Ri� as S�Ri�, that is, S�Ri� is the set of neighbors of im-mediate regions N�Ri� of target region Ri, N�Ri�� S�Ri�.In this work, dt�Ri� used as an effective statistical gaugeto judge whether a region adjacent to Ri comparing to therest of adjacent regions is more qualified to be merged.As will be shown later, this extension of adjacency of Rican help reduce the likelihood of erroneously merging tworegions, and even provides tolerance to input noise. Sta-tistically speaking, if any adjacent region of Ri with It�Rj�similar to that of Ri to such extent that their difference isless than dt�Ri�, then it is rather safe to state that R j ismore rightfully alike to Ri, compared to most of other ad-jacent regions. Thus, the rule of identifying an adjacentregion R j is simply this:

If��It�1�Ri�� It�1�Rj�

�� dt�1�Ri�� merge Ri and R j (7)

The rationale of using dt�Ri� in Eq. (7) as the local thresh-old is explained as follows. Because S�Ri� covers notonly immediate adjacent regions of Ri (i.e., N�Ri��, butalso the adjacent regions of N�Ri�, the value of dt�Ri� ina way stands for the variation of intensity across the areaS�Ri� centered at Ri. For an area S�Ri� comprising lots ofsmooth regions, the corresponding values of d t�Ri� shouldbe relatively small, regions in that area are more easily tobe merged. In contrast, an area S�Ri� has large valuesof dt�Ri� would indicate intensity variations in S�Ri� aresignificant, implying the existence of separated objects inS�Ri� is very likely, merging regions in S�Ri� is certainlynot encouraged.

After a larger region is generated due to merging, itsweight will be replaced with the average weight of thecomponent regions. And regions that were not merged atcurrent iteration will use the updated weights (via Eq. (3))in the subsequent iteration. Thus, except in the first it-eration calculating It�1�Ri� is region-based, rather thanpixel by pixel. Note that because the parameter d t�Ri� iscalculated on region basis, not pixel by pixel. Thus, cal-



culation of dt�Ri� only uses a small portion of the totalcomputation load. Furthermore, if some regions and theiradjacent regions are not merged at present iteration, theiraverage discrepancy parameters will not be recalculatedin the subsequent iterations. This also helps lessen com-putation load, especially in the case of huge number ofinitial regions. In fact, the speed of merging is fast duringthe beginning iterations, but as more regions are merged,it will slow down as the number of identified objects be-come close to the number of actual objects in the image.This property is useful as it provides a plausible way ofmanually setting the terminating condition for SOFNN.Namely, the iterative operation will be terminated whenthere are no more regions to be merged. In subsection 4.1,the convergence proof of SOFNN will be provided.

It is not unusual to see that some regions get mergedand some don’t at certain iteration. To proceed to the nextiteration, the updated values of connection weights andRAG will be recalculated for the merged regions alongwith the intact regions. In short, SOFNN mainly con-sists of three phases: counteracting learning, fusion, andmerging, they perform image segmentation in a mannerof relay race. Finally, it is worthy noting that the num-ber of neuron is fixed throughout the training process ofSelf-Organizing Map (SOM) [12], whereas the number ofneurons in SOFNN decreases during the iterative trainingprocess, and the final number of objects corresponding tothe number of neurons when SOFNN converges or manu-ally terminated.

3. Experimental Result

3.1. Image SegmentationWe use the benchmark image “peppers” (256�256, 256

gray levels) shown in Fig. 4(a) as the input. The result af-ter applying watershed pre-process is shown in Fig. 4(b).The over-segmentation phenomenon is quite vivid as thetotal number of initial regions is 1566. After applyingEq. (3) to blend the working features of adjacent regions.Larger regions are generated by the merging process us-ing Eqs. (5), (6) and (7). And Fig. 4(c) illustrates the179 objects painted with weights updated at the 1st iter-ation. After termination at iteration 5, Fig. 4(d) showsthe final result of 103 merged objects painted with theircorresponding original pixel intensities, the total compu-tation time is 0.328 sec. It is reasonable to conjecture thatthe parallel fashion indeed drastically reduce the compu-tation time. Fig. 5 shows the merging rate in “peppers”,it confirms that merging is fast during the beginning itera-tions, but slows down as the number of identified objectsbecome close to the number of actual objects in the image

3.2. Noise ToleranceIn conventional segmentation algorithms where the ba-

sic unit to be processed is region or pixel, often onlythe immediate neighbors are used for measuring similar-ity/homogeneity. In contrast, SOFNN explores the inter-

(a) (b)

(c) (d)

Fig. 4. (a) Input image “peppers”, (b) over-segmentation af-ter applying the watershed analysis, (c) segmentation resultof SOFNN, 179 regions at the 1st iteration, (d) result of 103regions when terminated at the 5th iteration.

Fig. 5. Merging rate of using input image “peppers”.

regional information; and the merging process is actuallyconducted in unit of regional cluster. That is, the lo-cal threshold is obtained in a wider sense by examiningnot only the immediate neighbors of target region but theneighboring regions surrounding them. This attribute notonly offers a more reliable merging criterion, but providesmore tolerance to input noise. Two types of noise aretested, namely impulsive and Gaussian. Fig. 6(a) is thetest input “house”. Figs. 6(b) and (c) are the corrupted im-ages by adding 20% impulsive and Gaussian noise (zeromean and variance�0.01), respectively. Fig. 6(d) is thetessellation result from applying median filter and water-shed analysis to Fig. 6(c). Using the output of apply-ing mean-filtered and watershed analysis to Fig. 6(b) astest input, Fig. 6(e) shows the segmentation result viaSOFNN. Using the output of applying mean-filtered to


Wang, J.-H. et al.

(a) (b)

(c) (d)

(e) (f)

(g) (h)

Fig. 6. (a) Input image “house”, (b) corrupted by 20%salt-and-pepper noise, (c) corrupted by Gaussian noise withmean�0 and variance�0.01, (d) output of applying a 3� 3median filter and watershed analysis to (c), (e) segmentedresult via SOFNN using the output of applying a mean fil-ter and watershed analysis to (b) as input, (f) segmented re-sult via SRM using the mean-filtered output of (b) as input,(g) segmented result via SOFNN using (d) as input, (h) seg-mented result via SRM using the median-filtered output of(c) as input.

Fig. 6(b) as test input, Fig. 6(f) is the result via SRM [13].Using Fig. 6(d) as test input, Fig. 6(g) shows the result viaSOFNN. Using the output of applying median-filtered toFig. 6(c) as test input, Fig. 6(h) is the result via SRM. De-spite the scattering false contours, these results verify thatSOFNN indeed is robust to noise.

3.3. Applying SOFNN to Gene-Expression DataClustering

As mentioned, one key attribute of SOFNN is the fu-sion behavior that facilitates identification of regions pre-sumably comprising a cluster. In this experiment, wedemonstrate that this property of SOFNN makes the ex-ploration of the substructure in the input data much sim-pler while preserving essential information such as hier-archical structure. In fact, SOFNN can be easily incor-porated with existing methods such as MST [14, 15] toextract the hierarchical structure that is highly preferredfor biologist in interpreting the clustering results.

When applied to clustering, the original MST tends tooverlook some local inter-data relations as it globally ex-ploits single-linkage structural information among sub-trees. To deal with this problem, we apply a modified ver-sion of MST (MMST) that mainly involves two stages.First, every data point is guaranteed to be linked to itsclosest neighbor. The edges thus formed are called es-sential edges. Also, depending on the input distributionscertain number of sub-trees might be generated at thisstage. Second, with these sub-trees a minimum span-ning tree can be generated by applying a greedy algo-rithm to sort all possible edges connecting two separatesub-trees in increasing order of Euclidean distance andthen an edge will be picked from the sorted list in such away that the edge, the previously picked ones, and the es-sential edges together cannot form a closed contour. Thisstrategy has been adopted in the classical Kruskal’s algo-rithm [16] to search the MST in a graph. The selectionprocess continues until the picked edges in conjunctionwith the essential edges are sufficient to form a spanningtree. The chosen edges will be labeled as necessary edges.In order to cluster multi-dimensional data into M f clus-ters, it is essential to remove M f � 1 necessary edges indecreasing order of Euclidean distance. The M f clustersare regarded as prototypes or sub-trees, hence the numberof the prototypes are much fewer than the original inputdata. Most importantly, these prototypes are significant tomulti-dimensional data which are difficult to analyze.

To demonstrate the capability of SOFNN in exploringthe substructure in the input data while preserving essen-tial hierarchical structure information, we purposely use agraphical output of a cluster analysis as the original geneexpression data input [17]. In order to apply MMST toclustering, the green components extracted from Fig. 7(a)are treated as a grey image of size 256�256 (Fig. 7(b)),each column can represent a gene condition, e.g. a pa-tient’s record. In order to perform clustering, each col-umn containing 256 distinct genes is mapped into multi-dimensional space. Note that the order of the column fromleft to right is arranged as the 1st to the 256th conditions.The right side of Fig. 7(a) shows the zoom-in of the 254th-256th columns. As widely known, filtering is a necessarypreprocess when dealing with Gene-expression data [18],Fig. 7(c) shows the filtered output using Fig. 7(b) as in-put. Fig. 7(d) shows the result of applying SOFNN toFig. 7(c). Fig. 7(e) illustrates the MMST results of three



(a)

(b) (c)

(d) (e)

Fig. 7. (a) Graphical output of a cluster analysis as theoriginal gene expression data input, (b) extracting the greencomponent treated as a grey image in (a), (c) result of filter-ing using input (b), (d) result of SOFNN using input (c), (e)clustering result using input (d).

separated clusters using Fig. 7(d) as input, wherein thenumber 1, 2, 3 represent the clustering label. These la-beled results clearly reveal that the outcome of SOFNNexactly corresponds to separate Gastric tumor lymph nodeand non-tumor Gastric tissues in Fig. 7(a), verifying theeffectiveness of SOFNN in grouping the similar regions.

The curves of Figs. 8(a) and (b) represent the grey leveldistribution of 255th(green)/256th(blue), 254th (red)/255th

(green) conditions in Fig. 7(a), respectively. In nakedeyes, the variation between two curves 255th/256th inFig. 8(a) seems quite vivid, but the two curves are actu-ally more similar in shape. On the other hand, though the

(a) (b)

(c) (d)

Fig. 8. (a) Grey level distribution of 255th (green)/256th

(blue), (b) grey level distribution of 254th (red)/255th

(green), (c) grey level distribution of 255th/ 256th after ap-plying filtering and SOFNN, (d) grey level distribution of254th / 255th after applying filtering and SOFNN.

variation between two curves of 254th / 255th in Fig. 8(b)looks smaller, the two curves are actually less alike inshape. In fact, by examining the zoom-in of Fig. 7(a),we can readily obtain the same separation with nakedeyes. Therefore, the proposed SOFNN indeed is capa-ble of reducing the variation between 255th/256th, andcan achieve more accurate clustering result. This is ver-ified by the almost overlapped curves in Fig. 7(c), i.e.,columns of 255th/256th are classified to the same cluster,yet columns of 254th /255th are classified as two differentclusters, as shown in Figs. 8(c) and (d), respectively. Theresults demonstrate that fusion provides desirable effectof reducing the variation of multi-dimensional data suchas gene-expression data to achieve better clustering.

4. Characterizations

This section characterizes SOFNN in order to revealother important properties and gain more in-depth under-standing of the method. First, the issue of stability analy-sis is addressed. We will also show how the parameter α iand β i affects the learning rate in further improving theperformance of SOFNN.

4.1. Stability AnalysisIt is widely acknowledged that judging whether a fi-

nal segmentation result is the best is pointless due to thelacking of objective metric. Nevertheless, in region-basedtechniques gauges such as maximum likelihood or leastsquare error have been developed to address the issue ofsegmentation quality. Our goal here is to study the dy-namics of SOFNN and prove SOFNN would converge toa stable state corresponding to a sensible segmentation.


Wang, J.-H. et al.

(a)

(b)

Fig. 9. (a) The average learning rate εav of each iteration byusing input image “peppers”, (b) the average learning rateεav of each iteration by using input image “house”.

Denote nt as the number of regions in image tessel-lation at the t th iteration. Before proceeding, we notethat once regions are merged, they will not separate af-terwards, namely nt�1 � nt . By Eq. (1), the similarityused as connection weight wt

i j is controlled by the λ (t�.And by Eq. (2), as the training iteration index increasesthe value of λ (t� goes up rather quickly, that will put astrong inhibitory effect on the value of wt

i j. Thus, byEq. (3) it is easily seen that for an arbitrary regions Riand R j� wt�1

i j � wti j. It follows that

��It�1�Ri�� It�Ri�� It�Ri�� It�1�Ri�

��. As λ �t� ∞, and Eq. (1) reduces to�wt

i j 0� j � iwt

i j � wtii �

1

eλ �t��It �Ri��It �R j��

� 1e0 � 1� j � i� (8)

Eq. (8) is called the identification equation, that is, regionsRi and R j being different guarantees t ∞�wt

i j � 0, andvice versa. Note that the convergence property also holdsfor local statistics of ξ t�1�Ri� � ξ t�Ri�

4.2. Convergence of Learning RateDuring the training process, the learning rate itself

also converges to a fixed value. Instead of convergingto a universal value, the value of converged rate varieswith different input data. We use input images “pep-pers” and “house” to demonstrate this, the results shownin Figs. 9(a) and (b) indicates that the learning rates con-verged to 9.248�10�3 and 7.445�10�3 for “peppers” and“house”, respectively. In addition, we can see clearlythat although the learning rate as a whole decreases toachieve stability, it slightly fluctuates at times, namely

Fig. 10. αT �α tav and β t

av�βT in each iteration.

the so called network plasticity that provides the desirableproperty for “forcing” SOFNN out of a local minimumwhile in search of a better convergence state.

4.3. Counteracting Parameters AnalysisUsing the input image “peppers”, Fig. 10 shows values

of αT�αtav and β t

av�βT which are defined as the averageof αT�αt

i and β ti �βT over all regions at iteration t. We

can see that β tav�βT is monotonically increased, in con-

trast, αT�αtav always decreases. The result clearly verifies

that α i favors region-merging while β i favoring region-splitting, and by counteracting to each other they pro-vide the effect of avoiding over-segmentation and under-segmentation.

As shown in Fig. 10, the value of α i is quite large at thebeginning iteration, thereby facilitating fast fusion withadjacent regions so as to favor regions merging. However,as many regions have been merged, excessive mergingshould be prevented, namely, in order to prevent under-segmentation the merging speed should be slow down af-ter certain training iterations by providing a mechanismcounteracting the merging. That is exactly the role playedby the parameter β i.

5. Conclusions

We have presented a new approach SOFNN that in-corporates simultaneously feature fusion, counteractinglearning, and extended adjacency strategy to solve theproblems of over-segmentation that plagues conventionalapproaches. To improve the segmentation accuracy, theproposed approach employs a pair of counteracting pa-rameters to avoid under-segmentation. Empirical resultshave shown that the strategy of extended adjacency effec-tively can acquire superior local statistics for determininga more reliable merging criterion. The strategy is effec-tive and robust even when noise is in presence. Throughthe fusion, regions comprising an object are identified andmerged in a self-organizing way, and the training process



will be terminated without manual intervention, therebyconveniently obviating the need of pre-specifying the ter-minating number of objects. Another major advantage ofSOFNN is that all regions are processed in parallel fash-ion, thus providing great potentiality for a fully parallelhardware implementation such as FPGA [19].

To compare, we note that another self-organizing neu-ral network, the Grow Cell Structure (GCS) [20] adoptsa competitive learning to update only the best-matchingunit and its neighborhood cells. In contrast, SOFNN un-dergoes a counteracting learning, and all neurons are si-multaneously updated. Furthermore, the number of neu-rons in SOFNN will only decrease after each merging;it never increases, whereas GCS regulates the number ofneurons to increase or decrease during the training. Inthe future, we will consider generalizing SOFNN to al-low the increase in the number of neurons intermittently(e.g. in splitting the erroneous merging), we conjecturethat by doing so more accurate and reliable segmentationresults can be obtained. We also attempt to apply SOFNNto color images, and examine the performance of SOFNNunder different color spaces. Furthermore, in order to pro-cess various input data with a dimension number higherthan 2D image data, the definition of a region should begeneralized to a high-dimensional sphere.

AcknowledgementsThis research was supported by the National Science Council ofTaiwan under grant NSC 94-2213-E-019-014.

References:[1] F. Hoppner, F. Klawonn, R. Kruse, and T. Runkler, “Fuzzy Cluster

Analysis,” New York, Wiley, 1999.[2] A. W. C. Liew, H. Yan, and N. F. Law, “Image Segmentation Based

on Adaptive Cluster Prototype Estimation,” IEEE Trans. Fuzzy Sys-tems, 13-4, Aug., 2005.

[3] A. M. Bensaid, L. O. Hall, J. C. Bezdek, L. P. Clarke, M. L.Silbiger, J. A. Arrington, and R. F. Murtagh, “Validity-Guided(Re)Clustering with Applications to Image Segmentation,” IEEETrans. Fuzzy Systems, 4-2, May, 1996.

[4] C. W. Chen, J. Luo, and K. J. Parker, “Image Segmentation viaAdaptive K-Mean Clustering and Knowledge-Based Morphologi-cal Operations with Biomedical Applications,” IEEE Trans. ImageProcessing, 7-12, Dec., 1998.

[5] S. A. Hojjatoleslami and J. Kitter, “Region Growing: A New Ap-proach,” IEEE Trans. Image Processing, 7-7, pp. 1079-1084, July,1998.

[6] R. C. Gonzalez and R. E. Woods, “Digital Image Processing,” 2nd

Ed., Prentice-Hall, 2002.[7] D. Hagyard, M. Razaz, and P. Atkin, “Analysis of Watershed Al-

gorithms for Greyscale Images,” IEEE Proc. Int. Conf. Image Pro-cessing, pp. 41-44, 1996.

[8] L. Vincent and P. Soille, “Watersheds in Digital Spaces: An Ef-ficient Algorithm Based on Immersion Simulations,” IEEE Trans.Pattern Anal. Machine Intell., 13-6, pp. 583-598, June, 1991.

[9] T. Pavlidis, “Structural Pattern Recognition,” New York, Springer,1980.

[10] K. Haris, S. N. Efstratiadis, N. Maglaveras, and A. K. Katsagge-los, “Hybrid image segmentation using watersheds and fast regionmerging,” IEEE Trans. Image Process., 7, (12), pp. 1684-1699,1998.

[11] A. P. Mendonca and E. A. B. da Silva, “Segmentation ApproachUsing Local Image Statistics,” Electronics Letters, 36, 14, pp. 1199-1201, July 6, 2000.

[12] T. Kohonen, “The Self-Organizing Map,” Proc. of IEEE, 78, 9,pp. 1464-1480, 1990.

[13] R. Nock and F. Nielsen, “Statistical Region Merging,” IEEE Trans.Pattern Analysis and Machine Intelligence, 26, 11, pp. 1452-1458,Nov., 2004.

[14] C. T. Zahn, “Graph-Theoretical Methods for Detecting and Describ-ing Gestalt Clusters,” IEEE Trans. on Computers, 20, pp. 68-86,Jan., 1971.

[15] C. F. Bazlamacci and K. S. Hindi, “Minimum-weight Spanning TreeAlgorithms – A Survey and Empirical Study,” Computers & Oper-ations Research, 28, pp. 767-785, 2001.

[16] K. A. Ross and C. R. B. Wright, “Discrete Mathematics,” Prentice-Hall, New Jersey, 1999.

[17] S. Y. Leung, X. Chen, K. M. Chu, S. T. Yuen, J. Mathy, J. Ji, A. S.Chan, R. Li, S. Law, O. G. Troyanskaya, I. P. Tu, J. Wong, S. So, D.Botstein, and P. O. Brown, “Phospholipase A2 group IIA Expres-sion in Gastric Adenocarcinoma is Associated with Prolonged Sur-vival and Less Frequent Metastasis,” Proc. of the National Academyof Science, 99, 25, pp. 16203-16208, Dec., 2002.

[18] D. Jiang, C. Tang, and A. Zhang, “Cluster Analysis for Gene Ex-pression Data: A Survey,” IEEE Trans. Knowledge and data engi-neering, 16-11, Nov., 2004.

[19] B. Zeidman, “Designing with FPGAs and CPLDs,” CMP Books,Sep., 2002.

[20] B. Fritzke, “Growing Cell Structure: A Self-organizing Networkfor Unsupervised and Supervised Learning,” Neural Networks, 7-9,pp. 1441-1460, 1994.

Name:Jung-Hua Wang

Affiliation:Professor, Soft-computing Lab, Department ofElectrical Engineering, National Taiwan OceanUniversity

Address:Rm-503, Bldg. EE-1, 2, Pei-Ning Road, Keelung, Taiwan 20224, R.O.C.Brief Biographical History:1983- Chung-Shan Institute of Science and Technology (CSIST)1985- Texas Instrument1991- Industrial Technology Research Institute (ITRI)1998-present Taiwan Intellectual Property Office (TIPO)1991-present Department of Electrical Engineering, National TaiwanOcean UniversityMain Works:� Image Processing, Neural Networks, Pattern RecognitionMembership in Academic Societies:� Institute of Electrical and Electronics Engineers (IEEE)� Optical Society of America (OSA)


Wang, J.-H. et al.

Name:Chun-Shun Tseng

Affiliation:Research assistant, soft computing lab, Depart-ment of Electrical Engineering, National TaiwanOcean University

Address:2 Pei-Ning Road, Keelung, Taiwan 20224, R.O.C.Brief Biographical History:2006 Received the B.S. degree in electrical engineering from NationalTaiwan Ocean University, Keelung, Taiwan, R.O.C.2006- Graduate student in department of Electrical Engineering in NTOUMain Works:� Neural Network, Image processing, pattern recognition, Applying neuralnetworks to current clustering/validation for gene expression dataNSC94-2213-E-019-014-, Applying Quantum Neural Networks toMicroarray Analysis NSC 95-2221-E-019-029Membership in Academic Societies:� Student Member in The Institute of Electrical and Electronics Engineers(IEEE)

Name:Sih-Yin Shen

Affiliation:Research assistant, soft computing lab, Depart-ment of Electrical Engineering, National TaiwanOcean University

Address:520 room, EE1, 2 Pei-Ning Road, Keelung, Taiwan 20224, R.O.C.Brief Biographical History:2006 Received the B.S. degree in electrical engineering from NationalTaiwan Ocean University, Keelung, Taiwan, R.O.C.2006- Graduate student in department of Electrical Engineering in NTOUMain Works:� Neural networks, image segmentation, bio-information, Applying neuralnetworks to current clustering/validation for gene expression dataNSC94-2213-E-019-014-, Applying Quantum Neural Networks toMicroarray Analysis NSC 95-2221-E-019-029Membership in Academic Societies:� Student Member in The Institute of Electrical and Electronics Engineers(IEEE)

Name:Ya-Yun Jheng

Affiliation:Research Assistant, Soft Computing Lab, De-partment of Electrical Engineering, National Tai-wan Ocean University

Address:EE-1 Building, 520R, 2 Peining Rd., Keelung, TaiwanBrief Biographical History:2006- Received B.S.E.E. from National Taiwan Ocean University2006- Graduate student, Soft Computing Lab, Department of ElectricalEngineering, National Taiwan Ocean UniversityMain Works:� Research and development of real-time motion object recognition andtracking. (NSC 95-2221-E-019-085)� Applying Quantum Neural Networks to Microarray Analysis. (NSC95-2221-E-019-029)Membership in Academic Societies:� Student Member in The Institute of Electrical and Electronics Engineers(IEEE)


self-organizing fusion neural networks

Documents