A Binary Neural Network Algorithm for the Graph Partitioning

Problem

Yasuhiro Tamaki, Nobuo Funabiki, and Seishi Nishikawa

Department of Informatics and Mathematical Science, Graduate School of Engineering Science, Osaka Un iversity,

Toyonaka, Japan 560

SUMMARY

The graph partitioning problem is an NP hard prob-

lem of deriving the partitioning of each vertex such that the

total stun of the edge weights among the groups is mini-

mized and the total sum of the vertex weights in each group

is less than the upper limit. In this paper, a neural network

solution is proposed in which the binary neurons are used

for the graph 2-partitioning problem. In the present neural

network, an energy function that is applicable to graphs

both with and without edge and vertex weights is defined.

For improvement of solution accuracy, shaking terms are

introduced into the operating equation. To evaluate the

solution search capability of the present method, simula-

tions are carried out for random graphs, together with the

KL method proposed by Kernighan and Lin, and the FM

method proposed by Fiduccia and Mattheyses. From the

simulation results, it is shown that the solutions obtained by

the present method are the best. © 1999 Scripta Technica,

Electron Comm Jpn Pt 3, 82(12): 34�42, 1999

Key words: Graph partitioning problem; NP hard;

neural network; shaking term; operating equation; KL

method; FM method.

1. Introduction

The graph partitioning problem is the problem of

deriving the partitioning of each vertex such that the total

sum of the edge weights among the group becomes a

minimum (objective condition) by making the sum of ver-

tex weights for each group less than the upper limit (con-

straint) for a given graph G = (V, E). Here, V is the ensemble

of the vertices, and E is that of the edges. The 2-partitioning

problem of a 12-node graph is shown in Fig. 1.

Let the vertex and edge weights be 1 and the upper

limit of the total sum of the vertex weights for each group

be 6. Then, the sum (Cmin) of the edge weights among the

groups takes the minimum value of 6 if nodes 1 to 6 are

assigned to Group 0 and nodes 7 to 12 to Group 1.

The graph partitioning problem is important for en-

gineering in such cases as the layout design of VLSI sys-

tems and object allocation in computers [1]. For instance,

when a neural network is installed on a parallel computer,

it is necessary to assign neurons (= vertices) to each proc-

essor such that the information exchange between the neu-

CCC1042-0967/99/120034-09

© 1999 Scripta Technica

Electronics and Communications in Japan, Part 3, Vol. 82, No. 12, 1999Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J80-A, No. 9, September 1997, pp. 1431�1438

Fig. 1. A 2-partitioning problem of a 12-node graph.

34

rons (= edges) becomes a minimum. This problem can be

formulated as a graph partitioning problem.

It has been shown that the graph problem belongs to

the NP hard class [2]. To date, several approximate analysis

methods have been proposed [3�5]. A typical approxima-

tion method for the graph partitioning problem is the KL

method proposed by Kernighan and Lin [1, 6]. In this

solution method, a relatively good solution can be obtained.

However, the solution is highly dependent on the initial

partitioning and has a high probability of being captured by

a local solution. The computational time for the KL method

is O(N2log N) when the number of nodes of the graph is N.

Another approximate method is the linear time solution

method (FM method) for the number of edges proposed by

Fiduccia and Mattheyses [7]. This method is also highly

dependent on the initial partitioning. In the graphs treated

by these methods, usually only two nodes exist at the end

points of each edge. A graph partitioning method has also

been proposed for hypergraphs in which more than two

nodes exist at the end points of each edge [8].

Many of the solution methods proposed to date are

intended to be applied to �graphs without weight� in which

the weights of the vertices and edges are fixed to 1. How-

ever, when engineering applications are taken into account,

it is necessary to treat �graphs with weight,� in which the

weights at the vertices and edges have some ranges of

values. This is because the sizes of the modules to be

partitioned and the number of signal lines between modules

are different in VLSI design. In this paper, graphs both with

and without weights are treated. A graph 2-partitioning

method by neural networks using binary neurons (NN

method) is proposed. By using binary neurons for which

either a 0 or a 1 output exists, implementation on digital

computers becomes easier. In the present NN method, a new

energy function is defined so that the constraints of graphs

both with and without weights can be satisfied. Also, to

improve the accuracy of the solution obtained, a shaking

term [9] is introduced into the operating equation.

In order to evaluate the solution search capability of

the NN method proposed in this paper, simulations are

carried out with it and with the KL and FM methods. The

graphs used in the simulation include one with the weights

at each vertex and edge fixed to 1, and one with random

weights.

2. Definition of Graph 2-Partitioning

Problem

In this paper, the nonoriented graph G(V, E) with

weights on the vertices and edges is considered. The

weights at the vertex and edge are 1 in the graph without

weight.

The symbols used in this paper are as follows:

V: Ensemble of vertices of the graph G

E: Ensemble of edges of the graph G

N: Total number of vertices belonging to the graph (=

|V|)

wi: The i-th vertex weight (i = 1 to N)

eij: The edge weight between vertices i and j

Gk: Ensemble of the vertices belonging to Group k

W: Total sum of the vertex weights (W = ¦ i 1N wi)

Wk: Sum of the weight of the vertices belonging to

Group k (k = 0, 1) (�Gk) (Wk = ¦i�Gk wi)

Cmin: Cut weight among the groups (Cmin = ¦i�G0

¦ j�G1 eij)

D: The sum of the vertex weights that can belong to

each group has a margin of DW from W/2. In this simula-

tion, D = 0.05 is used.

The constraints and objective conditions for the 2-

partitioning problems treated in this paper are as follows.

[Definition 1] For the graph 2-partitioning problems

with and without vertex and edge weights, the constraint is

that the sum of the vertex weights belonging to each group

is in the range

W/2 � DW d Wk d W/2 + DW (k = 0, 1)

while the objective condition is that the sum of the edge

weights over the groups (cut weight) Cmin is a minimum.

In this problem, it is required to derive a partitioning

that meets the objective condition while the constraint is

satisfied. Figure 2 is a diagram of the graph 2-partitioning

problem for the case in which the sum of the vertex weights

is 100 and D = 0.05. The graph in the upper part of the figure

is divided into Group 0 and Group 1 so that the sum of the

vertex weights for each group is 45 to 55, and then the cut

weight (Cmin) is minimized.

Fig. 2. An image of the graph 2-partitioning problem.

35

3. Neural Network Solution Method

3.1. Neuron expression

In the graph 2-partitioning problem, a total of N

vertices is divided into two groups (Groups 0 and 1). Hence,

a neural network with N neurons is used in response to N

vertices. The i-th neuron belongs to the group with vertex i

and has an input Ui and an output Vi.

For the input/output function of the neuron we use a

binary neuron model that takes values of 0 and 1 as the

neuron output values (see Fig. 3).

Hence, the input/output function of the neuron is

The neuron output Vi = 0 indicates that vertex i belongs to

Group 0, while Vi = 1 indicates that vertex i belongs to

Group 1.

3.2. Energy function

In the solution of the combinatorial optimization

problem using a neural network, an evaluation function

called the energy function, expressing the constraint and the

objective condition of the problem to be solved, is defined.

This energy function is a multivariable function of the

neuron output values:

As the energy function, a nonnegative function (usually of

bilinear form) that takes the minimum when the neuron

output represents the optimum solution and a large value

otherwise is chosen.

In this paper, in response to the constraint and the

objective condition of the graph partitioning problem, an

energy function consisting of the following two terms is

proposed:

Here, A and B are positive coefficients.

E1 is the energy function representing the constraint.

The function fi(x) used here is given by Eq. (6) (see Fig. 4).

The function E1 takes the value 0 when both the sum of wi

for the vertices i and the sum of Wi of the vertex weights for

each group are within the restricted region. This term keeps

the sum of the vertex weights for each group below the

upper limit of the restricted range.

E2 is the energy function expressing the objective condition

and indicates the value of the cut weight. When the vertices

i and j belong to different groups, the weight of the edge

between i and j is added. Hence, E2 is a function taking the

minimum value when the cut weight eij between the parti-

tions is the minimum. The present NN method is intended

Fig. 3. The binary neuron function.

(1)

(2)

(3)

(4)

(5)

Fig. 4. The function fi(x).

(6)

36

for ordinary graphs where each edge is connected to two

vertices. By modification of E2, the method can be applied

to hypergraphs. If the vertices of each edge in the hyper-

graph are divided into different groups, it is necessary to

compute the cut weight by assuming that this edge is

divided at one location.

3.3. Operating equation

The neuron input is updated by the operating equa-

tion. The operating equation for the i-th neuron is derived

by partial differentiation of the energy function by Vi:

The A term operates such that the constraint is satisfied

whichever group the vertex i belongs to. The B term acts to

reduce the sum of the edge weights across the groups.

3.4. Introduction of shaking term

In the present NN method, it is not necessarily pos-

sible to find the optimum solution when the state of the

neuron is varied by operating Eq. (7). The reason is that the

local optimum solution depending on the initial condition

is obtained when the solution process begins from the initial

condition. In order to relax the dependence on the initial

condition, the introduction of a noise term according to a

Gaussian distribution has been proposed [10]. However, in

the present method, such an approach is not practical since

an extremely long computation time is needed to find a

high-quality solution and a random number must be gener-

ated for each neuron.

In order to obtain a high-quality solution within a

limited computational time, the shaking term given by Eq.

(8) is introduced into the operating equation. By adding

vibrations to the neuron state, trapping by local optimum

solutions is relaxed so that the solution is sought over a wide

range [9]:

where C is a positive coefficient. Hence, the shaking term

operates in such a way that the vertex belonging to Group

0 is shifted to Group 1 and that belonging to Group 1 is

shifted to Group 0.

However, if this shaking term is always added to

update the neuron condition, the state changes excessively

and even a local solution may not be reached. Hence, as the

number of repeated calculations is increased, the number

of occasions on which the shaking term is added must be

reduced. Hence, if the number of repeated calculations is t,

the timing to add the shaking term to the operating equation

is as follows:

Let y = T0 � t u T0TMAX

� 1

Here, T0 is a positive coefficient, and TMAX is the upper

limit of the number of repetitive calculations. The variation

of the timing to add the shaking term for TMAX = T02 is as

shown in Fig. 5. When TMAX = n u T02, the same period is

repeated n times.

3.5. Restriction on the neuron input

When the solution is searched from the initial condi-

tion via the NN method, there is a problem of falling into

the local optimum solution, dependent on the initial condi-

tion. As a factor, the neuron input may take an extremely

large value or an extremely small value. If such a phenome-

non occurs, the state of the neuron output does not vary

easily. Hence, by restricting the neuron input value within

a certain range, the state of the neuron output is made to

change easily. In this paper, the upper limit (Umax) and the

lower limit (Umin) of the neuron input are set as follows by

using the average Uin of the edge weights connected to each

vertex:

(7)

(8)

Fig. 5. The timing of adding the shaking term.

(9)

(10)

(11)

37

From the B term in Eq. (7), the variation 'U of the input U

is dependent on the sum of the weights of the edges con-

nected to each vertex. The operating range of U is the

average of the edge weights from each vertex of the graph

[Eq. (10)]. Also, since each vertex can belong to either

Group 0 or 1 with an equal probability, the maximum and

minimum values of U are given by Eq. (11).

The neuron input takes the maximum value when it

is larger than the specified maximum value, and the mini-

mum value when it is smaller than the specified minimum

value:

This provision is effective in the hardware realization of the

NN method, since the necessary number of bits can be

reduced [11].

3.6. Setting of coefficients for the operating

equation

Unless coefficients A and B in the operating Eq. (7)

are appropriately set, no balance can be obtained between

the A and B terms and no high-quality solution can be

obtained. Hence, these coefficients are set as follows.

The B term is considered to depend on the average

Uin of the sum of the edge weights connected to each vertex.

Also, the A term depends on the average value wBB

[in (13)]

of the vertex weights:

Hence, from the above, it is necessary to set

with a proportionality constant K in order to achieve a

balance between the A and B terms. From the simulation,

we let K = 1 /10 for the graph with the vertex weights fixed

to 1 and K = 1/3 for the graph with random vertex weights.

Also, B is fixed to 1.

The value of the coefficient C used in the C term is

Uin/(3 + 10 u U) from the simulation. Here, U is the edge

generation rate of the graph (0 < U d 1).

The computation procedure of the NN method is

given in the Appendix.

4. Performance Evaluation by Simulation

4.1. Simulation condition

In the present simulation, a random graph is con-

structed under the following conditions.

x Number of vertices (N): 100, 200, 300, 400, 500

x Edge generating ratio (U): 0.05, 0.10, 0.30

x Vertex and edge weights: fixed to 1, or random

number in range 1 to 10

Also, the simulation of each solution method is car-

ried out under the following conditions.

x For the identical graph, the simulation is carried

out from 100 different types of initial distribu-

tions. The cut weight (total sum Cmin of the edge

weights across the groups) that is the smallest of

these distributions is chosen. For each solution

method, the identical 100 types are used for the

initial partitioning.

x As the initial partitioning, N/2 vertices are as-

signed to each group.

The upper limit TMAX of the number of repetitive

calculations significantly affecting the solution accuracy

and the computational time is determined from the results

in Fig. 6. This figure shows the degree of improvement of

the solution when TMAX is varied from 100 to 1000 for a

graph with 500 vertices, a 0.30 edge generation ratio, a

vertex weight of 1, and a side weight that is a random value

in the range 1 to 10. When TMAX is more than 200, the cut

weight of the solution obtained by the NN method, and

hence the solution accuracy, is found to be superior to other

methods. However, even if TMAX is set above 300, the

(12)

(13)

(14)

Fig. 6. The improvement of the solution quality by

increasing the number of iteration steps.

38

solution accuracy does not improve significantly in propor-

tion to the increase of the computational time. Therefore,

TMAX is chosen to be 200 in the simulation.

4.2. Effect of the shaking term

In order to show that the solution accuracy is im-

proved by the introduction of the shaking term, the simula-

tion is carried out for cases with and without the shaking

term in the operating equation. Table 1 shows the cut

weights with and without the shaking term for a graph with

N = 100. It is seen that better solutions are obtained if the

shaking term is added to the operating equation than with-

out this term. Hence, the introduction of the shaking term

is effective for improvement of the solution accuracy.

4.3. Comparison with conventional solution

methods

The solution accuracy of the NN method is compared

with those of the conventional KL and FM methods. The

cut weights obtained by each method are shown in Tables

2 to 5. The cut weights that are the minimum of those found

by the three solution methods are indicated by italics.

4.3.1. Case with vertex weight = 1

Tables 2 and 3 show the simulation results for the

graph in which the vertex weights are fixed to 1. It is found

that the results obtained by the NN method are about equal

to or somewhat superior to those obtained by the KL and

FM methods. On introduction of the shaking term, the

solution escapes from local optima dependent on the initial

partitioning and converges to an accurate solution.

4.3.2. Case with vertex weights in range 1 to 10

Tables 4 and 5 show the simulation results for graphs

with randomly assigned vertex weights in the range 1 to 10.

It is found that the solutions obtained by the NN method

are better than those obtained by conventional methods in

the graph with vertex weights. In particular, with the in-

crease of the edge density and the number of vertices, the

difference between results of the NN method and those of

the conventional method becomes larger. Also, the KL

Table 1. Cut weight solutions with and without shaking

term

Table 2. Cut weight solutions for graphs with equal

vertex weights and edge weights

Table 3. Cut weight solutions for graphs with equal

vertex weights and random edge weights

39

method is the least effective, apparently because the number

of degrees of freedom in exchange of the vertices is small.

4.4. Computational time and discussion

Table 6 shows the computational time on a

SONY/NEWS500 for each method, starting with 100 dif-

ferent types of initial partitioning for a graph whose vertex

and edge weights are fixed to 1. It is seen that the FM

method is the fastest and that the NN method becomes faster

than the KL method as the number of vertices is increased.

5. Conclusions

This paper has proposed a neural network (NN) so-

lution method using binary neurons for the graph 2-parti-

tioning problem applied to graphs with and without vertex

and edge weights. In order to evaluate the solution search

capability of the NN method in which a heuristic procedure

called the shaking term is introduced, numerical simula-

tions have been performed with this method and with the

conventional KL and FM methods. The solution search

capability of the proposed NN method is found to be about

equal to or slightly superior to the conventional methods in

the case of graphs with the vertex weights fixed to 1, and

much better in the case of graphs with the vertex weights

randomly assigned in the range of 1 to 10.

REFERENCES

1. Kernighan BW, Lin S. An efficient heuristic pro-

cedure for partitioning graphs. Bell Syst Tech J

1970;49:291�307.

2. Garey MR, Johnson DS. Computers and intractabil-

ity: A guide to the theory of NP-completeness. Free-

man; 1979.

3. Johnson DS, Aragon C, McGeoch L, Schevon C.

Optimization by simulated annealing: An experimen-

tal evaluation, part 1, graph partitioning. Oper Res

1987;37:865�892.

4. Van den Bout DE, Miller TK, III. Graph partitioning

using annealed neural networks. IEEE Trans Neural

Networks 1990;1:192�203.

5. Bui TN, Moon BR. Genetic algorithm and graph

partitioning. IEEE Trans Comput 1996;45:841�855.

6. Krishnamurthy B. An improved min-cut algorithm

for partitioning VLSI networks. IEEE Trans Comput

1984;C-33:438�446.

7. Fiduccia CM, Mattheyses RM. A linear-time heuris-

tic for improving network partitions. Proc 19th Des

Autom Conf, p 175�181, 1982.

8. Yeh C-W, Cheng C-K, Lin T-TY. A general purpose,

multiple-way partitioning algorithm. IEEE Trans

Comput Aided Des 1994;13:1480�1488.

Table 4. Cut weight solutions for graphs with random

vertex weights and equal edge weights

Table 5. Cut weight solutions for graphs with random

vertex weights and random edge weights

Table 6. CPU time for three algorithms (seconds)

40

9. Funabiki N, Nishikawa S, Tajima S. A binary neural

network approach for max cut problems. Proc Int

Conf Neural Inf, p 631�635, 1996.

10. Akiyama Y, Yamashita A, Kajiura A, Aiso H. Combi-

natorial optimization with Gaussian machines. Proc

Int Joint Conf Neural Networks p. 533�540, 1989.

11. Yamashita H, Kurokawa K, Koga Y. Hardware reali-

zation of mutually coupled binary neural network.

Trans IEICE 1994;J77-D-II:2130�2137.

APPENDIX

The computational procedure of the neural network

method for the graph partitioning problem is presented.

Here, CM(t) indicates the minimum cut weight found in

repetitive computations up to the t-th iteration.

[Step 0] Initialization of the number of iterations and

setting of the parameters.

[Step 1] Initial setting of the neuron output value

The outputs of N/2 randomly selected neurons are set

to 0, and those of the remaining N/2 are set to 1.

[Step 2] Initial setting of neuron input values:

In the above, random[x, y] is a function generating uniform

random numbers (integers) between xy.

[Step 3] Updating the input/output values of the

neurons and the minimum cut weight

(1) Updating the neuron input values:

(2) Updating of neuron outputs

If this partitioning satisfies the constraints, and if the

cut weight is smaller than the smallest previous value of

CM(t), the latter is updated.

[Step 4] Increment iteration number

[Step 5] Check of final condition

x The output condition of all of the neurons does not

change, or

x The number t of iterations is TMAX.

If either is satisfied, the process completes. Otherwise, the

process is returned to Step 3.

else

(A.1)

(A.2)

(A.4)

(A.5)

(A.7)

(A.8)

(A.3)

(A.9)

(A.6)

41

AUTHORS (from left to right)

Yasuhiro Tamaki (student member) graduated from Osaka University in 1995 and completed the M.S. course in 1997.

He then joined Sumitomo Electric. He worked on neural networks in his student research.

Nobuo Funabiki (member) graduated from the University of Tokyo in 1984 and joined Sumitomo Metals. In 1991, he

completed the M.S. course at Case Western Reserve University. In 1994, he became a lecturer in the Department of Information

Engineering, Faculty of Engineering Science, Osaka University, where he is now an associate professor. He holds a D.Eng.

degree. He has been engaged in research on neural networks, combinatorial optimization problems, and process control. He is

a member of IEEE and the Information Processing Society.

Seishi Nishikawa (member) graduated from the University of Tokyo in 1965 and completed the doctoral course in 1970.

He then joined Nippon Telegraph and Telephone (now NTT). After serving in the Basic Research Department, in the Information

Communication Processing Laboratory, and as director of the Communication Science Research Laboratory, he became a

professor on the Faculty of Basic Engineering, Osaka University, in 1990. He holds a D.Eng. degree. His research interests are

hardware design methods, parallel architectures, and solution of optimization problems. He is a member of the Information

Processing Society, IEEE, and ACM.

42