ELSEVIER

Advances in Water Resources, Vol. 19, No. 4, pp. 215-224, 1996 Copyright 0 1996 Published by Elsevier Science Limited

Printed in Great Britain. All rights reserved 0309-1708(96)00047-X 0309-1708/96/$15.00+0.00

Parameter structure identification using tabu search and simulated annealing

C. Zheng” & P. Wangb ‘Department of Geology, bDepartment of Mathematics. University of Alabama, Tuscaloosa, Alabama, USA

(Received 12 May 1995; accepted 27 November 1995)

In groundwater modeling the identification of an optimal flow or transport parameter that varies spatially should include both the values and structure of the parameter. However, most existing techniques for parameter identification only consider the parameter values. In this study, the problem of identifying optimal parameter structure is treated as a large combinatorial optimization problem. Two recently developed heuristic search techniques, simulated annealing and tabu seearch, are used to solve the large combinatorial optimization problem. The effectiveness and flexibility of these two techniques are evaluated and compared with simple grid search and descent search, using preliminary results from one- dimensional examples. Among the techniques examined in this paper, tabu search performs extremely well in terms of the total number of function evaluations required. Copyright 0 1996 Published by Elsevier Science Limited

Key words: parameter identification, parameter structure, global optimization, simulated annealing, tabu search, groundwater modeling.

1 INTRODUCTION

Parameter identification, or model calibration, is a critical step in the application of mathematical models in hydrologic sciences. Unfortunately, parameter iden- tification is an inherently difficult process and, as an inverse problem, it is plagued by the well-documented problems of nonuniqueness, nonidentifiability and instability. 1,30332 One of the most problematic aspects of parameter identification is that for most distributed- parameter systems the parameter structure is at least as uncertain as the parameter values. However, most existing approaches for parameter identification require the definition of parameter structure on an a priori basis.13>32 As a result, the parameter structure is often treated in a trial-and.-error manner that does not guarantee the optimality of the identified parameters.

The parameter structure is generally incorporated into the parameter identification process through either the zonation approach or the interpolation approach. In the zonation approach, the model grid is subdivided into a certain number of zones, and the aquifer properties are treated as constant within each zone.3,4 The initial number and boundarks of these constant parameter zones are usually defined on the basis of available information from geologic mapping and aquifer test- ing. Subsequent adjustment is made manually through

trial-and-error calibration runs, and sometimes using the results of parameter value estimation and sensitivity analysis. In the interpolation approach, the parameter structure is defined by a set of the nodal points where unique parameter values are specified initially and are subsequently adjusted. The parameter values at other locations are determined by interpolation from the estimated parameters. Various interpolation schemes have been used for this purpose; for example, inter- polation may be based on the linear basis function associated with a finite-element mode1,35 or on the spline function,26 or a kriging approach may be used.2)5J6

For most field-scale problems, the observed data are often so noisy and sparse that they do not justify the identification of aquifer parameters at a large number of locations. It may be adequate to represent the aquifer system by the zonation approach. After comparing the zonation approach with three other approaches, Keidser and Rosbjerg16 conclude that the zonation approach is not only much simpler than but also generally superior to the other three approaches in cases of limited data availability and poor data quality.

The number of zones with uniform parameter values is referred to as the dimension of parameterization. The necessity to limit the dimension of parameteriza- tion and optimize the zonation pattern has been clarified by many researchers.1,7335 However, due to the level of

215

216 C. Zheng, P. Wang

difficulty, the trial-and-error method is generally used to choose the best combination of the parameter structure and parameter values.‘5~18~28~31~33 The trial-and-error method of determining zonation is inefficient and labor-intensive. For example, even in an aquifer of only two zones, the trial-and-error determination of the zonal boundary is very tedious and time-consuming.‘4 Sun and Yeh29 are the first to propose a systematic way to identify the parameter structure. They transform the problem of optimizing the parameter structure into the problem of determining an optimum structure matrix. By introducing a new parameterization method, the problem of determining an optimum structure matrix is further formulated as a problem of combinatorial optimization. The approach is generally systematic, but the generation of the coefficients of the structure matrix involves a trial-and-error approach. It works well only when the number of zones is very small. When it is large, the rate of convergence is very slow and no global optimum is guaranteed. Therefore, development of an efficient and systematic procedure for parameter structure identification remains a challenging area of research.

During the last two decades, a new class of global optimization methods based on heuristic search tech- niques have emerged rapidly. Major representatives of these optimization methods are simulated annealing (SA), genetic algorithms (GA), and tabu search (TS). These optimization techniques mimic certain ‘natural’ systems, and have received much attention due to their considerable success in solving complex and difficult problems, and the relative ease and generality with which they can be implemented for a wide variety of problems. Although there are still relatively few applications in hydrologic sciences,6120-22’25 these optimization methods have shown promising poten- tial as an effective tool for optimal parameter identification because they are particularly well suited for incorporation of parameter structure in the parameter identification process.

In this paper, we discuss two global optimization methods, simulated annealing and tabu search, in the context of parameter structure identification in ground- water modeling. The emphasis of our discussion is on tabu search because this study is believed to represent the first effort in introducing this technique to the groundwater modeling literature. The effectiveness and flexibility of simulating annealing and tabu search in identifying the optimal parameter structure are assessed using preliminary results from one-dimensional examples.

t FORMULATION OF THE INVERSE PROBLEM

The partial differential equation (PDE) for groundwater flow, assuming constant fluid density and viscosity, can

be expressed as follows,

V - (KVh) + w = S,t3h/& (1)

where h is the hydraulic head, K the hydraulic conductivity tensor, w the fluid sink/source term, and S, the specific storage coefficient of the aquifer. Equation (l), along with a set of initial conditions and boundary conditions, forms the mathematical model of a groundwater flow system. Except for a few simple problems, numerical methods are required to solve the mathematical model. The numerical methods are based on spatial and temporal discretization which divides the continuous space and time domains into a network of discrete nodal points and a series of finite time intervals. When the various aquifer parameters are known, the hydraulic heads at any nodal points and time intervals can be obtained using a finite-difference or finite-element code.36

For most groundwater systems, the aquifer para- meters are not well defined. Thus, in general, an inverse problem must be solved to identify the aquifer para- meters that lead to the best match between model results and field observations. In this study, we concentrate on the type of inverse problem as defined next. Let R be the region of interest discretized into n nodes, 0 = {Sl,S2r. . . , sn}. Here the region 0 is a set of nodes whose coordinates may be one-, two-, or three-dimen- sional. For example, in a one-dimensional region, we simply have the set R = { 1,2,. . . , n}. For a two- dimensional region, each node has two indices, fl = ((1, l), (1,2), . . . , (i,j), . . .}. It is assumed that the region R consists of p sub-regions, or zones, Ri where i= 1,2,... ,p. Individual zones are mutually exclusive. Each zone, say pi, has a uniform hydraulic conductivity Ki and its shape may be irregular. It is further assumed that at selected nodes and times in the region ‘true’ hydraulic heads are observed. The inverse problem is then: find the shapes of the zones, or the boundaries between neighboring zones so that the calculated heads match the observed heads at these nodes. It should be pointed out that although the number of zones and the values of hydraulic conductivity in each zone are treated as known in this study, they can be considered jointly with the zonal boundaries in the identification procedure presented below and will be the subject of further research.

To avoid confusion, we will use the term ‘node’ to represent a physical location in the region. The term ‘solution’, denoted 0, is referred to as a configuration of the region, i.e. the hydraulic conductivity at each node in R is assigned to one of the p values, 6’ = {K(s,), K(S*), . . . , K(s,)}. A solution is also called a point in the solution space. Solution, configuration, and point are used interchangeably.

Associated with each solution is an objective function value that measures the quality of the solution. In our case, the objective function is chosen to be the sum of

Parameter structure idmhjication 217

squares of the weighted differences between the observed heads ho& t) and calculated ones hc(si, t)

E(B) = C Wj,, [fl’(Si, t) - h”(Sjr t)]* i, t

(2)

where wi,t is the weighting factor associated with an observation. The objective function is also referred to as the cost function in the following discussions. The determination of the zonation pattern becomes the determination of a solution such that its objective function, as expressed in eqn (2), is minimized. The problem is a combinatorial optimization problem. Thus, traditional optimization methods for continuous space such as the Newton’s method and gradient based search” are not applicable.

3 GLOBAL 0PTIMIP:ATION METHODS

Let 8 be the solution space and let N(0) be a neighborhood set of solution 19. The specific definition of neighborhood N(B) will be given in the next section. A solution 0 is said to be a local minimum if its objective function has the smallest value in its neighborhood N(0). A solution 0 is said to be a global minimum if its objective function has the smallest value in the entire solution space 8. The optimization of eqn (2) is complicated by the fact that the objective function may have many local minima.

The simplest method of finding the best (optimal) zonation pattern is grid search that assigns each node to all p possible values of hydraulic conductivity and then evaluates the objective function. The one with the smallest value will be the optimal solution. Grid search only works for problems with a very small number of nodes. For example, consider a 10 by 10 region with 3 zones, the total number of zonation configurations is 3’O”. Such an astronomic number rules out grid search for any practical purpose.

Another commonly used search strategy is the descent search in which the sohltion is updated by a new one if and only if the new solution improves the match. The descent search is effective if the objective function has only one minimum. When the function has multiple minima, however, it is very likely that the descent search will be terminated at a local minimum far from optimality. Therefore, global optimization methods must be employed. 111 this section, we will briefly discuss the fundamental idea and methodology for two of the global optimization methods, simulated anneal- ing and tabu search. In subsequent sections, we will present preliminary results from an initial study in which these methods, along with traditional grid and descent search, are used to define the zonation problem. The results obtained by different methods are compared in terms of the total number of function evaluations required.

3.1 Simulated annealing (SA)

Simulated annealing, as the name implies, simulates the thermodynamic process in which a solid metal is heated up to its melted temperature and then is cooled down slowly so that its structure is ‘frozen’ at the crystal configuration at which it has the lowest level of energy. In SA, one can simulate the behavior of a system of particles in thermal equilibrium using a stochastic relaxation technique developed by Metropolis et af.23 In optimization, the objective function to be minimized represents the energy in the thermodynamic process, while the optimal solution corresponds to the crystal configuration. The basic concept of SA lies in allowing the procedure to move occasionally ‘uphill’. This is done under the influence of a random number generator and a control parameter called temperature T. The basic steps of SA are outlined in Table 1.

It can be seen that the parameter temperature T plays an important role in the annealing process. For example, when the temperature T is high, the acceptance probability P is high, i.e. P x 1. Thus, any point is acceptable. This corresponds to a pure random walk. When the temperature T is low, i.e. T x 0, the acceptance probability is low, i.e. P x 0. Therefore, only downhill points are accepted, which represents a descent search.

The set of control parameters involved in simulated annealing is called an annealing schedule. Important control parameters are the initial temperature TO, the maximum number of iterations under a constant temperature L, and the temperature reduction factor A. Determination of the values of the control parameters requires extensive experiment. To guarantee the con- vergence of the annealing process, the annealing process should start at a high initial temperature To and then the temperature T should be dropped slowly.

Table 1. Simulated annealing procedure

Step 1 Select randomly an initial point, say B0 and calculate the corresponding initial value of the objective function, E,. Start the process at ‘temperature’ T= To.

Step 2 Introduce perturbations by choosing a random point 0, in the neighborhood of 0,. Calculate the objective function, El, at 8, and the change AE = El - E,.

Step 3 The point 8r is accepted if AE < 0. For AE > 0, the point 0r is accepted with probability P given by: P = e-*E/T. This means that ‘downhill’ steps (El < E,) are always accepted but the ‘uphill’ steps (E, > E,) are accepted by probability.

Step 4 Repeat Steps 2 and 3 under the same temperature for a suffkient number of perturbations, say L, to ensure that the process has reached the ‘thermal equilibrium’.

Step 5 Lower the temperature, e.g. T = XT, where 0 < X < 1. Go to Step 2 to continue the annealing process until a stop criterion is satisfied.

218 C. Zheng, P. Wang

In addition, the number of iterations under each constant temperature L should be sufficiently large.

SA has several advantages over other methods. First, it is easy to implement and does not require much computer coding. Second, it guarantees the identifica- tion of an optimal solution if an appropriate annealing schedule is selected. This is particularly important when the solution space is large and the objective function has local minima or changes dramatically with small changes in the parameter values. Examples of the SA application in the hydrologic literature include Dougherty and Marryott, Marryott et aL2’ and Mauldon et af.20

3.2 Tabu search (TS)

The ‘natural’ system on which tabu search is based is the human memory process. The modern form of TS derives from Glover.’ The basic principle of TS is to maintain a list of recent moves (transitions from solution to solution). This list is referred to as a tabu list, T. The purpose of keeping such a list is to prevent the search from moving back to where it was previously. Before a potential solution is selected, the search procedure will check the tabu list to see whether its attribute (which will be explained later) is in the tabu list or not. If it is, this move is prevented and will not be considered as the new solution.

The basic steps of the TS procedure consist of starting from a feasible point and then moving to the best neighbor. This is similar to the descent search except for the fact that it may move to a worse solution from the current one. There are two important elements in TS that differ from descent search. One is the construction of a tabu list T of tabu moves, and the other is the incorporation of an aspiration level function.

There are many different ways to form a tabu list. In our application, the elements in the tabu list T are the changes of the K values (or moves) in nodes from solution to solution during recent iterations. Once a move is put in the tabu list, it is prohibited from being selected for a fixed number of iterations depending on the length of the tabu list. The reason for maintaining a tabu list is to exclude moves which would bring the search back where it was at some previous iteration and keep it from being trapped in a local minimum. A move remains effective in the list T only for a certain number of iterations, so that we have in fact a cyclical list T. At certain iteration of the search, if the K value at node s is changed, say, from Kl to KS, then the opposite move that changes the K value at node s from K2 to KI is prohibited and the pair (s, KI) is added at the end of tabu list T while the oldest move in T is removed from T. The pair (s, K,) remains effective in the tabu list for the next 1 T 1 iterations where 1 T 1 represents the length of the tabu list.

The value of an aspiration level function o(s,e)

depends on a specified move s and the solution 8. The aspiration level is attained if E(B) < (Y(s, 0). The role of the aspiration level function is to provide additional flexibility to choose good moves by allowing the tabu status of a move to be overridden if the aspiration level is attained, meaning that the backward move is allowed. The goal is to do this in a way that retains the ability to avoid cycling.

The tabu restrictions and aspiration level criteria of tabu search play a dual role in constraining and guiding the search process. Tabu list T and aspiration level function (Y can be implemented by modifying the objective function expressed in eqn (2) as

E(B, T,(Y) = E(e) if 0 is not in T or E(0) < (Y

E(6) + A4 otherwise

(3)

where M is a large number and Q is the aspiration level which should be gradually decreased when the objective function drops during the search. The basic steps of TS can be summarized in Table 2.

The size of the tabu list T is an important parameter for success. A small size creates cycles, meaning that the solution comes back to the point where it has been. A larger size does not improve the procedure but increases unnecessary computational time. The size should depend on a number of factors such as the number of variables of the problem, the complexity, and the neighborhood structure. Some guidelines are given in Glover.’ For example, using the size 7 or fi where n is the number of estimated parameters has been successful in many applications, including this work.

Compared with SA and other optimization algo- rithms, TS has a number of attractive features. First, TS is not a probabilistic search; rather it is systematic and deterministic. When a local optimum is reached, tabu search structures its iterations in such a way that it permits itself to continue. Therefore, tabu search will never be trapped in a local minimum. As a consequence, repeated computations at the same point can be avoided. Second, TS identifies moves of high quality in

Table 2. Tabu search procedure

Step 1

Step 2

Step 3

(InitiaZization) Select a starting solution 0. Evaluate the modified function E(6J, T, a) for points in neighborhood set N(B) (see Fig. 1). Set the tabu list T empty. (Choice and termination) Evaluate the modified objective function for the solutions in N(8). Determine the best solution 0’ in neighborhood set N(B), i.e. 0’ = argmi; (E(8, T, a)}. (Update) Find the new N(0’) and update the tabu list T. Let e = 8’. Go back to Step 2 to continue the search until a stop criterion is satisfied.

Parameter structure ident$cation 219

the neighborhoods. In contrast, SA randomly samples among these moves in the neighborhood to apply an acceptance criterion that disregards the quality of other moves available in the neighborhood. Third, the number of control parameters involved in TS, such as the size of the tabu list, is fewer a:nd is easier to be determined. In contrast, the determination of an annealing schedule in SA is problem-specific and depends heavily upon experiment.

4 NEIGHBORHOOD STRUCTURE AND SENSITIVITY ANALYSIS

Thus far we have used the term neighborhood N(8) of solution 8 but not given a precise definition. In this section, we will define the neighborhood set and discuss how to evaluate the objective function in the neighborhood efficiently.

For each nodal point s in 0 we define a neighborhood set N(s) as the set of nodal points surrounding s. For example, a nodal point in a one-dimensional region may have two neighbors in its neighborhood set while a node in a two-dimensional region may have four or eight neighbors (Fig. 1). In a .three-dimensional region, a node may have six or 26 neighbors. Other neighborhood structures may be constructed.91’0

Since each zone sli is an integral portion of 0, the shape of the zone is uniquely determined by its boundaries with other zones. We now define the boundaries. Let q(i, j) denote the set of nodes that are in fli and that are neighbors of Clj, 9(i,j) = (8:s E Ri, and for some N(s) E !$}. Once Q(i, j) is defined, it is unnecessary to define Q(j,i). Let N[Q(i, j)] be the neighborhood set for 9( i, j). Then the neighborhood set

A-

-

-

-

Fig. 1. Definition of the neighborhood set.

for solution B is just the union of N[@(i, j)].

No) = U WQ(i, j)). (4) iJ

To increase the efficiency, the simulation code and the optimization code should be integrated. Note that the majority of the computational time associated with a heuristic search method is in evaluating the simulation model after the aquifer parameters are slightly changed. In a standard finite-difference or finite-element model, it is necessary to solve a large linear system of equations, [A] {h} = {b}, where {h} is the heads to be determined, [A] is the coefficient matrix incorporating the aquifer parameters, and {b} is a known column vector. Between successive simulation runs, only several entries in A and b may be changed. Let A’ and b’ denote the slightly changed matrices, i.e. A’ = A + AA and b’ = b + Ab, and h’ denote the new solution. By using matrix theory,24 the new solution h’ can be obtained from the sum of the previous one (h) and the perturbed portion (Ah), i.e. h’ = h + Ah. The perturbed solution Ah is much easier and more efficient to calculate than the entire new solution h’.

To be more specific, suppose that matrix A is an n by n matrix and each column of matrix A is perturbed by a small change, say S

A’ = A + Gee?, i= 1,2,...,n (5)

where e is a column vector with all elements equal to 1 and ei is a vector whose ith element is 1 and zero elsewhere. The superscript T represents matrix trans- pose. From Nobel and Danie124 (p. 191), the perturbed solution Ahi due to the perturbation on column i has the form

hoe? h A/q=-- 1 +eFh, ’

i= 1,2,...,n

where h, is the solution of [A] {h,} = {se}. To compute the n disturbed solutions, one only needs to solve the system of equations twice to obtain h and h, as opposed to solving the equations n times. There appears to be a great potential for reducing the computational time, particularly when the system is large.

5 ONE-DIMENSIONAL PROBLEMS

The model examined in this study is a one-dimensional region R discretized into II nodes with p zones of uniform hydraulic conductivities. All aquifer properties including the boundaries between neighboring hydraulic conductivity zones are first specified and used to obtain the ‘true’ head solution. The calculated heads at certain nodes are chosen as ‘observed’ heads for use in the subsequent identification process in which the bound- aries between neighboring hydraulic conductivity zones are assumed to be unknown. We analyze the uniqueness

I . ,.

Fig. 2. Illustration of a unique boundary between two neighboring zones.

of this type of inverse solution first, and then discuss the preliminary results for two specific examples. Since we believe tabu search has never been applied to groundwater modeling studies, we provide a more detailed description of the technique in the following discussions.

5.1 Uniqueness of the inverse solution

For simplicity, we only consider the steady-state solution for the discussions on the uniqueness question. It is assumed that the one-dimensional spatial variable x is continuous. Then the flow model as expressed in eqn (1) becomes a second-order ordinary differential equation in each zone

C. Zheng, P. Wang

5.2 Test problem 1

constants, C, and C2, which can be uniquely determined if two head observations are available in each zone (including boundaries). Thus, the intercept of the two curves hi(x) and hi+l(x) uniquely determines the boundary between the two adjacent zones (Fig. 2). If more than two observations are available, a least- squares method can be employed to determine the constants Ct and C,. The uniqueness is still guaranteed for time-dependent solutions.

K,d2hi(x)+w=o ’ dx2 7 i= 1,2,...,p

To test the applicability and efficiency of the heuristic search techniques, we first consider a small problem having 21 nodal points with a regular spacing of 5 m and 4 zones of constant hydraulic conductivity (Fig. 3). The values of the hydraulic conductivity associated with these zones are 2000, 100, 10, and lOOOm/day, respec- tively. The thickness of the confined aquifer is assumed to be 1 m and a constant storage coefficient of 0.02 is assumed for all four zones. The recharge rate w is assumed to be zero. The aquifer is bound by two constant-head boundaries with the initial heads both at 11 m. At t > 0, the head at the right boundary is instantaneously lowered to 10m. The transient head distribution is simulated using a simple finite-difference program with a time step of 1 day. The heads at columns 3, 10, 16 and 20, each of which is located in a unique zone, for the first five time steps are used as the ‘observed’ heads. The boundaries between different hydraulic conductivity zones are then treated as unknown to be determined through the optimization process.

where w is the area1 sink/source term. Its solution hi(x) is obtained by integrating the above equation twice

h,(x)=-&x2+CIx+C2, i= 1,2,...,p I

(8)

Let ri be the boundary nodal point between zone i and zone i + 1. Then we have the following optimization problem:

Minimize c [h’(i, t) - h”(i, t)12 i, f

Subjectto 1<1,<~~<~~<21. (9)

The general solution hi(x) contains two unknown Further investigation reveals that the objective function

1 2 3 4 5 6 78 9 10 11 12 13 14 IS 16 17 18 19 20 21

Node Number

Fig. 3. Configuration of the one-dimensional groundwater flow model.

Parameter structure identi~cation 221

Table 3. Distribution of local minima in the fkst test problem

Location of Value of objective local minima” function

(L&3) 7.2183 (1,13,18) 0*0300 C&3,4) 6.4804 (3,14,17) 0 (3,19,20) O-6219 (4,596) 5.0702 (4,18,19) 04419 (8,159 16) 0.0505

“The first numbmer inside the parenthesis is the nodal point separating zones 1 and 2; the second numkr zones 2 and 3; and the third number zone 3 and 4.

has 8 local minima for this simple problem. These minima are listed in Table 3.

To illustrate the process of the tabu search, we start with a solution at (5,10,15) (Step 0) where the value of the objective function is 0.685 and the tabu list is empty, T = qf~. After the neighborhood search the objective functions are evaluated at the six neighbors as shown in Table 4 (Step 1).

Apparently, the solution (5,10,16), indicated by (*) in Table 4 (Step 1) has the smallest objective function value. Thus it is selected as the new solution and the pair (3,16) is added to the tabu list, T = {(3,16)}. This pair (3,16) implies that the variable ZJ has been assigned to value 16, i.e. Z3 = 16. Meanwhile the neighborhood set is updated and the objective function values are calculated for the new neighbors (Step 2). At this step, the solution (5,10,17) is selected to be the current solution and the tabu list is updated by adding the pair (3,17), T = {(3,16), (3,17)}. And the tabu search proceeds.

Table 4. Objective fundion values at the neighbors of the trial solutions for tab0 search originating at the

poiillt (5,10,15)

Step

0 1

2

. . . 10

Neighbor Objective function

(5,10,15) 0.685

(4,10,15> O-675 (6,10,15> 0.697 (5,9,15) 0.662 (5,lL 15) 0.667 (5,10,14) 0.665 (5,10,16) 0.451 (*)

(4,10,16) 0444 (6,10,16) 0.460 (579916) 0448 (5,11,16) 0.405 (5,10,15) O-684 (5,10,17) 0.184 (*)

(2,13,1@ 0.016 (**) (1,12,18) 0.011 (*) (1,14,H) 0.058 (1,13,1’7) O-032 (***) (1,13, I!$) 0.081

To show the effect of tabu list, we skip the procedure to Step 10 at the beginning of which the solution is (1,13,18) and the neighborhood set and the associated objective function values are shown in Step 10 of Table 4. The length of the tabu list is selected to be 7 as suggested by Glover.’ At this step, the tabu list is full with T = ((2, ll), (2,12), (~4)~ (~3)~ (~2)~ (1, l), (2,13)1.

The smallest objective value occurs at the neighbor (1,12,18), but the change of Zz from the current value 13 to 12 is not allowed because the move (2,13) is in the tabu list. The second smallest objective value occurs at (2, 13, 18) which is also prohibited because of the move (1,2) in the tabu list. The new solution (1,13,17) is selected and the oldest element in the tabu list (2,ll) is replaced by (3,17). The process repeats.

The computational results using SA and TS are presented in Table 5 and compared with the grid search and descent search mentioned before. In descent search, the procedure starts at a random point and only moves to the point where the value of the objective function is smaller. In grid search, the search initially starts at the point (1,2,3). And then it evaluates at (1,2,4), (1,2,4), . . . , (1,2,21), . . . Except for descent search, the other three methods can find the optimal solution. The descent search is trapped in a local minimum point. The SA control parameters used are: the initial temperature TO = 1, the temperature reduction factor X = 0.9, and the number of iterations under each constant temperature equal to 100. To implement tabu search, the objective function is penalized if the solution is in tabu list by adding a large number. The aspiration level function is not employed in this problem since cycling does not occur. The variations in the objective function associated with SA and TS are shown in Fig. 4.

5.3 Test problem 2

The second example considered is identical to the first example except that the number of nodal points has increased from 21 to 100 and the number of constant hydraulic conductivity zones from 4 to 10. Since the nodal spacing remains 5 m, the distance between the two constant-head boundaries in the second example has increased nearly fivefold over that in the first example.

Table 5. Search results for the first test problem

Methods Optimality Number of model evaluations

Grid search yes 482 Descent search no N/A Simulated annealing yes 905 Tabu search yes 276”

‘For tabu search, the number of model evaluations is computed indirectly from the number of iterations times the maximum of six model evaluations per iteration in this test problem of three parameters and 2 neighbors. Thus, the actual number of model evaluations may be smaller than the number shown above.

222 C. Zheng, P. Wang

3.0

i 12 2.0

f 0

1 .o

0.0

0 200 400 600 800 1000

Number of Iterationa

0.0 0 10 20 30 40 50

Number 01 Iterations

0.8

0.6

2 s *s 0

; 0.4

$ 0 0.2

0 200 400 600 800 Number of iter8tiona (x100)

0.15

0.00 0 50 100 150

Number of itw’dons

Fig. 4. Variations in the objective function for the first test problem. In SA (top), each iteration involves one model evaluation; in TS (bottom), each iteration involves up to six

model evaluations.

Fig. 5. Variation in the objective function for the second test problem. In SA (top), each iteration involves one model evaluation; in TS (bottom), each iteration involves up to 18

model evaluations.

Table 6. Prescribed zonation for the second test problem

Zone 1 2 3 4 5 6 7 8 9 10

K W-W 2000 100 10 1000 10 2000 100 10 1000 10

Table 7. Search results for the second test problem

Methods Optimality Number of model The value of hydraulic conductivity associated with each zone is listed in Table 6. The boundaries between

Grid search no

evaluations

N/A

neighboring zones are at columns 10,20,. . , and 90. The heads at columns 5,15,. . . , and 95 for the first five

Descent search Simulated annealing Tabu search

no yes yes

N/A 74,334

26W

‘For tabu search, the number of model evaluations is computed indirectly from the number of iterations times the maximum of 18 model evaluations per iteration in this test problem of nine parameters and two neighbors. Thus, the actual number of model evaluations may be smaller than the number shown above.

time steps are taken as the ‘observed’ heads. For the second test problem, only SA and TS are able

to find the optimal solution as shown in Table 7. The grid search requires too many model evaluations, while descent search again is trapped in a local minimum point. The number of model evaluations required for tabu search is much smaller than that required for simulated annealing. The variations in the objective

Parameter structure ident$cation 223

function associated with simulated annealing and tabu search are shown in Fig. 5.

6 SUMMARY

We have proposed a combinatorial optimization model that identifies the best parameter structure for ground- water models. Traditio:nal, gradient based methods are not applicable for this type of inverse problem because of the difficulty in evaluating the function derivatives and the presence of many local minimum points in the objective function. Modern heuristic search techniques, such as simulated annealing, genetic algorithms and tabu search, on the other hand, are well suited for solving the proposed combinatorial optimization model. In this study, we have applied simulated annealing and tabu search to identify parameter structure in a one- dimensional groundwater flow model. Although we have focused on the identification of parameter struc- ture by treating the parameter values as known, our approach can be extended readily to include both parameter structure and parameter values.

For the one-dimensional problems examined in this study, it can be concluded that the inverse solution is unique if each zone has at least two observations (including the endpoints). The uniqueness for two- or three-dimensional problems is still an open question and needs to be further investigated. The preliminary results indicate that tabu search is much more effective than simulated annealing for the two one-dimensional examples.

ACKNOWLEDGEMENTS

The authors are grateful to Jiu J. Jiao of University of Alabama for his assistance in preparing the manuscript. Thanks are also due to Mary C. Hill of US Geological Survey for stimulating discussions on numerous occa- sions. This work is supported by research grants from the University of Alabalma and the US Environmental Protection Agency.

REFERENCES

Carrera, J. & Neum,an, S. P., Estimation of aquifer parameters under transient and steady-state conditions, 1. Maximum likelihood method incorporating prior information. Water Resow. Res., 22(2) (1986) 199-210. Clifton, P. M. & Neuman, S. P., Effects of kriging and inverse modeling on conditional simulation of the Avra Valley aquifer in southern Arizona. Water Resow. Res., 18(4) (1982) 1215-34. Coats, K. H., Dempsey, J. R. & Henderson, J. H., A new technique for determining reservoir description from field performance data. Sot. Pet. Eng. J., 10(l) (1970) 66-74.

4. Cooley, R. L., A method of estimating parameters and assessing reliability for models of steady state groundwater flow, 2. Application of statistical analysis. Water Resow. Res., 15(3) (1979) 603-17.

5. Delhomme, J. P., Spatial variability and uncertainty in groundwater flow parameters: a geostatistical approach. Water Resour. Res., lS(2) (1979) 269-80.

6. Dougherty, D. E. & Marryott, R. A., Optimal ground- water management, 1. Simulated annealing. Water Resour. Res., 27(10) (1991) 2493-508.

7. Emsellem, Y. & de Marsily, G., An automatic solution for the inverse problem. Water Resour. Res., 7(5) (1971) 1264-83.

8. Glover, F., Future paths for integer programming and links to artificial intelligence Camp. Op. Res., 5 (1986) 533-49.

9. Glover, F., Tabu Search. Annals of Operation Research, Vol. 41, 1993.

10. Glover, F. 8~ Laguna, M., Tabu search. In Modern Heuristic Techniques for Combinatorial Problems, ed. C. R. Reeves. Blackwell Scientific Publications, Oxford, UK, pp. 70-150.

11. Goldberg, D. E., Genetic Algorithms. Addison-Wesley, Reading, MA, 1989.

12. Holland, J. H., Adaptation in Natural and Arttficial Systems. MIT Press, Cambridge, MA, 1992.

13. Hill, M. C., A Computer Program (MODFLOWP) for Estimating Parameters of a Transient, Three-Dimensional, Ground-Water Flow Model Using Nonlinear Regression. U.S. Geological Survey Open-File Report 91-484, 1992.

14. Jiao, J. J., Sensitivity features of aquifer parameters and their implications on parameter estimation. PhD Thesis, University of Birmingham, UK, 1993.

15. Khan, I. A., Inverse problem in ground water: model development. Ground Water, 24(l) (1986) 32-48.

16. Keidser, A. & Rosbjerg, D., A comparison of four inverse approaches to groundwater flow and transport parameter identification. Water Resour. Res., 27(9) (1991) 2219-32.

17. Kirkpatrick, S., Gelatt, C. & Vecchi, M., Optimization by simulated annealing. Science, 220 (1983) 67 l-80.

18. Lu, A. H., Schmittroth, F. & Yeh, W. W.-G., Sequential estimation of aquifer parameters. Water Resour. Res., 24(5) (1988) 670-82.

19. Luenberger, D., Linear and Nonlinear Programming, 2nd Edn. Addison Wesley, 1984.

20. Mauldon, A., Karasaki, K., Martel, S., Long, J., Landsfeld, M., Mensch, A. & Vomuoris, S., An inverse technique for developing models for fluid flow in fracture system using simulated annealing. Water Resour. Res., 29(l) (1993) 3775-89.

21. Marryott, R. A., Dougherty, D. E. & Stollar, R. L., Optimal groundwater management, 2. Application of simulated annealing to a field-scale contamination site. Water Resour. Res., 29(4) (1993) 847-60.

22. McKinney, D. & Lin, M.-D., Genetic algorithm solution of groundwater management models. Water Resour. Res., 30(6) (1994) 1897-906.

23. Metropolis, N. A., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. & Teller, E., Equation of state calculations by fast computing machines. J. Chem. Phys., 21(6) (1953) 1087-92.

24. Nobel, B. & Daniel, J., Applied Linear Algebra. Prentice- Hall, Eaglewood Cliffs, NJ, 1977.

25. Ritzel, B. & Eheart, J., Using genetic algorithms to solve a multiple objective groundwater pollution containment problem. Water Resour. Res., 30(5) (1994) 1589-603.

26. Sagar, B., Yakowitz, S. & Duckstein, L., A direct method

C. Zheng, P. Wang

for the identification of the parameters of dynamic non- homogeneous aquifers. Water Resow. Res., ll(4) (1975) 563-70.

27. Sen, M., Gupta, A., Stoffa, P., Lake, L. & Pope, G., Stochastic reservoir modeling using simulated anneal- ing and genetic algorithm. Sot. Pet. Engng J., SPE 24754.

28. Shah, P. C., Gavalas, G. R. & Seinfeld, J. H., Error analysis in history matching: the optimum level of para- meterization. Sot. Pet. Engng J., M(3) (1978) 219-28.

29. Sun, N.-Z. & Yeh, W. W.-G., Identification of parameter structure in groundwater inverse problem. Water Resour. Res., 21(6) (1985) 869-83.

30. Sun, N. Z. & Yeh, W. W.-G., A stochastic inverse solution for transient groundwater flow: parameter identification and reliability analysis. Water Resow. Res., 28(12) (1992) 3269-80.

31. Weber, D., Easley, D. & Englund, E., Probability of plume interception using conditional simulation of hydraulic

head and inverse modeling. Math. Geol., 23(2) (1991) 219-39.

32. Yeh, W. W.-G., Review of parameter identification procedures in groundwater hydrology: the inverse problem. Water Resow. Res., 22(2) (1986) 95-108.

33. Yeh, W. W.-G. & Yoon, Y. S., A systematic optimization procedure for the identification of inhomogeneous aquifer parameters. In Advances in Ground Water Hydrology, ed. Z. A. Saleem. American Water Resources Association, New York, pp. 72-82.

34. Yeh, W. W.-G., Yoon, Y. S. 8c Lee, K. S., Aquifer parameter identification with kriging and optimum parameterization. Water Resour. Res., 19(l) (1983) 225-33.

35. Yeh, W. W.-G. & Yoon, Y. S., Aquifer parameter identi- fication with optimum dimension in parameterization. Water Resour. Res., 17(3) (1981) 664-72.

36. Zheng, C. & Bennett, G. D., Applied Contaminant Transport Modeling: Theory and Practice. Van Nostrand Reinhold, New York, 1995, 440 pp.