This article was downloaded by: ["Queen's University Libraries, Kingston"]On: 31 August 2013, At: 02:15Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Engineering OptimizationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/geno20

A surrogate-based particle swarmoptimization algorithm for solvingoptimization problems with expensiveblack box functionsYuanfu Tang a , Jianqiao Chen a & Junhong Wei aa Hubei Key Laboratory for Engineering Structural Analysis andSafety Assessment, Department of Mechanics, Huazhong Universityof Science and Technology, Wuhan, ChinaPublished online: 17 Jul 2012.

To cite this article: Yuanfu Tang , Jianqiao Chen & Junhong Wei (2013) A surrogate-based particleswarm optimization algorithm for solving optimization problems with expensive black box functions,Engineering Optimization, 45:5, 557-576, DOI: 10.1080/0305215X.2012.690759

To link to this article: http://dx.doi.org/10.1080/0305215X.2012.690759

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Engineering Optimization, 2013Vol. 45, No. 5, 557–576, http://dx.doi.org/10.1080/0305215X.2012.690759

A surrogate-based particle swarm optimization algorithm forsolving optimization problems with expensive black box

functions

Yuanfu Tang, Jianqiao Chen* and Junhong Wei

Hubei Key Laboratory for Engineering Structural Analysis and Safety Assessment, Department ofMechanics, Huazhong University of Science and Technology, Wuhan, China

(Received 6 November 2011; final version received 9 April 2012)

In engineering applications, computer experiments such as finite element analysis and computationalfluid dynamics are often used to model and analyse structural behaviours. In this article, a surrogate-basedparticle swarm optimization algorithm is proposed for solving optimization problems with expensive blackbox functions. An approximate optimization problem in which the black box functions are replaced by thehybrid surrogate models is efficiently solved to search and adjust the global optimum position during theiterative process. Since the presented method combines the merits of traditional optimization algorithmsand particle swarm optimization, only a small number of particles is needed to achieve the optimal positionafter several iterations. Therefore, the method shows great advantages in solving engineering optimizationproblems with expensive black box functions. Several examples are presented to demonstrate the feasibilityand effectiveness of the proposed method.

Keywords: hybrid surrogate models; surrogate-based particle swarm optimization; structural optimizationdesign; black box functions

1. Introduction

For several decades, computer experiments such as finite element analysis and computationalfluid dynamics have attracted intense attention as computers have developed. To some extent, theprocess of performing computer experiments is like a black box, which can be viewed solely interms of its input, output and transfer characteristics without any prior knowledge of its internalworkings. For complex systems, one simulation involving multiple disciplines may take severaldays. Therefore, solving an optimization design based on such computer experiments is a chal-lenging task. Not only must the overall optimization time be acceptable, but also the design mustbe applicable to various engineering applications, for example, situations with implicit or dis-continuous functions. The key point is to develop efficient optimization algorithms with flexibleapplicability, which can to obtain the global optimum of a complex engineering problem.

There exist several traditional optimization algorithms such as sequential quadratic program-ming, the trust region method, the method of feasible direction, the gradient projection method andpenalty methods for constrained optimization problems. Most of them require a priori knowledge

*Corresponding author. Email: mech-chen@263.net

© 2013 Taylor & Francis

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

558 Y. Tang et al.

of the optimization functions, e.g. explicit expression and gradient. When the functions involvedare black boxes, few of them are applicable. In addition, the majority of these traditional optimiza-tion algorithms belong to gradient-based, that is, local optimization methods (Yuan and Sun 1997).For problems with multiple local optimum solutions, setting different initial points may lead todifferent local optima. Recently, several intelligent evolution algorithms, such as genetic algo-rithms (GAs), simulated annealing, tabu search and particle swarm optimization (PSO), have beendeveloped to successfully solve problems with black box functions and multiple local optima.Since these population-based methods usually require a large number of function evaluations(Thomas 2009), it is not efficient to solve problems with expensive black box functions by usingthese evolution algorithms directly.

Because of the intensive computational cost involved in complex system analyses, surrogatetechniques have gained a lot of attention in recent years (Box and Draper 2007, Mullur andMessac 2005). Several researchers have focused on developing better sampling strategies, surro-gate models and surrogate-based optimization algorithms (Wang et al. 2004, Kazemi et al. 2011,Wang et al. 2001, Queipo et al. 2005). Wang and Shan (2007) have provided a detailed reviewin this area. Surrogate models are constructed based on the samples in a design space and thecomputational cost can be reduced considerably for solving optimization problems in term of thesurrogate models. A combination of response surface models and artificial neural networks hasbeen proposed by Varadarajan et al. (2000) to replace intensive analysis programs and thus to savecomputational time and cost. Zerpa et al. (2005) have used weighted average surrogate models toreduce the prediction variance with respect to that of the individual surrogates. Wang et al. (2004)have developed a mode-pursuing sampling method that systematically generates more samplepoints in the neighbourhood of a function mode while statistically covering the entire searchspace. Quadratic regression is performed to detect the region containing the global optimum.The sampling and detection process iterates until the global optimum is reached. Kazemi et al.(2011) have studied a mode-pursuing sampling method for problems with expensive objectiveand constraint functions. Wang et al. (2001) have used strategies to gradually reduce the searchspace. In the reduced region, an existing global optimization method is applied on the approxi-mation model to locate the optimum. Yannou and Harmel (2006) and Moghaddam et al. (2006)have used constraint programming to reduce the search space and employed surrogate techniquesto generate approximated mathematical models. However, constraint programming needs care-ful tuning, which brings extra difficulties for the designers. Jones et al. (1998) have taken thestochastic process model to develop an efficient global optimization algorithm. The key to thealgorithm lies in balancing the need to exploit the approximating surface with the need to improvethe approximation. Trust region-based approaches which combine classic trust region approacheswith surrogate models have been proposed by Alexandrov et al. (1998) and Gano et al. (2006).Wang and Simpson (2004) have proposed a fuzzy clustering-based approach, in which the fuzzyc-means clustering method is used to cluster the inexpensive point from the surrogate models,and an attractive cluster and its corresponding reduced design space are chosen.

Similar to other existing evolution algorithms, PSO (Kennedy and Eberhart 2001) is apopulation-based optimization method. It is known from the literature that it can eventually locatethe desired solution, but the convergence rate is typically slower than those of local direct searchtechniques [e.g. Hooke and Jeeves’ method (1961) and Nelder Mead simplex search method],owing to the fact that the latter ones simply depend on local information to determine the mostpromising search direction.

The present work is motivated by the shortcomings of the traditional optimization methodsand the evolution algorithms in solving problems with expensive black box functions.A surrogate-based particle swarm optimization (SBPSO) algorithm is proposed. A hybrid surro-gate model (HSM) is first established to approximate the expensive black box, and then integratedinto PSO to reduce the computational cost. The proposed method can search the global optimum

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 559

with significant reduction of the computational cost compared with the basic PSO. Severalexamples are presented to demonstrate the feasibility and efficiency of the proposed method.

2. Hybrid surrogate models

In the classic response surface method (RSM), a second order polynomial approximation of aresponse f (x) as expressed below is mostly utilized:

f̃rsm(x) = β0 +N∑

i=1

βixi +N∑

i=1

N∑j≤i

βijxjxj (1)

where the unknown coefficients β are usually determined by least squares regression analysis.The residual error between the existing data at point xj and the estimate defined in Equation (1)

is given as

e(xj) = f (xj) − f̃rsm(xj)

In spite of the continuality and smoothness, the polynomial approximation has the nature of alocal approximation; that is, it is generally inadequate over the entire domain. Thus, it is difficultfor the second order polynomial to approximate the true nonlinear response accurately, and largeerrors may arise. On the other hand, there exists a best sample size if RSM is used to approximatethe true response. Once the best sample size is achieved, the prediction accuracy of the responsesurface cannot be improved as the number of samples increases. This is because residual error isusually ignored after obtaining the coefficients (Box and Draper 2007). In order to make full useof the residual error information, an HSM based on RSM is proposed as follows. In the model,the radial basis function (RBF) is first established to interpolate these errors at the sample points.Then the derived RBF approximation is added to the response surface approximation. The HSMhas the properties of global approximation and smoothness. Thus, when HSMs are used to replacethe expensive black box functions in optimization problems, the globally optimal regions can becovered.

Based on the above strategy, the response approximation using HSMs can be formulated as

f̃ (x) = f̃rsm(x) + f̃rbf(x) (2)

f̃rbf(x) =Ns∑

i=1

wiϕi(‖x − xi‖) (3)

where f̃ (x) is the approximation to the true response, f̃rsm(x) is the response surface approximationwith quadric polynomials, f̃rbf(x) is the radial basis function approximation to the residual errorand w = [w1, w2, . . . , wNs ]T is the vector of coefficients for the radial basis function.

There are several types of radial function, such as Gaussian, multi-quadrics, reciprocal multi-quadrics, thin plate spline and logistic functions. In this article, the Gaussian function is adopted:

ϕi(r) = e−rz/c2i (4)

where r = ‖x − xi‖ and cj is the width factor.Finally, the actual response can be stated as

f (x) = f̃ (x) + ε = f̃rsm(x) + f̃rbf(x) + ε (5)

where ε is the independent error and is equal to zero at sample points.

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

560 Y. Tang et al.

Table 1. Test functions.

No. Function fi(x) Design space Ns

1 0.5x31 + x2

2 − x1x2 − 7x1 − 7x2 0 � xj � 10 10

2 x1 sin x2 + x2 sin x1 −2π � xj � 2π 64

3sin

√x2

1+x22√

x21+x2

2

−3π � xj � 3π 64

45∑

j=1

[0.3 + sin

(16

15xj − 1

)− sin2

(16

15xj − 1

)]−π � xj � π 80

Given a set of Ns samples, the values of the true response are equal to those of the approximationat each sample point xk:

f̃rbf(xk) = f (xk) − f̃rsm(xk) (6)

f̃ (xk) = f (xk) k = 1, 2, . . . , Ns (7)

For RSM with a second order polynomial, let β = [β0 β1 β2], where β0 is the constant coeffi-cient, β1 is the vector of coefficients associated with the first order term and β2 is the vector ofcoefficients associated with the second order terms. The response surface approximation can berewritten in a matrix form as

f̃rsm(x) = β0 + xβ1 + xTAx (8)

where A is a symmetric matrix.

2.1. Verification of the accuracy of surrogate models

In this section, the performance of three types of surrogate models: the RSM using a quadratic fit,the typical RBF, and the HSM are compared. In the following, four representative mathematicalfunctions as shown in Table 1 are chosen to show the accuracy of the HSM (Mullur and Messac2005).

For functions 1 and 4, 10 and 80 samples, respectively, are generated randomly in the designspace using Latin hypercube sampling. For functions 2 and 3, 8 × 8 grid samples are chosen. Inthe RBF and HSM, the width factors cj as shown in Equation (4) are fixed at 1. The plots of theactual function and the associated three types of surrogate model (for the two-variable functions)are shown in Figures 1–3.

Figure 1. Function 1: surrogate model surface plots: (a) actual function; (b) response surface method (RSM); (c) radialbasis function (RBF); (d) hybrid surrogate model (HSM).

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 561

Figure 2. Function 2: surrogate model surface plots: (a) actual function; (b) response surface method (RSM); (c) radialbasis function (RBF); (d) hybrid surrogate model (HSM).

Figure 3. Function 3: surrogate model surface plots: (a) actual function; (b) response surface method (RSM); (c) radialbasis function (RBF); (d) hybrid surrogate model (HSM).

To measure the accuracy of these surrogate models, the root mean square error (RMSE) andthe normalized root mean square error (NRMSE) are taken as the indicators:

RMSE ={

1

K

K∑k=1

[ f (xk) − f̃ (xk)]2

}1/2

(9)

NRMSE ={∑K

k=1[ f (xk) − f̃ (xk)]2∑Kk=1[ f (xk)]2

}1/2

(10)

where K denotes the number of additional samples for error measure, and f̃ is the surrogate modelvalue.

Small values of RMSE or NRMSE are desired. A value of 0 indicates a perfect fit, whereas alarge value of NRMSE indicates a poor fit. In the following, 1000 and 3125 samples are randomlychosen for functions 1–3 and function 4, respectively.

Table 2 compares the accuracy of surrogate models for the four test functions. It can be seen thatthe HSM can provide approximations with equivalent accuracy to the RSM surrogate model forlinear and low nonlinear problems, and with equivalent accuracy to the standard RBF surrogatemodel for high nonlinear problems. In addition, for function 1, which is a quadratic polynomial,HSM can provide a better approximation beyond the extended radial basis function (E-RBF)model. For function 2, the approximation accuracy for the HSM model and E-RBF model isapproximately the same. For function 4, HSM yields a worse approximation than E-RBF. Thereason is that HSM could not effectively capture sinusoidal terms over a large domain. In theE-RBF approach, a linear program is first solved, and if the linear program yields an unfeasiblesolution, the pseudo-inverse approach is used. Since the linear optimization problem requires morecomputational cost than HSM, in which one needs only to solve a system of linear equations, thecomputational cost for E-RBF is much greater than that for HSM. Other merits of HSM are thatit is easy to construct and clear on parameter sensitivity. In some cases, however, HSM is lessaccurate than the E-RBF model.

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

562 Y. Tang et al.

Table 2. Accuracy results of surrogate models.

RSM RBF E-RBF HSM

fi RMSE NRMSE RMSE NRMSE RMSE NRMSE RMSE NRMSE

1 1.9688 1.1667 16.5121 9.7848 3.9705 2.32 1.4270 0.84562 3.5339 90.07761 0.2904 7.4010 0.2816 7.63 0.3081 7.85413 0.1756 96.5667 3.0545e−2 16.8007 N/A N/A 3.0544e−2 16.79754 0.9319 19.8485 0.9121 19.4268 0.0180 1.62 0.8943 19.0483

2.2. Choice of width factors

Given a set of sample data, if RBF with the Gaussian function is used to approximate a nonlinearfunction, there exists an optimal vector of width factors [c1, c2, . . . , cNs]T. The component of thevector can either be a constant or vary independently. For example, the widths can be set as follows(Haykin 1999):

cj = dmax

2√

Ns(11)

where dmax is the maximum distance between those sample points. If the samples are uniformlydistributed in the design space, the above width factor would be close to the optimal width factor(Benoudjit et al. 2002). However, most real-life problems show non-uniform sample distributions.In this case, the following width can be adopted (Moody and Darken 1989):

cj = 1

nr

(nr∑

i=1

‖xci − xcj‖2

)1/2

(12)

where xci are the nr nearest neighbours of sample xcj.Another approach which unites both approaches as shown in Equations (11) and (12) was

proposed by Benoudjit et al. (2002). First, the standard deviation of each data cluster cdcj is

computed as shown in Equation (12). Subsequently, a width scaling factor q is determined:

cj = qcdcj , ∀j

For a set of samples, there exists an optimal width scaling factor qopt. A few samples can bechosen to compute the RMSE. For a width factor set Q, the optimal qopt corresponding to thesmallest error can be determined as

RMSE(qopt) ≤ RMSE(q), ∀q ∈ Q

The choice of width factors mainly depends on the number of samples, the sample distributionand the nonlinearity of functions. For the sake of simplicity, a uniform character as below isintroduced to distinguish the two situations intelligently. A smaller U corresponds to a betteruniformity of the samples. If there exist a lot of samples and the samples are uniformly distributedin the input space, i.e. U � ε1, a constant width factor can be chosen. Otherwise, the varied widthfactors or optimal width scaling factor may perform better:

U =maxx∈Cn

min1≤i≤Ns

d(x, xi)

R

R = 1

Ns − 1

Ns∑i=1

d(xi, x̄), x̄ =Ns∑

i=1

xi

where d( ) is the Euclidean distance and Cn is the design space of samples.

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 563

3. Surrogate-based particle swarm optimization

This section describes the proposed SBPSO. First, a brief overview of PSO is given, and then theprocedures of the proposed SBPSO are stated.

3.1. Basic particle swarm optimization

The basic PSO is a heuristic global optimization search technique based on community intelli-gence. Owing to its simple concept, easy implementation and quick convergence, compared withother evolution algorithms, PSO has gained much attention and has been successfully applied indifferent fields (Kennedy and Eberhart 2001, Chen et al. 2008, Tang et al. 2009). It was developedby Kennedy and Eberhart (2001) based on the simulation of birds flocking for food. The birdsfind food by flocking and not as individuals. This observation leads to the assumption that allinformation is shared inside the flock. Similar features exist in human groups, i.e. the behaviourof an individual (agent) is based on the behaviour patterns authorized by the group, such as customand other behaviour patterns depending on the experiences of each individual.

The search procedure of PSO can be described as follows. First, a group of random par-ticles (individuals) is generated. Then each particle adjusts its flying pattern according to itsown flying experience and that of its companions. Each particle represents a potential solutionfor the optimization design problem. Let the ith particle be a point in D-dimensional space,Xi = (xi1, xi2, . . . , xiD)T, and let the velocity (rate of position change, or displacement increment)be V i = (vi1, vi2, . . . , viD)T. The best previous position (the position possessing the best fitnessvalue) of any particle is recorded and denoted ‘pbest’ and the best position of the whole group isdenoted ‘gbest’. Each particle updates its own velocity and position according to the followingtwo formulae:

vk+1i = w × vk

i + c1 × rand1 × (pbesti − xki ) + c2 × rand2 × (gbest − xk

i ) (13)

xk+1i = xk

i + vk+1i (14)

where i denotes the ith particle, k denotes the kth iteration, vki and vk+1

i are the fly velocities ofthe particle in the kth and the next iteration, respectively, xk

i and xk+1i are the current and updated

positions of the particle, c1 and c2 are the acceleration constants (c1 = c2 = 2.05 in this article),rand1 and rand2 are random numbers uniformly distributed in the interval [0, 1], w is the inertiaweight, pbesti is the individual’s best position found so far, and gbest is the global best positionattained by the swarm found so far.

The stopping criterion is that either the iteration achieves the maximum iteration or the minimumerror requirement between the analytic optimal solution and the optimal solution in the currentiteration is satisfied. The maximum iteration is usually determined by user experience or testoperations. In this article, once either the iteration achieves the maximum iteration or the minimumerror requirement is satisfied, the algorithm terminates automatically.

3.2. Surrogate-based particle swarm optimization for unconstrained optimization problems

This section will discuss the infrastructure and rationale of SBPSO in detail. As mentioned earlier,PSO is a population-based optimization method. If basic particle swarm optimization (BPSO) isused directly in solving the optimization problems with expensive black box functions, the compu-tational cost will make the simulation impossible owing to a large number of function evaluations.An SBPSO method to solve such problems is developed. When solving an N-dimensional prob-lem, SBPSO generates Np particles at the beginning [Np ≥ (N + 1) is recommended]. At each

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

564 Y. Tang et al.

Approximate global position

Computer

experiments

Y

Convergence?

Output

Initiate population

Function evaluation

Best position

found by whole

population

Update velocity and position

Build surrogate

Global position

N

Figure 4. Flowchart of the proposed surrogate-based particle swarm optimization (SBPSO).

iteration, each particle updates its velocity and position according to BPSO and then the fitnessof the particles is calculated through function evaluations. These positions and the correspondingfunction values are stored in the sample database. In the meantime, an HSM is constructed toapproximate the true response surface. Since the explicit expression is derived in terms of theHSM, the traditional optimization algorithms, such as sequential quadratic programming or trustregion method, can be used to solve the approximate optimization problem. After the approximateglobal position has been determined, the simulated analysis is implemented at this position. In thekth iteration, there are three types of global position: the global position found by the particles,denoted as gbestk

pso, the approximate global position (gbestkhsm) and the global position found by

SBPSO algorithm (gbestk). The value of gbestk is adjusted and updated by gbestkhsm and gbestkpso.

If the better one of gbestkhsm and gbestkpso is better than gbestk=1, then this better global position

is chosen as gbestk , otherwise, gbestk = gbestk−1. In the next iteration, each particle updates itsvelocity and position based on the updated global position, and an HSM is constructed to approx-imate the true response surface using all the sample data including the particles’ positions and theapproximate global positions. It should be noted that in the BPSO, only gbestk and gbestkpso exist

and the gbestk is identical to gbestkpso.For an N-dimensional optimization problem, the proposed algorithm consists of following

detailed steps.

• Step 1: Generate a population of size Np using Latin hypercube sampling. Evaluate the functionvalue of each particle and find the global position. Store the positions searched by particles andthe corresponding function values in a sample database.

• Step 2: If the number of total samples Ns ≥ (N + 1)(N + 2)/2, construct the surrogate modelsby HSM and obtain the approximate global position, otherwise, go to Step 4.

• Step 3: Evaluate the function value at the approximate global position by performing thesimulation analysis and add this point to the sample database.

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 565

• Step 4: Apply BPSO to update the position and velocity. Update the global position and storethe updated positions and the corresponding function values in the sample database.

• Step 5: If a stopping criterion is satisfied, the algorithm terminates, otherwise, go to Step 2.

It should be noted that the stopping criterion of SBPSO is the same as that of BPSO. Theproposed SBPSO combines the advantages of traditional optimization algorithms and PSO, andcan converge after several iterations with a small number of particles. It is especially suitable forengineering applications with time-consuming implicit functions. The flowchart of the SBPSO isshown in Figure 4.

Figure 5 shows the relation between surrogate models and PSO in the proposed SBPSO. Theapproximate optimal solution obtained by solving the approximate optimization problems in termsof the surrogate models is used to adjust the global position. The PSO provides the rational sampledistribution such that the samples are distributed in the interested area.

The advantages of the proposed algorithms are listed below.

• For optimization problems with expensive black box functions, a global optimal solution isusually found after several iterations with a small number of particles. The computation costrequired by SBPSO is reduced considerably compared with BPSO.

• SBPSO makes full use of all the information about the particles. In BPSO, after obtaining theglobal position and the particle’s best positions, all the information about the particles, such asthe positions and function value, is discarded. However, in SBPSO, all the information aboutthe particles is employed to establish the surrogate models.

• The entire design space, including those that have never been explored by all the parti-cles, can be detected easily, since a surrogate model generates approximate optimum pointsprogressively during the iteration. This helps to improve the convergence to the globalposition.

• Particles are initiated by using Latin hypercube sampling, which maintains the randomnessin the population and guarantees particles from all parts of the design space. Latin hypercubesampling has even projection properties on any single dimension. It is usually more efficientthan other sampling strategies for problems with a large number of design variables becauseit requires fewer sample points to achieve the same prediction accuracy (Zou et al. 2008). Inaddition, an arbitrary number of particles can be sampled.

During the SBPSO optimization procedure, more and more samples are generated towardsthe global optimum. Unfortunately, for interpolation surrogate models, the matrix may be illconditioned and even singular, if the samples become too crowded. The numerical difficulty hasbeen addressed using singular value decomposition as suggested by Kiusalaas (2005).

For problems with expensive functions, each particle has a position at each iteration where anexpensive computer experiment is performed. Thus, the number of particles (Np) and numberof maximum iterations (Nmi) cannot be large. The typical ranges for Np and Nmi are (N + 1) −40 and 10–40, respectively. For most problems, 10 particles and 10 iterations are enough toobtain good results. For some difficult or special problems, such as optimization problems witha discontinuous feasible region, optimization problems with high nonlinear functions and high-dimensional optimization problems, 100–200 particles and 50–100 iterations may be required toobtain precise solutions.

Surrogate models

PSO Adjust the global position

Sampling

Figure 5. Relation between surrogate models and particle swarm optimization (PSO) in the proposed surrogate-basedparticle swarm optimization (SBPSO).

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

566 Y. Tang et al.

4. Surrogate-based particle swarm optimization for structural design optimization

A general structural constrained optimization problem can be formulated as

Min f (x) (15)

s.t. gi(x) ≤ 0, i ∈ E (16)

hj(x) = 0, j ∈ I (17)

where f (x) is the objective function, gi(x) and hj(x) are inequality and quality constraint functions,respectively, and E and I are index sets of inequality and quality constraint functions, respectively.The points satisfying Equations (16) and (17) are said to be feasible.

If BPSO is used to solve the constrained optimization problem given in Equations (15)–(17),it should be first transformed to an unconstrained optimization problem. The penalty functionmethod can be used to do this:

min F(x) = min

⎧⎨⎩f (x) + R

[∑i∈E

[max(0, gi(x))]2

]+ R

∑j∈I

[(hj(x))]2

⎫⎬⎭ (18)

where the second and third terms of the right-hand side are penalty terms and the penalty factor Ris greater than zero. The infinite penalty factor means that the constraints are satisfied rigorously.If the constraint conditions are violated at x, the value of the penalty terms becomes large such thatthe solution is pushed back towards to the feasible region. Otherwise, the constraint conditionsare satisfied and the value of penalty terms is equal to zero.

For problems with expensive black box functions, HSMs can be created. Once the surrogatemodels of objective function and constraint functions are available, the approximate optimizationproblem is formulated as

Min f̃ (x) (19)

s.t. g̃i(x) ≤ 0, i ∈ E (20)

h̃j(x) = 0, j ∈ I (21)

where f̃ (x), g̃i(x) and h̃j(x) are surrogate models of objective function, inequality constraints andequality constraints, respectively.

Traditional constrained optimization algorithms, such as sequential linear programming (SLP),sequential quadratic programming (SQP), the trust region method, method of feasible direction,gradient projection method, augmented Lagrangian method and intelligent evolution algorithms,can be used to solve problems (19)–(21) and an approximate global optimal solution is thenobtained. It should be noted that if the object function or constraint functions are explicit expres-sions in terms of design variables, there is no need to construct surrogate models; that is, theoriginal functions are employed in Equations (19)–(21).

After the approximate optimal point has been obtained by solving problems (19)–(21), theexpensive functions are called at this point. If any of the constraints is violated, the strategy ofconstraint correction (CC) algorithm or correction at constant cost (CCC) is adapted to search thenearest feasible point around the infeasible approximate optimal point (Arora 2004). CCC andCC need the gradients of the objective function and constraint functions. Fortunately, it is easyfor HSM to obtain the gradient.

If the maximum constraint violation is very large at a design point, i.e. MV ≥ ε2, the strategy ofCC can be adopted. In this article, ε2 is taken as 0.01. A quadratic programming (QP) subproblem

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 567

is first constructed to improve the constraint violation. The solution to the subproblem gives adirection with the shortest distance to the constraint boundary (linear approximation) from aninfeasible point. After the direction has been found, a step size can be determined to make surethat the constraint violations are improved. The QP subproblem is defined as

Min 0.5 ∗ dTd (22)

s.t. AT d ≤ b (23)

NT d = e (24)

where the left-hand sides are linear inequality and quality constraints, respectively; the columnsof matrices N and A contain gradients of inequality and quality constraints, respectively, at aninfeasible design point; and vectors b and e, which are determined by calculating the expensiveblack box functions, are the corrections of inequality and quality constraints, respectively, suchthat the constraint violation can be improved.

If the maximum constraint violation is not very large at a design point, i.e. MV � ε2, theCCC can be adopted to obtain a feasible point without any increase in the objective. Except for anadditional constraint of linear objective function, CCC is identical to CC. The following constraintis added to the QP subproblem given in the above equation:

σ · d ≤ 0 (25)

where σ is the gradient of the objective function. The constraint imposes the condition that thedirection d either is orthogonal to the gradient of the objective function or makes an angle between90◦ and 270◦ with it. The above two equations seek a shortest path to the feasible region that eitherreduces the linearized objective function or keeps it unchanged at an infeasible design point.

Each CC or CCC procedure requires expensive functions to be run once. For nonlinear con-strained problems, several procedures of CC or CCC are usually needed to correct the constraintviolation, since the gradient of black box functions determined by surrogate models is approxi-mate. In order to balance the efficiency and accuracy, a prescribed maximum number of proceduresof the CC or CCC strategy in each cycle should be set. If the feasible point derived fromthe CC or CCC strategy is better than the global optimal result, the latter is replaced by theformer.

Because of the randomness in the Latin hypercube sampling and the use of random numbersin Equation (13), random variation in the results may arise. It is a reasonable alternative foreach problem to carry out several trials to reduce the variation. For optimization problems withexpensive functions, if a small number of particles (Np) and number of maximum iteration (Nmi)

are chosen and only one trial runs, a suboptimal solution or local optimal solution may be found.In contrast, the computational cost will be prohibitive when more trials and large Np and Nmi

are performed. Thus, a trade-off between accuracy and efficiency must been made. In practice,one trial with appropriate Np and Nmi can provide satisfied optimal solutions for engineeringoptimization problems.

In the following, it is assumed that the objective functions and the constraint functions inExamples 5.1–5.5 are all expensive black box functions for the purpose of illustrative applicationof SBPSO, i.e. HSM is used to construct surrogate models and SBPSO is applied to obtain theresults. The computational cost and precision of SBPSO are discussed by comparing it with othermethods. Before constructing the surrogate models, all the design variables are scaled to the range[0,1] based on their upper and lower bounds.

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

568 Y. Tang et al.

5. Numerical examples

This section gives several examples to demonstrate the applicability and efficiency of the proposedmethod. The first example includes six well-known non-constrained optimization problems. Thenext two examples are constrained optimization problems with cheap explicit functions. The lasttwo examples need finite element analysis to determine the constraint functions. For constrainedoptimization problems, the strategies of CCC and CC are used to search the nearest feasible pointaround the infeasible approximate optimal point. In order to balance the efficiency and accuracy,the maximum number of procedures of the CC or CCC strategy in each cycle is set to 10.

5.1. Benchmark problems

The proposed SBPSO method was tested on six well-known benchmark optimization problems(Wang et al. 2004, 2005), including the six-hump camel back (SC) function, Branin (BR) function,Rastrigin (RS) function, generalized polynomial function (GF), Goldstein and Price (GP) functionand Hartman function (HN6). Except for HN6, the test benchmark functions are two dimensional.

Figures 6 and 7 show the initial particle positions and all the positions searched by SBPSO forthe SC and BR problems, respectively. The particles fly to the global point and congregate at theglobal point. More and more positions in the neighbourhood of the global point are searched. Ineach iteration, an HSM is constructed based on all the positions searched by particles at that step.In the first few iterations, the approximate optimal position obtained by the surrogate model maybe worse than that obtained by BPSO. As the iteration proceeds, more sample data can be used toconstruct a better surrogate model. The approximate optimal position obtained by the surrogatemodel in the latter iteration may be better than that in the former iteration.

For each problem, 10 trials are carried out to reduce random variation in the numerical resultsusing SBPSO, BPSO, GA, the improved adaptive response surface method (IARSM) (Wang2003) and efficient global optimization (EGO) (Jones et al. 1998). In SBPSO, the populationsize Np and the maximum iteration for two-dimensional benchmark functions are set to 4 and 8,respectively, and for HN6 they are set to 10 and 10, respectively. The total numbers of functionevaluations (nfe) are 4 × 8 + 7 = 39 and 10 × 10 + 8 = 108, respectively. In BPSO and GA,the population size Np and the maximum iteration for two-dimensional benchmark functionsare 30 and 10, respectively, and for HN6 they are 40 and 20, respectively. The results obtainedusing these methods are summarized in Table 3. It can be shown that the SBPSO can convergefairly close to the global optimum with a smaller number of function evaluations. In most cases,

Figure 6. Positions sought by particles for the six-hump camel back (SC) optimization problem: (a) initial particlepositions; (b) all the positions sought by particles.

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 569

Figure 7. Positions sought by particles for Branin optimization problem: (a) initial particle positions; (b) all the positionssought by particles.

SBPSO with HSM outperforms that with RBF on accuracy under the same computational cost.Nevertheless, the number of function evaluations required by SBPSO is significantly smaller thanthe BPSO and GA. When comparing SBPSO with the IARSM, it is obvious that for functionsSC, GF, GP and RS, either SBPSO/HSM or SBPSO/RBF can reach the global optimum withfewer function evaluations. For function HN6, SBPSO has achieved better accuracy with almostthe same function evaluation compared with IARSM.

Table 4 lists the best results of SC and BR benchmark functions during 10 trials using differentmethods including the mode pursuing sampling (MPS) method (Wang et al. 2004). It can be seenthat the same optimal result can be found, but the number of evaluations required by SBPSO isreduced significantly compared with BPSO and GA.

5.2. A design problem for a spring structure

The spring design problem (Kazemi et al. 2011) as shown in Figure 8 has been used as a testbenchmark in the literature. The design variables are the mean coil diameter D (= x1), the wirediameter d (= x2) and the number of active coils N (= x3). The objective is to minimize the weightof the spring subjected to constraints on shear stress, surge frequency and minimum deflection.The optimization problem can be formulated as:

Min f (x1, x2, x3) = (x3 + 2)x2x21

s.t. g1(x) = 1.0 − x32x3

71875x41

≤ 0

g2(x) = x2(4x2 − x1)

12566x31(x2 − x1)

+ 2.46

12566x21

− 1.0 ≤ 0

g3(x) = 1.0 − 140.54x1

x32x3

≤ 0

g4(x) = x1 + x2

1.5− 1 ≤ 0

Table 5 compares the SBPSO results with solutions reported by other researchers. In SBPSO,the population size Np and the maximum iteration are 10 and 40, respectively. The strategiesof CCC and CC are used to search the nearest feasible point around the infeasible approximateoptimal point in each cycle of SBPSO. Among 558 function evaluations, the total number of

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

570Y.Tang

etal.

Table 3. Summary of results on benchmark problems.

Optimum found Optimum found

f (x) Method Best Mean Median nfe f (x) Method Best Mean Median nfe

SC SBPSO/HSM −1.0316 −1.0314 −1.0316 39 GF SBPSO/HSM 5.2421e−3 6.9376e−2 7.0919e−2 39SBPSO/RBF −1.0316 −1.0276 −1.0305 39 SBPSO/RBF 1.4989e−3 0.1887 0.1143 39BPSO −1.0316 −1.0311 −1.0311 300 BPSO 4.5301e−4 5.34195e−3 3.6037e−3 300GA −1.0303 −1.0273 −1.028 330 GA 3.0760e−3 6.5385e−2 5.1058e−2 330IARSM −1.0290 N/A N/A 44 IARSM 0.0082 N/A N/A 46

BR SBPSO/HSM 0.3980 0.4059 0.3989 39 GP SBPSO/HSM 3.0377 3.7543 3.5917 39SBPSO/RBF 0.3979 0.4617 0.4011 39 SBPSO/RBF 3.0458 3.6820 3.2744 39BPSO 0.3980 0.4043 0.4001 300 BPSO 3.0079 3.0448 3.0157 300GA 0.3988 0.4219 0.4179 330 GA 3.0480 3.8926 3.5819 330IARSM 0.3980 N/A N/A 36 IARSM 3.0000 N/A N/A 77EGO 0.3980 N/A N/A 25 EGO 30000 N/A N/A 32

RS SBPSO/HSM −1.9945 −1.8307 −1.8494 39 HN6 SBPSO/HSM −3.1924 −2.9064 −2.9369 108SBPSO/RBF −1.9945 −1.8226 −1.8268 39 SBPSO/RBF −3.0214 −2.8852 −2.8539 108BPSO −1.9965 −1.9320 −1.9350 300 BPSO −3.3190 −3.1716 −2.9682 800GA −1.9770 −1.8403 −1.8458 330 GA −3.2695 −2.8966 −2.9117 840IARSM −1.8540 N/A N/A 60 IARSM −2.4560 N/A N/A 105

EGO −3.3200 N/A N/A 121

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 571

Table 4. Optimal results for the six-hump camel back (SC) and Branin problems.

x1 x2 f nfe

SC SBPSO −0.0899 0.7126 −1.0316 39BPSO −0.0903 0.7123 −1.0316 200GA −0.8915 0.6996 −1.0303 330MPS −0.0900 0.7130 −1.0316 48

Branin SBPSO 3.1414 2.2747 0.3979 39BPSO 3.1363 2.2707 0.3981 300GA 3.1514 2.2468 0.3988 330

function evaluations required in the procedures of CC or CCC is 108. In BPSO, the populationsize and the maximum iteration are 60 and 100, respectively. It is obvious that for obtaining theoptimal solution, SBPSO algorithm is most efficient in computational cost. The other comparedalgorithms include CCC (Arora 2004), GA (Coello and Mezura 2002), MPS (Wang et al. 2004)and CiMPS (Kazemi et al. 2011).

5.3. Design for a welded beam

A welded beam as shown in Figure 9 is designed for minimizing fabricating cost subject toconstraints on shear stress (τ ), bending stress (σ ), buckling load (PC), end deflection (δ) and sideconstraints (Mahdavi et al. 2007). There are four design variables: h (= x1), l (= x2), t (= x3)

and b (= x4). The problem can be stated as follows:

Min f (x) = 1.10471x21x2 + 0.04811x3x4(x2 + 14)

s.t. g1(x) = τ(x)/τmax − 1 ≤ 0

g2(x) = σ(x)/σmax − 1 ≤ 0

g3(x) = x1/x4 − 1 ≤ 0

g4(x) = δ(x)/δmax − 1 ≤ 0

Figure 8. A spring structure.

Table 5. Optimal results for the spring design.

Method x1 x2 x3 f nfe

SBPSO 0.0517 0.3568 11.3027 1.2689e−2 558BPSO 0.0573 0.5045 6.2386 1.3627e−2 6000CCC 0.0534 0.3992 9.1854 1.2681e−2 N/AGA 0.0520 0.3640 10.8905 1.2681e−2 80000MPS 0.0516 0.3536 11.4722 1.2665e−2 21027CiMPS 0.0515 0.3532 11.4969 1.2664e−2 1932

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

572 Y. Tang et al.

Figure 9. A welded beam structure.

g5(x) = P/PC(x) − 1 ≤ 0

0.125 ≤ x1 ≤ 0, 0.1 ≤ x2, x3 ≤ 10.01 ≤ x4 ≤ 5

where

τ(x) =√

(τ ′)2 + 2τ ′τ ′′ x2

2R+ (τ ′′)2

τ ′ = P√2x1x2

, τ ′′ = MR

J, M = P

(L + x2

2

)

R =√

x22

4+

(x1 + x3

2

)2

J = 2

{0.707x1x2

[x2

2

4+

(x1 + x3

2

)2]}

σ(x) = 6PL

x4x23

, δ(x) = 6PL3

x4x33

Pc(x) =4.013E

√x2

3x64/36

L2

(1 − x3

2L

√E

4G

)

P = 60001b, L = 14 in, δmax = 0.125 in, E = 30 × 106 psi, G = 12 × 106 psi, τmax = 13600 psi,σmax = 30000 psi

The optimal results with different methods are listed in Table 6. In SBPSO, 10 particles and40 maximum iterations are chosen. In BPSO, the number of particles and the maximum iterationare both set to 100. It is obvious that the SBPSO results are the best among these methods. Thecomputational cost is reduced significantly compared with BPSO. Radgsdell and Phillips (1976)compared the optimal results of different optimization methods, but the numbers of evaluationsare not available. In SBPSO, the total number of expensive function evaluations required in theprocedures of CC or CCC is 132 among 582 function evaluations.

5.4. Design of a two-member frame

Consider the design of a two-member frame subjected to out-of-plane loads as shown in Figure 10(Arora 2004). The design task is to determine the width w (in.), height h (in.) and thickness t (in.)of the rectangular section that minimizes the volume of the frame such that the frame works

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 573

Table 6. Optimal results for the welded beam example.

Optimal design variable (x)

Method x1 x2 x3 x4 f nfe

SBPSO 0.2294 6.3054 8.8475 0.2294 2.3497 582BPSO 0.2208 7.1898 9.5947 0.2251 2.5895 10,000APPROX 0.2444 6.2189 8.2915 0.2444 2.3815 N/ASIMPLEX 0.2792 5.6256 7.7512 0.2796 2.5307 N/ADeb (1991) 0.2489 6.1730 8.1759 0.2533 2.4328 N/ARANDOM 0.4575 4.7313 5.0853 0.6600 4.1185 N/A

Figure 10. A two-member frame.

subjected to von Mises stress and size limitation:

minw,h,t

f (d) = 2L(2wt + 2ht − 4t2)

s.t.σ 2 + 3τ 2

σ 2a

− 1 ≤ 0

2.5 in. � w, h � 10 in.

0.1 in. � t � 1 in.

The constraint functions are implicit expressions with respect to design variables. The stressesat points 1 and 2 are obtained by simple finite element analysis. In SBPSO, 10 particles and 30maximum iterations are chosen. In BPSO and GA, 50 particles and 100 maximum iterations arechosen. The optimal results are shown in Table 7. It can be seen that the proposed method canobtain the optimal solution with same degree of precision under much less computational effortcompared with BPSO or GA. In SBPSO, the total number of expensive function evaluationsrequired in the procedures of CC or CCC is 66 among 406 function evaluations.

5.5. Design of a Kiewette-6 spherical reticulated shell

The Kiewette-6 spherical reticulated shell as shown in Figure 11 is commonly used in engineeringapplications. The shell has a span of 40 m and a height of 8 m. The number of elements and nodes

Table 7. Optimal results for the two-member frame example.

Optimal design variable (x)

Method w h t f nfe

SBPSO 7.7940 10.0000 0.1000 7.0376e2 406BPSO 7.9229 9.8665 0.1013 7.1257e2 5000GA 7.6767 10.0000 0.1013 7.0813e2 7885

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

574 Y. Tang et al.

Figure 11. Structure of the Kiewette-6 spherical reticulated shell.

is 240 and 91, respectively. The distributed mass is 1.5 kN/m2. The shell is clamped at the supportand the element joints of shell are rigid. The bar element of shell is made of steel pipe with elasticmodulus 211 GPa, density 7850 kg/m2 and Poisson ratio 0.3. The safety factor is 3. The task is tominimize the total weight of the shell such that it satisfies constraints on strength and slendernessratio.Strength constraints:

σi = Ni

Ai≤ σu

where Ni and Ai are axial stress and cross-sectional area of the ith bar, and σu is ultimate stress.Stiffness constraints:

λi ≤ λt

where λi is the slenderness ratio of the ith bar, and λt is the maximum allowable slendernessratio.

Table 8. Optimal results for the Kiewette-6 spherical reticulated shell example.

SBPSO BPSO GA

(D1, t1) (0.0860,0.0030) (0.1135, 8.1114e−3) (0.0874, 4.0156e−3)

(D2, t2) (0.1057,0.0030) (0.1566, 5.2628e−3) (0.1073, 4.5450e−3)

(D3, t3) (0.1110,0.0030) (0.1291, 4.7986e−3) (0.1405, 3.4475e−3)

(D4, t4) (0.1127,0.0030) (0.1202, 3.9530e−3) (0.1390, 3.1697e−3)

(D5, t5) (0.1128,0.0030) (0.1212, 3.7744e−3) (0.1536, 3.0596e−3)

(D6, t6) (0.0988,0.0030) (0.2066, 4.9480e−3) (0.1803, 3.3557e−3)

(D7, t7) (0.0877,0.0030) (0.1557, 3.8508e−3) (0.0949, 4.9552e−3)

(D8, t8) (0.0868,0.0030) (0.1490, 3.6526e−3) (0.1162, 3.0332e−3)

(D9, t9) (0.0846,0.0030) (0.0959, 3.1311e−3) (0.0846, 3.0000e−3)

(D10, t10) (0.0818,0.0030) (0.0887, 4.9678e−3) (0.0818, 3.0000e−3)

f 8.3871e3 1.4482e4 1.1406e4g1 0 −4.7928e − 2 0nfa 490 10,050 11,232

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

Engineering Optimization 575

The mathematical formulation can be stated as

Min f =n∑

i=1

ρAiLi

s.t. g1 = max

(σi

σu

)− 1 ≤ 0

gj = λj

λt− 1 ≤ 0

The general commercial softwareANSYS is utilized to perform stress analysis. The bar elementof the shell is modelled by element pipe20. In calculation, MASS21 ELEMENT is used to modelthe distributed mass, which is equivalently transformed into nodal loads. The results obtainedusing the proposed method are shown in Table 8. The optimal results obtained by GA and BPSOare also listed in Table8. In SBPSO, 21 particles and 20 maximum iterations are chosen. In BPSOand GA, the numbers of particles and maximum iterations are set to 50 and 200, respectively.In engineering applications, performing a finite element analysis is a time-consuming process.Thus, the efficiency of three methods is compared by the number of finite element analyses (nfa).It can be seen that the proposed SBPSO outperforms GA and BPSO in efficiency and the solutionquality. In SBPSO, the total number of finite analyses required in the procedures of CC or CCCis 40 among 490 finite analyses.

6. Conclusions

Computer experimental techniques such as finite element analysis and computational fluid dynam-ics are used to model and analyse structure behaviours for complex structures or systems inengineering applications. The process is time consuming and may take days to run. If an opti-mization design is performed based on such techniques, the computational cost will be impractical.Therefore, it is important to seek a reasonable optimization algorithm, by which a reliable andaccurate optimal result can be obtained with practically allowable computation cost. In this article,an SBPSO algorithm was developed, which combines the advantages of the surrogate models andthe traditional optimization algorithm. First, an HSM based on the polynomial response surfacemethod and radial basis function is presented. In each cycle of SBPSO, HSMs for expensive blackbox functions are constructed using all the positions found by particles of BPSO. The approximateglobal position is obtained by solving an approximate optimization problem in which the expen-sive black box functions are replaced by the HSMs. The approximate position is compared withthe global position found by the BPSO. Since the approximate optimization problems are used todetect the region and adjust the global position, the computational cost is reduced considerably.Several examples are given to demonstrate the feasibility and efficiency of the proposed SBPSO.The same accurate optimal results can be obtained, while much less computational cost is requiredcompared with the other algorithms. Therefore, the proposed method provides an efficient toolfor the optimization of complex systems.

Future work will focus on improving the efficiency of the method and developing the algorithmswith a mixture of continuous and discrete variables.

Acknowledgements

This work is supported by the Natural Science Foundation of China (10772070), National Basic Research Program of China(2011CB013800), PhD Programs Foundation of Ministry of Education of China (20070487064) and the FundamentalResearch Funds for the Central Universities HUST (2010MS042).

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013

576 Y. Tang et al.

References

Alexandrov, N.M., et al., 1998. A trust-region framework for managing the use of approximation models in optimization.Structural and Multidisciplinary Optimization, 15 (1),16–23.

Arora, J.S., 2004. Introduction to optimum design. San Diego: Elsevier Academic Press.Benoudjit, N., Archambeau, C., and Lendasse, A., 2002. Width optimization of the Gaussian kernels in radial basis

function networks. In: ESANN’2002 proceedings – European symposium on artificial neural networks, 24–26 April2002 Bruges, Belgium, Brussels: d-side group, 425–432.

Box, G. and Draper, N., 2007. Response surfaces, mixtures, and ridge analyses. Hoboken, NJ: John Wiley & Sons.Chen, J., Ge, R., and Wei, J., 2008. Probabilistic optimal design of laminates using improved particle swarm optimization.

Engineering Optimization, 40 (8), 695–708.Coello, C. and Mezura, E., 2002. Constraint-handling in genetic algorithms through the use of dominance-based

tournament selection. Advanced Engineering Informatics, 16 (3), 193–203.Deb, K., 1991. Optimal design of a welded beam via genetic algorithms. AIAA Journal, 29 (11), 2013–2015.Gano, S., et al., 2006. Update strategies for kriging models used in variable fidelity optimization. Structural and

Multidisciplinary Optimization, 32 (4), 287–298.Haykin, S., 1999. Neural networks: a comprehensive foundation. 2nd ed. Englewood Cliffs, NJ: Prentice-Hall.Hooke, R. and Jeeves, T.A., 1961. Direct search solution of numerical and statistical problems. Journal of Association for

Computing Machinery, 8, 212–221.Jones, D., Schonalau, M., and Welch, W., 1998. Efficient global optimization of expensive black-box functions. Journal

of Global Optimization, 13 (4), 455–492.Kazemi, M., Wang, G.G., and Rahnamayan, S., 2011. Metamodel-based optimization for problems with expensive

objective and constraint functions. Journal of Mechanical Design, 133 (1), 1–7.Kennedy, J. and Eberhart, R., 2001. Swarm intelligence. San Diego: Academic Press.Kiusalaas, J., 2005. Numerical methods in engineering with MATLAB. New York: Cambridge University Press.Mahdavi, M., Fesanghary, M., and Damangir, E., 2007. An improved harmony search algorithm for solving optimization

problems. Applied Mathematics and Computation, 188 (2), 1567–1579.Moghaddam, R., et al., 2006. Applying constraint programming for design space reduction in metamodeling based

optimization. In: 16th international Institution for Production Engineering Research CIRP international designseminar, July 2006, Alberta, Canada. CIRP, Article No. 10081.

Moody, J. and Darken, C.J., 1989. Fast learning in networks of locally-tuned processing units. Neural Computation, 1,281–294.

Mullur, A.A. and Messac, A., 2005. Extended radial basis functions: more flexible and effective metamodeling. AIAAJournal, 43 (6), 1306–1315.

Queipo, N.V., et al., 2005. Surrogate-based analysis and optimization. Progress in Aerospace Sciences, 41 (1), 1–28.Ragsdell, K. and Phillips, D., 1976. Optimal design of a class of welded structures using geometric programming. Journal

of Engineering for Industries, 98 (3), 1021–1025.Tang, Y., Chen, J. and Wei, J., 2009. Probabilistic optimization of laminated composites considering both ply failure and

delamination based on PSO and FEM. Tsinghua Science and Technology, 14, 89–93.Thomas, W., 2009. Global optimization algorithms – theory and application [online]. Available from: http://www.

it-weise.de/projects/book.pdfVaradarajan, S., Chen, W., and Pelka, C.J., 2000. Robust concept exploration of propulsion systems with enhanced model

approximation capabilities. Engineering Optimization, 32 (3), 309–334.Wang, F., Qiu, Y., and Bai, Y., 2005. A new hybrid NM method and particle swarm algorithm for multimodal function

optimization. Lecture Notes in Computer Science, 3646, 497–508.Wang, G., 2003. Adaptive response surface method using inherited Latin hypercube design points. Journal of Mechanical

Design, 125 (6), 210–220.Wang, G.G. and Shan, S., 2007. Review of metamodeling techniques in support of engineering design optimization.

Journal of Mechanical Design, 129 (4), 370–380.Wang, G.G. and Simpson, T., 2004. Fuzzy clustering based hierarchical metamodeling for design space reduction and

optimization. Engineering Optimization, 36 (3), 313–335.Wang, G.G., Dong, Z., and Aitchison, P., 2001. Adaptive response surface method – a global optimization scheme for

computation-intensive design problems. Engineering Optimization, 33 (6), 707–734.Wang, L., Shan, S., and Wang, G.G., 2004. Mode-pursuing sampling method for global optimization on expensive

black-box functions. Engineering Optimization, 36 (4), 419–438.Yannou, B., and Harmel, G., 2006. Use of constraint programming for design. In: H.A. ElMaraghy and W.H. Elmaraghy,

eds. Advances in design. London: Springer, 145–157.Yuan, Y.X. and Sun, W.Y., 1997. Optimization theory and method. Beijing: Science Press.Zerpa, L.E., et al., 2005.An optimization methodology of alkaline–surfactant–polymer flooding processes using field scale

numerical simulation and multiple surrogates. Journal of Petroleum Science and Engineering, 47 (3–4), 197–208.Zou, T., et al., 2008.An indicator response surface method for simulation-based reliability analysis. Journal of Mechanical

Design, 130 (7), 1–11.

Dow

nloa

ded

by [

"Que

en's

Uni

vers

ity L

ibra

ries

, Kin

gsto

n"]

at 0

2:15

31

Aug

ust 2

013