Abstract—In engineering applications, Gaussian process (GP) regression method is a new statistical optimization approach, to which more and more attention is paid. It does not need pre-assuming a specified model and just requires a small amount of initial training samples. Based on the design of experiment (DOE), determining a reasonable statistical sample space is an important part for training the GP surrogate model. In this paper, a novel intelligent method of DOE, the translational propagation algorithm, is employed to obtain optimal Latin hypercube designs (TPLHDs). It also proved that TPLHDs’ performance is superior to other LHDs’ optimization techniques in low to medium dimensions. Using this method, the best settings of the process parameters are determined to train GP surrogate model in the injection process. A automobile door handle is taken as an example, and experimental results show that the proposed TPLHD performs much better than the normal LHD in the quality of fitting GP surrogate model, so taking TPLHDs instead of LHDs’ optimization technique for training GP model is practical and promising.

I. INTRODUCTION UMERICAL simulation models are widely used to simulate the process behavior in the domain of

injection molding. In order to perform efficient data analysis and prediction, it is necessary to determine the best settings for a number of design parameters. Generally a computer simulation of the mathematical models is costly and time-consuming, because there are a great number of possible input combinations. Therefore, a useful and quick choice of the design points is of primary importance.

Based on various approaches, a great variety of research works have been performed in the domain of the process parameters setting for injection molding[1], such as design of experiment (DOE), Artificial Neural Networks (ANN), Genetic Algorithm (GA) and so on. It is well known from these works that the DOE is used to generate the minimum number of samples to train the surrogate models while the characteristic of the process simulation can be adequately captured. Among the numerous DOE techniques, the Latin hypercube design (LHD), a stratified random sampling

Manuscript received April 10, 2010. This work was supported in part by

the National Natural Science Foundation of China under Grants No. 50965003 and the Postgraduate Research & Innovation Projects of Guangxi under Grants No. T32204

Xiaoping Liao, Xuelian Yan are with School of Mechanical and Engineering, Guangxi University, Nanning, Guangxi Province, China. (phone:0771-3239522; fax: 0771-3239522; e-mail: xpfeng@gxu.edu.cn)

Wei Xia, Bin Luo are with School of Material and Engineering, Guangxi University, Nanning, Guangxi Province, China. (e-mail: xiawei@gxu.edu.cn, luobin13579@126.com)

technique, which was proposed by McKay [2] and Iman Conover [3] is widely used for obtaining a small size DOE to train the surrogate models, such as Gaussian regression model and Kriging model. For example, Zhou Jian [4] adopted a LHD for initial training of the GP surrogate model in his PhD dissertation and some journal articles [5]. Gao [6] obtained the optimal LHD by exchanging active pairs to realize maximum entropy for Kriging model. Generally speaking, for constructing more accurate surrogate model, it is important to make the experimental designs optimal. However, owing to LHDs’ random character, the optimization of the space-filling qualities of the LHD is always a challenging problem. Park developed a row-wise element exchange algorithm[7], and then Morris and Mitchell adapted a version of simulated annealing (SA) algorithm [8] for constructing optimal LHDs. Ye, et al constructed optimal symmetrical LHDs using the column-wise-pair-wise (CP) algorithm in [9]. Bates et al employed the Genetic algorithm (GA) [10] and Jin R, Chen proposed the enhanced stochastic evolutionary (ESE) algorithm [11] to construct optimal LHDs. The optimal designs constructed by these algorithms may have a good space filling property，but the computational cost of these existing algorithms is generally high. In other words, such designs are usually obtained from time-consuming combinatorial optimization problems. Whereas, the translational propagation algorithm[12], which was first proposed by Felipe A.C.Viana, is presented as a computationally attractive strategy for obtaining optimal or near optimal LHDs. It does not need formal optimization which requires minimal computational effort.

In this paper, the translational propagation algorithm is introduced in details. And then it focuses on raising the method of translational propagation Latin hypercube designs (TPLHDs). Combined with the design of the proposed method, GP regression model is used to build an approximate function relationship between warpage and process parameters in the injection molding. After training the GP regression model, a comparison is made for model’s accuracy between normal LHDs and TPLHDs. Optimal parameters will be obtained from accurate models which are constructed by TPLHD samples. So it is significant and promising to take the method of TPLHDs for training GP surrogate model, achieving a competitive advantage of saving computational time and performing better performance.

II. LATIN HYPERCUBE DESIGN As previously indicated, Latin hypercube design is a very

popular method for the generation of the samples. Generally speaking, it operates in the following manner to generate a

A Fast Optimal Latin Hypercube Design for Gaussian Process Regression Modeling

Xiaoping Liao, Xuelian Yan，Wei Xia and Bin Luo

N

474

Third International Workshop on Advanced Computational Intelligence August 25-27, 2010 - Suzhou, Jiangsu, China

978-1-4244-6337-4/10/$26.00 @2010 IEEE

sample of size m from n variables with the distributions d1, d2,…,dn, input vector: x=[x1,x2,…, xn]. The range Xj of each variable xj is divided into m contiguous intervals Xij (i=1,2,…,m) of equal probability in consistency with the corresponding distribution dj. A value for the variable xj is selected at random from the interval Xij in consistency with the distribution dj for i=1, 2,…,m and j=1,2,…,n. Then, the m values for x1 are combined at random and without replacement with the m values for x2 to produce the ordered pairs [xi1, xi2](i=1, 2,…,m). Next, the preceding pairs are combined at random and without replacement with the m values for x3 to produce the ordered triples [xi1, xi2, xi3](i=1, 2,…,m). The process continues in the same manner through all n variables. The resultant sequence xi=[xi1,xi2,…,xin] (i=1,2,…,m) is an LHDs of size m from the n variables generated in consistency with the distributions d1, d2,…,dn [13].

III. OPTIMAL LATIN HYPERCUBE DESIGNS BASED ON TRANSLATIONAL PROPAGATION ALGORITHM

A. Basic Theory of Translational Propagation Algorithm The translational propagation algorithm is a new method

for obtaining optimal or near optimal Latin hypercube designs without using formal optimization which requires minimal computational effort with results virtually provided in real time. The aim of the procedure is to solve the optimization in an approximate sense, that means to obtain a good Latin hypercube quickly, rather than finding the best possible solution. However, it is found that for the lower-dimensional cases (up to six variables) the TPLHD approximates the optimal LHD very well, but in higher dimensions, the TPLHD does not approximate optimum solution well anymore. In the injection molding field, main variables less than six are usually under consideration. So using the TPLHD to obtain data samples is quickly and effectively.

The proposed approach is based on the idea of constructing the optimal nv-dimensional LHD from a fairly small nv-dimensional seed design for which there is no limitation on the number of points (normally one to five). In order to construct a LHD of np points from a seed design of ns points, the design spaces first divided into a total of nb blocks such that:

b p sn n / n= (1) Small building blocks, consisting of one or more points

each, are used to recreate these patterns by simple translation in the hyperspace. It means that each dimension is partitioned into the same number of divisions, nd:

1 v/ nd bn ( n )= (2)

where nv is the number of variables, s is the initial seed design, ns is the number of points in the seed design, np is the number of points in the design.

Fig.1 shows the division of the design space for the 9×2 LHD and which of the blocks is the first one to be picked in the algorithm.

Fig.1. 9×2 Latin hypercube mesh divided into blocks

(3 divisions in each dimension results in 9 blocks) Next each block is filled with the previously selected seed

design and the block with the seed design is iteratively shifted by np/nd levels in one of the dimensions. The shifting process continues in one of the dimensions until all the divisions in this dimension are filled with the seed design. In next step, the current set of points is used as a new seed design and the procedure of shifting the seed design is repeated in the next dimension. Fig.2 illustrates the shifting procedure of creating 9×2 LHD (ns=1) in the horizontal and vertical direction.

(a)Step1 (b)Step2

(c)Step3 (d)Step4

Fig.2. The shifting procedure of creating 9×2 LHD in the horizontal and vertical direction. (a) illustrates the initial placement of the seed;(b)-(c)show the translation of the seed in the horizon direction;(d) illustrates the translation of the new seed (c) in the vertical direction.

B. Generate Experimental Designs of Any Size To obtain a LHD with any number of points, the first step is

to generate a TPLHD that has at least the required number of points using the algorithm described above. If this design contains the required number of points, the process is completed. Otherwise, an experimental design larger than the required is created and a resizing process is used to reduce the number of points to the desired one. The points are removed one by one from the initially created TPLHD by discarding the points that are the furthest from the center of the hypercube, and then reallocate remaining points to fill the whole design (preserving the Latin hypercube properties). In the proposed algorithm removing the points furthest from the center does not reduce the area of exploration. After removing the points, the final design is rescaled to cover the whole design space. For example, a 12×2 Latin hypercube is

475

created using one point seed design. From Equations (1) and (2), 1 3 46v/ n

d bn ( n ) .= = does not result in an integer number in this case. Rounding nd ≈3.46 up would give nd ≈4 and the next largest design that can be constructed is the 16 ×2.The resizing process begins to calculate the distance between each of the 16 points and the center of the design space. To create a 12×2 design from a16×2 one, the four points furthest from the center have to be removed. When one point is removed, the levels occupied by its projection along each of the dimensions have to be eliminated. This preserves the Latin hypercube property that only a single point is found at any of the levels.

C. Seed Designs A seed design usually contains one to five points. It is

important to choose the initial seed before constructing the TPLHD. Fig.3 shows some possible two-dimensional seed designs. By changing the seed design and using the described algorithm can create different instances of the TPLHD. As it is not known beforehand which seed design will lead to the best design in terms of ~

pφ criterion( 0 1~

pφ≤ ≤ ) referred to equations (3) (4) and (5), one would not know what seed design to use. During the shifting process, the number of levels in a block is greater than the number of points in the seed design. This means that when creating design starting from seeds with more than one point (ns>1) it is necessary to reshape them to fit into one block of the TPLHD. No single seed size is always best. Table I shows the best TPLHD constructed by different seed design [12].

Fig.3. Examples of seed designs for two variables

1

1

1 1

p pn np / p

p iji j i

[ d ]φ−

−

= = +

= ∑ ∑ (3)

1

150 1

vn/ t

ij i j ik jkk

d d ( x , x ) [ x x ] , p ,t=

= = − = =∑ (4)

~p p

pp p

m i n ( )m a x ( ) m i n ( )

φ φφ

φ φ−

=−

(5)

Where np is the number of points of the design, and dij is the inter-point distance between all point pairs in the design, max(φ p) and min(φ p) are the maximum and minimum values ofφp found in the generated DOEs.

TABLE I THE BEST TPLHD CONSTRUCTED BY SEED DESIGN

No. of variables n No. of points n s 2 12 1 2 20 1 or 5 2 120 2 4 30 4 4 70 4 4 300 4 6 56 1 6 168 3 6 560 1

D. Latin Hypercube Designs via Translational Propagation Algorithm (TPLHDs)

The basic theory of the translational propagation algorithm is demonstrated in details above. The process of the algorithm to obtain optimal LHDs is shown in Fig.4 and summarized as follows:

Step 1: generate a TPLHD that has at least the required number of points using the basic algorithm;

Step 2: check whether or not a bigger experimental design is needed. The number of divisions nd and its rounded up value nd

* are compared; Step 3: the seed design is reshaped to fit in one block of the

TPLHD, and then a TPLHD of np* points is created; Step 4: if np*＞np, the TPLHD previously obtained is

resized so as to it has np points. The final TPLHD with np points X is constructed. In the

processs, np is the number of points of the desired LHD, np*and nd* represent the number of points and number of divisions of the first TPLHD to be created respectively.

Fig.4. Flowchart of the translational propagation Latin hypercube designs

E. LHD Optimization Techniques Comparison After introducing the TPLHDs in details, a performance

comparison is made between the TPLHD and two other different optimization techniques: (i) the enhanced stochastic evolutionary; (ii) the Genetic algorithm. Table II lists the details of the comparison.

TableII shows that TPLHD has the advantage in performance, which contains both design quality and computational cost. As to design quality, TPLHD offers good solutions in most of the designs with the value ~

pφ =0. This means that TPLHD finds the best result of the set of all simulations. In addition, referred to the computational cost, the TPLHD is superior in most of the cases. Even in the worse

Create an initial TPLHD using translational

propagation algorithm

Check whether or not a bigger experimental design is needed

Reshape the seed design to fit in one block of the TPLHD

Create a TPLHD of np* points

Resize the TPLHD

The final TPLHD with np points is obtained

np*＞ np

Yes

Compare np with np*

No

np*= np

Process completed

476

case, it is still cost less time than other optimization techniques. Clearly, the point density has the dramatic impact on the optimization techniques, such as ESEA and GA. For example, moving from 56 to560 points using ESEA with 6 variables, it takes the computational time from 2s to a little more than 18min. The GA tends to suffer more, as it can be seen that in six dimensions, when moving from 56 to 560 points the computational cost of GA increases more than 100 times.

TABLE II PERFORMANCE COMPARISON AMONG TPLHD, ESEA AND GA

No. of variables 2 4 6

No. of points 12 20 120 30 70 300 56 168 560

~

pφ

TPLHD 0.1 0.1 0 0 0 0 0.1 0 0

ESEA 0.2 0.1 0.1 0 0 0 0 0 0

GA 0.2 0.3 0.3 0.1 0.1 0 0 0 0

time (s)

TPLHD ≈0 ≈0 0.1 ≈0 ≈0 0.7 ≈0 0.5 2

ESEA ≈0 0.2 13 0.3 3 173 2 34 1096

GA 0.4 0.9 24 4 17 275 19 143 1509

Compared with the other LHD optimization techniques, it is clear that in low or medium dimensions the TPLHD offers a better design, not to mention a faster solution. Therefore it is recommended to generate TPLHD to construct the surrogates in small to medium dimensions.

IV. GAUSSIAN PROCESSES REGRESSION METHOD The Gaussian process (GP) regression, which employs a

Bayesian statistics approach, is adopted as a highly nonlinear regression technique. It assumes a distribution for the output rather than thinking the output as a point value, seeks to determine the parameters of the distribution, and predicts the most likely value as well as the variance for the desired response directly through statistical reasoning [14].

Given a training data set D, which consists of N pairs of L-dimensional inputs xN and scalar outputs t n for n=1,…,N, a GP prediction model is concerned with evaluating the probability P(tN+1|D, xN+1), where the new input vector is xN+1 and the corresponding output is tN+1. In this process, the P(tN| CN, {xN}) is assumed to follow a Gaussian distribution as referred to (6):

{ }( ) { }( ){ }( )

( )⎥⎦⎤

⎢⎣⎡ −−=

=

−+

−++

+++++

+NN

TNNN

TN

CC

NNN

NNNNNNNNN

tCttCt

CxtPCxxtPCxxttP

N

N

11

111

11111

21exp

2

1,|

,,|,,,|

1π

(6)

( ) ( ) 2

1 2 321

( )1( ) ( , ) exp[ ]2 =

−= = − + +∑

l lLi j

N ij i j ijl l

x xC C x x

rθ θ δ θ (7)

where tN= (t1(x1), t2(x2),…, tN(xN)), CN is the covariance matrix for P(tN|{xN}), which consisting hyperparameter θ=(θ1,θ2,θ3,r) needed to be optimized is given in Equation (7) and μ is the mean which will be zero for properly normalized data.

To find the optimal value of hyperparameters, it is easy to perform likelihood optimization of hyperparameters with common optimization algorithm [15]. Then the optimal value of the hyperparameters is used to construct the covariance

matrix, which, in turn, is inverted and used to give predictions for the predicted response as well as the predicted variance according to equations (8) and (9):

NNT

N tCt 11

ˆ −+ = κ (8)

λκλσ 12ˆ 1

−−=+ N

Tt CN

(9) where k =(C(x1 xN+1), C(x2 xN+1),…,C(x N xN+1))

κ =C(xN+1,xN+1). And

1ˆ

+Nt and 2ˆ 1+Ntσ are the expressions for the mean and

variance of the predicted target value. Thus, using GP is especially useful in engineering applications, since the engineers should always be concerned with the accuracy and applicability of the models created. The procedure of GP is shown in Fig.5.

Fig.5 The procedure of Gaussian Process (GP)

V. ROOT MEAN SQUARE ERROR The accuracy of the model is measured by root mean

square error (RMSE) [16]. The error is a very accurate estimate of the true prediction error of the model. It is calculated easily by squaring individual errors, summing them, dividing the sum by their total number, and then taking the square root of this quantity:

2( ) /= −∑ i i

predR M SE Y Y N (10)

Where xi a sample in the test set, iY is the simulation value at point xi and i

predY is the model prediction. The RMSE summarizes the overall error of the model and it is a fitness criterion in model prediction.

VI. ILLUSTRATIVE EXAMPLE An automobile door handle as an example is investigated.

Fig.6 shows its shape and mesh. Its length, width and height

Yes

Determine covariance functions

Select prior distribution on hyperparameters

Optimize hyperparameters

Calculate covariance matrix and computer inverse

Does the GP meet the acceptance?

End

Re-optimize hyperparameters

No

Run the TPLHDs

477

are 800, 70,45mm, respectively. And its average thinness is 3mm. It is a thin shell structure part and the warpage is a main defect for assembly and appearance. The material is Stylac-ABSR240A, and the material properties are given in Table III.

Fig.6 Mid-plane model of an automobile door handle

TABLE III MATERIAL PROPERTIES OF STYLAC-ABSR240A

Melt density 1.0511 g/cm3 Solid density 1.1667 g/cm3 Eject temperature 90oC Maximum shear stress 0.3 MPa Maximum shear rate 50000 (1/s) Thermal conductivity 0.14 W/m·oC Elastic module 5290.77MPa Poisson ratio 0.418 Fillers 20% Glass Fiber Filled

For this case, the warpage is quantified by the out-of-plane

displacement, which is the maximum upward deformation with reference to the default plane in MoldFlow software.

By defining the maxi-warpage as the objective function and the melt temperature, packing pressure, injection pressure, injection time, packing time as process conditions, the optimization problem can be expressed below together with the process ranges:

Minimize maxi-warpage (Y) Subject to: 220≤Tm≤240 oC 60≤Pin≤70MPa

50≤Pp≤60 MPa 4≤tin≤5s 10≤tp≤20s

Where Tm is melt temperature, Pin injection pressure, Pp packing pressure, tin injection time, tp packing time, the mold temperature and the cool time are fixed to 60 oC, 30s.

For constructing the GP surrogate model with five variables, 10 initial process combinations are selected by LHD and TPLHD respectively. By using MoldFlow analysis, the 10 initial samples and the simulation results are listed and then the proposed GP regression model will be constructed with the simulation results.

During the validation process, 10 samples are divided into a training set and a testing set randomly. The testing set is put asides, the created GP regression model constructed by the trainings set would subsequently be validated with the testing set data to assess the model’s accuracy. At last, the predictions of the warpage based on GP regression model are

compared with the simulation results, as tabulated in Table IV, where normal LHD, TPLHD are presented in (a) and (b) respectively.

Given the data in Table III and refer to (10), the RMSE can be calculated. The results are shown as follows:

RMSEa=0.4304 RMSEb=0.2534

The RMSE using the TPLHD between Ys and Ypred is 0.2534, which is 41.12% smaller than the RMSE（0.4304）using the normal LHD. The difference between these RMSE is statistically significant. Thus, the GP surrogate model approximates well with the simulation by obtaining samplings with TPLHD.

TABLE IV RESULTS OF SIMULATION AND PREDICTION WITH ALL SAMPLES

(a)Normal LHD No. T melt t in t p P in Pp WarpageYs Ypred

1 228.8 4.18 10.6 60.8 52.3 1.749 2.017

2 225.2 4.26 14.5 67.3 53.8 1.979 1.500 3 236.9 4.52 13.1 65.7 58.2 1.603 2.202 4 223.3 4.33 12.9 64.0 57.4 2.358 1.977 5 235.1 4.65 11.2 61.3 55.9 2.388 2.897 6 227.5 4.44 18.0 62.6 50.8 2.188 1.632 7 230.8 4.02 19.0 68.3 51.4 2.012 1.758 8 220.5 4.79 15.2 66.7 54.4 2.320 2.536 9 238.9 4.94 16.6 63.6 56.6 1.657 2.018

10 232.7 4.89 17.5 69.1 59.4 1.879 2.089

(b)TPLHD No. T melt t in t p P in Pp Warpage Ys Ypred

1 234 4.5 17 66 51 1.976 2.092

2 222 4.6 18 61 55 2.252 2.172 3 236 4.7 19 62 56 2.046 2.221 4 238 4.1 11 67 57 1.553 1.813 5 224 4.8 12 68 58 1.358 1.724 6 240 4.9 13 69 59 1.446 1.172 7 226 5.0 14 63 52 1.904 1.615 8 228 4.2 20 64 53 2.235 2.401 9 230 4.3 15 70 54 1.828 2.112

10 232 4.4 16 65 60 1.582 1.952

VII. CONCLUSION To improve the accuracy and computational efficiency of

GP surrogate model, a novel intelligent method is developed and applied in the domain of injection modeling.

Based on the LHD, the translational propagation algorithm to obtain optimal LHDs for training the GP surrogate model is presented. The new algorithm is exploited by simple translation of small building blocks in the hyperspace without using formal optimization, which is superior to other Latin hypercube optimization techniques. Implementation of the method has demonstrated that TPLHD for simulation is employed to select the most characteristic process conditions to enrich the information for training the GP model, while to minimize the time for sample acquisition. In addition, GP has the best overall performance for nonlinear regression. Training the GP surrogate model using TPLHD is a promising and practical research.

478

ACKNOWLEDGMENT The authors would like to thank the research group that

took part in the study for their generous cooperation. Also they would like to express appreciation to the referees for their helpful comments and suggestions.

REFERENCES [1] S.L. Mok, C.K. Kwong and W.S. Lau, “Review of research in the

determination of process parameters for plastic Injection molding,” Advances in Polymer Technology, vol. 18, no. 3, pp.225–236, March 1999.

[2] McKay MD and Beckman RJ, “A comparison of three methods for selecting values of input variables from a computer code,” Technometrics, vol. 21, no. 2, pp.239–245, May 1979.

[3] Iman RL and Conover WJ, “Small sample sensitivity analysis techniques for computer models with an application to risk assessment,” Communications in Statistics-Theory and Methods, vol. 9, no. 17, pp.1749–1842, 1980.

[4] J. Zhou and L.S. Turng, “Adaptive multi-objective optimization of process conditions for injection molding using a Gaussian process approach,” Advances in Polymer Technology, vol. 26, no. 2, pp. 71–85, April 2007.

[5] J. Zhou, L.S. Turng and A. Kramschuster, “Single and multiobjective optimization for injection molding using numerical simulation with surrogate models and genetic algorithm,” Int. Polym. Process, vol. 21, pp.509–520, 2006.

[6] Y.H. Gao and X.C. Wang, “Surrogate-based process optimization for reducing warpage in injection molding,” Journal of Materials Processing Technology, vol. 209, no. 3, pp. 1302–1309, Feb. 2009.

[7] Park JS, “Optimal Latin-hypercube designs for computer experiments,” Journal of Statistical Planning and Inference, vol. 39, no. 1, pp.95–111, April 1994.

[8] Morris MD and Mitchell TJ, “Exploratory designs for computational experiments,” Journal of Statistical Planning and Inference, vol. 43, no. 3, pp.381–402, Feb. 1995.

[9] Ye KQ, Li W and Sudjianto A, “Algorithmic construction of optimal symmetric Latin hypercube designs,” Journal of Statistical Planning and Inference, vol. 90, no. 1, pp.145–159, Sep. 2000.

[10] Bates SJ, Sienz J and Toropov VV, “Formulation of the optimal Latin hypercube design of experiments using a permutation genetic algorithm,” Forty-fifth AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, pp.19–22, April 2004.

[11] Jin R, Chen W and Sudjianto A, “An efficient algorithm for constructing optimal design of computer experiments,” Journal of Statistical Planning and Inference; vol. 134, no. 1, pp.268–287, Sep. 2005.

[12] Felipe A. C. Viana, Gerhard Venter and Vladimir Balabanov, “An algorithm for fast optimal Latin hypercube design of experiments,” International journal for numerical methods in engineering, vol. 82, no. 2, pp.135–156, Oct. 2009.

[13] C.J. Sallaberry, J.C. Helton and S.C. Hora, “Extension of Latin hypercube samples with correlated variables,” Reliability Engineering and System Safety, vol. 93, pp.1047–1059, July 2008.

[14] Yao Weixiong, Yang Yi and Zeng Bin, “Novel methodology for casting process optimization using Gaussian process regression and genetic algorithm,” China foundry, vol.6, no.3, pp.232–240, May 2009.

[15] C. E. Rasmussen and C. K. I. Williams, “Gaussian Processes for Machine Learning”, the MIT Press, 2006.

[16] Aynur O. Aptula and Nina G. Jeliazkova, “The Better Predictive Model: High q2 for the Training Set or Low Root Mean Square Error of Prediction for the Test Set,” QSAR &Combinatorial Science, vol. 24, no. 3, pp.385–396, Feb. 2005.

479