A Novel Nonlinear RBF Neural Network Ensemble Model forFinancial Time Series Forecasting
Donglin Wang and Yajie Li
Abstractβ In this paper, a novel nonlinear Radial BasisFunction Neural Network (RBF-NN) ensemble model based onπ-Support Vector Machine (SVM) regression is presented forfinancial time series forecasting. In the process of ensemblemodeling, the first stage the initial data set is divided intodifferent training sets by used Bagging and Boosting technology.In the second stage, these training sets are input to the differentindividual RBFβNN models, and then various single RBFβNNpredictors are produced based on diversity principle. In thethird stage, the Partial Least Square (PLS) technology is usedto choosing the appropriate number of neural network ensemblemembers. In the final stage, π-Support Vector Machine (SVM)regression is used for ensemble of the RBFβNN to predictionpurpose. For testing purposes, this paper compare the new en-semble modelβs performance with some existing neural networkensemble approaches in terms of two financial time series:S & P 500 and Nikkei 225. Experimental results reveal thatthe predictions using the proposed approach are consistentlybetter than those obtained using the other methods presentedin this study in terms of the same measurements. Those resultsshow that the proposed nonlinear ensemble technique providesa promising alternative to financial time series prediction.
I. INTRODUCTION
F INANCIAL time series modeling and forecasting isregarded as a rather challenging task because financial
time series are inherently noisy and nonstationary [1][2].Due to the high degrees of irregularity, dynamic mannerand nonlinearity, it is extremely difficult to capture theirregularity and nonlinearity hidden in financial time series bytraditional linear models such as multiple regression, expo-nential smoothing, autoregressive integrated moving average,etc. In recent years, Artificial Neural Network (ANN), whichemulate the parallel distributed processing of the humannervous system, have been successfully used for modelingfinancial time series [3][4][5]. Due to their powerful capa-bility and functionality, ANN provide an alternative approachfor many engineering problems that are difficult to solve byconventional approaches. An ANN with moderate numberof hidden layer(s) is capable of approximating any smoothfunction to any desired degree of accuracy without anyassumptions of traditional statistical approaches required. Inaddition, ANN are computationally robust, having the abilityto learn and generalize from examples to produce meaningfulsolutions to problems even when the input data contain errorsor are incomplete [6][7].
The application of an ANN, involves a complicated de-velopment process. As ANN approaches want of a rigorous
Donglin Wang and Yajie Li are with the Department of Mathematics, Bei-jing Vocational College of Electronic Science, No.1jia, 100024, ShaoyaojuStreet, Chaoyao District, Beijing (email: [email protected]).
theoretical support, effects of applications strongly dependupon operators experience. Even for some simple problems,different structures of neural networks (e.g., different numberof hidden layers, different hidden nodes and different initialconditions) result in different patterns of network gener-alization. In the practical application, the results of manyexperiments have shown that the generalization of singleneural network is not unique. That is, ANN results are notstable [8].
In order to overcome the main limitations of ANN, re-cently a novel ensemble forecasting model, i.e. artificial neu-ral network ensemble (ANNE), has been developed [9][10].Because of combining multiple neural networks learned fromthe same training samples, ANNE can remarkably enhancethe forecasting ability and outperform any individual neuralnetwork. It is an effective approach to the development of ahigh performance forecasting system [11].
The important motivation of ensemble learning method isintegrated different neural network models based on the fun-damental assumption that one cannot identify the true processexactly. Meantime, some linear ensemble learning methodson the standard feed-forward neural network models [12][13]such as back-propagation neural networks (BPNN), are alsopresented. Different from the previous work, this paperproposes a novel nonlinear Radial Basis Function NeuralNetwork (RBF-NN) ensemble forecasting method in terms ofπ-Support Vector Machine Regression (π-SVMR) principle.
The rest of this paper is organized as follows: SectionII describes the building process of the nonlinear RBFβNNensemble forecasting model in detail. For further illustration,two real financial time series are used for testing in SectionIII. Finally, some concluding remarks are drawn in SectionIV.
II. THE BUILDING PROCESS OF THE NONLINEAR
ENSEMBLE MODEL
In this section, a triple-phase nonlinear RBFβNN ensemblemodel is proposed for financial time series forecasting. Firstof all, many individual RBFβNN predictors are generatedin terms of diversification. Then an appropriate numberof RBFβNN predictors are selected from the considerablenumber of candidate predictors by the Partial Least Square(PLS) technology. Finally, selected RBFβNN predictors arecombined into an aggregated neural predictor in terms ofπ-Support Vector Machine Regression.
86
Third International Workshop on Advanced Computational Intelligence August 25-27, 2010 - Suzhou, Jiangsu, China
978-1-4244-6337-4/10/$26.00 @2010 IEEE
A. RBF Neural Network
The financial data is a complex nonlinear systems sothat it is difficult to model by using linear regressionmethodologies. Dissimilar to the regression, neural networksare nonlinear and their parameters are determined by somelearning techniques and searching algorithms such as errorback propagation and steep gradient algorithm. Radial basisfunction was introduced into the neural network literatureby Broomhead and Lowe [14]. The RBFβNN model is anetwork with local neurons which was motivated by thepresence of many local response neurons in human brain.Other motivation came from numerical mathematics. On thecontrary to the other type of NN used for nonlinear regressionlike back propagation feed forward networks, the RBFβNN learns quickly and has a more compact topology. TheGaussian RBβNN is found suitable not only in generalizinga global mapping but also in refining local features [15].
The RBF neural network is generally composed of threelayers: input layer, hidden layer and output layer. The maindifference from that of MLP is the absence of hidden-layer weights. The hidden neurons outputs are not calculatedby using the weighted-sum mechanism/sigmoid activation;rather each hidden neurons output ππ is obtained by thecluster of the input π to an m-dimensional parameter vectorπΆπ associated with the ith hidden unit. The response outputof ith hidden neurons is calculated by
ππ = ππ₯π{β 1
2π2πβ₯π β πΆπβ₯2} (1)
where πΆπ and ππ is centers and width of RBF in ith hiddenneurons, respectively. The distance is calculated by theEuclidean norm
β₯π β πΆπβ₯ =
ββββ·πβ
π=1
(π₯π β ππ)2 (2)
Given an input vector π , the output of the RBF network isgiven by
π‘π =
πβπ=1
ππππ + π0 (3)
where π0 is the bias, {ππ, π = 1, 2, β β β ,π} are the weightparameter, π is the number of nodes in the hidden layers ofthe RBF neural network. The sum of square errors (SSE) isgiven by
πππΈ =
πβπ=1
[π‘π β π¦π]2 (4)
where π is the number of training cases, π¦π is desired outputand π‘π is the actual output.
The training procedure of the RBF networks is a complexprocess, this procedure requires the training of all parametersincluding the centers of the hidden layer units (ππ, π =1, 2, β β β ,π), the widths (ππ) of the corresponding Gaussianfunctions, and the weights (ππ, π = 0, 1, β β β ,π) between thehidden layer and output layer. In this paper, the the orthog-onal least squares algorithm (OLS) is used to training RBF
based on the minimizing of SSE. The more detailed aboutalgorithm is described by the related literature [16] [17].
B. Generating individual RBF-NN predictors
With the work about biasβvariance tradeβoff of Breti-man [18], an ensemble model regression model consistingof diverse models with much disagreement is more likely tohave a good generalization [19]. Therefore, how to generatediverse models is a crucial factor. For RBFβNN model,several methods have been investigated for the generation ofensemble members making different errors. Such methodsbasically depended on varying the parameters of RBFβNNor utilizing different training sets. In this paper, there arethree methods for generating diverse models.
(1) Using different RBFβNN architecture, such as thedifferent number of nodes in hidden layer, diverse RBFβNNwith much disagreement can be created.
(2) Utilizing different the parameters of RBFβNN, such asdifferent cluster center π of the RBFβNN, : through varyingthe cluster center π of the RBFβNN, different cluster radiusπ of the RBFβNN, different RBFβNN can be produced.
(3) Using different training data: by re-sampling andpreprocessing data, different training sets can be obtained.Typical methods include bagging [20], cross-validation (CV)[21], and boosting [22]. With these different training datasets,diverse RBFβNN models can be produced.
C. Selecting appropriate ensemble members
After training, each individual neural predictor has gener-ated its own result. However, if there are a great number ofindividual members, we need to select a subset of represen-tatives in order to improve ensemble efficiency. In this paper,the Partial Least Square (PLS) regression technique [23] isadopted to select appropriate ensemble members. Interestedreaders can be referred to [23] for more details.
D. π-Support vector regression
SVM was a significant result of machine learning researchin recent years, which has been introduced by Cortes andVapnik in 1995 [24]. Compared with the traditional neuralnetwork, the support vector machine not only simple instructure but also all sorts of technical performances arebetter than neural network obviously, which were testified bylots of experiments [25]. It was developed on the foundationof small samples statistical learning theory that proposedby Vapnik etc., and its algorithm is based on the structuralrisk minimization principle [26], which minimizes the upperbound of the generalization error which is bounded by boththe sum of the training error and a confidence interval termwhich depends on the VapnikβChervonekis dimension.
With the introduction of Vapnikβs π-insensitive loss func-tion, the SVM can be extended to regression problem, say,support vector regression (SVR), which is constructed byminimizing Vapnikβs π-insensitive loss function residuals be-tween the outputs of SVR and the target values and the modelcomplexity, and SVR is applied to various fields like optimalcontrol, timeβseries prediction, and image segmentation.
87
If the proper hyperparameters are picked up, SVR willgain good generalization performance and vice versa, so itis important to select right model. Instead of selecting anappropriate π Schokopf et al [27] [28]. proposed a variant,called π-support vector regression, which introduces a newparameter π which can control the number of support vectorsand training errors without defining π a prior. To be moreprecise, they proved that π is an upper bound on the fractionof margin errors and lower bound of the fraction of supportvectors [29].
Given a data set {π₯π, π¦π, π = 1, 2, β β β , π} randomlygenerated from an unknown function, where π₯π β βπ is aninput variable and π¦π is the corresponding target value, Theregression problem of π-SVR can be described as follows:β§β¨β©
min ππ = β 12w
Tw + πΆ(ππ+ 1π
πβπ=1
(ππ + πβπ ))
π .π‘. ππ β (wTπ(π₯π) + π) β€ π+ πβπ(wTπ(π₯π) + π)β ππ β€ π+ ππππ, π
βπ β₯ 0, π = 1, 2, β β β , π, π β₯ 0.
(5)
where 0 β€ π β€ 1, πΆ is the regulator, and training data π₯πare mapped into a high (even infinite) dimensional featurespace by the mapping function π(β ). This primal optimizationproblem is a linearly constrained quadratic programmingproblem, which can be solved by introducing Lagrangianmultipliers and applying Karush-Kuhn-Tucker (KKT) condi-tions to solve its dual problem. Therefore, we can get theregression function formula as follows:
π(π₯) =β
π₯πβππ
(οΏ½οΏ½βπ β οΏ½οΏ½π)πΎ(π₯π, π₯π) + οΏ½οΏ½ (6)
Where πΎ(π₯π, π₯π) = ππ β ππ is kernel function, it is generallytaken as πΎ(π₯π, π₯π) = exp(β β₯ π₯π β π₯π β₯2 /2π2, whichis to meet the conditions of any symmetric kernel functioncorresponds to the feature space of the dot product, πΌβπ , πΌπ arethe Lagrangian multipliers associated with the constraints.
E. The Establishment of Combination Forecasting Model
To summarize, the proposed nonlinear RBF neural networkensemble forecasting model consists of four main stages.Generally speaking, in the first stage, the initial data setis divided into different training sets by used Bagging andBoosting technology. In the second stage, these trainingsets are input to the different individual RBFβNN models,and then various single RBFβNN predictors are producedbased on diversity principle. In the third stage, PLS modelis used to select choose the appropriate number of neuralnetwork ensemble members. In the four stage, π-SVR isused to aggregate the selected ensemble members. In sucha way final ensemble forecasting results can be obtained.For verification and testing purposes, four typical foreignexchange rates are used as testing targets for empiricalanalysis. The basic flow diagram can be shown in Fig.1.
III. EMPIRICAL ANALYSIS
In this section, two main two main financial time seriesare used to test the proposed nonlinear ensemble forecasting
Training SetTR1
RBF1
Bagging and BoostingTechnology
Training SetTR2
Training Set TRm
PLS SelectEnsemble Members
v-SVR NonlinearRegression Ensemble
Ensemble ResultOutput
Training SetTR1,TR2,...TRm
Input Data
. . . . . .
RBF2
RBFm
Fig. 1. A Flow Diagram of The Proposed Ensemble Forecasting Model.
model. First of all, we describe the data and evaluationcriteria used in this study and then report the experimentalresults.
A. Data description and evaluation criteria
The stock market is a complex system, which is alsoaffected by political, economic and social factors. Besidesthat, people are not sure of which variables have a significantimpact on the system for the lack of understanding of itsinternal operating mechanism. At present, technicalβstandardof the stock market are often used as neural network inputvariables to predict the stock market and they have achievedcertain effects. The model proposed by this paper chooses sixvariables as the neural network input based on reference [30].The description of six variables are presented in Table 1.
TABLE I
SIX VARIABLES AND THEIR FORMULAS
Variables indicators Formulas
Prices(π₯π‘) π₯π‘, (π‘ = 1, 2, β β β , π)Moving Average(ππ΄5)(π₯1) 1
5
βπ‘π=π‘β5+1 π₯π
Moving Variance(ππ5)(π₯2) 15
βπ‘π=π‘β5+1(π₯π β π₯π‘)
Moving Variance Ratio(πππ 5)(π₯3) ππ 2π‘ /ππ 2
π‘β5
Disparity5(D5)(π₯4) π₯π‘/ππ΄5
Disparity10(D10)(π₯5) π₯π‘/ππ΄10
Price Oscillator(PO)(π₯6) (ππ΄5 βππ΄10)/ππ΄5
The data set used for our experiment consists of two timeseries data: the S&P 500 index series, and the Nikkei 225index series. The data used in this study are obtained fromDatastream (http://www.datastream. com). The entire data setcovers the period from January 1, 2008 to December 31,2009. This paper established the predictive models based on421 daily data from January 1, 2008 to August 31, 2009as the training data sets, and test model based on 87 dailydata from September 1, 2009 to December 31, 2009 as thetesting data set, which are used to evaluate the good or badperformance of predictions.
In order to measure effectiveness of the proposed method,two typical indicators, normalized mean squared error(NMSE) and directional statistics (π·π π‘ππ‘) were used in thispaper. Given π pairs of the actual values (π¦π‘) and predicted
88
values (π¦π‘), the NMSE can be defined as
ππππΈ =
πβπ=1
(π¦π β π¦π)2πβ
π=1
(π¦π β π¦π)2(7)
where π¦π is the average of actual value.Directional change statistics (π·π π‘ππ‘) can be expressed as
π·π π‘ππ‘ =1
π
πβπ‘=1
πΌπ‘ β 100% (8)
where πΌπ‘ = 1 if (π¦π‘+1 β π¦π‘)(π¦π‘+1 β π¦π‘) β₯ 0, and ππ‘ = 0otherwise.
In order to investigate the effect of the proposed model, thesimple averaging ensemble, the mean squared error (MSE)based regression ensemble and variance-based weighted en-semble are established. Those are fitted the 421 samplesand forecasted the 87 samples by the those models, thecomparison results are used to test the effect of predictivemodels.
B. Analysis of the Results
In this study, four main ensemble methods are imple-mented on four exchange rates data-sets for comparison. Thestandard RBF neural networks with Gaussianβtype activationfunctions in hidden layer were trained for each training set,then tested as an ensemble for each method for the testing set.Each network was trained using the neural network toolboxprovided by Matlab software package.
TABLE II
THE NMSE COMPARISON WITH DIFFERENT MODELS FOR DIFFERENT
TIME SERIES ABOUT TRAINING SAMPLES
Ensemble S&P500 Nikkei 225
Method NMSE Rank NMSE Rank
Single RBF-NN 0.2389 5 0.1406 5Simple averaging 0.2274 4 0.0793 3MSE regression 0.2110 3 0.0932 4
Variance-based weight 0.1064 2 0.0757 2π-SVMR 0.0670 1 0.0530 1
TABLE III
THE π·π π‘ππ‘ COMPARISON WITH DIFFERENT MODELS FOR DIFFERENT
TIME SERIES ABOUT TRAINING SAMPLES
Ensemble S&P500 Nikkei 225
Method π·π π‘ππ‘(%) Rank π·π π‘ππ‘(%) Rank
Single RBF-NN 89.68 2 82.75 5Simple averaging 79.65 5 87.54 4MSE regression 88.45 4 89.90 3
Variance-based weight 89.53 3 94.27 2π-SVMR 93.75 1 95.93 1
Tables 2 and 3 show the fitting results of 421 trainingsamples for different models. In the two tables, a clear
comparison of various methods for the two financial timeseries is given via NMSE and π·π π‘ππ‘. From Tables 2 and3, the results show that π-SVM ensemble model better thanthose of the single RBF neural network model and otherensemble forecasting models for the two main financial timeseries in fitting.
The more important factor to measure performance of amethod is to check its forecasting ability of testing samplesin order. Tables 4 and 5 show the forecasting results of 87testing samples for different models about different measureindex. Generally speaking, the forecasting results obtainedfrom the two tables also indicate that the prediction perfor-mance of the proposed π-SVM ensemble forecasting modelis better than those of the single RBFβNN model and otherensemble forecasting models for the two main financial timeseries in forecasting.
TABLE IV
THE NMSE COMPARISON WITH DIFFERENT MODELS FOR DIFFERENT
TIME SERIES ABOUT TESTING SAMPLES
Ensemble S&P500 Nikkei 225
Method NMSE Rank NMSE Rank
Single RBF-NN 1.8527 4 1.9073 5Simple averaging 1.9708 5 1.6362 3MSE regression 1.5616 3 1.5873 2
Variance-based weight 0.3018 2 1.6967 4π-SVMR 0.0230 1 0.0331 1
TABLE V
THE π·π π‘ππ‘ COMPARISON WITH DIFFERENT MODELS FOR DIFFERENT
TIME SERIES ABOUT TESTING SAMPLES
Ensemble S&P500 Nikkei 225
Method π·π π‘ππ‘(%) Rank π·π π‘ππ‘(%) Rank
Single RBF-NN 69.75 5 63.10 5Simple averaging 74.21 3 66.71 3MSE regression 70.89 4 76.09 2
Variance-based weight 78.98 2 65.28 4π-SVMR 83.65 1 80.93 1
In detail, the NMSE of the our proposed π-SVMR ensem-ble model reaches 0.02303 in the testing samples for S &P500 time series, however the NMSE of the Single RBF-NNmodel is 1.8527; the NMSE of the Simple average model is1.9708; the NMSE of the MSE regression model is 1.5616and the NMSE of the Variance-based weight model is 1.5616;These results show the NMSE of the π-SVMR ensemblemodel is less than other model, which has obvious advantagesover other models for S & P500 time series forecasting.
Similarly, for π·π π‘ππ‘ efficiency index, the proposed π-SVMR ensemble model has higher valuer in the testingsamples for S & P500 time series forecasting. From Table5, the π·π π‘ππ‘ for the π-SVMR ensemble model reaches only83.65%, while for the Simple average model, the π·π π‘ππ‘ is69.75%; the π·π π‘ππ‘ of the MSE regression model is only
89
74.21%; the π·π π‘ππ‘ of the MSE regression model is 70.89%;the π·π π‘ππ‘ of the Variance-based weight model is 78.89%.Those show that the π-SVMR ensemble model is closeto their financial series data for S & P500 time seriesforecasting.
From Tables 4 and 5, the results show the ensemble modelperforms the best for Nikkei 225 time series data forecasting.we can conclude that
(1) in all the ensemble methods the π-SVMR based ensem-ble model performs the best, followed by the neural networkbased ensemble method and other three linear ensemblemethod from a general view and;
(2)the nonlinear ensemble methods including RBFβNNβbased and π-SVMR-based method outperform all the linearensemble methods, indicating that the nonlinear ensemblemethods are more suitable for financial time series forecast-ing than the linear ensemble approaches due to high volatilityof the financial time series.
IV. CONCLUSIONS
Stock market faces a complex external environment ofrapid change. Because of the factors of uncertainty in fore-casting increasing, forecasting from single prediction modelcan not achieve the ideal effects from purely linear or nonlin-ear models due to the nonβextensive information sources andnonβsensitivity to the model set. In this study, we proposea novel nonlinear RBF neural network ensemble predictorbased on π-SVMR for financial time series forecasting. Ex-amples of calculation shows that the method can significantlyimprove the systemβs predictive ability, prediction accuracy,and with a high prediction accuracy of the rising and fallingtrend of the stock market. Empirical results obtained revealthat the proposed nonlinear combination technique is a verypromising approach to financial time series forecasting.
REFERENCES
[1] E. H. Francis and L. J. Chao, β Modified support vector machine infinaancial time series forecasting,β Neurocomputing, vol. 48, pp. 847-861, 2002.
[2] L. J. Cao and E. H. Francis, βSupport vector machine with adaptiveparameters in financial time series forecasting,β IEEE Transactions onNeural Networks, vol. 14, no. 6, pp. 15063-1518, 2003.
[3] K. A. Smith and N. D. Gupat Jatinder, βNeural Networks in business:techniques and applications for the operations researcher,β Computers& Operations Research, vol. 27, pp. 1023-1044, 2000.
[4] G. P. Zhang and V. L. Berardi, β Time series forecasting with neural net-work ensembles: an application for exchange rate prediction,β Journalof the Operational Research Society, vol. 52, pp. 652-664, 2001.
[5] K. K. Lai, L. Yu, W. Huang and S. Y. Wang, βA novel support vectormachine metamodel for business risk identification,β In: Q. Yang andG. Webb, (eds.) PRICAI 2006. Lecture Note in Computer Science, vol.4099, pp. 980-984, Springer, Heidelberg, 2006.
[6] L. Yu, S. Y. Wang and K. K. Lai, βA novel nonlinear ensembleforecasting model incorporating GLAR and ANN for foreign exchangerates,β Computers & Operations Research, vol. 32, pp. 2523-2541,2005.
[7] A. Jain and A. M. Kumar, βHybird neural network models for hy-drologic time series foreasting,β Applied Soft Computing, vol. 7, pp.585-592, 2007.
[8] Jiansheng Wu, βA novel nonparametric regression ensemble for rainfallforecasting using particle swarm optimization technique coupled withartificial neural network,β Lecture Note in Computer Science, vol. 5553,no. 3, pp. 49-58, Springer-Verlag Berlin Heidelberg, 2009.
[9] P, Sollich and A. Krogh, βLearning with Ensembles: How Over-fittingcan be useful,β In: Touretzky D, Mozer M, Hasselmo,M.(eds). Advancesin Neural Information Processing Systems 8, Cambridge, MA: MITPress, pp. 190-196, 1996.
[10] J. Mao, βA case study on bagging boosting and basic ensemblesof neural networks for OCR,β In Processing of International JointConference on Neural Networks-98, Anchorage, AK, vol. 3, pp. 1828-1833, 1998.
[11] Z. Zhou, J. Wu and W. Tang, βEnsembling neural networks: Manycould be better than all,β Artificial Intelligence, vol.137, no.1-2, pp.239-263, 2002.
[12] J. Carney and P. Cunningham,, βTuning diversity in bagged ensem-bles,β International Journal of Neural Systems, vol. 10, pp. 267-280,2000.
[13] B. E. Rosen, βEnsemble learning using decorrelated neural networks,βConnection Science, vol. 8, pp. 373-384, 1996.
[14] D. S. Broomhead and D. Lowe, βMultivariable functional interpolationand adaptive networks,β Complex Systems, vol. 2, pp. 321-355, 1988.
[15] P. K. Dash, A. K. Pradhan and G. Panda,, βApplication of minimalradial basis function neural network to distance protection,β IEEETransactions on Power Delivery, vol. 16, no. 1, pp. 68-74, 2001.
[16] Z. Moravej, D. N. Vishwakarma and S. P. Singh, βApplication ofradial basis function neural network for differential relaying of a powertransformer,β Computers and Electrical Engineering, vol. 29, pp. 421-434, 2003.
[17] A.M.S. Barreto, H. J. C. Barbosa and N. F. F. Ebecken, βGOLSβGenetic orthogonal least squares algorithm for training RBF networks,βNeurocomputing, vol. 69, pp. 2041-2064, 2006.
[18] L. Breiman, βCombining Predictors,β Proceedings of Combining Artifi-cial Neural Nets-Ensemble and Modular Multi-net Systems, the SpringerPress, Berlin, vol. 1, pp. 31-50, 1999.
[19] J. A. Benediktsson, J. R. Sveinsson, O. K. Ersoy and P. H. Swain,βParallel Consensual Neural Neural Networks,β IEEE Transactions onNeural Networks, vol. 8, pp. 54-64, 1997.
[20] L. Breiman, βBagging predictors,β Machine Learning, vol. 24, pp. 123-140, 1996.
[21] A. Krogh and J. Vedelsby, βNeural network ensembles, cross valida-tion, and active learning,β G. Tesauro, D. Touretzky and T. Leen (Eds.),Advances in Neural Information Processing Systems, vol. 7, pp. 231-238, MIT Press, Cambridge, MA, 1995.
[22] R. E. Schapire, βThe strength of weak learnability,β Machine Learning,vol. 5, pp. 197-227,1990.
[23] D. M. Pirouz. βAn overview of partial least square,β Technical report,The Paul Merage School of Business, University of California, Irvine,2006
[24] C. Cortes and V. Vapnik, βSupport vector networks,β Machine Learn-ing, vol.20, pp.273-295, 1995.
[25] B. Holkopf, K. Sung and C. J. C. Burges, β Comparing support vectormachine with Gaussian kernel to radial basis function classifiers,β IEEETranslated Signal Processing, vol.45, pp.2758-2765, 2001.
[26] P. L. Bartlett and J. S. Taylor, βGeneralization performance on supportvector machines and pattern classifiers,β B. Sholkopf, C. Burges, andA. Smola Eds., Advances in Kernel Methods-Support Vector Learn-ing,Cambridge, MA: MIT Press, 1999.
[27] B. Schokopf and A. J. Smola, βGeneralization performance on supportvector machines and pattern classifiers,β B. Sholkopf, C. Burges, andA. Smola Eds., Advances in Kernel Methods-Support Vector Learn-ing,Cambridge, MA: MIT Press, 1999.
[28] C. C. Chang and C. J. Taylor, Learning with kernels,Cambridge, MA:MIT Press, 2002.
[29] P. L. Bartlett and J. S. Lin, βTraining π-support vector regression:theory and algorithms,β Neural Computation, vol.14, 1959-1977, 2002.
[30] K. J. Kim, βFinancial time series forecasting using support vectormachines,β Neurocomputing, vol. 55, pp. 307-319, 2003.
90