prediction of foreign exchange rates by neural … · prediction of foreign exchange rates by...

Prediction of foreign exchange rates by neuralnetwork and fuzzy system based techniques

V. Kodogiannis, A. Lolis*

University of Westminster, Dept. of Computer Science, London, HA1 3TP, U.K*University of Greenwich, Dept. of Engineering, Chatham, ME4 4TB

Abstract: Forecasting currency exchange rates are an important financial problem thatis receiving increasing attention especially because of its intrinsic difficulty andpractical applications. This paper presents improved neural network and fuzzy modelsused for exchange rate prediction. Several approaches including multi-layerperceprtons, radial basis functions, dynamic neural networks and neuro-fuzzy systemshave been proposed and discussed. Their performances for one-step a-head predictionshave been evaluated through a study, using real exchange daily rate values of the USDollar vs. British Pound.

1. Introduction

An estimation problem of particular importance in the field of financial engineering isthe problem of forecasting or predicting trends in the foreign exchange market. Theforecasting of exchange rates is actually a very difficult task because of the manycorrelated factors that get involved.Many techniques have been proposed in the last few decades for exchange rateprediction. The drawbacks of the linear methods as well as the development ofartificial intelligence, have led to the development of alternative solutions utilisingnon-linear modelling. Two of the forecasting techniques that allow for the detectionand modelling of non-linear data are fuzzy systems and neural networks [1-2]. Neuralnetworks (NNs) have recently gained popularity as an emerging and challengingcomputational technology and they offer a new avenue to explore the dynamics of avariety of financial applications. However, NNs as models for forecasting exchangerates have been investigated in a number of previous studies. The main characteristicof these studies is the use of a simple network using the basic BP algorithm fortraining. Additionally, weekly data have been used although it is well known that suchdata contain substantially less noise and are less volatile than real daily data.In this paper we present algorithms that trace previous “currency” patterns and predicta “currency” pattern using recent data. The datasets of two currencies studied in thisresearch comprise 1000 daily rates from the end of 1997 to the end of March 2000.For a univariate time-series forecasting problem, the inputs of the network are the pastlagged observations of the data series and the outputs are the future values. Each inputpattern is composed of a moving window of fixed length along the series. In thissense, the feedforward network used for time series forecasting is a generalautoregressive model. The balance of this paper contains a comparative study of

ESANN'2001 proceedings - European Symposium on Artificial Neural NetworksBruges (Belgium), 25-27 April 2001, D-Facto public., ISBN 2-930307-01-3, pp. 245-250

various prediction techniques used to develop a forecasting tool for exchange rateprediction.

2. Non-linear Modelling

Some artificial neural network architectures exhibit the capability of forming complexmappings between input and output that enable the network to approximate generalnon-linear mathematical functions. The multi-layer perceptron (MLP) neural network,trained by the standard back-propagation (BP) algorithm, is probably the most widelyused network and its mathematical properties for non-linear function approximationare well documented [3]. In order to provide sufficient information for modellingusing an MLP, a structure with two hidden layers and 5 inputs was used.

2.1 Radial Basis Functions

An alternative model to the multilayer networks for the time series identification isthe neural network employing radial basis functions (RBFs). An RBF is a functionwhich has in-built distance criterion with respect to a centre. The present study adoptsa systematic approach to the problem of centre selection. Because a fixed centrecorresponds to a given regressor in a linear regression model, the selection of RBFcentres can be regarded as a problem of subset selection. The orthogonal least squares(OLS) method can be employed as a forward selection procedure that constructs RBFnetworks in a rational way. OLS learning procedure generally produces an RBFnetwork smaller that a randomly selected RBF network [4]. Due to its linearcomputational procedure at the output layer, the RBF is faster in training timecompared to its BP counterpart.

A major drawback of this method is associated with the input spacedimensionality. For large numbers of inputs units, the number of radial basisfunctions required, can become excessive. To avoid this problem, an efficientcombination of the zero-order regularisation and the OLS algorithm proposed byChen et al [5]. In our case, the ROLS algorithm was employed to model the exchangerate problem. Best results were obtained using 5 past previous values as inputs.

2.2 Autoregressive recurrent neural network

Recurrent neural networks have important capabilities such as attractor dynamics andthe ability to store information for later use. A new model called the autoregressiverecurrent network (ARNN), which can converge in reasonable training time, isproposed and a generalised BP algorithm is developed to train it. The idea is to use arecurrent neural network model but the recurrent neurons are decoupled so that eachneuron only feedbacks to itself. With this modification, the ARNN model isconsidered to converge easier and to need less training cycles than the fully recurrentnetwork [6]. The ARNN is a hybrid feedforward / feedback neural network, with the


feedback represented by recurrent connections appropriate for approximating thedynamic system. The structure of the ARNN is shown in Fig. 1.

Fig. 1: ARNN architecture

There are two hidden layers, with sigmoidal transfer functions, and a single linearoutput node. The ARNN topology allows recurrency only in the first hidden layer. Forthis layer, the memoryless backpropagation model has been extended to include anautoregressive memory, a form of self-feedback where the output depends also on aweighted mum of previous outputs. The mathematical definition of the ARNN is

shown below:

(t)j

HjlW=lS),(flQ(t),

l=O(t)=y(t) jZlSlQO

lW �=� (1)

and

�� −i

Iij

n=k

1=k

Djkjjj W+)(W=)(H,))(f(H=)(Z ij IktZttt (2)

where Ii(t) is the ith input to ARNN, Hj(t) is the sum of inputs to the jth recurrentneuron in the first hidden layer, Zj(t) is the output of the jth recurrent neuron, Sl(t) isthe sum of inputs to the lth neuron in the second hidden layer, Ql(t) is the output of thelth neuron in the second hidden layer and O(t) is the output of the ARNN. Here, f(• ) isthe sigmoid function and WI, WD, WH and WO are input, recurrent, hidden and outputweights, respectively. The memories in each node at the first hidden layer allow thenetwork to encode state information. The ARNN was trained as a one-step-aheadprediction model for the currency exchange using a structure of 4/16/8/1 nodes.

2.3 ELMAN Network

The recurrent network developed by Elman has a simple architecture and it can betrained using the standard BP learning algorithm. This network has been proved to beeffective for modelling linear systems not higher than the first order. For this reason, amodified Elman network that is shown in Fig. 2 has been developed. Here, self-connections are introduced in the context units of the network in order to give theseunits a certain amount of inertia. The introduction of self-feedback in the context unitsincreases the possibility of the Elman network to model high-order systems. Thus theoutput of the jth context unit in the modified Elman network (M.ELMAN) is given by

5)-(txα+)4(xα+3)-(txα+)2(xα+)1(xα+)(x=)1( j5

j4

j3

j2

jj −−−+ tttttxcj (3)


Usually, α is between 0 and 1. A value of α nearer to 1 enables the network to tracefurther back into the past. The five ‘memories’ in each node at the context layer allowthe network to encode state information [7].

Fig. 2. M.ELMAN architecture

In order to enhance network’s performance an extra hidden layer has been added, andthe linear output function was replaced with a standard sigmoidal one. Therefore a4/16/24/1 Modified Elman network was applied with self-feedback in the 16 contextunits, with α equal to 0.25.

2.4 Adaptive Fuzzy Logic System (AFLS)

The various neural architectures presented in previous sections illustrated theirstrength for modelling the exchange rate forecasting problem. Recently, theresurgence of interest in the field of NNs has injected a new driving force into the‘fuzzy’ literature. An adaptive fuzzy logic system (AFLS) is a fuzzy logic systemhaving adaptive rules. Its structure is the same as a normal FLS but its rules arederived and extracted from given training data. In other words, its parameters can betrained like a neural network approach, but with its structure in a fuzzy logic systemstructure. Since we have general ideas about the structure and effect of each rule, it isstraightforward to effectively initialise each rule. This is a tremendous advantage ofAFLS over its neural network counterpart. The AFLS is one type of FLS with asingleton fuzzifier and a defuzzifier. The centroid defuzzifier cannot be used becauseof its computation expense and that it prohibits using the error BP-training algorithm.The proposed AFLS consists of a new defuzzification approach, balance of area(BOA) [8]. The proposed MIMO-AFLS has a feedforward structure as shown in Fig.3 with an extra “fuzzy basis” layer. In general form, the calculation of the output, y,will be

�

�

=

==M

m

mpm

M

m

mp

mpm

p

L

yL

y

1

1

µ

µ(4)

where yp : the pth output of the network, µm : the membership value of the mth

rule, mpL : the spread parameter of the membership function in the consequent part of


the pth output of the mth rule, mpy : the centre of the membership function in the

consequent part of the pth output of the mth rule.

Fig.3. AFLS architecture

For a Gaussian shaped membership function, the BOA defuzzifier gives results closerto the COA’s than other defuzzification methods.

3. Results

Several structures of neural networks with algorithms ranging from backpropagationlearning to Neuro-fuzzy methods were tested. The results and the statistics offorecasts obtained from the application of the developed neural and fuzzy models forthe one step ahead exchange rate prediction of US$ and GBP£. The complete resultsfor the exchange rate prediction problem of forecast are illustrated in Table 1.

BP RBF ARNN ELM AFLS% Relative error 0.31 0.2646 0.2711 0.2683 0.2514

Root mean square error 0.421 0.3905 0.4021 0.4010 0.3939Standard error deviation 0.0025 0.0023 0.0024 0.0024 0.0024

Table 1: ResultsThe widely used for such an application, standard MLP with BP learning algorithm,was considered in this work as a testbed case. The introduction of hybrid learningalgorithms imposed a new dimension to this specific problem. The main advantagesof the proposed AFLS method are the ability to learn from experience and a highcomputation rate. These results are illustrated in Fig. 4.The balance of area defuzzifier (BOA) uses the shape information of fuzzymembership functions in the consequence part of the IF-THEN rules to obtain theresult. Its output is close to the centroid of area defuzzification (COA) while requiringmuch less computation.With these choices of components, the AFLS can be trained by several trainingalgorithms such as error backpropagation or genetic algorithms. In this research thedesigned AFLS was trained by an error backpropagation algorithm.


Fig. 4: Prediction using AFLS-BOA system

4. Conclusions

This study is based on the comparative analysis of neural network and fuzzy systems.These methods were developed for a one-step-ahead prediction of US$ and GBP£daily exchange rates. Several neural architectures were tested including, multilayerperceptrons, fuzzy-neural-type networks, radial basis and memory neuron networks.The introduction of hybrid learning algorithms imposed a new dimension to exchangerate prediction. The main advantages of the proposed AFLS with the inclusion of aninnovative defuzzification method are the ability to learn from experience and a highcomputation rate. In a future work, the present approach will be enhanced by usingadvanced neuro-fuzzy models to develop multiple step ahead prediction exchange ratesystems and with the inclusion of additional features, such as interest rates, oil prices.

References

1 G. Bojadziev, M. Bojadziev: Fuzzy logic for Business, Finance, andManagement. World Scientific, (1997).

2 A. Refenes, D. Bunn, Y. Bentz, A. Zapranis: Financial Time Series ModellingWith Discounted Least Squares Backpropagation. Neurocomputing 14, 123-138,(1997).

3 S. Haykin: Neural Networks A Comprehensive Foundation. Macmillan CollegePubl. Company, New York, (1994).

4 S. Chen, C. Cowan, P. Grant: Orthogonal Least-Squares Algorithm for RadialBasis Function Networks. IEEE Transactions on Neural Networks, 2, 2, 302-309(1991).

5 S. Chen, E. Chang, K. Alkadhimi: Regularised Orthogonal Least SquaresAlgorithm for Constructing Radial Basis Function Networks. Int. J. Control, 64,5, 829-837 (1996).

6 V.S. Kodogiannis, E.M Anagnostakis: A study of Advanced LearningAlgorithms for Short-Term Load Forecasting. Engineering Applications of AI,12, 2, 159-173 (1999).

7 V.S. Kodogiannis: Comparison of advanced learning algorithms for short-term loadforecasting. Journal of Intelligent & Fuzzy Systems, 8, 4, 243-261 (2000).

8 A. Lolis: A Comparison Between Neural Network and Fuzzy System BasedTechniques for Exchange Rate Prediction. MSc dissertation, Univ. of Greenwich,UK, (2000).

0

0.2

0.4

0.6

0.8

116

31

46

61

76

91

106

121

136

151

166

181

196

Patterns

Values


prediction of foreign exchange rates by neural … · prediction of foreign exchange rates by...

Documents