a multivariate forecasting method for short-term load using chaotic features and rbf neural network

A multivariate forecasting method for short-term load usingchaotic features and RBF neural network

Yuming Liu1*,y, Shaolan Lei2, Caixin Sun1, Quan Zhou1 and Haijun Ren1

1The State Key Laboratory of Power Transmission Equipment and System Security and New Technology,

Chongqing University, Chongqing 400044, China2Chongqing University of Technology, Chongqing 400050, China

SUMMARY

This paper presents a multivariate forecasting method for electric short-term load using chaos theory andradial basis function (RBF) neural networks. To apply the method, the largest Lyapunov exponent andcorrelation dimension are firstly calculated which show the electric load series is essentially a chaotic timeseries. Then, a multivariate chaotic prediction method is proposed taking historical load and temperature intoaccount. Phase space reconstruction of a univariate time series is extended to construct a multivariate timeseries. Delay time and embedding dimension of the historical load series and temperature series aredetermined by mutual information and minimal forecasting error, respectively. Finally, a three-layer RBFneural network is employed to forecast the load of one day ahead and one week ahead. Real load data ofChongqing Power Grid are tested. Daily and weekly forecasting results show that the proposed multivariateapproach improves the accuracy of forecasting significantly comparing with the univariate methods.Discussion of forecasting error and future work are also presented. As an efficient and effective alternativefor STLF, the chaos theory based multivariate forecasting is feasible for potential application. Copyright#2010 John Wiley & Sons, Ltd.

key words: chaotic feature; electric short-term load forecasting; multivariate time series; radial basisfunction neural network

1. INTRODUCTION

Electric short-term load forecasting (STLF) is to predict future load ranging from several hours to

several weeks depending on the time scale of interest. As an important routine for power system

dispatch department, STLF has significant impact on real-time control and economic scheduling.

Accurate and reliable forecasting result in appropriate operational and planning strategy, and

ultimately achieve lower operational cost, higher reliability, and better quality of power supply. With

the current trend of deregulation, utilities have become more concerned to the accuracy of STLF than

ever before[21].

Reported methods can roughly be classified into traditional methods and modern artificial

intelligence (AI) based approaches. Based on statistical theory, the former methodology extracts

underlying statistical regularities contained in load series and deduces formulas as well as parameters

for load forecasting. These kinds of forecasting methods, e.g., regressive models [1-2], time series

analysis [3-7], have become mature for STLF. An excellent discussion of traditional STLF models is

reported in Reference [8], and comparison of methods using regional load data, e.g., European data and

Brazilian data, can be found in References [9] and [10], respectively. Short-term load is significantly

influenced by different environmental factors such as seasonal change, weather condition, equipment

EUROPEAN TRANSACTIONS ON ELECTRICAL POWEREuro. Trans. Electr. Power 2011;21:1376–1391Published online 28 September 2010 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/etep.502

*Correspondence to: Yuming Liu, The State Key Laboratory of Power Transmission Equipment & System Security and

New Technology, College of Electrical Engineering, Chongqing University, Chongqing 400044, China.yE-mail: [email protected]

Copyright # 2010 John Wiley & Sons, Ltd.

failure and maintenance, and big events [11]. Most of the statistical methods are based on linear

analysis and difficult to reveal the relationship between load change and influence factors. Besides,

load series is usually a complicated nonlinear function of exogenous variables and behaves

nonlinearity and nonstationarity. Aiming at these features, AI is introduced offering better forecasting

results. Neural network (NN) [12-13] and fuzzy modeling [14-17] represent two main AI methods due

to strong nonlinear mapping ability and deductive capability. The NNs not only learn the load profiles

but model the complicated nonlinear relationship between load and influence factors, while the fuzzy

methods deal with uncertainty of load and extract similarity contained in massive data, which is the just

thing required by STLF. Hybrid methods incorporating NN with fuzzy logic have also been proposed.

Reference [18] presents a hybrid correction method in which fuzzy logic is used to modify the output of

NN so as to predict the next day load accurately; a series of combinatorial methods can be found in

References [19–21]. Moreover, optimization techniques are bringing forward to optimize the structure

and parameters of NNs during training process, e.g., in Reference [22] genetic algorithm (GA) is

reported to obtain optimal parameters of NN considering different day types and weather information;

subsequently, membership functions and rule numbers are generated automatically. Other optimization

methods, such as simulated annealing, particle swarm optimization have been introduced in

References [23] and [24]. Besides the NN variant methods, support vector machines [25,26] and

Bayesian modeling [27] have also reported excellent forecasting performance.

Just as mentioned before, load data exhibit complicated behavior. Better understanding of the load

dynamics and statistical properties will be a perspective to implement appropriate models. With the

development of phase space reconstruction and chaos dynamics, complexity of a nonlinear time series

has been given new explanation. For a chaotic time series, long-term evolution is unpredictable.

However as for short-term evolution, it is feasible to predict since orbit divergence in phase space is

small within limited time range. Thus this short-term predictability is potential for STLF. Chaotic

forecasting method fulfills short-term prediction by using self-similarity of chaotic attractors in

different levels. When the method is applied, phase space reconstruction is usually adopted to restore

the original chaotic series approximately, which avoids learning the relationship between load and its

influencing factors. Work of Reference [28] is the earlier exploration of chaotic forecasting method for

STLF in which the Lyapunov spectrum analysis provides a good guideline for input variables selection,

and a MLP realizes one-step ahead prediction; whereas in Reference [29] correlation dimension and

the largest Lyapunov exponent are used to determine the input variables of ANN; in Reference [30]

authors use the Grassberger-Procaccia (GP) algorithm and least squares regression method to obtain

the correlation dimension, and employ a fuzzy neural network for load prediction. To make the

differences, this paper analyzes chaotic features contained in electric load and proposes a multivariate

method for STLF. Based on the underlying chaotic features of the load data, phase space reconstruction

of a univariate time series is extended to reconstruct a multivariate time series. Delay time and

embedding dimension of the load and temperature data series are also discussed respectively. Finally,

ability of function approximation of a radial basis function (RBF) neural network is adopted to map

evolutionary behavior in the reconstructed phase space to fulfill the forecasting process.

The rest of the paper is organized as follows. Section 2 verifies the chaotic characteristic contained in

a regional load series, and the Section 3 establishes the multivariate time series reconstruction.

Section 4 describes the RBF structure and training procedures while the Section 5 presents application

examples. Finally, conclusions are drawn in Section 6.

2. CHAOTIC CHARACTERISTIC OF ELECTRIC SHORT-TERM LOAD

In load forecasting modeling, mathematical characteristic of load data is the key factor for model

accuracy. Long time observation indicates the electric short-term load presents multivariate dynamic

evolutionary behavior and multilevel structures, and is prone to be influenced by internal and external

factors. Different from the external factors, such as weather, major holidays, spot price, policy, and

economic situation etc., internal factors refer to deterministic nonlinear factors contained in load

series. Therefore, electric load is the result of two kinds of factors, and the accuracy of forecasting is not

only influenced by extraneous factors but mainly determined by dynamics characteristic implied in

Copyright # 2010 John Wiley & Sons, Ltd. Euro. Trans. Electr. Power 2011;21:1376–1391DOI: 10.1002/etep

A MULTIVARIATE FORECASTING METHOD FOR SHORT-TERM LOAD 1377

system itself. Because of various influencing factors and highly nonlinearity of power system, the

electric load seems to be of random features. However, chaos theory and phase space reconstruction

provide a novel approach for nonlinear time series analysis and rectify this misconception. It reveals

that the electric load series, essentially, exhibit deterministic chaotic dynamic behavior.

2.1. Preprocessing of load series

In practice, nonlinear dynamics theory set high requirements to a time series, e.g., phase space

reconstruction based deterministic chaotic analysis require noise-free and smooth time series, which is

difficult to achieve for power systems. Due to various interferences original data usually contain

pseudo-data which will make attractors in reconstructed phase space dispersive and unsmooth, and

subsequently influence the accuracy and quality of forecasting. Considering the fact that historical load

data are collected off-line, preprocessing of the load data focuses on outlier data elimination and

normalization. The former mainly use three-point mean method to adjust load value, and is expressed

as

xi ¼ xi�1 þ xiþ1

2(1)

where xi is supposed as an outlier and xi stands for adjusted value. The normalization can be expressed

as follows:

Supposed fxk : k ¼ 1; 2; � � � ; 2160g is a load series observedwith even interval, the normalization is:

yi ¼ xi�xmin

xmax�xmin

; i ¼ 1; 2; � � � ; 2160 (2)

where xmin ¼ minfxjg, xmax ¼ maxfxjgReal load data from January1 to March 31, 2003 of Chongqing Power Grid are collected. In order to

demonstrate chaotic characteristic contained in load series, a randomly generated time series with the

same length which satisfy the standard normal distribution is introduced. Two time series are shown as

Figures 1 and 2, respectively.

One of the most important aspects of load analysis is to identify the underlying mathematical

characteristic of load data, which is the foundation of forecasting. If chaotic features can be verified

from load data, short-term predictability of chaotic time series can be used for STLF. In the following

paragraphs, we will calculate the largest Lyapunov exponent and correlation dimension to

quantitatively describe the chaotic characteristic of the load series.

1000

1500

2000

2500

3000

Hou

rly lo

ad/M

W

0 500 1000 1500 2000Time/h

Figure 1. Electric load from January 1 to March 31, 2003.


1378 Y. LIU ET AL.

2.2. Deterministic validation of short-term load series

Chaotic behavior is a bounded and non-periodic motion which is superimposed of finite periodic

motions with different frequency. Its power spectrum is consecutive with spikes and of random

characteristics. Power spectrum of the load and the simulated time series are shown in Figures 3 and 4,

respectively. In these two figures, both of them are of continuous spectrum which means the load series

are of similar random characteristic compared with the random time series. However, power spectrum

analysis can only determine a random motion and is not capable to show the reasons behind

randomness—whether it is due to external disturbances or inner deterministic behavior in system

itself. Thus, it can infer that the load series may have other characteristic besides this superficial

randomness and further identification is needed. Here the largest Lyapunov exponent is calculated for

justification purpose, which has been proven to be a distinct feature for a chaotic system.

Lyapunov exponent is the average exponential rate of divergence or convergence of nearby orbits in

phase space, which describes the sensitivity of chaotic behavior to initial states. Since the nearby orbits

correspond to two near states, exponential orbit divergence means that a system whose initial

differences we may not be able to solve will soon evolve quite differently with each other. Value of the

Lyapunov exponent will discriminate a chaotic system from other systems. Any system possessing at

least one positive Lyapunov exponent is believed to be chaotic with the magnitude of the exponent

0 500 1000 1500 2000

-2

0

2

Time/h

Am

plitu

de

Figure 2. Simulated random time series.

0.1 0.2 0.3 0.4 0.5-5

0

5

10

15

20

x 104

Frequency/kHz

Pow

er S

pect

rum

Figure 3. Power spectrum of load series.



reflecting time scale in which system dynamics become unpredictable [31]; zero value indicates

periodic motions while negative value implies a system will converge on a balance point in future.

In numeric calculation, only the largest Lyapunov exponent is needed. A well-known method

proposed byM. T. Rosenstein [32] is adopted here. To obtain the exponent in framework of phase space

reconstruction, two parameters, namely, delay time and embedding dimension, should be determined

firstly since they link directly to the quality of phase space and forecasting accuracy. Mutual

information [33] and the Cao’s method [34] are used here to calculate these two parameters and briefly

presented in the following:

For two systems A and B, there are two measurements presented by aj and bk with probability PAðajÞand PBðbkÞ, respectively; the joint probability is PABðaj; bkÞ. Mutual information quantitatively

describes the mutual influence between system A and B, and is defined by:

IAB ¼Xaj; bk

PABðaj; bkÞln PABðaj; bkÞPAðajÞPBðbkÞ (3)

As for a time series fxk : k ¼ 1; 2; � � �Ng and its time-delayed series fxkþt : k ¼ 1; 2; � � �Ng, wheret is delay time, xi and xiþt is observed with probability PðxiÞ and PðxiþtÞ, and the joint probability is

Pðxi; xiþtÞ. Then, mutual information is the function of delay time t:

IðtÞ ¼XNi¼1

Pðxi; xiþtÞln Pðxi; xiþtÞPðxiÞPðxiþtÞ (4)

The function measures the dependence of consecutivemeasurements, and delay time t is determined

by finding the first minimal value of IðtÞ.In Cao’s method, delay time should be selected in advance, and aði;mÞ is firstly defined by:

aði;mÞ ¼ Xiðmþ 1Þ�Xnði;mÞðmþ 1Þ�� XiðmÞ�Xnði;mÞðmÞ

�� (5)

where Xiðmþ 1Þ is the i� th vector in a reconstructed phase space withmþ 1 dimension, nði;mÞ is aninteger which enables Xnði;mÞðmÞ to be the nearest neighbor of XiðmÞ in them dimensional reconstructed

phase space; nði;mÞð1 � nði;mÞ � Nm�mLÞ is dependent on i and m, �k k represents Euclidean

distance. Then, average value of aði;mÞ and variation from m dimension to mþ 1 dimension are

0.1 0.2 0.3 0.4 0.5

-50

0

50

100

150

200

Frequency/ kHz

Pow

er S

pect

rum

Figure 4. Power spectrum of simulated series.


1380 Y. LIU ET AL.

defined by:

EðmÞ ¼ 1

N�mt

XN�mt

i¼1

aði;mÞ (6)

E1ðmÞ ¼ Eðmþ 1ÞEðmÞ (7)

Besides, another parameter is defined in Cao’s method to discriminate chaotic series and random

series, that is

E�ðmÞ ¼ 1

N�mt

XN�mt

i¼1

xiþmt�xnði;mÞþmt

�� (8)

E2ðmÞ ¼ E�ðmþ 1ÞE�ðmÞ (9)

For a chaotic time series, mþ 1 will be selected as the smallest embedding dimension if E1ðmÞbecomes saturated with the increase of m. In practice, however, E1ðmÞ of a random time series may

also become saturated whenm is increasing. Thus E2ðmÞ is defined to discriminate this situation. For a

random time series, E2ðmÞ is always around one for allm, whereas for a chaotic series, the value cannotremain constant for allm because E2ðmÞ is a function ofm. It means E2ðmÞ 6¼ 1 will hold true for some

m for a chaotic time series.

The largest Lyapunov exponent can be calculated when delay time and embedding dimension are

selected. Detailed information of calculation is referred to Reference [35]. Delay time, embedding

dimension, and the largest Lyapunov exponent are shown in Figures 5–7, respectively.

In Figure 5 the first minimum of mutual information function appears at 7 in the horizontal axis.

Hence, delay time of the load series t is 7; result of embedding dimension in Figure 6 shows that E1ðmÞ(real line) increases gradually and tends to be saturated when the embedding dimension m is

increasing. When m becomes 10, variation of E1ðmÞ is minimal. Here the embedding dimension is

chosen as m ¼ 11; E2ðmÞ (dashed line) in Figure 6 is not constant but fluctuates around 1, which also

indicates the load series is chaotic time series.

It should be pointed that delay time and embedding dimension selected here are optimal and not

necessary the best value. This is not due to the methods but the original data. Theoretically, noise-free

data is a prerequisite of complete approximation of attractors. In practice, however, it is impossible to

collect noise-free data and thus the obtained delay time and embedding dimension are approximation

of the best value.

0 10 20 30

3.2

3.4

3.6

3.8

4

Mut

ual i

nfor

mat

ion

Delay Time

Figure 5. Delay time using mutual information.



In Figure 7, k is evolutionary step of discrete time and yðkÞ stands for the mean logarithm distance

after k steps evolution for all the nearest points. When k is less than 100, the curve of yðkÞ seems like a

straight line with a slope of 0.0126 determined by least square technique. The slope represents the

largest Lyapunov exponent of load series and it is a positive value which strongly demonstrates chaotic

behavior existing in the electric load series.

Moreover, the G-P algorithm [36] is used to calculate correlation dimension of chaotic attractor. The

value is 2.89, a non-integer, which also proves that the load series is of chaotic features. Therefore,

short-term predictability of chaotic time series can be applied for STLF.

3. MULTIVARIATE TIME SERIES RECONSTRUCTION METHOD

Reconstruction of a univariate time series is a common approach in chaotic forecasting. For univariate

prediction, only observed load data are used for dynamic forecasting modeling. As long as the delay

time and embedding dimension are appropriately chosen, a univariate time series can theoretically

reconstruct phase space well and achieve perfect results. In real application, however, the collected

load data contains noise and is of finite length. Consequently, the phase space reconstructed by a

univariate time series cannot accurately describe orbit evolution of state change in a dynamic system.

Moreover, it is unknown whether a univariate series contains adequate information of a constructed

5 10 15 20

0.2

0.4

0.6

0.8

1

Embedding Dimension

E1(m)

E2(m)

E(m

)

Figure 6. Embedding dimension using Cao method.

100 200 300

0

0.2

0.4

0.6

0.8

1

k

y(k)

Figure 7. The maximal Lyapunov exponent cure of load time series.


1382 Y. LIU ET AL.

dynamic system. Compared with the univariare series, a multivariate series contains richer information

and is capable of reconstructing more accurate phase space. Hence a multivariate forecasting method is

developed in this paper.

3.1. Phase space reconstruction of a multivariate time series

The application of chaos theory to nonlinear time series is based on phase space reconstruction. To

some extent, a univariate time series can be considered as a special case of a multivariate time series.

Hence the reconstruction of a univariate series can be extended to a multivariate series. For a given time

series fxk : k ¼ 1; 2; � � � ;Ng, it can be embedded into Euclid space according to Takens’ theory [37].

Phase points can be expressed as:

X1þðm�1Þt ¼ ðx1þðm�1Þt; x1þðm�2Þt; . . .; x1Þ...

Xi ¼ ðxi; xi�t; . . .; xi�ðm�1ÞtÞ...

XN ¼ ðxN ; xN�t; . . .; xN�ðm�1ÞtÞ

8>>>>>><>>>>>>:

(10)

where t and m are delay time and embedding dimension, respectively; N�ðm�1Þt is amount of phase

points, and Xi is one phase point in reconstructed phase space (i ¼ 1þ ðm�1Þt; � � � ;N).As for a M dimensional multivariate time series X1;X2; � � �XN , where Xi ¼ ðx1;i; x2;i; � � � xM;iÞ,

i ¼ 1; 2; � � � ;N. When M equals to 1, the series degenerates to a univariate one. Phase points of a

multivariate time series in time-delayed reconstruction can be expressed as:

Vn ¼ ðx1;n; x1;n�t1 ; :::; x1;n�ðm1�1Þt1 ; :::; xM;n; xM;n�tM ; :::; xM;n�ðmM�1ÞtM Þ...

Vi ¼ ðx1;i; x1;i�t1 ; :::; x1;i�ðm1�1Þt1 ; :::; xM;i; xM;i�tM ; :::; xM;i�ðmM�1ÞtM Þ...

VN ¼ ðx1;N ; x1;N�t1 ; :::; x1;N�ðm1�1Þt1 ; :::; xM;N ; xM;N�tM ; :::; xM;N�ðmM�1ÞtM Þ

8>>>>>><>>>>>>:

(11)

where n ¼ max1�i�M

ðmi�1Þti þ 1, ti and mi (i ¼ 1; 2; � � � ;M) are delay time and embedding dimension,

respectively; m ¼ m1 þ m2 þ � � � þ mM is the total embedding dimension of the multivariate series.

According to Takens’ theory, if m > 2D (D is dimension of attractor), there exists a mapping

F : Rm ) Rm, that is

Vnþ1 ¼ FðVnÞThen, the change of Vn to Vnþ1 reflects evolutionary behavior of the original dynamic system, which

means geometrical characteristic of attractor remains the same in the original dynamic system and in

the m-dimensional reconstructed phase space. Thus, any differential or topological invariate in the

original dynamic system can be calculated in the reconstructed phase space. When m or

m1;m2; � � � ;mM are big enough, it equals to

x1;nþ1 ¼ F1ðVnÞx2;nþ1 ¼ F2ðVnÞ

..

.

xM;nþ1 ¼ FMðVnÞ

8>>><>>>:

(12)

where xi;nþ1; i ¼ 1; 2; � � � ;M is the i� th forecasting value of the reconstructed phase vector Vnþ1.The

value of x1;nþ1; x2;nþ1; . . .; xM;nþ1 can be calculated in the above equation if the mappings

F1;F2; . . .;FM are known. As for STLF, only the load component is of interest, which means

only x1;nþ1 andF1 is required. In the following section, the ability of function approximation of a RBF

neural network is used to approximate the mapping F1.



3.2. Delay time of a multivariate time series

As stated before, delay time and embedding dimension are two essential parameters for phase space

reconstruction. Delay time determines the difference between each component of a state point. Hence

it influences the information contained in a reconstructed phase space. For the multivariate series, delay

time is determined by calculating the mutual information of each sub-series.

Suppose there is a multivariate time series fxi;j; j ¼ 1; 2; � � � ;Ng (i ¼ 1; 2; � � � ;M) with delay time ti,

the sub-series is presented by fxi;jþti ; j ¼ 1; 2; � � � ;Ng; pðxi;kÞ and pðxi;kþtiÞ stand for the probability ofxi;k and xi;kþti appeared in the series fxi;j; j ¼ 1; 2; � � � ;Ng and fxi;jþti ; j ¼ 1; 2; � � � ;Ng. When xi;k and

xi;kþti appear simultaneously, there is a joint probability pðxi;k; xi;kþtiÞ. Then mutual information

function, like formula (4) for a univariate time series, can be defined by:

IðtiÞ ¼X

k¼1!N

pðxi;k; xi;kþtiÞlnpðxi;k; xi;kþtiÞpðxi;kÞpðxi;kþtiÞ

(13)

This function describes dependence of measurements acquired one after the other. The first minimal

value of IðtiÞ will be chosen as the delay time of fxi;j; j ¼ 1; 2; � � � ;Ng. Accordingly, delay time of the

load and temperature series are calculated with the value t1 ¼ 7 and t2 ¼ 2 shown in Figures 8 and 9,

respectively.

3.3. Embedding dimension of a multivariate time series

Forecasting error varies with different embedding dimensions. When embedding dimension is small,

improper embedding will make attractor folding and self-intersection in some place; reconstructed

phase space cannot reflect dynamic characteristics contained in the original system and cause a large

forecasting error. If embedding dimension is approaching the optimal value, forecasting errors

decrease synchronously. However, if it is beyond the optimal value, the error will increase significantly

due to noise magnifying effect of a larger embedding dimension. Hence, optimal embedding

dimension can be determined by minimizing the forecasting error [38]. The procedure is as follows:

step 1 for each time series fxi;1; xi;2; � � � ; xi;Ng i ¼ 1; 2; � � � ;M, determine the delay time ti and range

of embedding dimension mi i ¼ 1; 2; � � � ;M;

step 2 for a given embedding dimension, reconstruct phase space according to formula 11, and

search the nearest neighbor point of Vn using Euclidean distance; the obtained nearest neighbor point

Vj can be expressed as

Vn�Vj

�� ¼ minf Vn�Vik kgi ¼ max

1�i�Mðmi�1Þti þ 1; � � � ;N:ði 6¼ nÞ

n ¼ max1�i�M

ðmi�1Þti þ 1; � � � ;N(14Þ

where �k k stands for the Euclidian norm.

0 10 20 30 40 50 60

2

2.5

3

3.5

4

Delay time/h

Mut

ual i

nfor

mat

ion

Figure 8. Delay time of the load time series.


1384 Y. LIU ET AL.

step 3 label the next evolutionary point of Vj as Vjþ1, use the first component of Vjþ1, x1;jþ1, as the

forecasting value of x1;nþ1, and calculate the mean one-step absolute error Eðm1;m2; � � � ;mMÞ

Eðm1;m2; � � � ;mMÞ ¼ 1N�kþ1

PNj¼k

x1;nþ1�x1;jþ1

�� k ¼ max

1�i�Mðmi�1Þti þ 1

(15Þ

where x1;nþ1 and x1;jþ1 are real value and forecasting result respectively.

step 4 increase the embedding dimension and repeat steps 2 and 3 until all the embedding dimension

are covered;

step 5 determine the optimal embedding dimension according to ðm10;m20; � � �mM0Þ ¼minfEðm1;m2; � � � ;mMÞg.Suppose the maximum embedding dimension of load and temperature series are both 10, then the

mean one-step absolute errors in different dimension pairs are calculated. Results show that the

minimal absolute forecasting error is 0.1748, and the corresponding optimal embedding dimension of

load and temperature shown in Figure 10 are 10 and 3, respectively.

4. ESTABLISHMENT OF FORECASTING MODEL

Sensitive dependence of initial condition is the typical characteristic of chaos, which leads to

unpredictability for long term. Nevertheless, it can be applied for short-term forecasting. This paper

adopts artificial neural network for STLF. Considering the shortcomings of traditional BP network

0 5 10 15 20 25 30

6.1

6.15

6.2

6.25

6.3

Delay time t/h

Mut

ual i

nfor

mat

ion

Figure 9. Delay time of the temperature time series.

2 4 6 8 105

100

0.1

0.2

0.3

0.4

Erro

r

m2

m1

(a)

0 50 1000.1

0.2

0.3

0.4

0.5 (b)

Erro

r

Figure 10. Mean one-step absolute prediction error curve: (a) different embedding dimension and(b) different embedding dimension pairs.



such as slow convergence and local maximum problem, radial basis function neural network (RBFNN)

is employed. RBFNN is a multi-input-single-output feed forward network with three layers. Hidden

layer performs nonlinear transform by mapping inputs into a new space for feature extraction while the

output layer realizes linear combination. Structure of the RBFNN for STLF is shown in Figure 11.

In Figure 11 xi;j is the input variable, where i ¼ 1; 2; � � � ;M, j ¼ max1�i�M

ðmi�1Þti þ 1; :::;N; amount

of input variables is sum of embedding dimension of each sub-series, namely, m ¼ m1 þ m2 þ � � � þmM; wi is weight between the hidden and the output layer and x1;nþ1 stands for output result. Excitation

function used in the hidden layer is the Gaussian function:

Fð Vn�Cj

�� Þ ¼ expð� Vn�Cj

�� 22s2

j

Þ (16)

In formula 16, Vn ¼ ðx1;n; x1;n�t1 ; :::; x1;n�ðm1�1Þt1 ; :::; xM;n; xM;n�tM ; :::; xM;n�ðmM�1ÞtM Þ is the n� th

input of training samples; Cjand sj are center and width parameters of the radial basis function; the

output of network is expressed as:

x1;nþ1 ¼Xkj¼1

wjFð Vn�Cj

�� Þ (17)

where k is number of the hidden layer, and output error can be defined as:

ej ¼ x1;jþ1�Xki¼1

wiFð Xj�Ci

�� Þ (18)

where x1;jþ1 is real load value.

The common method to determine parameters of Gaussian function is the k-means clustering

algorithm. Supposed the k-means algorithm has already finished input sample clustering, and ui stands

for all the samples in the i� th group, then Cj and sj can be expressed as:

Cj ¼ 1Mi

Px2ui

x

s2j ¼ 1

Mi

Px2ui

ðx�CjÞTðx�CjÞ (19)

where Mi is the pattern number in ui.

In order to minimize the forecasting error, total error function is defined

min E ¼ 1

2

XNj¼n

e2j (20)

Figure 11. Structure of RBFNN for STLF.


1386 Y. LIU ET AL.

Gradient descent method is used to adjust the three parameters. Learning rules of basis function

center, width, and weight in output layer are given as follows:

Cjðnþ 1Þ ¼ CjðnÞ�h1@EðnÞ@CjðnÞ

s2j ðnþ 1Þ ¼ s2

j ðnÞ�h2@EðnÞ@s2

jðnÞ

wjðnþ 1Þ ¼ wjðnÞ�h3@EðnÞ@wjðnÞ

(21)

where the revised variables are

@EðnÞ@CjðnÞ ¼ 2wjðnÞ

XNi¼r

eiðnÞF0ð Xi�CjðnÞ�� Þs�2

j ðnÞ½Xi�CjðnÞ�

@EðnÞ@s2

j ðnÞ¼ �wjðnÞ

XNi¼r

ejðnÞF0ð Xi�CjðnÞ�� Þ½Xi�CjðnÞ�T

@EðnÞ@wjðnÞ ¼

XNi¼r

ejðnÞFð Xi�CjðnÞ�� Þ

n is iteration number; F0ð�Þ is differential of Fð�Þ; jmeans hidden layer node (j ¼ 1; 2; � � � ; k);r ¼ max

1�i�Mðmi�1Þti þ 1 stands for initial starting point in phase space reconstruction; h1; h2; h3 are

learning steps within the range of 0 and 1 and had been chosen different values during learning.

The whole procedure stops if convergence condition is satisfied:

Eðnþ 1Þ�EðnÞj j < " (22)

where " is a given permissible error.

5. TESTING EXAMPLE

Twenty-four points load data in each day and temperature data from January 1 to March 31, 2003 in

Chongqing power grid, China, are used to verify the proposed method. This period of data is typical

since load patterns were complicated due to week-long national major holidays, unstable weather

condition and fast growing industrial demands. Different factors influence the forecasting results, and

only the weather factor is chosen for the sake of simplicity. Apparently, variation of temperature leads

to sensitive change of load. Other meteorological factors, e.g., rainfall, wind speed and humidity also

represent the variation of temperature in some degree. Based on the previous research, those

meteorological factors are correlated with each other and can mainly be represented by temperature.

Put simply, only the temperature data is taken into account.

Suppose the load series is X1;X2; � � � ;X2160 where Xi ¼ ðx1;i; x2;iÞ. Xi ¼ ðx1;i; x2;iÞ is combination of

load series x1;i and temperature series x2;i. Load data is preprocessed as presented in Section 2. The

time-delayed phase space reconstruction can be expressed as:

Yn ¼ ðy1;n; y1;n�t1 ; :::; y1;n�ðm1�1Þt1 ; y2;n; y2;n�t2 ; :::; y2;n�ðm2�1Þt2Þ (23)

where n ¼ maxi2f1;2g

ðmi�1Þti þ 1; :::; 2160; yi;n stands for the ith normalized time series;ti and mi are

delay time and embedding dimension respectively.

According to calculation method of delay time and embedding dimension presented in section 3,

four parameters of load and temperature series are obtained as t1 ¼ 7; t2 ¼ 2 and, m1 ¼ 10;m2 ¼ 3.

Total embedding dimension of system is 13, learning rates of h1; h2; h3 are selected as 0.2, 0.01, and 0.1by trial and error, respectively.

Twenty-four hours load data in each day from January to March in 2003 were used as training data.

Daily load in April 1, 2003 are forecasted with the time lag of 1 hour, and relative error is used to

measure accuracy. Two univariate time series prediction methods [35] are presented to compare with



the multivariate methods. It should be pointed that the univariate method here means chaotic local

region forecasting, which use one-order linear regression model for polynomial fitting problem in a m-

dimensional phase space; the only difference between Method 1 and Method 2 lies in the former only

uses the Euclidean distance to find the nearest neighbor point, while the latter combined the Euclidean

distance with correlation degree. At the same time, linear regression in Reference [39] is adopted to

compare with RBFNN in the multivariate framework. The comparison of forecasting errors is shown in

Table I.

From Table I, absolute mean error of univariate and multivariate series are 1.2875, 1.0900, 1.0162,

and 0.9458%, respectively. Forecasting results indicate the multivariate method achieves better

accuracy than the univariate counterparts and the RBFNN obtains a better performance than the linear

regression.

Moreover, one week loads from April 1 to April 7 in the year 2003 are also forecasted using the

proposed method. Table II gives the results where the data represent the ratio of numbers of forecasting

points within a given error range to the total forecasting points. Two numbers correspond to the

multivariate method and the univariate method, respectively. It clearly shows the accuracy of results,

for example in Monday, given an error range less than 2%, the first number is 91.67, which means

91.67% of the total forecasting points possess an error within 2%. From the mean value (last column in

Table II), the multivariate forecasting method obtained a good accuracy, and 94.04% of the total

forecasting results possess an error less than 3%.

Meanwhile, the multivariate method obtains better results than the univariate method for weekend

load forecasting. 91.67 and 87.5% of the forecasting results are within 3% error for Saturday and

Sunday, respectively. During the whole practice, it finds quality of the original load data will influence

the forecasting accuracy. Usually, historical load data are chosen from the SCADA database, and

inevitably include some abnormal data, e.g., spikes, abrupt change of a segment of data, or abnormal

fluctuation of data, etc. Additive noise will make load patterns difficult to be understood. Further in

phase space reconstruction, the noise is prone to be magnified by improper embedding dimension.

Hence, effective data preprocessing is essential for chaotic prediction method, and data preprocessing

Table I. Comparison of forecasting results in April 1, 2003.

Time(hour)

Real load(MW)

Univariate method (%) Multivariate method (%)

Method 1 Method 2 Linear regression RBFNN

0 1929.24 0.11 �0.64 0.37 0.501 1865.15 �0.91 �0.52 �0.62 �0.652 1725.03 3.39 �0.82 0.38 0.393 1675.57 0.34 0.33 �1.91 �1.904 1642.45 0.78 4.41 2.14 2.105 1726.82 �0.78 0.11 1.35 1.306 2011.51 �6.5 1.88 1.28 1.387 2089.91 0.03 �0.01 �0.56 �0.688 2070.63 0.24 3.80 �1.57 �2.569 2416.42 �2.22 �0.17 �0.41 �0.4110 2473.58 0.25 �1.05 1.45 1.5511 2653.25 �2.02 �2.47 0.57 0.4712 2638.97 �1.91 0.23 �0.78 �0.7813 2354.70 3.45 3.65 �0.69 �0.6914 2459.37 0.56 �0.56 1.24 1.2715 2483.36 0.17 �0.29 �0.59 �0.5916 2514.83 0.24 0.07 0.96 0.9017 2463.97 0.08 �0.54 �1.20 �1.2018 2683.79 �0.17 0.03 0.53 0.5119 2602.49 2.39 1.64 2.75 0.6720 2933.86 2.37 �0.02 0.25 0.2521 28580.31 0.84 �0.25 1.57 1.5222 2580.47 �0.07 0.66 �0.54 �0.2523 2221.31 1.08 2.01 0.68 0.18


1388 Y. LIU ET AL.

in Section 2.1 still needs further exploration. Moreover, parameter selection for phase space

reconstruction is another source of forecasting error. Selection of delay time and embedding dimension

may vary slightly for different methods, e.g., embedding dimension obtained by the Cao’s method and

minimal forecasting error are 11 and 10 for the same load data. Quantitative estimation of forecasting

error due to different methods is also an interesting question.

6. CONCLUSION

Seeking method with better accuracy for STLF is always a significant task for theoretical research and

practical application. Besides periodicity, electric short-term load is sensitive to environmental factors,

which requires taking not only the extraneous factors but the inner mathematical characteristic of load

data into account to learn load change information as adequate as possible. From this perspective, a

typical period of load data is collected from Chongqing Power Grid compared with a randomly

generated time series. Power spectrum of two time series is compared. Delay time and embedding

dimension are selected to obtain the largest Lyapunov exponent in the framework of phase space

reconstruction. Results reveal the electric load series contains chaotic behavior besides superficial

randomness. Then, phase space reconstruction of a univariate time series is extended for a multivariate

time series; delay time and embedding dimension of a multivariate time series are also discussed.

Finally a RBFNN is adopted to forecast one day and one week load ahead. Experiment results indicate

the multivariate method achieves better accuracy than the univariate counterpart for weekday and

weekend load forecasting. During the entire forecasting, weather factor is taken into account and only

the historical temperature data is considered for simplicity purpose.

This paper only tests a three-month length data and considers a single influence factor into account,

which might be the shortcomings of this report. It is scheduled to analyze other typical period of load

data (e.g., the summer season) to test the proposed method, and find the influence of meteorological

condition to load forecasting as exactly as possible; a comparison of different calculation methods for

delay time and embedding dimension is also needed to figure out the impact on forecasting; moreover,

refinement of neural network is also an attractive research topic since effective method for optimal

structure selection, parameter tuning, and neural networks training will definitely facilitate the

forecasting task.

7. LIST OF SYMBOLS

t delay time

Eðm1;m2; � � � ;mMÞ mean one-step absolute error

IðtiÞ mutual information function

sj width parameter of RBFNN

Cj center parameter of RBFNN

k evolution step

m embedding dimension

w weight of RBFNN

y(k) mean lag distance after k steps evolution

Table II. Forecasting results in April 1–April 7 in 2003.

Errorrange

Mon Tue Wed Thu Fri Sat Sun Mean

< 2% 91.67/79.16 87.5/75 83.33/70.83 75/66.67 70.83/66.67 66.67/62.5 45.83/45.83 74.4/66.672–3% 8.33/16.67 8.33/20.83 12.5/20.83 20.83/25 20.83/20.83 25/25 41.67/31.34 19.64/23.22> 3% 0/4.17 4.17/4.17 4.17/8.34 4.17/8.34 8.34/12.5 8.33/12.5 12.5/20.83 5.96/10.12



ACKNOWLEDGEMENTS

The authors acknowledge the support of the National Natural Science Foundation (Project 50607023) of China;the first author is also very grateful to the China Scholarship Council. Constructive comments and suggestions bythe two anonymous reviewers as well as the help of the editor are highly appreciated.

REFERENCES

1. Papalexopoulos AD, Hesterberg TC. A regression based approach to short term system load forecasting. IEEE

Transaction on Power Systems 1990; 5(4):1535–1547.2. Charytoniuk W, Chen MS, Van Olinda P. Nonparametric regression based short-term load forecasting. IEEE

Transaction on Power Systems 1998; 13(3):725–730.3. Galiana F, Handschin E, Fiechter A. Identification of stochastic electric load models for physical data. IEEE

Transaction on Automatic Control 1974; 19(9):887–893.4. Hagan MT, Behr SM. The time series approach to short term load forecasting. IEEE Transaction on Power Systems

1987; 2(2):25–30.5. Paarmann LD, Najar MD. Adaptive online load forecasting via times series modeling. Electric Power Systems

Research 1995; 32(3):219–225.6. Huang S-J, Shih K-R. Short-term load forecasting via ARMA model identification including non-Guassian process

considerations. IEEE Transaction on Power Systems 2003; 18(2):673–679.7. Yang H-T, Huang C-M, Huang C-L. Identification of ARMAXmodel for short term load forecasting: an evolutionary

programming approach. IEEE Transaction on Power Systems 1996; 11(1):403–408.8. Gross G, Galiana FD. Short term load forecasting. Proceeding of the IEEE 1987; 75(12):1558–1573.9. Taylor JW, McSharry PE. Short-term load forecasting methods: an evaluation based on European data. IEEE

Transaction on Power Systems 2007; 22(4):2213–2219.10. Soares LJ, Medeiros MC. Modeling and forecasting short-term electricity load: A comparison of methods with an

application to Brazilian data. International Journal of Forecasting 2008; 24:630–644.11. Kang C, Xia Q, Zhang BM. Review of power system load forecasting and its development. Automation of Electric

Power Systems 2004; 28(17):1–11.12. Hippert HS, Pefreira CE, Souza RC. Neural network for short-term load forecasting: A review and evaluation. IEEE

Transaction on Power Systems 2001; 16(2):44–54.13. Kim C-i, Yu I-k, Song YH. Kohonen neural network and wavelet transform based approach to short-term load

forecasting. Electric Power System Research 2002; 63:169–176.14. Ranaweera DK, Hubele NF, Karady GG. Fuzzy logic for short term load forecasting. Electrical Power and Energy

Systems 1996; 18(4):215–222.15. Mastorocostas PA, Theocharis JB, Bakirtzis AG. Fuzzy modeling for short term load forecasting using the

orthogonal least squares method. IEEE Transaction on Power Systems 1999; 14(1):29–36.16. Papadakis SE, Theocharis JB, Bakirtzis AG. A load curve based fuzzy modeling technique for short-term load

forecasting. Fuzzy Sets and Systems 2003; 135:279–303.17. Pandian SC, Duraiswamy K, Christober Asir Rajan C, et al. Fuzzy approach for short term load forecasting. Electric

Power System Research 2006; 76:541–548.18. Senjyu T, Mandal P, Uezato K, et al. Next day load curve forecasting using hybrid correction method. IEEE

Transaction on Power Systems 2005; 20(1):102–109.19. Kim K-H, Youn H-S, Kang Y-C. Short-term load forecasting for special days in anomalous load conditions using

neural networks and fuzzy inference method. IEEE Transaction on Power Systems 2000; 15(2):559–565.20. Alireza K, Enwang Z, Hassan E. A neuro-fuzzy approach to short-term load forecasting in a price-sensitive

environment. IEEE Transaction on Power Systems 2002; 17(4):1273–1282.21. Zhang Y, Zhou Q, Sun C, et al. RBF neural network and ANFIS-based short-term load forecasting approach in real-

time price environment. IEEE Transaction on Power Systems 2008; 23(3):853–858.22. Ling SH, Leung FHF, Lam HK, et al. A Novel Genetic-algorithm-based neural network for short-term load

forecasting. IEEE Transaction on Industrial Electronics 2003; 50(4):793–799.23. Liao G-C, Tsao T-P. Application of a fuzzy neural network combined with a chaos genetic algorithm and simulated

annealing to short-term load forecasting. IEEE Transaction on Evolutionary Computation 2006; 10(3):330–340.24. Mohammed EI-T, Fowwaz EI-K. Short-term forecasting of Jordanian electricity demand using particle swarm

optimization. Electric Power Systems Research 2008; 78:425–433.25. Chen B-J, Chang M-W, Lin C-J. Load forecasting using support vector machines: a study on EUNITE competition

2001. IEEE Transaction on Power Systems 2004; 19(4):1821–1830.26. Fan S, Chen L. Short-term load forecasting based on an adaptive hybrid method. IEEE Transaction on Power Systems

2006; 21(1):392–401.27. Cottet R, Smith M. Bayesian modeling and forecasting of intraday electricity load. Journal of the American

Statistical Association 2003; 98(464):839–849.28. Hiroyuki M, Shouichi U. Short term load forecasting with chaos time series analysis. International Conference on

Intelligent Systems Application to Power Systems, ISAP’96, 1996; 133–137.


1390 Y. LIU ET AL.

29. Michanos SP, Tsakoumis AC, Fessas P, et al. Short term load forecasting using a chaotic time series. International

Symposium on Signal, Cricuits and Systems (SCS), Vol. 2, 10–11 July 2003; 437–440.

30. Yang HY, Ye H, Wang G, et al. Fuzzy neural very-short-term load forecasting based on chaotic dynamics

reconstruction. Chaos, Solitons and Fractals 2006; 29:462–469.31. Wolf A, Swift JB, Swinney HL, et al. Determing Lyapunov exponents from a time series. Physica 1985; 16D:285–

317.

32. Rosenstein MT, Colins JJ, De luca. CJ. A practical method for calculating largest Lyapunov exponents from small

data sets. Physica D 1993; 65:117–134.33. Liu Y-z, Chen L-q. Nonlinear Dynamics. Shanghai Jiao Tong University Press: Shanghai, 2000.

34. Liangyue C. Practical method for determining the minimum embedding dimension of a scalar time series. Physica D

1997; 110:43–50.35. Shaolan L. Research on short-term load forecasting model and method based on chaotic characteristic of electric

load time series. Ph.D Dissertation, Chongqing University, Chongqing, China, 2005.

36. Grassberger P, Procaccia I. Measuring the strangeness of strange attractors. Physica 1983; 9D:189–208.37. Takens T. Detecting strange attractors in turbulence. Lecture Note in Mathematics 1981; 898:366–381.38. Liangyue C, Alistair M, Kevin J. Dynamics from multivariate time series. Physica D 1998; 121:75–88.39. Lei S, Sun C, Zhou Q, Zhang X. The research of local linear model of short-term electrical load on multivariate time

series. Proceeding of the CSEE 2006; 26(2):25–29.



a multivariate forecasting method for short-term load using chaotic features and rbf neural network

Documents