Elman’s Recurrent Neural Network Applied to Forecasting the Quality Of Water Diversion in the Water Source Of Lake Taihu

Heyi Wang* College of Hydrology and Water Resources

Hohai University Nanjing, China

e-mail: wanghy@hhu.edu.cn

Yi Gao Bureau of Hydrology and Water Resources Monitoring

Taihu Basin Management Bureau Wuxi, China

e-mail: taihuliuyu@vip.163.com

Abstract—This paper describes the training, validation and application of recurrent neural network (RNN) models to computing the total nitrogen (TN), total phosphorus (TP) and dissolved oxygen (DO) at three different sites in Gonghu Bay of Lake Taihu during the period of water diversion. The input parameters of Elman’s RNN were selected by means of the principal component analysis (PCA). Sequentially, the conceptual models for Elman’s RNN of different simulated parameters were established and the Elman models were trained and validated on daily data set. The values of TN, TP and DO computed by the models were closely related to their respective values measured at the three sites. The results show that the PCA can efficiently ascertain appropriate input parameters for Elman’s RNN and the Elman’s RNN can precisely compute and forecast the water quality parameters during the period of water diversion.

Keywords-Elman’s recurrent neural network; principal component analysis (PCA); water quality; water diversion

I. INTRODUCTION Water diversion has been widely applied to irrigation,

flood control, water supply, power generation, and so on, all around the world. In many countries, diverting a large quantity of low-nutrient water to a eutrophic lake is considered as an approach to lake restoration. Lake Taihu is the third largest fresh water lake in china. After experimental water diversion, the long-time project for water diversion in the lake is carried out.

Without careful modeling, forecasting, and analysis, even the most acceptable conceptual and physically-based models are too difficult to arrive the end results during the period of water diversion. A conceptual and physically-based model requires a lot of data and parameters that are often unknown, while data-driven techniques provide an effective alternative to conventional conceptual and physically-based model. Models developed by data-driven techniques are computationally very fast and require fewer input parameters than process-based models. Data-driven modeling techniques become more and more popular in the past 20 years. ANN models are such data-driven models with particular characteristic which are greatly suited to dynamic nonlinear

*Corresponding author

system modeling. Compared with conventional simulation methods, the advantages of ANN models have been discussed in details by French et al. ANN models have been widely applied to solving water quality problems [1-7].

In this paper, recurrent neural network approach, named as Elman’s net [8], was used for modeling to predict and forecast total nitrogen (TN), total phosphorus (TP) and dissolved oxygen (DO) in water resource of Lake Taihu during the period of water diversion. These water quality parameters were measured daily at various locations. The results of the Elman prediction and forecasting model in water resource of Lake Taihu during water diversion were discussed in this paper.

II. MATERIALS AND METHODS

A. Study area and water quality data set Lake Taihu is the third largest freshwater lake in China,

with a total water surface area of 2338 km2, an average water depth of 1.9m and a volume of 44×108 m3. Bounded by Jiangsu, Zhejiang, Anhui Province, and Shanghai, its basin area which considered as the fast developing area in China is about 36,900 km2[9][10]. Gonghu Bay is located in the northeast of Lake Taihu. In the northeast of the bay, the Wangyu River is the channel to transfer the water from the Yangtze River to Lake Taihu. Until now, there are three main waterworks scattered along shore of Gonghu Bay and supplied approximately 0.7 billion m3 drinking water annually from the lake to the surrounding cities, such as Suzhou and Wuxi.

The data set used in this study was collected from continuous monitoring of water quality from May 30 to Sep 19 in 2007, Apr 16 to Jun 19 in 2008, and May 5 to Jun 30 in 2009 at four sites during water diversion in Gonghu Bay of Lake Taihu (Fig. 1). Site 1, the farthest water diverting site, is closing to the Gonghu waterworks. Located in the north east of Gonghu Bay, site 2 is the first water diverting site. Near the Jinshuwan waterworks covered with higher aquatic macrophytes is site 3. Site 4 lies in the area of Wangyu River before entering the Gonghu Bay.

283

2010 International Conference on Biology, Environment and Chemistry IPCBEE vol.1 (2011) © (2011) IACSIT Press, Singapore

Figure 1. Map showing the geographical setting of the present survey area

with four sites

Water samples were collected from a depth of 50 cm below the surface at four different sampling sites in Gonghu Bay. During the study period all samples collected daily were analyzed for sixteen different parameters. Sampling network and analytical procedures are executed by standard.

As for Elman modeling, ten parameters have been included: water temperature(Tem, ℃), water pH(pH), secchi depth(SD), dissolved oxygen(DO, mgL-1), permanganate index(CODMN, mgL-1), total nitrogen(TN, mgL-1), total phosphorus(TP,mgL-1), ammonical nitrogen(NH3-N, mgL-1), Chl-a(mgm-3), and the average input rate of dilution water(WQ, m3d-1)

B. Structure of neural network model The Elman neural network (feedback connection) is

obtained by another feedback loop from the output of hidden layer to the input of this layer, which constitutes the “context layer” that retains information between observations [8]. The result of processing in a previous time step can be used at the current time step. This property of the Elman type RNN provides very important advantage, especially, in real time applications to follow the dynamical change of water resources variables in practice.

In this study, three-layer Elman’s neural networks were constructed for prediction of the water quality TN, TP and DO in three sites of Gonghu Bay during the period of water diversion, as shown in Fig.2. The model was composed of one input layer optimized input variables selected by the method of principal component analysis(PCA), one hidden layer and one output layer with one output variable in three sites. In order to determine the optimum number of nodes in the hidden layer and transfer functions, different Elman models were constructed and tested.

Figure 2. The architecture of the Elman model for water quality in Lake

Taihu during water diversion

1) Data processing: The data set used in this study was normalized for the

input layer using the method of linear insert-value, as expressed in equation (1).

T=2(X-Xmin)/(Xmax-Xmin)-1 (1) Here, X and T are original data of factors for the input

layer and their normalized data respectively; Xmin and Xmax are the minimum and maximum value for input layer, respectively. Moreover, if the original data is 0 or not be tested, its corresponding normalized values are regarded to be 0.1.

The collected data sets from four sites were randomly divided into two sets. One set contained 70% of the records and was used as a training set, the other contained 30% of the records and used as a validation set. If a no measured or untested factor was included in one data set, it would be cut from training or validation set.

2) Elman parameter selection Initially, the default parameter values of the learning rate

(lr=0.1) and momentum constant (mc=0.9) as well as training goal (1e-2) were used. Determining the number of hidden layers and nodes is a usually a trial and error task in ANN modeling. A rule of thumb for selecting the number of hidden nodes relies on the fact that the number of samples in the training set should at least be greater than the number of synaptic weights [11]. A one-hidden-layer network is commonly adopted by most ANN modelers; the number of hidden nodes M in this model is between I and 2I +1, where I is the number of input nodes [12]. As a guide, M should not be less than the maximum of I/3 and the number of output nodes O. The optimum value of M is determined by trial and error. During the training the weights and biases are iteratively adjusted using the momentum method to minimize the network performance, and evaluated with the mean squared error (MSE) between the network outputs and the target outputs. If the MSE was found to be small enough and stable at the end of each learning epoch by adjusting lr, mc, epochs, the neurons and number of hidden layers, the parameter set was determined and post-process were carried out.

3) Selection of input variables based on PCA In an ANN, one of main task is to determine the model

input variables that affect the output variable(s) significantly. In addition to input variables (i1, i2… in), individual immediate past historical records may also influence the output variables. The choice of input variables is generally based on a priori knowledge of causal variables, inspections of time series plots, and statistical analysis of potential inputs and outputs. In this paper, the choice of input variables for the neural net-work modeling is based on Principle component analysis (PCA) of the field data, the prediction accuracy of water quality variables, and the domain knowledge. Principle component analysis is a technique widely used for reducing the dimensions of multivariate problems [13]. The purpose of PCA is to synthesize the information included in the matrix by a certain optimum method so as to simplify the matrix and reduce the dimensions to reveal its structure as well as to give out reasonable interpretation and answer the problems.

284

In this paper, PCA computed ten water parameters in each predicting site and site 4. It is shown in Table 1 that the input variables of Elman mode were the result of PCA. Table 1 shows that all Elman model’s input variables in site 2 include water parameters of site 4. It is concluded that because site 2 is the point where water diversion is first arrived, it is affected significantly by water diversion. The input variables of Elman model of site 1 and site 3 included more water parameters related to site 4, while the input variables of Elman model of site 2 included fewer water parameters related to site 4. In particular, the results of PCA computed for TN Elman model of site 1 and site 3 has no water parameters of site 4. The results were suggested that site 1 and 3 is less affected by water diversion than site 2.

TABLE I. DIFFERENT ELMANS FOR FORECASTING OF SELECTED WATER QUALITY VARIABLES BASED ON PCA

Modeled WQ

variable

Inputs Output

site 1 TN [(TN,chl-a)t-1] at site 1 TN TP [ (TP, DO)t-1] at site 1 and [ (TP, WQ)t-1]

at site 4 TP

DO [ (DO,Tem)t-1] at site 1 and [ (DO, WQ)t-1] at site 4

DO

Site 2 TN [(TN,DO, pH, Tem, SD)t-1] at site 2 and

[ (TP, WQ)t-1] at site 4 TN

TP [(TP,DO, pH, chl-a)t-1] at site 2 and [ (TP, pH, WQ)t-1] at site 4

TP

DO [ (DO,Tem)t-1] at site 1 and [ (DO, WQ)t-1] at site 4

DO

Site 3 TN [(TN, pH, chl-a)t-1] at site 3 TN TP [(TP,TP)t-1] at site 3 and [ (TP, NH3-N,

WQ)t-1] at site 4 TP

DO [ (DO,Tem, CODmn, chl-a)t-1] at site 3 and [ (DO, WQ)t-1] at site 4

DO

4) Model performance evaluation

To determine the performance of each of the selected network model, two different criteria were used: the root mean square error (RMSE) and the coefficient of determination (R2) [14]. The RMSE represents the error associated with the model and can be computed as:

∑=

−=n

i

ipi

nxxRMSE

1

2)(

(2) The coefficient of determination (R2) represents the

percentage of variability that can be explained by the model and is calculated as:

∑ ∑

∑

= −

=

−

−−= n

i

n

ipii

n

ipii

xn

x

xxR

1

2

1

1

2

2

)1(

)(1 (3)

where xpi and xi represent the model computed and measured values of the variable, and N represents the number of observations. The RMSE, a measure of the goodness-of-fit, best describes an average measure of the error in predicting the dependent variable. Depending on

sensitivity of water quality parameters and the mismatch between the forecasted water quality variable and that measured, an expert can decide whether the predictability of the Elman model is accurate enough to make important decisions regarding data usage.

III. RESULTS AND DISCUSSION

A. TN model results The Elman model was developed to simulate daily DO

concentrations during the period of water diversion at three sites in Gonghu Bay of Lake Taihu. The architecture of the best Elman model for the TN is shown in Fig.2. The Elman for TN model is composed of one input layer with input variables selected by PCA (Table 1), one hidden layer with optimized nodes and one output layer with one output variable. The parameters of Elman model which produced the “best results” for validation data set was explained in section of Elman parameter selection.

The developed Elman models accurately simulated the TN concentrations during water diversion at three sites in Gonghu Bay of Lake Taihu. The results are described in Fig.3. Using optimized input variables, the Elman TN prediction model accurately simulated the range of TN values at site 1(R2=0.91; RMSE=0.13), site 2(R2=0.72; RMSE=0.27) and site 3 (R2=0.92; RMSE=0.07). The model simulated TN concentrations with a good accuracy. The Elman model was able to simulate the TN concentration with an accuracy of a degree or less (RMSE<0.30 and R2>0.7).

According to input variables selected by PCA, TN concentration was less affected by water diversion in site 1 and 3 than site 2. The result of Elman model shows that it is possible to predict TN concentration in three sites during water diversion.

B. TP model results The Elman model was developed to simulate daily TP

concentrations at three sites in Gonghu Bay of lake Taihu. It used an Elman architecture with one input layer, with input variables selected by PCA (Table 1), one hidden layer with optimized nodes and one output layer with one output variable. The parameters of Elman model were selected as explanation in section of Elman parameter selection. It produced the “best results” for validation data set.

Fig.4 shows the measured and predicted values of TP concentration for the best Elman model during transferring water at three sites in Gonghu Bay of Lake Taihu. The TP prediction model simulated the range of TP values at site 1(R2=0.68; RMSE=0.038), site 2(R2=0.45; RMSE=0.028) and site 3(R2=0.61; RMSE=0.016). Except for the prediction site 2 (R2=0.45), the neural network was able to simulate the TP concentrations with RMSE<0.03 and R2>0.6.

The results can be accepted. The Elman model of site 2 can be improved by adding more data for the training and testing or inputting variables related to water diversion.

C. Do model results The Elman model was developed to simulate daily DO

concentrations at three sites in Gonghu Bay of lake Taihu.

285

The architecture of the best Elman model for the TN is stated in Fig.2. Input variable in three sites optimized by PCA were given in Table 1. The parameters of Elman model were selected as explained in section of Elman parameter selection.

Fig.5 shows the measured and predicted values of DO concentration for the best Elman model during transferring period at three sites in Gonghu Bay of lake Taihu. The DO prediction model imitated the range of DO values at site 1(R2=0.30; RMSE=1.10), site 2(R2=0.39; RMSE=1.55) and site 3(R2=0.29; RMSE=0.83).

Because of unknown local factors, it is possible that there was a small amount of uncertainty in the DO concentration and prediction for some field measurements and Elman-predicted values. The accuracy of the model can be improved not only by adding more data for the training and testing of three sites but also by inputting variables related to water diversion.

IV. CONCLUSIONS In this paper, using continuous daily measurements of

water quality parameters at different sites, Elman models were created to imitate dissolved oxygen, total nitrogen and total phosphorus of Gonghu Bay during water diversion. Based on PCA, the factors affecting the change of water quality were selected to be input variables. In spite of largely unknown factors controlling transferring water quality variation and the limited data set size, a relatively good correlation was observed between the measured and predicted values. The discussion shows that the Elman can be used to extract, recognize and predict related patterns of limnological time series. It is also stated that the input variables computed by PCA is acceptable. The accuracy of the predictions is improved with increasing event and time resolution of training data. We suggest that the neural networks is the effective tool for the computation of transfer water quality and used in other areas to improve the understanding of water quality change during the period of water diversion. The Elman can be as a powerful predictive alternative to traditional modeling techniques.

The limitation of this study include the predict model of DO. The lack of fit between the predicted and measured date indicates that new patterns should be incorporated into the model, and the model should be recalibrated and revalidated as more data are collected. Further studies that apply

multivariate models and incorporate new key input variables are necessary to arrive at better result of DO predict model.

ACKNOWLEDGMENT I wish to thank Bureau of Hydrology and Water

Resources Monitoring, Taihu Basin Management Bureau, for providing data of the lake Taihu.

REFERENCES [1] Rogers, L.L., Dowla, F.U., 1994. Optimization of groundwater

remediation using artificial neural networks with parallel solute transport modeling. Water Resour.Res. 30, 457–481.

[2] Raman,H., Chandramouli, V., 1996. Deriving a general operating policy for reservoirs using neural networks. J.Water Resour. Plann. Manage. 122, 342–347.

[3] Wen, C.W., Lee, C.S., 1998. A neural network approach to multiobjective optimization for water quality management in a river basin. Water Resour. Res. 34, 427–436.

[4] Lek, S., Guegan, J.F., 1999. Artificial neural networks as a tool in ecologicalmodelling, an introduction. Ecol. Model. 120, 65–73.

[5] Bowers, J.A., Shedrow, C.B., 2000. Predicting stream water quality using artificial neural networks.WSRC-MS-2000-00112.

[6] Kuo, Y., Liu, C., Lin, K.H., 2004. Evaluation of the ability of an artificial neural network model to assess the variation of groundwater quality in an area of blackfoot disease in Taiwan.Water Res. 38, 148–158.

[7] Kuo, J., Hsieh,M., Lung,W., She, N., 2007. Using artificial neural network for reservoir eutriphication prediction. Ecol. Model. 200, 171–177.

[8] [9]Elman, J., 1990. Finding structure in time. Cognitive Science 14, 179–211.

[9] Hu, W., Jorgensen, S.E., Zhang, F., 2006. A vertical-compressed three-dimensional ecological model in Lake Taihu, China. Ecol. Model. 190, 367–398.

[10] Hu, W., Zhai, S., Zhu, Z., Han, H., 2008. Impacts of the Yangtze River water transfer on the restoration of Lake Taihu. Ecol. Eng. 34, 30–49.

[11] Tarassenko, L., 1998. A Guide to Neural Computing Applications. Arnold Publishers, London.

[12] Hecht-Nielsen, R., 1987. Kolmogorov’s mapping neural network existence theorem. Proceedings of 1st IEEE International Joint Conference of Neural Networks. Institute of Electrical and Electronics Engineers, New York, NY.

[13] Johnson, R.A., Wichern, D.W., 1992. Applied Multivariate Statistical Analysis, third edn. Prentice Hall, Englewood Cliffs, NJ.

[14] Chenard, J.F., Caissie, D., 2008. Streamtemperaturemodelling using neural networks: application on Catamaran Brook,NewBrunswick, Canada. Hydrol. Process. (DOI: 1002/hyp.6928).

286

Figure 3. Measured and predicted TN(mg/L) values for training(column 1) and validation(column 2) tests of three sites

287

Figure 4. Measured and predicted TP(mg/L) values for training(column 1) and validation(column 2) tests of three sites

Figure 5. Measured and predicted DO(mg/L) values for training(column 1) and validation(column 2) tests of three sites

288