developing ann

7/27/2019 Developing ANN

1/8

6thEnvironmental Engineering Specialty Conference of the CSCE

& 2ndSpring Conference of the Geoenvironmentla Divisionof the Canadian Geotechnical Society

6econgrs spcial sur le gnie de l environmnement de la SCGCet 2econgrs de pr intemps de la secti on goenvi ronnementale

de la societcanadi enne de gotechnique

______________________________________________________

London, Ontario

7-10 June 2000 / 7 10 juin 2000

376

DEVELOPING ARTIFICIAL NEURAL NETWORK PROCESS MODELS: AGUIDE FOR DRINKING WATER UTILITIES

C.W. Baxter, S.J. Stanley, Q. Zhang, D.W. SmithDepartment of Civil and Environmental Engineering, University of Alberta, Canada

ABSTRACT: Due to the complex nature of drinking water treatment unit processes, utilities are often unable toquantify the interactions and relationships that exist between process inputs and process outputs. Processmodels, where they exist are site-specific and are unable to fully handle continuous and extreme variations in

more than one or two key process parameters. As such, process control strategies are based upon the results ofbench-scale and pilot-scale experiments rather than on process models. Presented is a framework fordeveloping artificial neural network (ANN) models of drinking water treatment processes. Then ANN modelling

technique is robust and can handle the dynamic nature of the treatment processes. The utility of ANN models isdemonstrated through a case study where a successful ANN model to predict filtration performance wasdeveloped.

1. INTRODUCTION

All unit processes in the drinking water treatment

industry are complex, involving many biological,physical, and chemical phenomena. While thenumerous parameters that directly influence the

performance of each process are known to plantoperators and researchers, the interactions andrelationships between process inputs and outputs is

often poorly understood and can not be easilyquantified. Process models, where they exist, areoften site-specific and developed using the results of

bench-scale and pilot-scale experiments where many

of the process parameters are held constant. Suchmodels are unable to cope with simultaneous

fluctuations in more than one or two key parameters.

The inability of water treatment utilities to quantify

process interactions and relationships can causegreat difficulty for water treatment process control. Ifeach unit process is considered to be a multiple-input

multiple-output (MIMO) non-linear optimizationproblem, effective control can only be achieved

through strategies that can accommodate thefluctuations of multiple input and output parameters ona real-time basis. In the absence of such strategies,

utilities resort to using bench-scale tests that provideonly a snapshot of the process conditions in order toachieve process control.

As utilities strive to meet more stringent customer-imposed and regulatory demands on finished water

quality, there is an urgent need for new tools toimprove process knowledge and provide effectivealternatives to current process control methodologies.

One such tool is artificial neural network (ANN)

modelling, a robust technique that allows for thedevelopment of multiple-variable, non-linear models

that can be integrated into real-time process controlstrategies.

Presented are guidelines for the development of ANNprocess models in the drinking water treatmentindustry. More specifically, the requirements and

applicability of the modelling technique are discussed,and the utility of the technique is demonstrated


2/8

377

through a sample process model and an associatedoffline process control application.

2. ARTIFICIAL NEURAL NETWORK PROCESS

MODELS

ANNs are categorized as an artificial intelligence

modelling technique due to their ability to recognizepatterns and relationships in historical data andsubsequently make inferences concerning new data.

ANNs can be used for two broad categories ofproblems: data classification and parameterprediction. For data classification problems, the ANN

uses a specified algorithm to analyze data cases orpatterns for similarities and then separates them into apre-defined number of classes. Credit agencies, for

example, often use ANN models to separate apotential list of clients into classes according to theirlevel of credit risk or buying patterns. For parameter

prediction problems, the ANN learns to accuratelypredict the value of an output parameter givensufficient input parameter information. The main

applications of the ANN technique in the watertreatment industry are in the development of processmodels and model-based process-control andautomation tools. These applications can be

categorized as parameter prediction problems, towhich the ensuing discussion is dedicated.

2.1 Advantages of ANN Modelling

The ANN modelling technique holds several

advantages over mechanistic modelling that make itparticularly suitable to process modelling in thedrinking water treatment industry. In developing

models of MIMO water treatment processes, a keyconsideration is the ability of the model to adapt to thedynamic nature of the process on a real time basis.

ANN models can handle non-linear relationships, andcan provide predictions of output parameters in realtime in response to simultaneous and independent

fluctuations of the values of model input parameters.ANN models are also fault-tolerant in model buildingand in end-use. Data patterns where the value of one

or more of the model inputs are missing can be

incorporated into model building if necessary,although model prediction will improve if only

complete data patterns are used. Similarly, whencompleted models are applied to new data patternswhere values are missing, the value of model outputs

can still be predicted. This feature is particularly usefulin process control applications where input data is fedto models in real time by instruments that are subject

to periodic failure. Finally, ANNs do not requirecomplicated programming, logical inference schemes,

or the development of complex algorithms to developa successful model. Many user-friendly ANN software

packages exist, offering the user a myriad ofmodelling options and allowing the user to customizethe modelling process to suit his or her knowledge of

modelling heuristics.

2.2 Challenges of ANN Modelling

Some aspects of the ANN modelling technique maypresent challenges to drinking water treatment utilities

that wish to develop successful process models. ANNmodels can only be developed where sufficienthistorical data for each of the process parameters

exists. Since the modelling technique requires that.While data requirements will be addressed further on,it is important to note at this point that not all utilities

have detailed historical records of influent waterquality, process control actions, and treated waterquality. As more utilities realize that historizing such

data is important for increasing process knowledgeand improving process efficiency, this challenge willbecome less influential with time. Perhaps the

greatest challenge that must be overcome is theperception of ANN models to be black-box modelsthat can not be understood by the end-user. Thisperception is fed by a new host of ANN software

packages that use proprietary algorithms to developquick and easy models with minimal user interference.From a public health standpoint, utility managers are

rightfully concerned about trusting plant operations tosuch a model. By using less sophisticated softwarepackages that feature well-understood training

algorithms, this challenge can be overcome.

2.3 Existing ANN Models

In recent years, water treatment utilities have come torealize the potential for the ANN modelling

techniques, resulting in a number of published modelapplications in water quality, treatment, anddistribution. With respect to water quality, models

have been developed for trihalomethane (THM)formation and speciation (Hutton, Sandhu and Chung1996), source water salinity forecasting (DeSilets et

al. 1992), and raw water colour forecasting (Zhang

and Stanley 1997). Process models have beendeveloped for alum and polymer dose forecasting in

coagulation (Mirsepassi, Cathers and Dharmappa1995), and turbidity and colour removal throughenhanced coagulation (Stanley, Baxter and Zhang

2000). Finally, for distribution systems, a model hasbeen developed for the prediction of residual chlorinein the distribution system (Rodriguez et al. 1997).


3/8

378

3. Building ANN Models of Drinking WaterTreatment Processes

3.1 Types of ANN Models

Each drinking water treatment unit process can beconsidered a non-linear MIMO process, as previouslydiscussed. The process inputs include influent water

parameters, such as pH and turbidity, as well asoperational parameters, such as chemical dosinglevels and flow rate. The process outputs are the

effluent water parameters. ANN models of drinkingwater treatment process can assume two distinctforms: process models and inverse process models.

In the former, the model predicts the value of one ormore process outputs, given the values of the processinput parameters. An example of this type of model is

the prediction of clarifier effluent turbidity usinginfluent water parameters and operational parameters.In the latter, the ANN model predicts the value of one

or more process inputs, given the values of theremaining process inputs and process output(s). Thistype of model is often used to predict the value of an

operational parameter required to reach a targeteffluent quality. An example of this type of model isthe prediction of alum dose required to maintain adesired value for clarifier effluent turbidity.

3.2 ANN Modelling Requirements

In order to build successful ANN process models,specific data, software, hardware, and personnelrequirements must be met. The key requirement of

the ANN modeling approach is the availability ofrelevant data to describe the process being modeled.Data must be available in a useable electronic format

for each of the process input and output parameters.The data used in model development must berepresentative of plant operations, spanning the range

of operating conditions that may be encounteredduring both routine operations and process upsetconditions. Model development requires the use of

appropriate ANN software. Many commercial softwarepackages are currently available; site-specificrestrictions on operating systems and data format, as

well as desired options will dictate the best software

choice for each utility. As previously indicated,software packages that use only proprietary

algorithms should be avoided in favour of those thathave a high degree of user input in model-building.With respect to hardware requirements, recent

advances in computing technology have made eventhe most modest new home computing systemcapable of performing the modelling calculations. To

ensure optimal performance however, a 500 MHzprocessor and 128 M of RAM should be considered to

be the minimum system requirements. With respect topersonnel requirements, the model developer should

have expert knowledge of the process being modelledand should understand basic modelling heuristics. Assuch, a plant operator or operations engineer with

some training in ANN model development would be asuitable candidate to develop process models.

3.3 Key Components of ANN Models

There are several key components to ANN models

that are collectively referred to as the ANNarchitecture. Processing units or neurons performprimitive operations such as scaling data, summing

weights, and amplifying or thresholding sums.Neurons are organized into layers with each layerperforming a specific function. The input layer serves

as an interface between the input parameter data andthe ANN model. Most models also contain one or twohidden layers, although more are possible. These

layers perform most of the iterative calculations withinthe network. The output layer serves as the interfacebetween the ANN model and the end-user,

transforming model information into an ANN-predictedvalue of the output parameter(s). Each neuron isconnected to every neuron in adjacent layers byweights; links that represent the strength of

connection between neurons. Each ANN model has apropagation rule that defines how the weightsconnected to a neuron are combined to produce a net

input. The propagation rule is generally a simplesummation of the weights. As discussed, the inputlayer serves as an interface between input parameter

data and the model. In this layer, a scaling function isused to scale data from their numeric range into arange that the network deals with efficiently, typically 0

to 1. The hidden and output layers contain anactivation function that defines how the net inputreceived by a neuron is combined with its current

state of activation to produce a new state of activation.The most common activation function used in processmodelling is the sigmoidal activation function. Finally,

each network has a learning rule that defines how theweights are modified in order to minimize predictionerror. As will be discussed, the backpropagation

algorithm is the most common learning rule employed

in process modelling.

3.4 ANN Learning

In developing ANN process models, a supervised

learning paradigm is employed. In supervisedlearning, historical data patterns, consisting of valuesfor each of the model input and output parameters,

are used to train the ANN. The goal of supervisedlearning is to minimize the error between the model-


4/8

379

predicted value and the actual value of the outputparameter(s). The error minimization takes place by

modifying the weights between neurons according toa learning rule, typically the backpropagationalgorithm. This algorithm increases or decreases the

value of the weights in order to minimize the squareddifference between the network-predicted value andthe actual value of the output parameter(s), summed

over all of the data patterns. ANN learning or trainingproceeds through repeated presentation of datapatterns to the ANN and subsequent weight

modification until the prediction error is sufficientlysmall, as defined by the user, or until a maximumnumber of iterations has been reached. A more

detailed description of the ANN learning process ispresented by Baxter, Stanley, and Zhang (1999).

3.5 Data Collection and Analysis

In order to develop successful ANN models of drinking

water treatment processes, careful attention must bepaid to the details of data collection and analysis. Incollecting data, several factors need to be considered.

First, the availability of the data must be ascertained.For data availability, the parameters for whichhistorical data exists, the time frame of historicalmeasurements, and the frequency of data

measurement must all be determined. The format andreliability of the data are also key considerations indata collection. Historical data can originate from of

grab-samples or real-time measurements, andmeasurements can be discrete or aggregated from anumber of samples. The reliability of the available

data should be ascertained through an examination ofquality assurance and quality control protocols.Finally, it is of considerable importance to note any

process changes that may have been implementedduring the time frame for which data are available.

With the above considerations in mind, it is possible todelineate a number of guidelines for selecting data tobe used in ANN process modelling. First and

foremost, data for each of the parameters known orsuspected to affect the process being modelled mustbe available. The quantity of data required to develop

a model is site specific, and is affected by seasonal

fluctuations in influent water quality, and the frequencyof process upsets. As such, it is more important to

ensure that the data are fully representative of therange of conditions that can be expected duringperiods of routine and upset operations. As a general

guideline, at least one full cycle of data must beavailable in order to ensure a representative data set.In temperate areas with four distinct seasons for

example, a data cycle would encompass a full year ofhistorical data. With respect to the format of the data,

the variability of the process as well as dataavailability will dictate whether to use hourly data,

daily averages, or some other frequency for each ofthe parameters. Successful process models can oftenbe made using the daily average or some daily

percentile value of each of the model parameters.With respect to the effect of major process changeson data selection, data collected prior to major

process changes should not be incorporated if theyare believed to differ significantly from current processdata. This guideline is necessary to ensure that the

data are representative of current process conditions.Finally, in order to maintain the integrity of the dataset, appropriate quality assurance and quality control

protocols for each model parameter must be in place.

Once an appropriate historical data set has been

selected, it should be fully characterized andsubjected to a comprehensive statistical analysis.Data characterization involves a qualitative

assessment of hourly, daily, and seasonal trends ofeach potential model parameter. The statisticalanalysis involves the determination of measures of

central tendency, measures of variation, percentileanalysis, and identification of outliers, erroneousentries, and non-entries for each data parameter. Incombination, the data characterization and statistical

analysis help to identify the boundaries of the studydomain as well as potential deficiencies in the dataset.

3.6 Model Building Protocol

Unfortunately, there is no widely accepted bestmethod of developing ANN models. When all of thepossible options in building an ANN architecture are

considered, an infinite number of distinct architecturesare possible. As such, each model developer mayuse a different protocol to reduce the number of

architectures that are evaluated. What follows is a six-step protocol that the authors have found to be usefulin developing drinking water treatment ANN process

models.

3.6.1 Selection of Model Inputs and Outputs

The first step involved in the selection of model inputsand outputs is the selection of the model output(s).

The output(s) are selected on the basis of operationalneeds, relevant literature, and data availability. Aspreviously discussed, drinking water treatment

processes can be considered MIMO processes. Assuch, models of such processes can have multipleoutput parameters. Since ANN models train by

minimizing the error between the predicted and actualvalues of model output parameters, the technique


5/8


6/8

381

being modelled however, model fine-tuning can makethe difference between acceptable and unacceptable

prediction performance.

3.6.6 Comparison of Candidate Models

Where more than one candidate model has beenidentified, the candidate models are compared and

the final model is selected. The selection processinvolves statistical and graphical analyses of themodelling results for each of the candidates. Plots of

actual versus predicted values of the model outputcan be useful in highlighting problem areas andprediction inconsistencies. Overall, the model that

best meets the goals of the modelling process isselected.

4. Case Study - ODP Plant

The Oxidation Demonstration Project (ODP) facility,owned and operated by the Metropolitan WaterDistrict of Southern California (MWD), is located on

the premises of the F.E. Weymouth Filtration Plant, inLa Verne, California. The facility began operation in1992 in order to evaluate the use of ozone andPEROXONE (a combination of ozone and hydrogen

peroxide) as primary disinfectants at the full-scaleMWD facilities. The facility has a capacity of 20.8ML/d and consists of three types of ozone generators,

three feed gas systems, an external mix ozonecontacting system, two ozone contactors, two types ofozone destruct systems, a hydrogen-peroxide

injection system, and conventional coagulation andfiltration unit processes. With respect to theconventional process, the ODP plant has the ability to

feed sulfuric acid, sodium hydroxide, metal coagulant(alum, ferric chloride, or polyaluminum chloride),polymer, filter aid, chlorine, and ammonia. The ODP

plant is operated as a stand-alone facility, although

treated water is currently pumped from the clearwell tothe head works of the Weymouth facility.

In addition to the evaluation of alternativedisinfectants, the ODP facility has been used to study

biological filtration, filter air binding, enhancedcoagulation, arsenic removal, and other filtrationissues. Each of these studies required different plant

configurations and had different data collectionrequirements. As such, while the plant has been incontinuous operation for over 7 years the data set was

often incomplete. A complete list of the parametersthat have been historically measured at the facility ispresented in Table 1.

Table 1. Data parameters historically measured atthe ODP plant

Parameter Classification

Source Water Plant Influent

Turbidity (NTU) Plant Influent

pH Plant Influent

Dissolved Oxygen (mg/L) Plant Influent

UV254(cm-1

) Plant Influent

Temperature (C) Plant Influent

Alkalinity (mg/L) Plant Influent

Bromide (mg/L) Plant Influent

Plant Flow (MGD) Operational

Coagulant Dose (mg/L) Operational

Polymer Dose (mg/L) Operational

Filter-aid Dose (mg/L) Operational

Total Ozone Dose (mg/L) OperationalFiltration Rate (gpm/ft

2) Operational

90th

Percentile Turbidity (NTU) Filter Effluent

50th

Percentile Turbidity (NTU) Filter Effluent

90th

Percentile Particle Counts (PC/mL) Filter Effluent

50th

Percentile Particle Counts (PC/mL) Filter Effluent

4.1 Model Development

The objective of this case study is to develop ANN

models to predict the 50

th

percentile filter effluentturbidity at the ODP using historical operational data.All percentile parameters are based on a 24-hour time

period. The six-step protocol previously defined wasused in model development. As per the protocol, themodel input and output parameters were selected

based on the results of a literature review oncoagulation filtration and data availability. A list of theinputs used in model development is presented in

Table 2

Table 2. 50thpercentile filter effluent model

inputs

Parameter Classification

Turbidity (NTU) Plant Influent

pH Plant Influent

Temperature (C) Plant Influent

Alkalinity (mg/L) Plant Influent

Coagulant Dose (mg/L) Operational

Polymer Dose (mg/L) Operational

Filter-aid Dose (mg/L) Operational

Total Ozone Dose (mg/L) Operational

Filtration Rate (gpm/ft2) Operational


7/8

382

0

0.02

0.04

0.06

0.08

0.1

0.12

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79

Pattern #

FilterEffluentTurbidity(NTU

)

Actual Model Prediction

Figure 1. Model results for 50th percentile filter effluent turbidity model

0.05

0.055

0.06

0.065

0.07

0.075

0.08

0.085

0.09

0.095

1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5

Polymer Dose (mg/L)

50thPercentileFilterEffluentTurbid

ity(NTU)

Optimized Polymer Dose

Figure 2. Effect of polymer dose on filter effluent turbidity for typical spring water conditions

Based on the protocol, the best candidate model was

a three-layer backpropagation network with 32 hiddenlayer neurons. The model predicted the 50

thpercentile

value of filter effluent turbidity with a mean absolute

error of 0.003 NTU and a coefficient of multiple

regression (R2) of 0.85, based on the production data

set. The model results are presented graphically inFigure 1.


8/8

383

While a detailed description of the numerous processcontrol and related applications of developed models

is beyond the scope of this paper, the trained modelcan be used as a virtual full-scale laboratory toprovide information about process operations at the

ODP facility. A graphical representation of the effect ofaltering the polymer dose on filter effluent turbidity fortypical spring water conditions is presented in Figure

2. The values of the filter effluent turbidity used tocreate this figure were generated from the trainedANN process model.

Based on typical raw water and operational conditionsexperienced in springtime at the ODP facility, a

polymer dose of approximately 2.4 mg/L wouldproduce a filter effluent with the lowest turbidity. Itshould be noted that this example of the virtual-lab

capability of the ANN models is extremelyrudimentary; more advanced virtual-lab systems canbe developed where the effects of simultaneously

changing the values of multiple input parameters canbe assessed.

5.0 CONCLUSION

While the challenges involved in developing ANN

process models may be prohibitive for some utilities,the current trend towards historizing process data, incombination with recent advances in computing

technology, have made the technology appropriate formany utilities. As demonstrated through the casestudy, The ANN modelling technique allows utilities to

improve process knowledge, and therefore facilitateprocess control, by allowing for the development ofmultiple-parameter, non-linear models of complex unit

processes. As customer and regulatory demands onfinished water quality continue to become morestringent, ANN models will continue offer drinking

water utilities a sophisticated process modelling andcontrol alternative to conventional methodologies.

6.0 ACKNOWLEDGEMENTS

The authors an indebted to the American Water

Works Association Research Foundation (AWWARF),the Metropolitan Water District of Southern California,

and EPCOR Water Services for their continuedlogistical and financial support of ANN research in thedrinking water treatment industry.

7.0 REFERENCES

Baxter, C.W., S.J. Stanley, and Q. Zhang.Development of a Full-Scale Artificial NeuralNetwork Model for the Removal of Natural Organic

Matter by Enhanced Coagulation. AQUA

48(4):129-136.

DeSilets, L., B. Golden, Q. Wang and R. Kumar.1992. Predicting Salinity in the Chesapeake BayUsing Backpropagation. Computers and

Operations Research19(3-4): 277-285.

Hutton, P.H., N. Sandhu, and F.I. Chung. 1996.

Predicting THM Formation With Artificial NeuralNetworks. In Proceedings of the North AmericanWater and Environment Conference, Anaheim,

CA.: ASCE.

Mirsepassi, A., B. Cathers and H. B. Dharmappa.

1995. Application of Artificial Neural Networks tothe Real Time Operation of Water TreatmentPlants. In IEEE International Conference on Neural

Networks: Proceedings. Perth, Australia: IEEE

Rodriguez, M. J., J. R. West, J. Powell and J. B.

Serodes. 1997. Application of Two Approaches toModel Chlorine Residuals in Severn Trent WaterLTD (STW) Distribution Systems. Water Science

and Technology36(5): 317-324.

Stanley, S.J., C.W. Baxter, and Q. Zhang. 2000.

Process Modeling and Control of Enhanced

Coagulation. Denver, Colo.: AWWARF andAWWA.

Zhang, Q. and S. J. Stanley. 1997. Forecasting Raw-water Quality Parameters for the North

Saskatchewan River by Neural Network Modeling.Water Research31(9): 2340-2350.

developing ann

Documents