bayesian belief networks as a meta-modelling tool in integrated river basin management — pros and...

E C O L O G I C A L E C O N O M I C S 6 6 ( 2 0 0 8 ) 9 1 – 1 0 4

ava i l ab l e a t www.sc i enced i rec t . com

www.e l sev i e r. com/ l oca te /eco l econ

ANALYSIS

Bayesian belief networks as a meta-modelling tool inintegrated river basin management — Pros and cons inevaluating nutrient abatement decisions under uncertainty ina Norwegian river basin

D.N. Bartona,⁎, T. Salorantaa, S.J. Moea, H.O. Eggestadb, S. Kuikkac

aNorwegian Institute for Water Research (NIVA), Gaustadalléen 21, NO-0349 Oslo, NorwaybNorwegian Institute for Agricultural and Environmental Research (Bioforsk), Fr. A. Dahlsvei 20, NO-1432 Ås, NorwaycFEM Group, University of Helsinki, Department of Biological and Environmental Sciences,P.O. Box 56, FI-00014 University of Helsinki, Finland

A R T I C L E I N F O

⁎ Corresponding author. Tel.: +47 924 42 111;E-mail address: [email protected] (D.N

0921-8009/$ – see front matter © 2008 Elsevidoi:10.1016/j.ecolecon.2008.02.012

A B S T R A C T

Article history:Received 23 November 2006Received in revised form5 February 2008Accepted 5 February 2008Available online 18 April 2008

A Bayesian network approach is used to conduct decision analysis of nutrient abatementmeasures in the Morsa catchment, South Eastern Norway. The paper demonstrates the useof Bayesian networks as ameta-modelling tool in integrated river basinmanagement (IRBM)for structuring and combining the probabilistic information available in existing cost-effectiveness studies, eutrophication models and data, non-market valuation studies andexpert opinion. The Bayesian belief network is used to evaluate eutrophication mitigationcosts relative to benefits, as part of the economic analysis under the EU Water FrameworkDirective (WFD). Pros and cons of Bayesian networks as reported in the literature arereviewed in light of the results from our Morsa catchment model. The reported advantagesof Bayesian networks in promoting integrated, inter-disciplinary evaluation of uncertaintyin IRBM, as well as the apparent advantages for risk communication with stakeholders, areoffset in our case by the cost of obtaining reliable probabilistic data and meta-modelvalidation procedures.

© 2008 Elsevier B.V. All rights reserved.

Keywords:Decision analysisInfluence diagramsBayesian networksBenefit–cost analysisEutrophicationUncertaintyWater Framework Directive

1. Introduction

In the last decade Bayesian networks have increasingly beenapplied to environmental management problems under un-certainty, and recently also to integrated water managementissues (see for example Varis et al., 1990; Varis and Kuikka,1999; Borsuk et al., 2001; Varis and Lahtela, 2002; Borsuk et al.,2004; Henriksen et al., 2004; Ames et al., 2005; Bromley et al.,2005; Labiosa et al., 2005). Common to these studies is the use

fax: +47 22 18 52 00.. Barton).

er B.V. All rights reserved

of Bayesian networks to integrate probabilistic informationderived from data sets, model simulations and expert opinionin the study of water allocation or pollution problems. As analternative to extensive scenario analysis using deterministicmodels (e.g. Hein, 2006) Bayesian networks hold the promiseof a more complete accounting of integrated model uncer-tainty. In some cases, Bayesian networks are used to study theproperties of integrating a number of sub-models for purposesof targeting data collection or joint risk analysis. In other

.

mailto:[email protected]

http://dx.doi.org/10.1016/j.ecolecon.2008.02.012

92 E C O L O G I C A L E C O N O M I C S 6 6 ( 2 0 0 8 ) 9 1 – 1 0 4

cases, Bayesian networks are extended to include uncertaintyregarding the costs and benefits of management decisions inwhat is known as influence diagrams.

The main aim of this paper is to evaluate the advantagesand disadvantages of using Bayesian networks for integratedassessment of the uncertain costs and benefits of eutrophica-tion abatement measures through its application in a casestudy. The Vansjø lakes in the Morsa catchment in South-Eastern Norway (Fig. 1) are ‘at risk’ of not meeting the EUWater Framework Directive's (WFD) requirement for ‘goodecological status’ (GES) by 2015 due to eutrophication. Giventhe continued algal blooms, even several years after imple-menting nutrient mitigation measures in upstream agricul-ture and wastewater treatment, managers wonder whethercurrent and supplementary measures will reduce nutrientconcentrations as predicted in previous impact assessments

Fig. 1 –Morsa catchment draining to the Va

(e.g. Lyche Solheim et al., 2001). Decision-makers are alsointerested in the relative uncertainty of mitigation costsversus benefits. We evaluate how well suited Bayesiannetworks are to deal with this task.

The WFD requires that a cost-effectiveness analysis of aprogramme of measures for attaining GES be conducted. Theenvironmental objectives can be lowered (objective deroga-tion) or delayed in time (time derogation) if the costs of thesemeasures can be shown to be ‘disproportionate’ (Sections 3–7,art. 4 WFD). As a first step in the assessment of costdisproportionality we conduct a simple test of whetherexpected costs exceed expected benefits under uncertaintyin our study. Consistent evaluation of the joint uncertaintiesunderlying the cost and benefit estimates in the economicanalysis has been identified as one of the main researchchallenges of WFD implementation (Brouwer, 2005). Bayesian

nsjø lakes including Lake Storefjorden.

93E C O L O G I C A L E C O N O M I C S 6 6 ( 2 0 0 8 ) 9 1 – 1 0 4

networks offer a framework for documenting and assessingthe probability of non-compliance with GES against uncertainabatement costs, as well as the probability that expected costsexceed expected benefits, by jointly assessing conditionalprobability distributions for abatement costs, water qualityeffects and benefits.

The paper is structured as follows. Section 2 reviewslimitations and advantages found in other applications ofBayesian networks to water management problems. Section 3provides a brief introduction to the use of Bayesian networksin a driver-pressure-state-impact (DPSI) approach to inte-grated river basinmodelling, as well as introducing the object-oriented model for the Morsa catchment. Section 4 presentstheMorsa catchment inmore detail, including a description ofthe data sets. Simulation models, surveys and expert evalua-tions are used to populate the network nodes with probabilitydistributions. Section 5 presents results of the application tocost-effectiveness and cost–benefit analysis. Section 6 dis-cusses some of the limitations of the Bayesian networkapproach in the Morsa catchment in assessing costs andbenefits in relation to some of the previous findings in theliterature. Section 7 provides some conclusions and directionsfor future work.

2. Bayesian networks in river basin modellingand management

Bayesian networks (BNs) have a number of generic featureswhich make them well suited for integrating the results ofdiverse models for the purpose of river basin management.The handful of studies that have focused on applications ofBNs to watershed management are summarised in the nextsection. Varis and Kuikka (1999) report their lessons learnedusing belief networks in nine case studies, several of whichinclude water resource management (restoration of a tempe-rate lake; real time monitoring system for a river; cost-effective wastewater treatment for a river). The authors notethat while methodological development in Bayesian decisionanalysis is progressing rapidly, the way to empirical applica-tion tends to be long. They see the lack of acceptance ofBayesian approaches among established scientific specialistsand in institutions with established management approachesas the main barrier to application. While research noveltymay be a limitation to policy application, previous studieshave argued that BNs applied to model integration in watermanagement also have limitations inherent to the methoditself.

Borsuk et al. (2001, 2004) used belief networks to integrate acombination of process-based models, multivariate regres-sions and expert opinion of river eutrophication to predictprobability distributions of policy-relevant ecosystem attri-butes. In addition to the strengths of BNs in eliciting expertinformation, the authors point out the tendency of experts to‘overload’ a network with “pet processes” related to theirown research, and the problems this can lead to in terms ofmodel complexity and information costs in defining a sto-chastic model. For parsimonious network design they recom-mend evaluating whether variables are (1) controllable, (2)observable, and (3) predictable at the scale of themanagement

problem, before inclusion in the network. Other limitationsidentified include BNs inability to explicitly represent systemfeedback relationships — BNs describe relationships as one-way causal influences at a particular instant in time or as netinfluences on eventual steady-state conditions. Also BNs facemodel validation problems, given that uncertainty may be inthe causal structure itself, as well as in parameter uncertaintyand natural variation that are captured by probability dis-tributions. For checking uncertainty in model structure, theauthors recommend Bayesian model averaging, learning fromadditional data, as well as rigorous model testing.

Varis and Lahtela (2002) use BNs to conduct scenarioanalysis for basin-wide policy impacts on different usergroups in the Senegal River. They show how BNs can dealwith the fact that benefit–cost analysis of basin-wide mea-sures do not generally account for non-quantifiable external-ities, and often do not conduct risk or uncertainty analysis.However, the authors reach the conclusion that little changecan be expected relative to the baseline scenario in the riverbasin even with rather strong management actions. Theirmodel is quite big with 45 variables and 840 linkages. Lowpolicy effectivenessmay have been due to the large number oflinkages combinedwith a low degree of discretization of only 6qualitative probability classes. A large number of model nodesdoes not automatically lead to higher uncertainty, if each stepis correctly evaluated, but does increase the probability thatsome processes in the overall network will be evaluatedincorrectly.

Ames et al. (2005) use BNs to model watershed manage-ment decisions with a case study application to phosphorusmanagement in a small catchment in Utah. Their BNsintegrated headwater and reservoir state variables with costof wastewater treatment and revenues from recreationallake use. Only a 1% increase in the probability of improvedrecreational conditions in the target lake is observed under themost effective scenario to reduce non-point source nutrientloading. Here too the authors point out the effect thatdiscretization has on the potential loss of information in aBN. On the other hand, discretization is also seen as a par-ticularly useful property in modelling variables with break-points or thresholds relevant to management. The authorsemphasise the need to validate completed BNs using inde-pendent information, but admit that this can be a challengewhen the BNs probability distributions are derived fromsources other than observed data, or when no new databecomes available for assessing the BN model. They there-fore recommend testing the model against adaptive manage-ment, third party expert opinion or at a minimum sensitivityanalysis.

The issues pointed out in the literature which are alsoraised in the present case study, include (i) limitations of anacyclical causal structure, (ii) a tendency of over-complexity ofnetwork structure relative to the scale of the managementproblem, (iii) sensitivity to discretisation of probability dis-tributions, (iv) cumulative uncertainty and resulting insen-sitivity of environmental objective variables to measures(v) selecting model validation approaches. In addition, thepaper addresses (vi) the implicit assumptions of geographicaland temporal scale in BN modelling, and (vii) correctspecification of correlation between probability distributions.


3. Methodology

Discrete BNs represent factorisations of joint probability dis-tributions over finite sets of discrete random variables. Var-iables are represented by nodes of the BN (describing variablesof the problem at hand), and the links of the network repre-sent the properties of conditional dependencies and indepen-dencies among the variables as dictated by the distribution.Each variable is specified as a probability distribution condi-tional on the configuration of its conditioning parent variables(Kjærulff and Madsen, 2005). The conditional probabilities areused tomodel how precisely we can describe the relationshipsof the variables. If uncertainties are high (e.g. low correlationsin statistical analysis) and the problem (model) is complex, theend result may be a model that does not promise markedchanges in the system impact variables due tomanagement ofdriver or pressure variables.

The nodes and links of a BN form a directed acyclic graph.The acyclical property is required to carry out the probabilitycalculus that make Bayesian decision analysis software soefficient, but implies that feedback effects are not modelled inone and the same network (Jenssen, 2001).

Fig. 2 illustrates the difference between a Bayesian net-work and influence diagram in the context of a generic driver-pressure-state-impact model (OECD, 1993; Eurostat, 1999), forexample for water quality management. In this stylised ex-ample the management context is made up of the states of anexogenous variable X conditioning water quality state S, andthe decision D on whether the pressure P mitigating measureis implemented or not. In this framework prior knowledgeof water quality can be expressed as a probability of a stateS given pressure P and exogenous variable X: Pr (S | P, X).Similarly the probability of nutrient loading pressure P de-pendent on the decision D is Pr (P | D). In an impact analysisa manager may be interested in determining the posteriorprobability for a state given a pressure and the states of

Fig. 2 – Influence diagrams and Bayesian networks in the contexDecision nodes are represented by rectangles, utility nodes by d

context variables c = c(D, X): Pr (S | P, c), or conversely alikelihood, expressed as a probability of pressure given thestate of given context variables: Pr (P | S, c). Bayes' rule (Eq. (1))expresses the relationship between the prior, likelihood andposterior probabilities.

Pr SjP; cð Þ ¼ Pr PjS; cð Þ � Pr Sjcð ÞPr Pjcð Þ ð1Þ

In this case study we use commercially available software(Hugin Expert®: www.hugin.com) to implement Bayesian cal-culus using influence diagrams to evaluate the expected valueof utility functions of decision alternatives. Whereas a BN is amodel for reasoning under uncertainty, an influence diagram(ID) is a probabilistic network for reasoning about decision-making under uncertainty (Kjærulff and Madsen, 2005).Influence diagrams are close relatives of decision trees (e.g.Clemen, 1996), but their more explicit graphical presentationof cause–effect linkages offer non-specialists a better chanceto understand integrated model structures. Referring back tothe generic influence diagram in Fig. 2, the software depictsdecision nodes as rectangles (exogenous policy drivers). Utilitynodes representing impacts of decisions (costs and benefits)are depicted as diamonds. Chance nodes (ovals) are used todepict exogenous variables described by unconditional prob-ability distributions, as well as endogenous variables de-scribed by joint probability distributions conditional on thestates of one or more parent nodes. Influence diagrams withdecision and utility nodes estimate expected (net) utility ofdecisions accounting for all probability distributions of thenetwork. The BN structure is driven by the conceptualunderstanding of the problem, but also the detail of existingstudies or models that are used to describe each link of theDPSI chain.

When multiple sources of information at different scalesand levels of abstraction (model simulations, data-correla-tions and expert judgement) need to be combined, an object-

t of DPSIR (Driver–Pressure–State–Impact–Respons). Note:iamonds and chance nodes by ovals.

http://www.hugin.com


oriented Bayesian network (OOBN) is convenient (Kjærulff andMadsen, 2005). This is a hierarchically specified probabilisticnetworkwhere the objects can be either BNs or IDs, and can beconsidered as sub-models of the overall network. In this studywe use OOBNs to organise information from a number ofmodels, data sets and expert evaluations. The various OOBNscan be developed by different experts and then linked togetherto describe the expected behaviour of the whole system.

As discussed in the literature review, model validation is achallenge in Bayesian decision models. This is especially thecase if we are interested in predicting the outcomes ofmanagement options which have never been applied to thegiven area. We use the information analysis1 function inHugin Expert® to determine which variables contribute mostto uncertainty in the OOBN. Uncertainty is reduced withadditional information defined in terms of changes in theprobability distribution of a hypothesis variable of manage-ment interest (e.g. willingness to pay for increased bathingsuitability). If P(T) is the probability density function of thehypothesis variable T then an indicator of information (V)— orconversely uncertainty (H) — can be defined as:

V Tð Þ ¼ �H Tð Þ ¼X

TP Tð Þ log P Tð Þð Þ ð2Þ

A higher value of H(T) is interpreted as more uncertaintyabout the true state of, in our example above, willingness topay. An indicator of the reduction in uncertainty about T fromobservations of a conditional variable X (e.g. backgroundnutrient loading) can be defined as I(X,T) where:2

V TjXð Þ ¼ � H Tð Þ � I X;Tð Þð Þ ð3Þ

In the example above I(X1,T) N I(X2,T) would tell us thatuncertainty about the expected value of willingness to pay isreduced more by new observations on background nutrientloading (X1) than some other conditional variable such asnutrient-reduction effect of tillage (X2). This is a convenientapproach to sensitivity analysis in large OOBNs, indicatingwhich variables should be observed first to decrease theuncertainty in any hypothesis variable of interest.

Fig. 3 shows the highest level of the OOBN for the Morsacatchment, with an indication of some of the network objects atmoredisaggregate levels.TheOOBNcanbeunderstoodasameta-model (Varis, 1997) — probability distributions of the networknodes may be thought of as ‘response surfaces’ that summariseunderlyingmodel simulation results, data distributions or expertopinion, without having to process the model code, data orexperts themselves each time the network is evaluated.

The overall network shows the decision nodes for the fourmanagement measures under consideration — changedtillage practices, sedimentation dams, vegetation buffer strips,and individual residential waste water treatment. As anexample of an OOBN, the ‘tillage cost/effect’ object in turncontains an object representing a phosphorus-loss regressionmodel. Each abatement measure is represented by an object

1 Not to be mistaken with value-of-information analysis, whichindicates the utility (monetary or otherwise) of decreasing theuncertainty in a variable that specifically conditions a manage-ment decision entailing costs and benefits (Clemen, 1996).2 The Hugin Expert® manual refers to H as an “entropy

indicator” and I as a “mutual information statistic”.

containing the BNs used to calculate its cost and phosphorusloading effect, which is output to two effect nodes and a cost/effect node. These effect nodes are aggregated to non-pointand point source reduction nodes for dissolved (DIP) andparticulate (PIP) inorganic phosphorus. The cost/effect node isused as such for cost-effectiveness analysis (Fig. 3).

DIP and PIP effects are input to Lake Storefjorden object,which contains the BN that represents eutrophication andalgal bloom models and criteria for bathing suitability.Suitability for bathing is determined by the water quality ofthe lake, which in turn depends on baseline loading andreductions in loading due to measures (Fig. 4). The tempera-ture of the lake has an impact on bathing suitability aswell, i.e.if water temperature happens to be less than is suitable in agiven bathing season, a higher probability of suitable waterquality does not lead to a higher probability of recreationalbenefits. Bathing suitability is in turn input into the ‘economicimpact recreation’ object which contains a BN describinghousehold willingness-to-pay (WTP) for bathing suitability.Notice that this impact does not cycle back as a response orfeedback effect to the decision node due to the acyclicrequirement of BNs (the R is missing from the well-knownDPSIR framework for river basin modelling).

4. Case study area and available data

4.1. Catchment description

The Morsa catchment area is approximately 700km2, locatedin south-eastern Norway, including outskirts of Oslo in thenorth and a number of smaller lakes draining into the RiverHobøl which runs into Lakes Storefjorden and Vanemfjorden(collectively known as Vansjø) in the south, which in turndrain into the Oslofjord through the town of Moss (Fig. 1).Land-use in the catchment is mainly forest and agriculturewith the urban areas found in the southern end of thecatchment. Population in the headwaters is mainly found indispersed residential areas and farm housing. Lake Vanemf-jorden is a highly eutrophic lake with frequent cyanobacteriablooms coinciding with the bathing season in June–August.Water enters Lake Vanemfjorden from Lake Storefjordenwhich has been the focus of upstreammanagementmeasuresto date. Total phosphorus (Tot-P) loading into Storefjorden inthe period 1984–2000 varied between 5000 and 25,000kg Tot-P/year (median 12,000kg/yr) (Lyche Solheim et al., 2001). In 2000the main sources of nutrient loading were agriculture (57%),septic tanks from individual households (11%), municipalwastewater (6%) and natural background run-off (26%). Thelimiting nutrient in freshwaters, and focus of abatementefforts in the upper watershed, is phosphorus. The catchmenthas little industry of significance and other water qualityissues are marginal compared to nutrient loading.

Lake Storefjorden provides drinkingwater supply for 60,000people through the MOVAR water supply utility. Thirty onethousand people live within the catchmentmunicipalities andconstitute a conservative estimate of the potential recrea-tional user population of Lakes Storefjorden and Vanemfjor-den . Due to algal blooms and heavy turbidity in spring floods,MOVARhas recently invested approximately 10million Euro in

Fig. 3 –Object-oriented Bayesian network model for nutrient abatement in Morsa catchment. Note: grey ovals representunderlying sub-networks; orange boxes represent decisions on implementation of measures(true/false); white ovals representnodes with conditional probability distributions.


additional active coal, flushing and ozone treatment ofdrinking water. Basic nutrient abatement measures thathave been implemented in the catchment in the last decadehave increased annual costs in agriculture (i.e. yield loss,maintenance of buffer strips, sedimentation dams, grassywater courses), and for households (septic tank upgrades,connection to municipal treatment). Monitoring data for 2005show a reduction in Tot-P concentrations in Lake Storefjordenrelative to 2000 (Bjørndalen et al., 2006), but this reduction fallswithin the range of variation in Tot-P concentrations in thelake since 1984. The most recent measurements of Chlor-ophyll a (ChlA) concentrations are as low as the lowest valuesince 1984, but variability relative to Tot-P concentrations alsosuggests that it is too early to tell whether management

measures taken so far actually have had an impact on waterquality. Our analysis focuses on Lake Storefjorden usingavailable monitoring data for upstream measures until theyear 2000. Conclusions in this study therefore do not reflecttrends or variability after this point in time. In the followingsections the data is summarised along with some features ofthe OOBN. A fuller and more detailed presentation of thenetwork and the data can be found in Barton et al. (2006).

4.2. Abatement measures' costs and effects

Upstream abatement cost data are based on an original cost-effectiveness analysis conducted by Lyche Solheim et al. (2001)in the catchment. In that study the authors estimated the total

Fig. 4 –Lake Storefjorden state (instance) describing eutrophication and bathing suitability. Note: DIP:dissolved inorganic P; PIP:particulate inorganic P; SIS: suspended inorganic sediment; ChlA: Cholophyll A; %cyano: cyanobacteria as % of algal biomass.Nodes with grey rings indicate input and output nodes to other sub-networks.


costs of implementing agricultural and individual wastewatertreatment measures comparable to the ones considered herein the order of 11–13million NOK/yr.3 The sub-networks foragricultural measures have been revised in the study reportedhere compared to Barton et al. (2005) based on more recentdiscussions with experts on run-off. Probability distributionsreflect a variety of data sources including expert opinion,empirical data and regression model results. Where available,the minimum–maximum range for costs of measures wereevaluated as uniform probability distributions. Where onlypoint estimates are available, additional studies were exam-ined and data points assessed as discrete distributions.Whereno other studies are available the authors were advised byexperts to use triangular distributions (min, median, max).Abatement costs are financial costs to farmers, corrected forany transfers, and are a first order assessment of economicopportunity costs common to feasibility analyses carried outat farm level. Conditional probabilities predicting Tot-P in run-off as a function of soil P, run-off and erosion-risk wereadapted from the USLE-based regression model applied in

3 In 2000 1 NOK equalled approximately 0.11 US$. For thepurpose of future mitigation decisions, downstream investmentsin drinking water treatment are considered sunk costs in this casestudy. Because water treatment operating costs depend onlymarginally on the severity of an algal bloom thanks to the watertreatment upgrades, additional benefits of nutrient mitigation arefurthermore expected to be small and therefore left out of theanalysis.

Eggestad et al. (2001), including parameter distributions(mean, std. error) obtained from these authors. Uncertaintyregarding erosion risk inMorsawasmodelled as the variabilityin erosion risk classes for a statistical region, which includesmunicipalities inside and outside the Morsa catchment. A flatdistribution of variation in soil P was used in the model as thedistribution across the catchment is unknown. Changes in theland use erosion factors (C-factors) define the effectiveness ofchanged tillage practices. Uncertainty regarding C-factors wasbased on expert opinion.

4.3. Water quality simulation

The ‘Storefjord lake state’ object, describing eutrophicationand bathing suitability, is driven by changes in PIP and DIPloading as shown in Fig. 4. Themainmodel forcing in our casestudy are the daily time series of meteorological, hydrologicaland nutrient loading data for the 16-year simulation period1985–2000. The dynamic process-based model MyLake (Salor-anta and Andersen, 2007) was used to simulate the watersurface temperature, as well as the relationship betweendifferent PIP and DIP loads and the daily concentration of Tot-P and ChlA in Storefjord Lake. The corresponding conditionalprobability tables (CPTs) for these relationshipswere producedby running themodel repeatedlywith different parameter andinput factor values in a Monte Carlo simulation. MyLake is aone-dimensional model code for the simulation of the dailyvertical distribution of lake water temperature and thus


density stratification, the evolution of seasonal lake ice andsnow cover, sediment–water interactions, and phosphorus–phytoplankton dynamics. The basic idea behind MyLake hasbeen to include only the significant physical, chemical andbiological processes in a well-balanced and robust way. Themodel code, its calibration and application to Lake Storefjor-den is described in more detail in Saloranta (2006) andSaloranta and Andersen (2007).

Water quality criteria (Tot-P, ChlA, the proportion ofcyanobacteria and temperature) were used as a proxy forrecreational suitability in general. This ‘lake state’ object iscentral to the whole network and allows a comparison of costsof measures and benefits of improvements in recreationalsuitability. The impact of reductions in loading on bathingsuitability is given as the difference in probability of suitableconditions between the state of Lake Storefjorden before andafter implementation of a programme of measures. Theeffectiveness of measures to improve water quality in LakeStorefjorden is measured against the water authorities'operational limit values for ‘good ecological status’ (GES) for(i) cyanobacteria as a percentage of algal biomass (10%) and(ii) Tot-P (11–14μg Tot-P/l) (pers. com. H. Gunnarsdottir , Morsaproject).

The conditional probabilities for Secchi depth reflectingbathing water transparency and the percentage of cyanobac-teria and ChlA were computed using data from Norwegianlakes, all collected by NIVA. The sampling period spans 1972 to2002, but a large proportion of the samples were taken duringthe national eutrophication survey in 1988. Earlier analyses ofthese data are reported by Lyche Solheim et al. (2004). Theproportion of cyanobacteria is calculated as the biomass of allcyanobacteria (except the genus Merismopedia) divided by thetotal phytoplankton biomass. The proportion of cyanobacteriagenerally increases with ChlA concentration, but the phyto-plankton community may also be affected by factors such asalkalinity and humic content (Lyche Solheim et al., 2003). LakeStorefjorden belongs to the low-alkalinity, high-humus lakegroup. However, the lake chemistry is close to the limit forboth typology parameters (4mg/L Ca and 5mg/L TOC).We havetherefore included two states (high and low) for both of thesetwo parameters, so that the network can cover all four lakegroups assuming an equal probability for each.

In order to obtain a data set that is representative for LakeStorefjorden, and at the same time contains enough samplesto parameterise the probability tables, we first compared therelationship between ChlA and the proportion of cyanobac-teria for different combinations of geographic ranges and laketypes.4 This relationship was estimated by a non-parametricgeneralised additive regression model. The smallest data set(Eastern Norway, Lake Storefjorden type) was not sufficient tocover the whole eutrophication gradient of interest, while thefull data set (Nordic countries, all lake types) resulted in morenoise (data obtained from www.rbm-toolbox.net/rebecca).The best fit (n = 481, R2 = 0.50) was obtained with data fromlowland lakes (b 200m above sea level) of all types in EasternNorway, including alkalinity and humus levels as categoricalcovariables. Next, the entries for the CPT for the percentage of

4 Based on the Lakes database taken from the EU projectREBECCA (www.rbm-toolbox.net/rebecca).

cyanobacteria were calculated as the number of observationsper cyanobacteria state for each combination of states of ChlA,alkalinity and humic content. Likewise, the CPT for Secchidepth states was calculated as the number of observations perSecchi state for each state of ChlA. Note that the CPTs forcyanobacteria and Secchi depth are based on raw observations,contrary to the CPT for ChlA, which is based on modelsimulations. The empirically observed CPTs for the formercontainmore natural variability than themodel-based CPT forChlA.

4.4. Willingness to pay for bathing suitability

Mean householdWTP as an indicator of the economic benefitsof improvements in the suitability of water in the Vansjø lakeswas obtained from Magnussen et al. (1995). WTP wasdetermined through a representative contingent valuation(CV) survey conducted in 1994 targeting 300 randomly selectedhouseholds in the Morsa catchment. Households were askedfor their incremental WTP over and above their current waterbill for a programme of measures that would improve lakequality from a currently “poorly suited” state to a “well suited”state for bathing, boating, fishing and drinking water. A waterquality ladder was used to depict the different states, wherethe baseline situation showed water quality suitable forboating and fishing, but not for bathing and drinking. Whileincremental change in suitability is related to multiple wateruses in the CV scenario, the upgrade of drinking watertreatment since the original study was conducted meansthat the main use and user benefits in the future will berelated to water-based recreation, with bathing requiring thehighest water quality (higher even than water to be treated fordrinking). Non-user values for ecosystem improvements ofhouseholds living outside the catchment were not sampled.However, the large number of lakes in the region and resultsfrom other valuation studies of fjord water quality improve-ment suggest that for local water bodies in Norway both userand non-user values are predominant (Magnussen and Berg-land, 1996). While the benefit transfer literature suggests thatWTP estimates may not be ‘time stable’ for periods of oversome 5years (Brouwer and Bateman, 2005), the available WTPvalues were not corrected for any possible changes incontextual circumstances. Given that water quality hasdeteriorated since the mid-nineties in the Vestre Vansjø andStorefjorden lakes in the catchment it is possible that thetransfer of the WTP values from 1994 results in an under-estimation of current WTP for recreational water quality.Overall, the transferred WTP may on the one hand be anunderestimate of the worsened conditions during the pastdecade since the original study was carried out and fail tocapture user and non-user values of households outside thecatchment, while on the other hand it may overestate therecreation use value in Lake Storefjorden as part of the widerVansjø lakes. While the standard error of the original WTPestimate is incorporated in our BN, the uncertainties related tothe benefits transfer described above have not. While it is rareto have an on-site water quality valuation study available forbenefits assessment — even one that is more than tenyearsold — the explicit temporal and geographical context of theabatement andwater qualitymodels in our BN highlight some

http://www.rbm-toolbox.net/rebecca

http://www.rbm-toolbox.net/rebecca


un-quantified uncertainties inherent in the explicit transferand integration of economic value estimates.

In our network uncertainty about the total benefits estima-tion is determined jointly by the error of the average WTPestimate in the original study (E(WTP) = 2269NOK per house-hold per year, standard error (WTP) = 200NOK per householdper year5), aswell as the error in the aggregation procedure, i.e.the total number of households overwhich the average benefitvalue is to be aggregated (between 10,000 and 11,000 house-holds by 2015 (Norwegian Bureau of Statistics, 2007). Theexpected aggregate WTP for water quality if suitability passedfrom 100% unsuitable to 100% suitable for bathing would be inthe order of magnitude of 24million NOK/yr. In the modelobject “Economic impact on recreation” expected WTP forimprovements in bathing suitability is determined by theprobability that bathing in the lake goes from “unsuitable” instate 0 without measures to “suitable” in state 1 withmeasures. We calculate expected aggregate WTP as the jointprobability of bathing suitabilitywithmeasures being true andbathing suitabilitywithout measures being false, multiplied byWTP aggregated across the number of households in munici-palities neighbouring the lake. In other words, theWTP resultsonly play a role if the mitigation measures lead to an increasein the probability of a positive binary change in suitability.While bathing suitability could also have been defined on astepwise scale, suitability was defined to fit the binary natureof the originalWTP results. If P is interpreted as the proportionof days in a season likely to have bathing conditions, then weare assuming that WTP is proportional to the length of theseason. Ideally, a CV study conducted specifically for applica-tion in a BN would provide a continuum of WTP estimates fordifferent probabilities of bathing conditions in any givenseason, but this is not the case for the results taken fromMagnussen et al. (1995). Finally, due to natural variability ofprocesses in the catchment and water body there are randomincreases in water suitability between two periods, i.e. notrelated to the implementation ofmeasures. In otherwords, theBN also calculates expected benefits of a water qualityimprovement which cannot be attributed to the implementa-tion of the measures. We subtract such “windfall” benefitsfrom expected benefits “with measures” in order to get acorrect estimation of the incremental benefits.

5. Results

5.1. Cost-effectiveness of measures accounting foruncertainty

A convenient feature of Hugin software for model evaluationis the viewing of probability distributions within the BN. InFig. 5, the nodes called “kr/kg” (NOK per kilogram) shownpreviously in Fig. 3, have been expanded to show eachmeasure's cost-effectiveness. Cost-effectiveness is measuredhere as kr/kg phosphorus on-site at the ‘end-of-pipe’ for each

5 WTP values were adjusted to the year 2000 using theconsumer price index (CPI) to make them comparable with thecosts of measures in the same year.

nutrient abatement measure, i.e. without regard to theeutrophication effect downstream in Lake Storefjorden. Visualinspection of the probability distributions at the top of Fig. 5indicate that the measure “buffer strip” is the most cost-effective, followed by similar cost-effectiveness for tillage andsedimentation dams, while individual wastewater treatmentis least cost-effective.While the ranking ofmeasures is similarto that of Lyche Solheim et al. (2001), the difference from theoriginal deterministic cost-effectiveness analysis is that anunequivocal ranking of measures is more difficult due to theuncertainty incorporated both in the cost and effect assess-ment. If budget constraints forced managers to choosebetween the similarly cost-effective tillage and sedimentationdam measures, inspection of probability distributions in theirunderlying network nodes help identify and visualise whatcost or effect assumptions dominate cost-effect uncertaintyand where to gather more information.

5.2. Costs and benefits of measures accounting foruncertainty

In the scenario presented in Fig. 5, the expected net benefits ofall the measures taken together is − 4.9million NOK per year.Thenet benefits due to the incremental effect of implementingall measures while accounting for the random water qualityimprovements caused by natural variation in the absence ofmeasures (2.7million NOK/yr) is − 7.6million NOK per year. In adeterministic benefit–cost analysis, i.e. where the costs ofmeasures are assumed certain and abatement measures areexpected to be 100% effective in reducing eutrophication, andthere is no uncertainty regarding householdWTP, net benefitswould be around 16million NOK per year. Hence, the imple-mentation of the proposed measures results in a welfareimprovement assuming there is no uncertainty.

When uncertainty in the DPSI chain is modelled explicitly,the probability of bathing suitability increases from 18%without to 28% with abatement measures (i.e. in a 100daybathing season abatement measures would provide anexpected 10 additional days suitable for swimming). Thislow probability (multiplied with the aggregate WTP value) isthemain reason for the net benefits to turn out negative underuncertainty. The uncertainty in the nodes for P-loading, P-concentrations, algal biomass and cyanobacteria blooms ispropagated through the network and substantially reducesthe expected value of the benefits involved. In a deterministicanalysis, this low probability of reaching suitable bathingconditions would simply be ignored and the aggregate WTPvalue applied unconditionally in the CBA.

An inspection of the expected benefits of each measure onits own reveals a more varied picture of net benefits (bufferstrips: 0.9million kr/yr; reduced tillage: − 0.1million kr/yr;sedimentation dams: 0.5million kr/yr; and individual waste-water treatment: − 6.0million kr/yr). Due to non-linearities inthe environmental part of the network, expected net benefitsof the whole programme of measures is not equal to the sumof the expected benefits for each individual measure. Ofinterest here is furthermore how costs and benefits relate toreaching GES. When only agricultural measures are imple-mented there is a 49% probability that Tot-P concentrations,and a 19% probability that cyanobacteria as a percentage of

Fig. 5 –Evaluating the uncertain benefits of a programme of measures. Note: nodes in networks are presented with selectedprobability density functions (first row probabilities; second row discretisation intervals). Square decision nodes displayexpected utilities of decisions. Expected net benefits of implementing all themeasures is −4. 9million kroner (decision nodes attop are set “true”). This is due to an increase in bathing suitability from 17.8% to 28.4% of summer season with fullimplmentation of measures.


algal biomass exceed the water authorities' limit values forGES (see Section 4.3). With all measures, including individualwastewater treatment plants, this drops only a little to 44 and18% respectively. Hence, agricultural measures dominate themodel outcomes under uncertainty. However, while some ofthe agricultural measures have positive expected net benefitswhen implemented individually they are insufficient to reachGES. After the first agricultural measure has been implemen-ted, successive measures do little in the model to increasebathing suitability. Net benefits of implementing all agricul-tural measures together are − 0.6million NOK per year. Apartfrom non-linearities in the overall network, the coarse

discretization of the baseline nutrient loading nodes (intervalsof 3000kg P/yr) relative to the effect of the individual measures(median effectiveness of 100–500kg P/yr) is the principleexplanation for the lack of sensitivity of bathing suitabilityin the model.

6. Discussion

Which variables contribute most to the reduction of uncer-tainty about a hypothesis variable such as WTP? Of all nodesin the main network in Fig. 3, the information-analysis in


Hugin reveals that the “baseline PIP loading” has the highestinformation statistic for the hypothesis variable ‘WTP total’.We evaluated the effect of using different probability distribu-tions and discretizations in the ‘baseline PIP loading’ node(Table 1). This addresses some of the concerns raised byauthors in previous case studies regarding assumptions aboutprobability distributions in chance nodes. Referring to theresults in Table 1, a uniform distribution has the highestinformation score — new observations of “baseline PIPloading” have the highest impact on WTP, because a uniformdistribution contains the least information about the truevalue of the variable. Finer discretization intervals have ahigher information score. An empirically derived distributionhas a higher information score than a parametric normal

Table 1 – Value of information analysis example

Note: Entropy indicator H(WTP) is score for the information in WTP — a higher score is interpreted as more uncertainty about the true state ofthe node. Themutual information statistic (I(WTP|PIP baseline load)) scores how sensitiveWTP is to new observations of “PIP baseline load”. Thehigher the score, the more information is gained aboutWTP by getting new observations on “PIP baseline load”. The analysis in Table 1 has beenrun without any other instantiations than those shown — the results therefore differ from the scenario analysed in Fig. 5.

Table 1–Value of information analysis example

distribution (holding discretization constant). Finally, evi-dence in the form of a 100% probability of a given intervalprovides most information of all because we assume knowl-edge of the discrete interval in which the true state of the”baseline PIP loading” node lies.

While increasing discretization and empirical distributionsare preferred to estimated distributions in terms of themutualinformation score, the impact on the uncertainty of thehypothesis variable (WTP) does not follow directly. Table 1shows that expected WTP is bimodal as it is conditional onthreshold definitions of bathing suitability. When a reductionin “PIP baseline loading” leads to an increase in expectedWTP(from a large probability mass at zero) this also increasesbimodality as probability at zero is shifted to a probability of


positive aggregate WTP. For example, uncertainty aboutexpected WTP is greater when the interval for “PIP baselineloading” is known but relatively low (H = 0.4214; last row inTable 1) compared to when we use the empirical distributionwith relatively high but variable historical PIP loading (H =0.2687; third row in Table 1).

Propagated uncertainty in our linked models is largeenough to cancel out net benefits of the programme ofmeasures in a deterministic model. While this is in partdue to discretization assumptions in the network, the loweffectiveness of measures in our model is in accordancewith the lack of effectiveness of eutrophication measuresobserved in Lake Storefjorden (Bjørndalen et al., 2006).Hysteresis has been suggested as an explanation for thiskind of lack of response to measures in shallow lakes(Scheffer and Carpenter, 2003). This implies that theecological degradation is not simply reversible — degrada-tion of a lake will start when the phosphorus pressure hasreached an ecological threshold, but the ecological restora-tion process will not start until the phosphorus pressure isreduced to a level much below this threshold. A hysteresissituation can be indicated by multiple thresholds andalternative stable states, given sufficient data. The BNapproach is in principle suitable for detecting alternativestable states (as peaks in different categories). For our casestudy, however, there is currently no sufficient data todemonstrate such multiple states and to support thehypothesis of hysteresis.

In addition to uncertainty due to lacking data, causalstructures and natural variability, our study revealed anumber of modelling challenges that will likely be commonto any application of BNs (Uusitalo, 2007) some of which havebeen addressed in studies reviewed earlier.

6.1. Model complexity

There may be too many nodes in the network relative to anoptimal problem formulation (Borsuk et al., 2004; Varis andLahtela, 2002). A further evaluation of network techniques istherefore needed (e.g. parent-divorcing versus simplifica-tion of discretization intervals). In our case, some nodesmay be redundant due to a lack of abatement effect (e.g.dissolved organic phosphorus is not affected by agriculturalmeasures).

6.2. Discretization

Relative to a continuous probability function, there is someinformation loss at each node due to discretization assump-tions (Varis and Lahtela, 2002; Ames et al., 2005). How muchinformation is lost overall in the network, and how discretiza-tion of a particular node is propagated to the variables ofinterest to decision-making will depend inter alia on whetherthe resolution is increasing or decreasing in the causalitydirection of the network, and particularly the resolution of themost sensitive variables. In our case, especially the coarsediscretization of nutrient loading nodes in the lake waterquality model significantly affected the estimated effective-ness of certain abatement measures compared to the baselinesituation.

6.3. Parametrization of conditional probability tables

Simple empirical data correlations (e.g. ChlA and percentageof cyanobacteria) embody more uncertainty than modelpredictions (e.g. erosion risk run-off regression model) andsimulations (e.g. MyLake water quality model). The choicebetween empirical data correlations andmodel simulations inthe meta-model is a topic that belongs to scientists, but theuncertainties caused by these choices should be evaluated anddiscussed with decision-makers. What best represents “cur-rent knowledge” about the system and whether it can beindependently verified using other data or third party expertopinion will determine the model's credibility as a basis fordecision-making (Ames et al., 2005).

6.4. Implicit temporal and geographical scale andvariability

The BN practitioner has to make sure that the variabilitydescribed in chance nodes reflects the same temporal andspatial scales and resolution throughout the network. Thisconsistency requirement is often hard to meet for a modelintegrating different disciplines with different samplingregimes. An example in our case study concerns the numberof assumptions required to couple existing WTP estimates topredictions of bathing suitability from thewater qualitymodels.

6.5. Probabilistic analysis of the models used to estimatesome of the input–output relationships

As we used for instance published regression models in someparts of the model and a validated and calibrated lake modelin another part of the model, it is crucial that the jointuncertainty analysis of thesemodels is carried out correctly. If,for example, the correlation of the regression parameters isnot included in the analysis using the variance–covariancematrix results, it may be that the probabilistic model predic-tions show too high uncertainties. Markov chain Monte Carlo(MCMC) techniques (e.g. Gamerman (1999)) enable a morecorrect analysis of parameter uncertainties, but this techniquewas not used in our model applications (e.g. MyLake).

6.6. Modelling response variables and feedback

For basin-wide modelling of eutrophication problems theacyclical properties of BNs may not be a major limitation,because of the predominantly one-way causality of hydro-logical processes, the fact that eutrophication is dominated byintra-annual processes (exceptwhere for example sediment P-source is involved), and the fact that the economic analysis ofabatement measures is undertaken on a progressive annualbasis. In other words, a single network object can capture themost important eutrophication processes and economicconsiderations. For the analysis of the abatement of persistentpollutants or inter-annual water allocation problems, a time-slicing approach (Jenssen, 2001) will need to be used to capturesome of the relevant dynamics underlying these watermanagement problems. Another alternative is to use aprobabilistic simulation model that includes loops and timesteps, and to use only the input–output combinations of these


models in the meta-model of BN. Because each OOBN isreplicated for each time period of interest, parsimony inmodelspecification is crucial.

7. Conclusions

In this paperwe demonstrated the use of Bayesian networks forthe evaluation of the costs and benefits of the implementationof theEUWaterFrameworkDirective.Anumberof limitationsofBayesian networks identified in the literature are elaboratedthrough our application to benefit–cost analysis of nutrientabatementmeasures in theMorsa catchment.While a determi-nistic calculus shows the annual net benefits of a programmeofmeasures to be positive, explicitly accounting for the uncer-tainty across integrated models in a BN, i.e. probability basednetwork, shows that expected costs exceed the expectedbenefits. The relative lack of effectiveness of the programmeof measures shown by our model may be counter-intuitive tomanagers used to working with deterministic models. Thisunderlines the point that the integration andmulti-disciplinaryprocess of defining a network, determining its probabilitydistributions and conducting sensitivity analysis may be moreimportant than the results of the analysis itself. In the reviewprocess for this paper the Bayesian network was qualitycontrolled by external reviewers, who spotted potential draw-backs and errors in a matter of a few hours, showing that aBayesian network can portray a complexmanagement problemin an easily accessible fashion. While we see Bayesian decisionanalysis as an important addition to river basin managers'toolbox, we feel that further work is needed on the limitationsidentified above before Bayesian networks gain wider appeal inintegrated management of water resources.

Acknowledgements

Thisworkwas supported by the EU researchprojects “North SeaRegional and Local Implementation of the Water FrameworkDirective” (NOLIMP-WFD) and “Benchmark Models for theWater Framework Directive (BMW)” and “Bayesian networkintegration of nutrient loading and lake eutrophication modelsin cost-effectiveness analysis of abatement measures” (Eutro-Bayes), supported by the Norwegian Research Council. We aregrateful to two anonymous referees for their useful commentson an early draft of the paper and in particular Jan Vermaat andRoy Brouwer from the Institute for Environmental Studies, VrijeUniversiteit Amsterdam for their extensive critical reviewof thevarious paper versions.

R E F E R E N C E S

Ames, D.P., Nielson, B.T., et al., 2005. Using Bayesian networks tomodel watershedmanagement decisions: an East Canyon Creekcase study. Journal of Hydroinformatics 07 (04), 267–282.

Barton,D.N.,Saloranta,T.,Bakken,T.H.,Lyche-Solheim,A.,Moe,J.,Selvik,J.R.,Vagstad,N.,2005.UsingBayesiannetworkmodelstoincorporateuncertaintyintheeconomicanalysisofpollutionabatementmeasuresundertheWaterFrameworkDirective.WaterScienceandTechnology:WaterSupply5(6),95–104.

Barton, D.N., Saloranta, T., et al., 2006. Using Bayesian beliefnetworks in pollution abatement planning. Example fromMorsacatchment, South Eastern Norway. NIVA Report SNO 5213–2006.

Bjørndalen, K., Andersen, T., Borgvang, S.A., Hobæk, A., Kaasa, B.,Rohrlack, T., Skjelbred, B., 2006. Utredninger Vansjø 2005.Kartlegging av vannkvaliteten. Delrapport 1. NIVAReport 5143–2006.

Borsuk, M., Clemen, R., Maguire, L., Reckhow, K., 2001. Stakeholdervalues and scientific modeling in the Neuse River watershed.Group Decision and Negotiation 10, 355–373.

Borsuk, M.E., Stow, C.A., Reckhow, K.H., 2004. A Bayesian networkof eutrophication models for synthesis, prediction, anduncertainty analysis. Ecological Modelling 173, 219–239.

Bromley, J., Jackson,N.A., Clymer,O.J., Giacomello, A.M., Jensen, F.V.,2005. The use ofHugin to develop Bayesian networks as an aid tointegrated water resource planning. Environmental Modelling &Software 20, 231–242.

Brouwer, R., 2005. Uncertainties in the economic analysis of theEuropean Water Framework Directive. IVM report E05-03.July, 2005. Vrije Universiteit Amsterdam.

Brouwer, R., Bateman, I.J., 2005. The temporal stability andtransferability of models of willingness to pay for flood controland wetland conservation. Water Resources Research 41 (3),W03017. doi:10.1029/2004WR003466.

Clemen, R.T., 1996. Making hard decisions, 2nd edition. PWS Kent,Boston. 664 pp.

Eggestad, H.E., Vagstad, N., Bechmann, M., 2001. Losses of Nitrogenand Phosphorus from Norwegian Agriculture to the OSPARproblem area. Jordforsk report 99/01.

Eurostat, 1999. Towards environmental pressure indicators for theEU. Panorama of the European Union. European Commissionand Eurostat. First edition 1999.

Gamerman, D., 1999. Markov Chain Monte Carlo: Stochasticsimulation for Bayesian inference. Chapman & Hall, London,UK.

Hein, L.G., 2006. “Cost-efficient eutrophication control in a shallowlake ecosystem subject to two steady states". EcologicalEconomics 59 (4), 429–439.

Henriksen, H.J., Rasmussen, P., Brandt, G., von Bülowb, D., Jensen,F.V., 2004. Engaging stakeholders in construction andvalidation of Bayesian belief networks for groundwaterprotection. Mimeo, Geological Survey of Denmark andGreenland (GEUS).

Jenssen, F.V., 2001. Bayesian networks and decision graphs.Statistics for Engineering and Information Science. Springer,New York, p. 263.

Kjærulff, U.B., Madsen, A.L., 2005. Probabilistic Networks — AnIntroduction to Bayesian Networks and Influence Diagrams,Department of Computer Science, Aalborg University andHUGIN Expert A/S.

Labiosa, W., Leckie, J., Shachter, R., Freyberg, D., Rytuba, J., 2005.Incorporating Uncertainty in Watershed ManagementDecision-Making: A Mercury TMDL Case Study. ASCEWatershedManagement Conference,ManagingWatersheds for Human andNatural Impacts: Engineering, Ecological, and EconomicChallenges. Williamsburg, VA, July 19–22, 2005.

Lyche Solheim, A., Vagstad, N., Kraft, P., Løvstad, Ø., Skoglund, S.,Turtumøygard, S., Selvik, J., 2001. Tiltaksanalyse for Morsa.Vannsjø-Hobøl Vassdraget. Sluttrapport, NIVA, Jordforsk,Limnoconsult. NIVA rapport 4377–2001.

Lyche Solheim, A., Andersen, T., Brettum, P., Erikstad, L., Fjellheim,A., Halvorsen, G., Hesthagen, T., Lindstrøm, E.-A., Mjelde, M.,Raddum, G., Saloranta, T., Schartau, A.K., Tjomsland, T., ogWalseng, B., 2003. Foreløpig forslag til system for typifisering avnorske ferskvannsforekomster og for beskrivelse avreferansetilstand, samt forslag tilreferansenettverk.Oppdragsgivere, SFT, DN og NVE. NIVA-rapport 4634-2003.

Lyche Solheim, A., Andersen, T., Brettum, P., Bækken, T., Bongard,T., Moy, F., Kroglund, T., Olsgard, F., Rygg, B., Oug, E., 2004.

http://dx.doi.org/10.1029/2004WR003466


BIOKLASS — Klassifisering av økologisk statu i norskevannforekomster: forslag til aktuelle kriterier og foreløpigegrenseverdier mellom god og moderat økologisk status forutvalgte elementer og påvirkninger. NIVA Rapport Lnr. 4860.

Magnussen, K., Bergland, O., 1996. Verdsetting av miljøgifter ivann. Stiftelsen Østlandsforskning (STØ) OR 51.96.

Magnussen, K., Bergland, O., Navrud, S., 1995. Overføring av nytte-estimater: status i Norge og utprøving knyttet til vannkvalitet.Del II Utprøving knyttet til vannkvalitet, NIVA.

Norwegian Bureau of Statistics, 2007. Population forecasts athttp://www.ssb.no/befolkning/.

OECD, 1993. OECD core set of indicators for environmentalperformance reviews. OECD Environment Monographs No. 83.OECD, Paris.

Saloranta, T.M., 2006. Highlighting the model selection andapplication process in policy-relevant water quality modelling.Ecological Modelling 194, 316–327.

Saloranta, T.M., Andersen, T., 2007. MyLake — A Multi-year Lakesimulation model code suitable for uncertainty and sensitivityanalysis simulations. Ecological Modelling 207, 45–60.

Scheffer, M., Carpenter, S.R., 2003. Catastrophic regime shifts inecosystems: linking theory to observation. Trends in Ecologyand Evolution 18 (12), 648–656 (Dec).

Uusitalo, L., 2007. Advantages and challenges of Bayesian networksin environmental modelling. Ecological Modelling 203 (3–4),312–318.

Varis, O., 1997. Bayesian decision analysis for environmental andresource management. Environmental Modelling & Software12, 177–185.

Varis, O., Kuikka, S., 1999. Learning Bayesian decision analysis bydoing environmental and natural resources management.Ecological Modelling 119, 177–195.

Varis, O., Lahtela, V., 2002. IntegratedWater ResourcesManagementalong the Senegal River: Introducing an Analytical Framework.International Journal of Water Resources Development 18 (4).

Varis, O., Kettunen, J., Sirviö, H., 1990. Bayesian influence diagramapproach to complex environmental management includingobservational design. Computational Statistics & Data Analysis9 (1) (January).

http://www.ssb.no/befolkning/

bayesian belief networks as a meta-modelling tool in integrated river basin management — pros and...

Documents