auser guide to the risk and uncertainty decision tree wiki site

A USER GUIDE TO THE RISK AND UNCERTAINTY DECISION TREE WIKI SITE

Keith Beven Lancaster Environmental Centre Hamish Harvey Newcastle University Florian Pappenberger Lancaster Environmental Centre and ECMWF, Reading David Leedal Lancaster Environmental Centre Jim Hall Newcastle University

June 2007 FRMRC Research Report UR13

Project Web: www.floodrisk.org.uk

User Guide to the Risk and Uncertainty Decision Tree Wiki Site FRMRC Research Report UR13

UR13_risk_uncertainty_decision_tree_wiki_WP9_1_v1_0.doc 19 June 2007 ii

FRMRC Partners The FRMRC Partners are: • University of Bristol • Heriot Watt University • HR Wallingford • Imperial College, London • University of Lancaster • University of Manchester • University of Nottingham • University of Sheffield

Project Secretariat ARP Directorate of Planning and Academic Services University of Manchester Sackville Street, Manchester PO Box 88 M60 1QD Tel: +44 (0)161 306 3626 Fax: +44 (0)161 306 3627 Web: www.floodrisk.org.uk


UR13_risk_uncertainty_decision_tree_wiki_WP9_1_v1_0.doc 19 June 2007 iii

Summary This report provides the second User Focused Measurable Output (UFMO) report from the Risk and Uncertainty Research Priority Area (RPA9) of the UK Flood Risk Management Research Consortium (FRMRC). It is presented in the form of a guide to the Wiki pages at (http://www.floodrisknet.org.uk/methods/Introduction) that include a decision tree, catalogue and guide to different methods of uncertainty estimation, Case Studies describing the use of different methods, and a glossary of terms. The presentation of the information about risk and uncertainty methods in this form is important in that it allows practitioners and researchers to add and edit material. This report also provides instructions as to how to use, edit and add to the Wiki pages. The list of Case Studies will be extended during the remainder of the RPA9 project. In addition, some recommendations arising out of the RPA9 project and a summary of the Reframe software, partly developed under RPA9 funding, are given (further details may be found linked to the Wiki pages). In particular the way in which uncertainty estimation might be embedded in the wider decision making processes is considered. Issues involved in the development of Codes of Practice for different types of application, that require risk and uncertainty analysis as an integral part of practice, are outlined. The RPA9 expect to continue to support the Wiki pages for the foreseeable future.


UR13_risk_uncertainty_decision_tree_wiki_WP9_1_v1_0.doc 19 June 2007 iv

Document Details

Document History Version Date Lead Author Institution Joint authors Comments

1 June 07 Keith Beven Lancaster Environmental Centre

Newcastle University Hamish Harvey Lancaster Environmental

Centre and ECMWF, Reading

Florian Pappenberger

Lancaster Environmental Centre

David Leedal

Newcastle University Jim Hall v1_0 June 07 HR Wallingford Ltd Final formatting

End user approval: Dr Kate Scott, Environment Agency, 27 July 2007

Acknowledgements This research was performed as part of a multi-disciplinary programme undertaken by the Flood Risk Management Research Consortium. The Consortium is funded by the UK Engineering and Physical Sciences Research Council under grant GR/S76304/01, with co-funding from: • Defra and the Environment Agency through their Joint R&D programme on Flood and Coastal

Erosion Risk Management, • UKWIR • NERC • The Scottish Executive • Rivers Agency Northern Ireland We would particularly like to thank the practitioner members of the RPA9 Advisory Group: Ian Meadowcroft and Kate Scott, Environment Agency (lead end-users), Kevin Sene (Atkins); Rob Lamb and Barry Hankin (JBA); Dag Lohmann (RPA) and Peter von Lany (Halcrow) for their input to the project.

Disclaimer This document reflects only the authors� views and not those of the FRMRC Funders. This work may rely on data from sources external to the FRMRC Partners. The FRMRC Partners do not accept liability for loss or damage suffered by any third party as a result of errors or inaccuracies in such data. The information in this document is provided �as is� and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and neither the FRMRC Funders nor any FRMRC Partners is liable for any use that may be made of the information.

© Copyright 2007 The content of this report remains the copyright of the FRMRC Partners, unless specifically acknowledged in the text below or as ceded to the funders under the FRMRC contract by the Partners.


UR13_risk_uncertainty_decision_tree_wiki_WP9_1_v1_0.doc 19 June 2007 v

Table of Contents Title page i

FRMRC Partners ii Summary iii Document Details iv Table of Contents v

1. Introduction................................................................................................................1 2. A Guide to the Wiki Site.............................................................................................2

2.1 The Decision Tree ..............................................................................................2 2.2 Catalogue of Methods for Uncertainty Analysis..................................................3 2.3 Example Case Studies .......................................................................................4 2.4 The Glossary ......................................................................................................5

3. How to use a Wiki site ...............................................................................................6 3.1 Navigation...........................................................................................................6 3.2 Change Notifications ..........................................................................................7 3.3 How to Modify Text.............................................................................................7 3.4 Adding pages .....................................................................................................8

4. Recommendations.....................................................................................................8 4.1 A code of practice for uncertainty assessment...................................................9 4.2 Realising the potential of Reframe ...................................................................10 4.3 Maintaining the Wiki site...................................................................................10

5. References ..............................................................................................................12 Table of Figures Figure 1 Decision tree for uncertainty analysis. Rounded boxes represent

questions to derive a decision for an uncertainty method. Double boxes show the major classifications of methods. Normal boxes represent small sub-groups of these or individual methods. In the version on the web site, most of the boxes are hyperlinked to further information. ......................3

Figure 2 Demonstration of the use of pop-up boxes from the Glossary for terms highlighted on other pages of the Wiki...........................................................5

Figure 3 Accessing the page history from the Menu Bar at the top of each Wiki page...............................................................................................................6

Figure 4 Accessing the page history from the Menu Bar at the top of each Wiki page...............................................................................................................6

Figure 5 Accessing the Wiki change pages from the Menu Bar at the top of each Wiki page .......................................................................................................6

Figure 6 An illustration of the information available at the Wiki change pages............6 Figure 7 Link to register for notification of changes on the Risk and Uncertainty

Wiki site .........................................................................................................7 Figure 8 Links to switch between view and edit modes for each Wiki page. ...............7 Figure 9 Editing a Wiki page, with link to help information for editing and mark-up

conventions at bottom....................................................................................8


UR13_risk_uncertainty_decision_tree_wiki_WP9_1_v1_0.doc 19 June 2007 vi


UR13_risk_uncertainty_decision_tree_wiki_WP9_1_v1_0.doc 19 June 2007 1

1. Introduction Risk and Uncertainty analysis is a cross-cutting and overarching theme in the Flood Risk Management Research Consortium (FRMRC). During the course of FRMRC considerable work has been done to catalogue the range of available uncertainty methods, assess and test their applicability and demonstrate their use, with a view to promoting more widespread and rigorous use by the practitioner community. The research has been documented as an internet based Wiki site, that can be found at: http://www.floodrisknet.org.uk/methods/Introduction. The site includes • background on a variety of different risk and uncertainty assessment methods; • a decision tree to help users in choosing methods; • a glossary of terms; • a collection of Case Studies to demonstrate the use of different methods. This report is intended to provide an overview of the Wiki site and information on how to use it. A Wiki site may be modified by the users, which means that any user can add comments or modify the text on the pages of the site, and also add case studies to augment the information on the site. The information added is recorded and then moderated by the hosts of the site (in this case, the Universities of Newcastle and Lancaster). The Wiki site has been publicised internationally through journals and web sites (e.g. Pappenberger et al., 2006, 2007). This report, and the Wiki site (developed during the 2004 � 2007 period), that it describes is the second �user focused measurable outcome� (UFMO) of the FRMRC Research Priority Area on Risk and Uncertainty (RPA9). The first UFMO report which outlined the range of risk and uncertainty methods to be considered and issues involved in software implementation in a general way is available at: http://www.floodrisk.org.uk/images/stories/docs/UR2_tools_for_uncertainty_evaluation_WP9_1_v1_0.pdf. In addition to research on cataloguing and rationalising uncertainty methods, as documented on the Wiki site, RPA9 has undertaken collaborative research to demonstrate uncertainty methods across FRMRC in support of the objective �to make uncertainty analysis a routine aspect of flood risk modelling activities, so that decision-makers are always provided with information on uncertainties in model predictions in a standard format�. • In work with RPA3 (Real Time Flood Forecasting ) we have demonstrated, in a flood forecasting

application to the River Severn, how uncertainties can be constrained by real-time data assimilation cascading through models of different parts of the system (see Romanowicz et al., 2006a,b).

• With RPA4 (Infrastructure Management) we have collaborated on systems risk assessment

methodologies including the treatment of uncertainty in RASP-type methodologies and system optimisation problems.

• With RPA5 (Towards Whole System Modelling), we have demonstrated how flood inundation

uncertainties can be assessed and constrained by historical inundation data and how to react when it is not possible to match all of the historical data (see Pappenberger et al. 2005, 2006a,b). Work has begun on how faster, coarser scale models can mimic detailed fine scale models in urban areas (Hall et al 2007).



• With RPA6 (Urban Flood Management ) we have introduced the concept of risk attribution in urban flooding and developed the first probabilistic methods for risk attribution (Dawson et al. 2006, Hall et al., 2006, 2007).

• With RPA7 (Stakeholder and Policy), we have held a Practitioner�s workshop on Risk and

Uncertainty in Flood Risk Management and have developed concepts for making uncertainty analysis a routine aspect of modelling activities (see Pappenberger and Beven, 2006; Faulkner et al., 2007).

The insights from these applications and demonstration studies are also described on the Wiki site. Finally, RPA9 research has included a major theme with the objective to resolve the uncertainty-handling and software issues associated with the construction of composite risk models of flooding systems. This research is now reaching fruition, in the form of a meta-model (called Reframe) and associated software that supports: • coupling of software components for flood system simulation (e.g. hydrological, hydraulic,

structural reliability and economic models) from multiple sources; • incorporation of methods for risk, uncertainty, robustness and sensitivity analysis; • collaboration, including collaboration across organisational boundaries. The report follows the following structure. Section 2 provides a description of the content of the Wiki site including a description of the decision tree, catalogue of methods, case studies, and glossary for uncertainty analysis. Section 3 provides a guide to the practical issues of making the most of the Wiki site including navigating, formatting and modifying text, and adding pages. A summary of Reframe is given in Section 4.2.

2. A Guide to the Wiki Site 2.1 THE DECISION TREE The decision tree (see Figure 1) is one of the most important resources in the Wiki. Using it, the practitioner without extensive experience of uncertainty analysis or background knowledge can select which other parts of the Wiki to focus their attention on.



Figure 1 Decision tree for uncertainty analysis. Rounded boxes represent questions to derive a decision for an uncertainty method. Double boxes show the major classifications of methods. Normal boxes represent small sub-groups of these or individual methods. In the version on the web site, most of the boxes are hyperlinked to further information.

The version of the decision tree on the web site has active hyperlinks from many of the boxes to pages providing more detailed information. Method boxes link to pages describing those methods; group boxes to pages providing information general to the class of methods in that group; and question boxes link to explanatory text about the question. For those questions with answers other than �yes� and �no�, the answers may also be links. The decision tree is intended to help with the process of choosing a method for quantitative uncertainty analysis. Uncertainty also has qualitative aspects. Both may be important to dcision making and should be recorded wherever possible (e.g. using the NUSAP methodology of van der Sluijs et al., 2005).

2.2 CATALOGUE OF METHODS FOR UNCERTAINTY ANALYSIS Each of the methods or classes of method listed in the terminal nodes of the decision tree is described in its own page. These pages can be found by following the link �Methodologies for uncertainty



analysis� from the front page of the Wiki. The methods are clustered into broad categories, as they are in the decision tree. Each method page contains an outline description, an indication of software available which implements the method, a list of notable advantages and disadvantages, a list of case studies in the Wiki which involve the method, and references for further reading. Case studies can be reached from the decision tree with only two mouse clicks. Uncertainty analysis methods included at the time of writing are: • Error propagation equations • Monte Carlo methods, including sensitivity and model emulation methods. • Fuzzy and Interval methods • Nonlinear regression • Bayesian methods • Generalise likelihood uncertainty estimation (GLUE) • Kalman filter and Extended Kalman filter • Sequential Monte Carlo methods including Ensemble Kalman Filter and Particle Filter • NUSAP The intimately related issue of sensitivity analysis is also treated, with one of its major roles � constraining the uncertainty analysis problem by ranking uncertainties � clearly indicated in the decision tree. There are many different ways in which uncertainties might be used in decision making. An additional guide to some decision making strategies can also be found on the Wiki pages (Click on <Decision making under uncertainty> go directly to http://www.floodrisknet.org.uk/ methods/DecisionMakingUnderUncertainty). Info-gap decision theory (Ben-Haim, 2006) is presented as an example of a technique which allows a decision problem to be recast directly in terms of uncertainty, in this case specifically the robustness of decisions under extreme uncertainty.

2.3 EXAMPLE CASE STUDIES A number of case studies are presented. These can be found from the front page via the �Case studies� link. Case studies have been chosen to highlight particular methods, and links are included between method and related case study pages. Each case study page summarises a case study from the literature, with references provided. Links are included from case studies back to the method or methods which they exhibit. Case studies included at the time of writing are: • Risk Assessment of flood and coastal defence for Strategic Planning (RASP) (Forward

uncertainty propagation) • Assimilating satellite rainfall data (Forward uncertainty propagation) • Estimating Design Discharges (Bayesian Methods) • Flood frequency estimation under climate change (GLUE) • Flood forecasting cascade from ensemble rainfall forecasts to inundation maps (GLUE) • Vulnerability weighted flood inundation risk estimation (GLUE) • Fitting Flood Frequency distributions (Nonlinear regression) • Assimilating satellite rainfall data (Forward uncertainty propogation) • Data assimilation for real-time runoff forecasting (Kalman filter) • Real time flood forecasting (data assimilation / Kalman filter)



• Uncertainty in rating curve estimation (Fuzzy methods) • Real time flood forecasting (Ensemble Kalman Filter / Monte Carlo methods) The Wiki philosophy allows users of the pages to add further Case Studies that they think might be useful to others. A template is provided for that purpose.

2.4 THE GLOSSARY A glossary is provided for use alongside the uncertainty methods Wiki. The glossary itself can be found at http://www.floodrisknet.org.uk/glossary. Glossary entries are not Wiki pages, and are thus not freely editable; comments can be left, however, enabling discussion and debate. The contents of the glossary are used to annotate the pages of the uncertainty methods Wiki (and indeed all pages on floodrisknet.org.uk), as shown in Figure 2. Where words or phrases from the glossary appear in the text of a page, these are marked with a dotted underline and blueish background (1, �uncertainty�). Holding the mouse cursor over these words will show the definition from the glossary in a pop-up text box (2). For convenient access to the glossary, all terms found in the page are also listed in a �related terms� box in the left margin (3). Clicking on a highlighted word or its entry in the �related terms� box will take you to the full glossary entry.

Figure 2 Demonstration of the use of pop-up boxes from the Glossary for terms highlighted on

other pages of the Wiki



3. How to use a Wiki site 3.1 NAVIGATION Navigation within a Wiki is exactly like that within a normal web site. Some additional features are present, however. These can be found in the green bar at the top of each Wiki page (Figure 3).

Figure 3 Accessing the page history from the Menu Bar at the top of each Wiki page

The history function lists recent edits to the page, along with the person who made them if they were logged in at the time. �Related pages� (Figure 4) lists, among other things, �backlinks� which are other pages in the Wiki which link to this one.

Figure 4 Accessing the page history from the Menu Bar at the top of each Wiki page

�Wiki changes� provides a list of the latest changes across the whole Wiki (Figure 5). Freshly added pages are clearly marked �new� (Figure 6):

Figure 5 Accessing the Wiki change pages from the Menu Bar at the top of each Wiki page

Figure 6 An illustration of the information available at the Wiki change pages



3.2 CHANGE NOTIFICATIONS Keeping up with changes on a web site can be hard work, particularly if it is only occasionally updated. The floodrisknet web site provides a general notification service. Visitors can subscribe to receive email alerts of changes. These emails are sent out daily if (and only if) changes have been made during the previous day. The subscription link may be found at the bottom of the left hand column (Figure 7)

Figure 7 Link to register for notification of changes on the Risk and Uncertainty Wiki site

3.3 HOW TO MODIFY TEXT All Wiki content can be edited by anyone at any time. At the top of the main area of each page in the Wiki you will find a pair of tabs labelled �view� (1) and �edit� (2) in Figure 8. The solid green tab indicates the current �mode� (view by default). There may be other tabs in certain circumstances. A click on the �edit� tab will enable you to modify the content of the page. A complete history of page edits is recorded, however, providing protection against vandalism (past versions can be reinstated) and the ability to review past versions (3).

Figure 8 Links to switch between view and edit modes for each Wiki page.

Pages are written in plain text supplemented with simple mark-up conventions (as examples: ��italic�� for italicised text, and [a page title] to link to the page with that title). Help on these conventions can be reached by following the link at the bottom of the edit view.



Figure 9 Editing a Wiki page, with link to help information for editing and mark-up

conventions at bottom

Users registered with floodrisknet.org.uk and logged in will also be able to upload supporting files and images. When a file or image is uploaded, a link is automatically added to the bottom of the page. Return to the edit view to move this link to the appropriate place in the page.

3.4 ADDING PAGES Pages can be added to the Wiki very easily. Any links included in existing pages, to pages which do not exist, are displayed followed by a question mark link. Clicking this will create a page with that title and take you directly to its edit view. Method and case study pages should follow the existing layouts. Method pages should include references, including web links wherever possible. Full case studies can be presented within a Wiki page, but where the case study is fully documented elsewhere, a summary with references and links is more appropriate.

4. Recommendations RPA9 has made considerable progress in promoting the implementation of uncertainty methods in practice in flood risk management. The challenge of ensuring that uncertainty methods are widely and appropriately used should not however be under-estimated; nor should the importance of doing so. Increasingly government is requiring a careful consideration of uncertainty in major planning and investment decisions. Defra�s guidance repeatedly calls for proper consideration of uncertainty in



appraisal decisions. Guidance document FCDPAG1 (Defra, 2001) on �Good decision making� (p5) states: Good decisions are most likely to result from considering all economic, environmental and technical issues for a full range of options, together with a proper consideration of risk and uncertainty. FCDPAG3 (Defra, 1999) has a section on �Sensitivity analysis and robustness testing� which stresses the importance of seeking options whose benefits are robust to uncertainty. FCDPAG4 (Defra, 2000) calls for explicit treatment of uncertainty in risk analysis (p8): All risk assessments are predictive and, therefore, the results are inherently uncertain. In undertaking risk assessment work, it is important to acknowledge explicitly the degree of uncertainty. Making Space for Water (Defra, 2005) does not expand upon the topic of uncertainty but states that �Decisions will reflect the uncertainty surrounding a number of key drivers�� [p14] and, furthermore, the glossary (p41) defines appraisal as �The process of defining objectives, examining options and weighing up the costs, benefits, risks, and uncertainties before a decision is made� (our italics). Research in RPA9 has helped to identify necessary next steps in responding to this government requirement for uncertainty analysis.

4.1 A CODE OF PRACTICE FOR UNCERTAINTY ASSESSMENT Faulkner et al. (2007), in a paper arising out of the FRMRC RPA7/RPA9 research, have suggested that the process of developing a Code of Practice is one way of encouraging communication about uncertainty estimation methodologies between researchers, practitioners and end users. The process will necessarily involve the resolution of communication difficulties by the development of an agreed vocabulary and sets of tools for the different applications involved in Flood Risk Management. It is possible to identify considerable commonalities between different approaches, which tend to involve the following steps: 1. Choice of model(s) to be considered 2. Prior assessment of uncertainties in driving variables and model parameters 3. Evaluation of prior predictions against observations, with either implicit or explicit assumptions

about nature of errors, to derive posterior uncertainty estimates of inputs and/or model outputs. 4. Use of posterior uncertainties to make predictions and inferences about the future behaviour of the

system under study. These steps are common whether the assessment is being made within a formal statistical framework with prior and posterior probability distributions, or within a fuzzy framework, or within a scenario framework (including dealing with epistemic uncertainties), or for simulation or real-time forecasting purposes. However, there is no readily available guidance about how to do uncertainty analysis in hydrologic and hydraulic modeling or infrastructure management. This makes it difficult for a practitioner to choose a method, and for the end-user of modelling results to interpret the resulting uncertainties. There is also the issue of cost and resources. Uncertainty estimation is currently considered to be an added component to an analysis, at extra cost. It should, however, be an intrinsic and expected part of any modelling exercise. We expect mature guidance on the use of different methods will develop over time, but it is not too early to suggest the development of Codes of Practice for different types of application that makes uncertainty analysis an integral part of the modelling process.



From the modeller�s point of view, a Code of Practice would (mostly) represent an extension of normal practices for the quality assurance and version tracking of computer codes, with similar documentation requirements. Such a Code of Practice would need to address the following issues (see Pappenberger and Beven, 2006): Taking account of model context; taking account of uncertainty in model choice; taking account of uncertainty in model drivers; taking account of uncertainty and dependencies in model parameters; choice of uncertainty in uncertainty estimation methodology; taking account of uncertainty in observations used in constraining uncertainty; and taking account of uncertainty in future conditions; and the communication of results to users.

4.2 REALISING THE POTENTIAL OF REFRAME In the proposal for RPA9 it was recognised that making uncertainty analysis more routine in practical flood risk analysis and flood risk management decisions would require a fundamental review of the use of software for flood risk modelling. In response to this review, the Reframe metamodel has been developed which supports: • coupling of software components for flood system simulation (e.g. hydrological, hydraulic,

structural reliability and economic models) from multiple sources; • propagation of uncertainty through coupled flood risk models and incorporation of methods for

risk, uncertainty, robustness and sensitivity analysis; • collaboration, including collaboration remotely and across organisational boundaries. By the end of the FRMRC phase 1 research the following developments will have been completed: 1. Establishing the Reframe metamodel 2. Writing the Reframe language 3. Development of a graphical user interface 4. Coding of modules for �translation� from the most common data formats used in flood modelling 5. Demonstration for a realistic flood risk calculation. Reframe will be made freely available under an Open Source licence in September 07. Following this initial phase of research and development, further work is required to realise the full potential of Reframe in practice: 1. The number of data formats the Reframe can incorporate needs to be extended to include all

commonly used formats. 2. Functionality needs to be developed to support the execution of complex sequences of

computations. 3. The potential for use of the Reframe meta-model to support high level reasoning about the use of

composite models needs to be explored. In addition to this technical work adoption of Reframe will depend on �buy-in� from the flood risk management industry, in particular the Environment Agency and their primary consultants and software providers. The problems which Reframe is designed to address are widely recognised; achieving this buy-in will involve demonstrating that it indeed addresses these problems successfully. Collaboration with HR Wallingford in developing the Thames case study represents a first step in this process. This case study will also provide the basis for a wider discussion when it is sufficiently well developed.

4.3 MAINTAINING THE WIKI SITE The Wiki site described in this report has now been operational for six months. The development phase of the site is now complete, but it will require ongoing maintenance if it is to continue to be an up to date resource for use by the flood risk management community. The research team have a good track record in maintaining web sites. The http://www.floodrisknet.org.uk has been maintained by



Newcastle University since the completion of the FloodRiskNet project in 2004. Keeping the methodologies and case studies on the site up to date will require addition intermittent effort and, whilst a Wiki site can be edited by anyone, some occasional dedicated effort will be required in order to guarantee that this takes place.



5. References Ben-Haim, Y, 2006, Info-Gap Decision Theory, 2nd Edition, Academic Press: Amsterdam. Dawson, R.J., Hall. J.W., Speight, L., Djordjevic, S., Savic, D. and Leandro, J. Flood risk analysis to support integrated urban drainage, in Emerging Environmental Issues and Future Challenges: Proc 4th CIWEM Conf., Newcastle, 12-14 September 2006. in press. Defra, 1999. Flood and Coastal Defence Project Appraisal Guidance: Economic Appraisal, FCDPAG3, December 1999. Defra, 2000. Flood and Coastal Defence Project Appraisal Guidance: Approaches to Risk, FCDPAG4, February 2000. Defra, 2001. Flood and Coastal Defence Project Appraisal Guidance: Overview (including general guidance), FCDPAG1, May 2001. Defra, 2005, Evidence-based policy making: if it exists, what makes it robust?, Evidence-based policy making project, Defra Science Strategy Team, London, 1-10. Faulkner, H, Green, C, Parker, D, and Beven, K J, 2007, Developing a translational discourse to communicate uncertainty in flood risk between science and practitioner. Submitted to Ambio Hall. J.W., Dawson, R.J., Speight, L., Djordjevic, S., Savic, D. and Leandro, J. Sensitivity based attribution of flood risk, in NOVATECH 2007: Proc. 6th Int. Conf. Sustainable Techniques and Strategies in Urban Water Management, Lyon, 25-28 June 2007. in press. Hall, J.W., Dawson, R.J., Speight, L., Djordjevic, S., Savic, D., and Leandro, J. Attribution of flood risk in urban areas, in Hydroinformatics 2006: Proc. 7th Int. Conf. on Hydroinformatics, Nice, September 4-8, 2006. Gourbesville, P., Cunge, J., Guinot, V. and Liong, S.-Y. Research Publishing Services, Chennai, India, Volume I, pp.280-287. Pappenberger, F and Beven, K J, 2006, Ignorance is bliss: 7 reasons not to use uncertainty analysis, Water Resources Research, 42, W05302, doi:10.1029/2005WR004820, 2006 Pappenberger, F, Harvey, H, Beven K, Hall, J and Meadowcroft, I, 2006, Decision tree for choosing an uncertainty analysis methodology: a Wiki experiment http://www.floodrisknet.org.uk/methods Hydrological Processes (HPToday), 20, 3793-3798. Pappenberger, F, Harvey, H, Beven, K J, Hall, J, 2007, Uncertainty Analysis in Environmental Modeling Made Easy, EoS Trans. AGU, 88(3) p26, doi:10.1029/2007EO030003 van der Sluijs J., Craye M., Funtowicz S., Kloprogge, P., Ravetz, J., Risbey, J. 2005. Experiences with the NUSAP system for multidimensional uncertainty assessment. Water Science and Technology 52(6), 133-144.

auser guide to the risk and uncertainty decision tree wiki site

Documents