ewater guidelines rrm (v1 0 interim dec 2011)
TRANSCRIPT
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
1/50
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
2/50
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
3/50
eWater CRC
Copyright Notice
eWater Ltd 2011
Legal Information
This work is copyright. You are permitted to copy and reproduce the information, in an
unaltered form, for non-commercial use, provided you acknowledge the source as per thecitation guide below. You must not use the information for any other purpose or in any other
manner unless you have obtained the prior written consent of eWater Ltd.
While every precaution has been taken in the preparation of this document, the publisher and
the authors assume no responsibility for errors or omissions, or for damages resulting from
the use of information contained in this document. In no event shall the publisher and the
author be liable for any loss of profit or any other commercial damage caused or alleged to
have been caused directly or indirectly by this document.
Citing this document
Vaze, J ., J ordan, P., Beecham, R., Frost, A., Summerell, G. (eWater Cooperative Research
Centre 2011) Guidelines for Rainfall-Runoff Modelling: Towards Best Practice ModelApplication.
Publication date: December 2011 (Interim Version 1.0)
ISBN 978-1-921543-51-7
Acknowledgments
eWater CRC acknowledges and thanks all partners to the CRC and individuals who have
contributed to the research and development of this publication.
We acknowledge the inputs from the hydrology group in DERM, Queensland, and Mark
Alcorn from SA Department for Water. We thank Matthew Bethune, Peter Wallbrink, DugaldBlack, J in Teng, J ean-Michel Perraud, Melanie Ryan, Bill Wang, David Waters, Richard
Silberstein, Geoff Podger, David Post, Cuan Petheram, Francis Chiew and Andrew Davidson
for useful discussions.
eWater CRC gratefully acknowledges the Australian Governments financial contribution to
this project through its agencies, the Department of Innovation, Industry, Science and
Research, the Department of Sustainability, Environment, Water, Population and
Communities and the National Water Commission.
For more information:
Innovation Centre, Building 22
University Drive South
Bruce, ACT, 2617, Australia
T: 1300 5 WATER (1300 592 937)
T: +61 2 6201 5834 (outside Australia)
www.ewater.com.au
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
4/50
Best Practice Guidelines for Rainfall-Runoff Modelling
Table of Contents
Table of Contents .......................................................................................................................... 21 Introduction ............................................................................................................................ 5
1.1 Background ................................................................................................................... 51.2 Definition of Best Practice ............................................................................................. 51.3 Scope ............................................................................................................................ 6
2 Overview of procedure for rainfall-runoff modelling .............................................................. 82.1 Problem definition .......................................................................................................... 8
2.1.1 Problem statement and setting objectives ............................................................ 82.1.2 Understanding the problem domain ...................................................................... 82.1.3 Metrics and criteria and decision variables ........................................................... 92.1.4 Performance across multiple catchments and subcatchments ............................. 9
2.2 Option modelling ........................................................................................................... 92.2.1 Methodology development .................................................................................... 92.2.2 Collate and review data ....................................................................................... 102.2.3 Setting up and building a model .......................................................................... 102.2.4 Calibration and Validation ................................................................................... 102.2.5 Sensitivity/uncertainty analysis............................................................................ 122.2.6 Documentation and Provenance ......................................................................... 122.2.7 Model acceptance and accreditation ................................................................... 132.2.8 Use of accepted/accredited model ...................................................................... 13
3 Model choice ....................................................................................................................... 143.1 Model selection ........................................................................................................... 143.2 Available models ......................................................................................................... 15
3.2.1 Empirical methods ............................................................................................... 153.2.2 Large scale energy-water balance equations ..................................................... 163.2.3 Conceptual Rainfall-Runoff Models ..................................................................... 163.2.4 Landscape daily hydrological models ................................................................. 173.2.5 Fully distributed physically based hydrological models which explicitly modelhillslope and catchment processes ..................................................................................... 17
4 Collate and Review Data ..................................................................................................... 214.1 Catchment details ........................................................................................................ 22
4.1.1 Location of gauges (streamflow, rainfall and evaporation) ................................. 224.1.2 Topography and Catchment Areas ..................................................................... 224.1.3 Soil types ............................................................................................................. 224.1.4 Vegetation ........................................................................................................... 224.1.5 Water Management Infrastructure ...................................................................... 23
4.2 Flow data ..................................................................................................................... 234.3 Rainfall ......................................................................................................................... 244.4 Evapotranspiration ...................................................................................................... 25
5 Statistical Metrics for Testing Performance......................................................................... 26
Page 2
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
5/50
eWater CRC
6 Calibration and validation .................................................................................................... 286.1 Calibration ................................................................................................................... 286.2 Validation ..................................................................................................................... 286.3 Calibration and Validation of Models to Single Gauge Sites, Multiple Gauge Sites andRegionalisation of Model Parameter Sets ............................................................................... 306.4 Automated, Manual and Hybrid Calibration Strategies ............................................... 31
6.4.1 Manual Calibration .............................................................................................. 316.4.2 Automated Calibration ......................................................................................... 326.4.3 Hybrid Calibration Strategies............................................................................... 336.4.4 Selection of Objective Functions for Automated and Hybrid Calibration ............ 346.4.5 Further Guidance on Calibration and Validation of Conceptual Rainfall RunoffModels 36
6.5 Calibration of Regression Models ............................................................................... 397 Uncertainty and Sensitivity Analysis ................................................................................... 40
7.1 Sensitivity Analysis ...................................................................................................... 417.2 Application of Multiple Parameter Sets ....................................................................... 427.3 More Advanced Quantitative Uncertainty Analysis ..................................................... 427.4 Consideration of Uncertainty in Practical Applications of Rainfall Runoff Models ...... 43
8 Concluding remarks ............................................................................................................ 449 References .......................................................................................................................... 45
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
6/50
Best Practice Guidelines for Rainfall-Runoff Modelling
Page 4
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
7/50
eWater CRC
1 Introduction
1.1 Background
Reliable estimates of stream flow generated from catchments are required as part of
the information sets that help policy makers make informed decisions on water
planning and management. The characteristics of the streamflow time series that
influence water resources system modelling and planning can include the sequencing
of flows on daily and longer time steps, spatial and temporal variability of flows,
seasonal distribution and characteristics of high and low flows.
The best available estimate of streamflow would be expected to come from water level
observations made at a gauging station, converted to flow estimates using a well
defined and stable rating curve. However, these observations are only available for
limited number of gauging locations and for limited time span. Estimates for ungauged
locations and for a much longer time period are needed for contemporary watermanagement, and ways to make estimates for future possible conditions are also
required.
A range of methods are available to estimate streamflow from catchments, using
observed data wherever possible, or estimating by empirical and statistical techniques,
and more commonly using rainfall-runoff models. The modelling approach used to
estimate streamflow by different water agencies and consultants varies across
Australia and depends on the purpose of the modelling, time and money available, and
the tools and skills available within the organisation. With increasing levels of inter-
agency collaboration in water planning and management, development of a best
practice approach in rainfall runoff modelling is desirable to provide a consistent
process, and improve interpretation and acceptability of the modelling results.
The purpose of this document is to provide guidance on the best practice for
implementing fit for purpose rainfall-runoff models, covering topics such as setting
modelling objectives, identifying data sources, quality assuring data and understanding
its limitations, model selection, calibration approaches, and performance criteria for
assessing fitness for purpose
1.2 Defini tion of Best Practice
Best Practice Modelling can be defined as a series of quality assurance principles and
actions to ensure that model development, implementation and application are the best
achievable, commensurate with the intended purpose (Black et al., 2011).
What is in practice best achievable, commensurate with the intended purpose may be
subject to data availability, time, budget and other resourcing constraints. Hence, what
is meant by the term Best Practice Modelling can vary. Not only does it depend on
the circumstances of the project, particularly providing results that are fit for the
intended purpose, but it also depends to a great degree on interpretation in peer
review. This, in turn, will be influenced by the general state of knowledge and
technology in the modelling field, which can be expected to progressively develop overtime (such as new remote sensing data sources coming on line, and new computing
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
8/50
Best Practice Guidelines for Rainfall-Runoff Modelling
hardware and software), as well as data, time, budget and resourcing constraints. Best
Practice Modelling provides for a strategic approach to modelling which enables the
trade-offs that may be imposed by these constraints to be better managed, and assists
in identifying priorities for addressing model and data limitations.
1.3 Scope
The eWater CRC has prepared generic Best Practice Modelling guidelines (Black et
al., 2011). They aim to provide for an integrated approach that enables interactions
and feedbacks between all domains relevant to water management (e.g. hydrological,
ecological, engineering, social, economic and environmental) to be considered.
The procedure in that guidance is intended to be flexible enough to accommodate
variations in the meaning of the term Best Practice Modelling and also allow for
continuous improvement as the state of knowledge and technology in the modelling
field develops and improves.
The eWater CRC will also provide guidelines to support the BPM guidelines in specificareas of hydrological modelling that relate to the products that they are developing.
This guideline is intended to address rainfall-runoff model application with the
objectives being to provide water managers, consultants and research scientists with
information on rainfall-runoff models and how to choose one that is fit for purpose, the
data available to develop them, and the calibration and validation methodologies.
There are a number of different purposes that a rainfall runoff model may be applied
within an overall water resources or catchment modelling framework, such as eWater
Source IMS. Most of these purposes relate to providing information to support decision
making for some water management policy. In particular, this can involve:
Understanding the catchment yield, and how this varies in time and space,
particularly in response to climate variability: seasonally, inter-annually, and
inter-decadally.
Estimating the relative contributions of individual catchments to water
availability over a much larger region, e.g. valley or basin scale.
Estimating how this catchment yield and water availability might change over
time in response to changes in the catchment, such as increasing development
of farm dams, or changes in land-use and land management.
In some instances with a high quality network of long term stream gauges, most of thistype of information can be estimated from the observations. However, the more
common case is where there is some combination of short term stations, variable
quality data, and gaps in spatial coverage. In these cases, spatial and temporal gaps in
the information can be estimated using rainfall runoff models to:
Infill gaps caused by missing or poor quality data in an observed data series for a
gauged catchment.
Estimate flows for a gauged catchment for periods before the observed flow record
started or after when the observed flow record ends.
Estimate flows for an ungauged catchment.
Page 6
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
9/50
eWater CRC
Estimating flows from ungauged subcatchments within an overall gauged
catchment.
Forecast flows for some immediate future period (typically for a period of between
a few days and a few months), conditioned on current (or recent) observations of
the catchment state.
Assess the impacts of human influences within a gauged catchment (for examplelanduse or vegetation cover change) and simulating the flows that would have
occurred for the historical climate sequence with modified catchment conditions.
This may include assessment of catchment conditions that may be non-stationary
in either the observed record or for the simulation.
Assess the potential impact of climate variability and/or climate change on flows
from a gauged catchment.
In some cases, several of the above purposes may be satisfied by rainfall runoff
modelling for the same catchment. There are similarities in the approach that is taken
to rainfall runoff modelling, even though the purpose may differ.
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
10/50
Best Practice Guidelines for Rainfall-Runoff Modelling
2 Overview of procedure for rainfall-runoff modelling
The generic guidelines (Black et al., 2011) outline a procedure for applying ahydrological model. This can be summarised as occurring in 4 phases:
1. Project management,
2. Problem definition,
3. Option modelling,
4. Compare Options and select the best.
This guidance deals only with problem definition and option modelling because the first
and last phases are discussed sufficiently for the purpose of rainfall-runoff modelling in
the generic guidelines. A further reason is that rainfall-runoff modelling is usually only a
part of a larger hydrological modelling project and these phases would be most
appropriately considered in the context of that larger project. Specific aspects of project
management and option comparison that are directly applicable to the development of
a rainfall-runoff model, such as accreditation, are dealt with at appropriate points in this
guidance.
2.1 Problem definit ion
2.1.1 Problem statement and setting objectives
The problem to be addressed must be clearly articulated to minimise the risk that thewrong tool will be used for the job. The problem statement will give direction on what
objectives will be considered in developing the rainfall-runoff model. As many water
management decisions will often have more than one goal it will be important to ensure
these are all identified.
Sometimes it can be useful to express objectives in a hierarchy that shows primary
objectives, secondary objectives and so on. In this regard, consideration should also
be given to possible additional future objectives and goals that could be met based on
the current project or on future projects that build upon the model established in the
current project. The decision on which option offers the best solution should be based
upon whether, or how well, each option meets the agreed objectives (also see section
2.2.1 and 2.2.2 in the generic guidelines).
2.1.2 Understanding the problem domain
The choice of the rainfall-runoff model will vary based on the purpose the modelling is
being done for, e.g., to understand seasonal low flow characteristics for an in-stream
environmental need; or to assess over-bank flow frequency; or to estimate overall
catchment yield on an average annual basis. The model selected, data required, and
calibration approach adopted should reflect this requirement. Where the same model
may be used for two or more different purposes, there may also be a need to focus thecalibration on a number of different flow regimes. If rough flow estimates are required
Page 8
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
11/50
eWater CRC
over large areas and the runoff generation methodology should be consistent then the
data and modelling process will differ again.
2.1.3 Metrics and criteria and decision variables
Model calibration is a process of systematically adjusting model parameter values to
get a set of parameters which provides the best estimate of the observed streamflow(in the case of rainfall-runoff models). The process of determining which particular set
of parameter values are best for the intended purpose depends on what comparison
metrics are used. Metrics should be used to quantify the acceptability of the developed
model. In all cases graphical assessment and statistical results of some sort will be
assessed to identify the ability of the calibrated model to reproduce the flows calibrated
against.
Different metrics will be more effective in determining model appropriateness to meet
different objectives. What these are should be considered when the problem is being
defined. Understanding appropriate metrics allows model acceptance criteria to be
identified.
2.1.4 Performance across multiple catchments and subcatchments
In some situations, the purpose of rainfall runoff modelling is to produce an estimate of
the runoff at a single location where there is a streamflow gauge. If this is the case, the
calibration and validation process may be performed for the single gauged catchment.
This approach is justifiable in situations where gauged data is available for most of the
period that flow results are required for and the purpose of the rainfall runoff model is to
infill missing data during the period of record. It may also be justifiable where there is a
requirement to extend the period of record at the single gauge.
A much more common situation is that flow time series estimates are required at
several locations and that gauged streamflow data is also available at several
locations. The locations where flow estimates are required may or may not overlap with
the locations where the flow data is also available. An objective of any project that
involves the application of rainfall runoff models to multiple catchments or
subcatchments should be to demonstrate consistency in the rainfall runoff model
response between those catchments and to explain systematic differences in the
hydrological response between catchments and subcatchments in an appropriate and
logical manner.
2.2 Option modelling
This section describes the process of developing a rainfall-runoff model, further details
on key components are provided in later sections.
2.2.1 Methodology development
The models and methodology employed should be appropriate for the purpose that the
model will be used for. The choices made will be directed by the problem definition
developed and any other information at hand to the modeller. Detail on the models
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
12/50
Best Practice Guidelines for Rainfall-Runoff Modelling
available and guidance on selecting models and methodology that is fit for purpose is
provided in Section 3.
2.2.2 Collate and review data
The amount and quality of data available to develop a model should be determined at
the outset of the project. This can influence the selection of models, the performancecriteria, and the approach to calibrate models. A bare minimum data set sufficient to
make an approximate estimate of mean annual catchment yield would include
catchment area along with spatial and temporal characteristics of rainfall and potential
evapotranspiration (PET). A comprehensive data set would include long-term
streamflow measurements and rainfall and PET data collected at one or more locations
within the catchment along with land use coverage, vegetation cover and impervious
area information, including changes over time.
The quality of the data should be reviewed prior to using to detect errors, non-
stationarity if any, and understand uncertainties that may influence estimates. Some
methods are discussed in section 4.
2.2.3 Setting up and building a model
The catchment characteristics are considered along with the knowledge on data
available and any other information available to the modeller. The rainfall-runoff model
chosen is conceptualised and an initial parameter set is identified.
When the model is first set up consideration should be given to all constraints which
are limiting and their effects on the modelling. Section 5 provides more details
associated with this step.
2.2.4 Calibration and Validation
Model calibration is a process of systematically adjusting model parameter values to
get a set of parameters which provides the best estimate of the observed streamflow
(in the case of rainfall-runoff models).
The term validation, as applied to models, typically means confirmation to some
degree that the calibration of the model is acceptable for the intended purpose
(Refsgaard and Henriksen, 2004). In the context of rainfall runoff modelling, validation
is a process of using the calibrated model parameters to simulate runoff over anindependent period outside the calibration period (if enough data is available) to
determine the suitability of the calibrated model for predicting runoff over any period
outside the calibration period. If there is not enough data available, the validation may
be performed by testing shorter periods within the full record.
It is normal in research studies to split the observed data sets into calibration and
validation period prior to the study, to demonstrate the performance of the model under
both sets of conditions. Use of this approach can cause problems in practical
applications if a model demonstrates acceptable performance for the calibration data
set but produces unsatisfactory results for the validation data set. An alternative
approach in this situation is to calibrate the rainfall runoff model to all available data but
Page 10
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
13/50
eWater CRC
to demonstrate that the performance of the model is satisfactory over different sub-sets
of the period that observed data is available.
Further discussion of model calibration and validation is provided in Section 6.
It is a very common situation in a project that involves rainfall runoff modelling for flow
time series to be required for several catchments or subcatchments within the model
domain and for data to be available from two or more stream flow gauges to facilitate
calibration and validation. At locations where gauged flows are available and flow
estimates are required, two options are available to the modeller:
The rainfall runoff models may be calibrated independently for each gauged
catchment. In this case, independent parameter sets will be derived for the
rainfall runoff models of each catchment; or
A joint calibration may be performed, with rainfall runoff models calibrated with
consistent parameters to fit to the gauge records from two or more gauges. In
this case, a single set of rainfall runoff model parameters will be produced for all
of the catchments that represent a compromise to fit the flows at all of the
gauges within that group.
Consideration should be given at the outset of modelling to the approach that will be
used for dealing with flows from multiple catchments and subcatchments and from
multiple gauges. The strategy for dealing with this issue should be documented at this
point and revised, if necessary, during the process of calibrating and validating the
models.
Calibration of a rainfall runoff model normally involves running the model may times,
trialling different values of parameters, with the aim of improving the fit of the model to
the calibration data. Calibration can be facilitated:
Manually, with the modeller setting the parameter values, running the model to
inspect the results and then repeating this process many times;
Using automated optimisation, with an optimiser algorithm running the model
hundreds or thousands of times with different parameter values; or
Using a hybrid approach of automated optimisation phases, interspersed with
manually implemented trials of parameter sets.
Defining the calibration and validation approach before commencing a modelling
project can maximise the efficiency of the calibration process, whilst avoiding the
temptation to overfit the model to noise in the observational data. A calibration
strategy should therefore outline the:
gauge locations where model calibration and validation will be performed;
viable or allowable ranges for each model parameter value;
known constraints, dependencies or relationships between parameter values
(for example, the total of the three partial area parameters in AWBM, A1, A2
and A3 must sum to 1);
period for calibration at each gauge location;
period for validation at each gauge location;
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
14/50
Best Practice Guidelines for Rainfall-Runoff Modelling
expected level of uncertainty in observations introduced by measurement
uncertainty;
metrics to be used to test calibration and validation performance;
whether manual or automated calibration strategy will be adopted, or how a
hybrid strategy of progressive manual and automated calibration will be
implemented. If an automated or hybrid optimisation strategy is to be used, further details
should be defined at the outset of the calibration process on:
algorithms to be used for optimisation of parameter values;
objective function(s) that will be used to test the calibration performance;
weightings that may be applied in computation of objective functions, to
encourage fitting to different parts of the flow regime (typically the relative
weightings to high, medium and low flows); and
the set of model parameters that will be optimised during calibration and
constraints on the allowable range of values for each parameter.Ideally, calibration strategy should be documented prior to the commencement of the
calibration process. It may be appropriate for the calibration strategy to be reviewed
during the calibration.
2.2.5 Sensitivity/uncertainty analysis
Relevant sources of uncertainty in typical order of importance include:
5. Model input data including parameters, constants and driving data sets,
6. Model assumptions and simplifications of what the model is representing,
7. The science underlying the model,
8. Stochastic uncertainty (this is addressed under variability below),
9. Code uncertainty such as numerical approximations and undetected software
bugs.
The potential impacts of the above sources of uncertainty on the decisions that will be
made using the model should be considered early in the modelling process and then
re-examined once the model has been calibrated, validated and applied for scenario
runs. Uncertainty becomes more important for estimation of events in the tails of the
probability distribution, floods and droughts, than it is for consideration of events that
are closer to the centre of the probability distribution (such as estimation of the meanannual runoff from a catchment).
2.2.6 Documentation and Provenance
Documentation is an important requirement for model acceptance. Its role is:
1. To keep a record of what was done so that it can be reviewed and reproduced,
2. To provide source or background material for further work and research,
3. To effectively communicate the results from models, and
Page 12
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
15/50
eWater CRC
4. To effectively communicate the assumptions made during the modelling
process and the decisions made by the modeller during implementation of the
model.
Good documentation supports a study and it will also enable someone coming along
later to see what decisions were made, what was done to underpin the decisions and
why, particularly if aspects of the project are revisited at some later time.
Provenance, as it might relate to hydrological modelling studies simply means the
ability to trace the source/lineage of the data, model and modelling results. Reasons for
providing provenance in rainfall runoff modelling include:
1. Accountability and a full audit trail for all modelled results.
2. Repeatability; ability to re-create a results data set using current data or better
understanding.
3. Reproducibility; ability to re-create a results data set exactly using all original
data, workflow ordering, assumptions and parameters.
4. Versioning of both entire workflow and systems implementation. Versioning ofthe subcomponents and data sets will be the responsibility of those who govern
them but must be captured by the system.
The degree of provenance required depends on the application and/or how the
modelling system is intended to be used by the individual or organisation in future.
Current best practise provenance is to save all input data and model/parameters
version and workflow history such that the outputs can be reproduced in future if
required. In the future the ability to register and resolve the type and identity of objects
within the modelling process should reduce the requirement to capture and archive
these objects, especially as modellers take greater advantage of services based point
of truth data streams and modelling systems, and rely less on ad hoc locally managedresources.
2.2.7 Model acceptance and accreditation
The aim of model acceptance is to gain agreement that the model is fit for purpose.
Information available from the model accreditation process (Reporting, QA
documentation, Peer review) provides model development details and review results
which will enhance model acceptance.
Peer review plays an important part, especially with stakeholders that are external to
the organisation undertaking the model development. It is important for establishing
the credibility, reliability and robustness of results and the methodology used to obtain
the results. It is undertaken by people with specialist understanding in fields relevant to
the project.
2.2.8 Use of accepted/accredited model
Once a calibrated model is evaluated against good quality data and has undergone
thorough review process (model acceptance and accreditation), it can be used for
modelling to support water management planning and policy decisions (provided that
the model was accredited for similar purpose).
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
16/50
Best Practice Guidelines for Rainfall-Runoff Modelling
3 Model choice
3.1 Model selection
Model selection is made based on an understanding of the objectives and the system
being modelled (http://www.toolkit.net.au/Tools/Category-Model_development;
CRCCH 2005a, b). The WMO (2008, 2009) report include the following factors and
criteria as being relevant when selecting a model:
1. The general modelling objective; e.g. hydrological forecasting, assessing
human influences on the natural hydrological regime or climate change impact
assessment.
2. The type of system to be modelled; e.g. small catchment, river reach, reservoir
or large river basin.
3. The hydrological element(s) to be modelled; e.g. floods, daily average
discharges, monthly average discharges, water quality, amongst others.
4. The climatic and physiographic characteristics of the system to be modelled.
5. Data availability with regard to type, length and quality of data versus data
requirements for model calibration and operation.
6. Model simplicity, as far as hydrological complexity and ease of application are
concerned.
7. The possible transposition of model parameter values from smaller sub
catchments of the overall catchment or from neighbouring catchments.
8. The ability of the model to be updated conveniently on the basis of currenthydrometeorological conditions.
Other things that should be considered are:
1. The level of modelling expertise available.
2. Whether the model is going to be used on its own, or if it is going to be used in
conjunction with other models.
3. Freedom of choice may be limited by a desire to minimise problems of different
models for much the same purpose in the same project area, or to avoid
problems of different models in adjoining project areas, particularly where the
models are linked in some way in the future or results compared in some way.
4. Whether uncertainty will be explicitly modelled. If uncertainty is to be explicitly
included, what types of uncertainty are to be modelled (e.g. climatic uncertainty,
uncertainty in climate change projections, uncertainty in rainfall runoff model
parameter values); what approaches will be used to generate the replicates to
represent uncertainty and how many replicates will be required to adequately
quantify uncertainty.
5. Whether simulation or optimisation, or a combination of both, is adopted.
6. Whether the model is to be used for hindcasting or forecasting when being
applied in predictive mode.
Page 14
http://www.toolkit.net.au/Tools/Category-Model_developmenthttp://www.toolkit.net.au/Tools/Category-Model_development -
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
17/50
eWater CRC
In essence the governing principle in choosing a model should be that it should not have
more parameters requiring calibration or a greater level of detail than the available data can
support, to minimise problems of spurious results and false calibrations.
3.2 Available models
Rainfall runoff models can be represented by a range of approaches, in order ofincreasing complexity as:
simple empirical methods (e.g., curve number and regression equations);
large scale energy-water balance equations (e.g., Budyko curve);
conceptual rainfall-runoff models (e.g. SIMHYD, Sacramento, AWBM)
landscape daily hydrological models (e.g., VIC, WaterDyn);
fully distributed physically based hydrological models which explicitly model
hillslope and catchment processes (e.g., SHE, TOPOG).
These categories have been used for ease of description, and there is overlap betweenthese model types. Although these approaches vary in terms of the complexity with
which they represent the rainfall-runoff transformation processes, all of them
conceptualise the real processes using some sets of mathematical equations (and
hence are all conceptual models of the physical environment). Similarly, conceptual
rainfall-runoff models run in distributed mode can be classed as being landscape daily
hydrological models. This section provides a discussion of the characteristics of each
of these model types, along with a broad assessment of the strengths and weaknesses
of each approach for rainfall runoff modelling (Table 3-1).
3.2.1 Empirical methodsEmpirical methods to rainfall runoff modelling typically involve the fitting and application
of simple equation(s) that relate drivers of runoff response to flow at the catchment
outlet. Empirical equations are most often derived using regression relationships.
Common predictor variables may include rainfall for the catchment, flow observed at
another gauge in the vicinity, evapotranspiration, groundwater levels, vegetation cover
and the impervious area within the catchment. Where rainfall is used as a predictor
variable, regression relationships derived almost always include a non-linear
relationship between rainfall and runoff.
All catchments incorporate storage elements, including interception by vegetation,
storage within the soil column, groundwater storage and storage within stream
channels. Catchment storage typically results in runoff from the catchment being an
integrated function of the climatic conditions for the catchment over some period prior
to the period for which runoff is to be calculated by the model. Therefore, empirical
models that produce acceptably accurate simulations of runoff are either applied at
sufficiently long time steps that changes in internal water storage within the catchment
can be ignored (e.g. annual time step) or applied to represent an integration of the
climatic conditions that occurred over some time period prior. As a practical example,
for most catchments a regression model that only includes daily rainfall on the current
day is likely to produce a very poor estimation of daily runoff but a model for predicting
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
18/50
Best Practice Guidelines for Rainfall-Runoff Modelling
daily runoff that used individual values of daily rainfall for several days prior may
produce acceptable runoff estimates.
Empirical regression relationships are often developed using spreadsheets. They can
also be fitted using more sophisticated statistical analysis packages, which may more
easily facilitate the investigation of predictor variables. For general information on the
development of regression relationships, the modeller is referred to NIST/SEMATECHe-Handbook of Statistical Methods (NIST and SEMATECH, 2010) or to a University
Level statistics text book.
Empirical regression equations are best suited to situations where there are two flow
gauges on the same stream with partially overlapping periods of record, which are
therefore subject to similar climatic drivers, and the regression equation is used to
extend the simulated flow to the combined period of record from both sites. They can
also produce adequate simulations for neighbouring gauged catchments with
overlapping periods of record in situations where the two catchments are subject to
similar rainfall timeseries and are relatively similar hydrologically.
3.2.2 Large scale energy-water balance equations
The large scale energy-water balance methods are based on the hypothesis of
available energy and water governing large scale water balance (precipitation,
evaporation and runoff). These are usually developed using large scale observed data
sets, eg. the Budyko curve (Budyko, 1958) was developed using mainly European
data, and numerous other forms have been proposed to improve estimates in local
regions and to account for different land cover types (Arora, 2002). One of the more
popular forms of the Budyko method is the Fu (1981) rational function equation (Zhang
et al., 2004) where a single parameter, , in the equation can be calibrated againstlocal data to tune the method for the local conditions. The inputs to these equations are
rainfall and potential evapotranspiration (PET) and the output is runoff at mean annual
time step.
3.2.3 Conceptual Rainfall-Runoff Models
Conceptual rainfall runoff models represent the conversion of rainfall to runoff,
evapotranspiration, movement of water to and from groundwater systems and change
in the volume of water within the catchment using a series of mathematical
relationships. Conceptual rainfall runoff models almost always represent storage of
water within the catchment using several conceptual stores (or buckets) that can
notionally represent water held within the soil moisture, vegetation, groundwater or
within stream channels within the catchment. Fluxes of water between these stores
and in and out of the model are controlled by mathematical equations.
Most applications of conceptual rainfall runoff models treat the model in a spatially
lumped manner, assuming that the time series of climatic conditions (notably rainfall
and potential evapotranspiration) and the model parameter values are consistent
across the catchment. There have been implementations in more recent times of
conceptual rainfall runoff models in semi-spatially distributed and spatially distributed
frameworks. In distributed application, the catchment is defined by grid cells orsubcatchments within the catchment that are assigned the same rainfall runoff
Page 16
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
19/50
eWater CRC
parameter values but different time series of climatic inputs so that different grid cells
or subcatchments within the catchment produce different contributions to the overall
runoff. This is effectively a series of lumped rainfall runoff models, with lumped sets of
model parameters that are applied with spatially distributed rainfall.
Conceptual rainfall-runoff models have been widely used in Australia for water
resources planning and operational management because they are relatively easilycalibrated and they provide good estimates of flows in gauged and ungauged
catchments, provided good climate data is available.
In Australia there are six widely used conceptual rainfall-runoff models; AWBM
(Boughton 2004), IHACRES (Croke et al. 2006), Sacramento (Burnash et al. 1973),
SIMHYD (Chiew et al. 2002), SMARG (Vaze et al., 2004) and GR4J (Perrin et al.
2003). The input data into the models are daily rainfall and PET, and the models
simulate daily runoff. The models are typical of lumped conceptual rainfall-runoff
models, with interconnected storages and algorithms that mimic the hydrological
processes used to describe movement of water into and out of storages. They vary in
terms of the complexity of the catchment processes that they try to simulate and in
terms of the number of calibration parameters which vary from four to eighteen.
3.2.4 Landscape daily hydrological models
These models are based on the concept of landscape processes and they model the
typical landscape processes using simplified physical equations (VIC model, Liang et
al., 1994; 2CSALT, Stenson et al., 2011; AWRA-L, Van Dijk, 2010). A catchment is
usually conceptualised as a combination of landscapes which are delineated using
some combination of outputs from digital elevation model analysis, underlying geology,
soil types and land use. Often these models have been designed to reproduce othervariables in addition to streamflow (e.g. distributed evapotranspiration, soil moisture,
recharge, salinity), and as a result have a greater complexity to methods that target
streamflow alone.
3.2.5 Fully distributed physically based hydrological models which explicitly
model hillslope and catchment processes
The physically based models are based on our understanding of the physics of the
hydrological processes which control the catchment response and use physically
based equations to describe these processes. A discretisation of spatial and temporalcoordinates is made at a very fine scale for the entire catchment and the physical
equations are solved for each discretised grid to obtain a solution.
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
20/50
Best Practice Guidelines for Rainfall-Runoff Modelling
Table 31 Assessment of Strengths and Weaknesses of Different Rainfall RunoffModel Structures
Criteria Model Type
Empirical Large Scale
Energy-Water
Balance
Conceptual Landscape
Daily
Fully
Distributed
Physically
Based
Typical Run
Time Step
Can be
daily if
daily flow
from
another
gauge is
used as a
predictor
variable.
Otherwisetypically
only
applied at
annual (or
longer)
time scale
Typically only
applied for
mean annual
runoff,
although
pattern of
flows from a
nearby gauge
may be used
todisaggregate
annual totals
to monthly or
daily time
steps
Daily,
although
shorter run
time steps
are
possible if
sufficient
climatic
data is
available atthis shorter
time step
Daily, although
shorter run
time steps are
possible if
sufficient
climatic data is
available at
this shorter
time step
Minutes to
hours to
maintain
numerical
stability,
although often
forced with
daily data and
assumed
patterns usedto
disaggregate
to shorter time
steps
Typical
Number of
Parameters
1 to 5 2 to 4 4 to 20 10 to 100 10 to 1000's
Risk of over-
fitting or over-parameterising
the model.
Low Very Low Moderate High Very High
Need for high
resolution
spatial data
layers
None to
Moderate
Low to
Moderate
Low High Very High
Strength of
Apparent
Connection
between Model
Parametersand
Measurable
Physical
Catchment
Characteristics
None None Weak for
most
parameters
(although
imperviousarea or
interception
may be
exceptions)
Moderately
weak
Claimed to be
strong by
proponents
but can be
difficult tovalidate this
claim
Run time on
typically
available
computer
platforms for
100 years ofdaily data
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
21/50
eWater CRC
Criteria Model Type
Empirical Large Scale
Energy-Water
Balance
Conceptual Landscape
Daily
Fully
Distributed
Physically
Based
Ability toimplement
multiple runs
for automated
calibration
Nottypically
required -
optimum
parameters
can be
obtained
by least
squares
fitting that
does not
require
multiple
runs
Not typicallyrequired
Very Good.Run times
are
typically
sufficiently
low to
facilitate
this and
tools are
available
(Rainfall
Runoff
Library and
Source
IMS) to
facilitate
this
Good. Runtimes likely to
be sufficiently
low to facilitate
this in most
circumstances,
however tools
for calibrating
such models
using
automated
routines are
not as widely
available
Poor. Runtimes are
generally too
long to
consider
automated
calibration
Typical
Performance in
Regionalisation
Moderate
at annual
time steps.
Usually
very poor
at shorter
time steps
(e.g. Daily)
Good at
annual time
steps. Usually
very poor at
shorter time
steps (e.g.
Daily)
Moderate
at daily
time steps
Proponents
claim to be
superior for
regionalisation
to conceptual
rainfall runoff
models
Proponents
claim this to
be a strength
of distributed
models but in
reality the
large number
of parameters
required may
compromise
the application
of distributed
models to
regionalisation
Representation
of non-
stationarity in
catchmentconditions
Not
possible
Often applied
to explicitly
represent
non-stationarity in
vegetation
cover for
mean annual
runoff signal
Usually
difficult,
due to lack
of physicalmeaning
for many
model
parameters
Possible Possible
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
22/50
Best Practice Guidelines for Rainfall-Runoff Modelling
Criteria Model Type
Empirical Large Scale
Energy-Water
Balance
Conceptual Landscape
Daily
Fully
Distributed
Physically
Based
Typicalperformance of
model when
applying to a
very different
climatic period
to that used for
calibration
Poor Moderatewhen used to
estimate
impact on
mean annual
flow but flows
disaggregated
to shorter
time steps are
likely to be
poorly
estimated
Variable -can be
good in
some
catchments
but poor in
others
Variable - canbe good in
some
catchments
but poor in
others
Variable - canbe good in
some
catchments
but poor in
others
Typical level of
expertise with
this type of
model within
Australian
water industry
Strong Moderate Strong Weak Very weak
Likelihood that
previously
calibrated
models are
available for
catchment to
be modelled.
Moderate
to Low
Moderate Very High Low Very low
Page 20
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
23/50
eWater CRC
4 Collate and Review Data
Climatic data is the most important driver of any rainfall runoff modelling process. The
calibration and validation of models also involves comparison to observed streamflowdata. Major causes of difficulty in calibrating rainfall-runoff models are errors and
uncertainties in the input data (see Kavetski et al, 2003). A discussion of these
problems can be found in the collection of papers in Duan et al 2003. Checks should
therefore be performed on the input data and the comparison data set for calibration
and validation to be used in rainfall runoff modelling before any attempt is made to
apply or calibrate the models. The intent here is to investigate the integrity of the data,
whether observations are in the first instance plausible (e.g., is the volume in a
hydrograph less than the product of the rainfall and catchment area). Investigations
into data to be used for rainfall runoff modelling should include checks of:
Stationarity of the data time series , i.e. has there been any systematic or step
change in the statistical properties over the time of data collection, and if so
why;
Spatial coherence of data, i.e., is the data consistent with regional spatial and
temporal patterns and trends;
Accuracy of the spatial location of the gauging site;
Consistency in the approach used to date and time stamp the data, particularly
for data provided by different agencies;
Procedures use for spatially interpolation of point observations to gridded data
estimates or estimated series across catchment areas
e.g., time series plots at different levels of temporal aggregation, ranked plots, residual
mass curves, double mass curves, etc. This will pick up patterns as well as identify
anomalies which may be potential data QA issues.
Other checks and analysis, including regional consistency of runoff depths, rain-runoff
ratios, rating confidence limits, period of record, whether rainfall and PET is observed
or interpolated, base-flow characteristics, checks for stationarity and variability over
time, etc would also be useful. It is important that prior knowledge is considered.
One major factor which will apply across all types of time series data used is that thetime base must be kept consistent so that the data applies to the same time period. An
example is where flow data time steps should be matched to the rainfall data time step.
In Australia, daily rainfall data is commonly recorded as the depth of rainfall that
occurred in the 24 hours preceding 9 am on the date of the recorded data. In contrast,
daily streamflow totals are often quoted for the 24 hour period commencing on the
nominated date, resulting in the recorded flow data being offset by 1 day forward of the
rainfall data. Where possible the flow data should be extracted at a time step to match
the rainfall dataset. HYDSTRA databases allow this where the records are at short time
intervals. In other cases shifting the recorded time series by one day for either the
rainfall or flow time series may be required to produce consistent time series for
modelling.
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
24/50
Best Practice Guidelines for Rainfall-Runoff Modelling
The remainder of Section 4 outlines the data types, sources, availability, accuracy,
manipulations (such as gap filling) and any other issues.
4.1 Catchment details
4.1.1 Location of gauges (streamflow, rainfall and evaporation)
The streamflow recorded at the catchment outlet is a combined response to the spatial
distribution of rainfall and evaporation across the catchment. There are uncertainties
associated with the streamflow measurements due to rating curve errors as well as due
to extrapolation outside the limits of the rating curve. There is spatial variability in
rainfall (and to smaller extent evaporation) across a catchment which is not captured
when undertaking lumped catchment modelling using a single rain gauge. There might
be problems with the location of the rain gauge in terms of capturing a representative
rainfall for all the rainfall events especially for catchments with high rainfall gradients.
4.1.2 Topography and Catchment AreasThe catchment area for a catchment represents the contributing area to the catchment
outlet where the streamflow is measured. The catchment boundaries (and the
corresponding catchment area) can either be derived from topographic map layers or
using the catchment digital elevation model (DEM) and a standard package such as
ARCGIS. It is usually easier to determine catchment area for the catchments located in
steeper terrain compared to those located in very flat areas (especially when using
DEM).
Slope and dominant aspect may provide useful explanatory variables for estimating
routing parameters or for regionalisation of rainfall runoff parameters betweencatchments.
4.1.3 Soil types
A catchments rainfall-runoff response is related to the soil types in the catchment. The
surface soil characteristics determine the infiltration rates and so the contributions from
different flow components (surface runoff, throughflow and base flow). Soils information
can be obtained from any soils field work that has been undertaken in the catchment or
from large scale soil properties maps (e.g. Australian Soils Atlas, Northcote et al.,
1960). In most practical applications of conceptual rainfall-runoff models in Australia,
soil information is seldom directly used as input in the calibration process because the
inherent spatial variability in soil properties within a catchment is typically sufficiently
large that it has been difficult to demonstrate statistically robust relationships between
conceptual model parameters and soil properties.
4.1.4 Vegetation
Land cover/vegetation cover in a catchment can often be correlated with the amount of
interception storage/loss and actual evapotranspiration in a catchment. The land cover
across the catchment can be derived from large scale vegetation mapping based on
satellite imagery or remotely sensed data. Vegetation cover data has not typically beenused explicitly in directly determining rainfall runoff model parameters, although there
Page 22
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
25/50
eWater CRC
have been some recent studies which have demonstrated the importance of catchment
land cover in rainfall-runoff model calibration and for predictions in ungauged basins
(Zhang and Chiew, 2008; Vaze et al., 2011c).
4.1.5 Water Management Infrastructure
Water management infrastructure within a catchment can allow humans to make verysubstantial modifications to flows within a catchment. Water management infrastructure
may include large dams, farm dams and off stream storages, extractions, man-made
canals or diversion pipelines and discharges from sewage treatment plants. Locations
of these infrastructures should be identified where they exist within the catchment so
that their potential impact on streamflows may be assessed. Recorded flows at the
catchment outlet may require adjustment to allow for the influence of water
management infrastructure located upstream of each of the flow gauging locations.
4.2 Flow data
Reliable measurements of streamflow data are critical for successfully calibrating arainfall-runoff model to a catchment. The streamflow data for all the gauged locations
can be obtained from the respective state government water management agencies or
from the Bureau of Meteorology (in Australia). Considerations in checking streamflow
data include:
the agency collecting the data and the quality assurance procedures (if any)
implemented by that organisation during data collection;
reliability of the rating of levels to flows for the gauge;
the accuracy, extent and currency of cross sections surveyed at the gauge site.
(Surveyed cross sections may only extent to the top of the stream bank and
gauging for flows extending onto the floodplain may use a cross section that is
inaccurate);
vegetation and substrate material for the channel bed, channel banks and
floodplain and the influence of assumptions made about these on gauged flows;
the ratio of the highest flow outputs to the highest flow that the gauge has been
rated for;
how hydraulically stable (variable over time) the rating site is;
examination of potential backwater effects for the site from influences that are
downstream of the site, such as stream confluences, bridge crossings, culverts,dams or weirs;
hysteresis effects leading to different flow rates for the same recorded level on
rising and falling limbs of hydrographs;
how well maintained the gauging site and instrumentation used for measuring
water levels has been;
any changes to the gauging instrumentation over time;
the length of time since the last rating at high flows;
length of record at the site;
availability of quality codes with the flow data record;
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
26/50
Best Practice Guidelines for Rainfall-Runoff Modelling
proportion of missing data;
trends in when data is missing from the record (i.e. Is there any bias toward
high or low flow periods, particular seasons, or are the gaps just random?) and
how this might influence any infilling procedures; and
if there are a number of gauges closely located that basically represent the
same catchment the data sets may be able to be combined to give a longerrecord for the site.
Assessment of the above factors will inform whether the data is useful in calibration of
the model, independent validation of the model or whether the data should be ignored.
4.3 Rainfall
Rainfall is the main driver of runoff and so reliable measurements of rainfall are critical
for successfully calibrating a rainfall-runoff model to a catchment. There are several
sources for obtaining climatic data for a particular catchment:
Site observations from Bureau of Meteorology climate database.
Site observations taken from monitoring sites collected by other organisations
that may exist outside of the Bureau of Meteorology database. Many
jurisdictional databases contain rainfall records.
Gridded data products, such as the Bureau of Meteorologys Australian Water
Availability Project (AWAP) or Queensland Centre for Climate Applications
SILO data set.
It is important to be aware of how this data has been collected and what data quality
control methods have already been applied to the data prior provision of the data set
as this may influence the modelling results. This is particularly relevant to gridded
products, such as SILO and AWAP (SILO, J effrey et al., 2001; AWAP, J ones et al.,
2009), as these data sources generally use different algorithms to convert time-series
observations at data points to gridded data products.
In a small catchment, considerably better results may be obtained from using rainfall
station data from the BOM (http://www.bom.gov.au/climate/) or locally collected data
than a gridded data set that smoothes observations from a smaller number of more
sparsely located sites. In some cases it may be appropriate to adjust the station data,
normally by a percentage, if the mean catchment rainfall can be defined using other
sources e.g. isohyetal detail.
In large catchments there is spatial variability in rainfall across a catchment which is
not captured when undertaking rainfall-runoff modelling using rainfall time series from
the rain gauges. If using a single rain gauge, there might be problems with the location
of the rain gauge in terms of capturing a representative rainfall for all the rainfall
events. If using a spatial rainfall product (SILO or AWAP in Australia), there can be
uncertainties introduced because of the method used for interpolating rainfall between
rain gauges and changes in the rain gauge network over time. Interpolation methods
currently used are more suited to areas where rainfall varies less over space and in
time. They do not account well for orographic effects, and rainfall networks in Australia
historically have not captured the spatial and temporal variations in tropical andmonsoonal areas well.
Page 24
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
27/50
eWater CRC
Vaze et al., 2010b discusses testing carried out considering the effects of using
different rainfall data sets on the calibration and simulation of conceptual rainfall-runoff
models. They conclude that considerable improvements can be made in the modelled
daily runoff and mean annual runoff with better spatial representation of rainfall. Where
a single lumped catchment-average daily rainfall series is used, care should be taken
to obtain a rainfall series that best represents the spatial rainfall distribution across the
catchment. However where only estimates of runoff at the catchment outlet arerequired, there is little advantage in driving a rainfall-runoff model with different rainfall
inputs from different parts of the catchment compared to using a single lumped rainfall
series for the catchment.
4.4 Evapotranspiration
The measured pan evaporation data can be obtained for all the locations with the
evaporation gauges installed (in Australia from the Bureau of Meteorology (BoM) basic
records). In Australia there are also some spatial climate products which use point
evaporation measurements recorded by the BoM and use an interpolation schemes to
produce spatial evaporation surfaces (SILO, J effrey et al., 2001; AWAP, J ones et al.,
2009).
The network of pan evaporation recording stations in Australia is sparse in comparison
to stream flow and rainfall networks, although there is some compensation in that
typically potential evapotranspiration exhibits substantially higher spatial correlation
than rainfall or stream flow. This limits the ability to accurately represent the true spatial
and temporal variability in evaporation in models however the spatial variability in
evaporation is much smaller compared to the variability in rainfall.
The BoM network records pan evaporation. Modelling requires potentialevapotranspiration (PET). There are a number of methods to convert pan evaporation
to PET including Penman Monteith, Mortons and accepted pan factors. These use
climatic variables in the conversion calculation including solar radiation, temperature,
vapour pressure, and wind speed which are recorded at some pan recording stations
but not all. This further limits the network available to draw data from.
When all the required data is available the conversion calculations will use the records
but often some variable is missing and estimates of that variable are made and used.
Where there is no data for the climatic variables, calculated pan to PET conversion
factors from a nearby station can be used to derive PET from pan evaporation.
Commonly the spatial products have interpolated layers for a range of climatic factors
and the spatial PET layer is calculated from data in these layers rather than
interpolating PET calculated at recording stations.
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
28/50
Best Practice Guidelines for Rainfall-Runoff Modelling
5 Statistical Metrics for TestingPerformance
There are many performance measures used to consider the acceptability of a rainfall-runoff model. In all cases visual assessment and statistical results of some sort will be
assessed to identify the ability of the model to reproduce the flows it is calibrated or
validated against. All may contribute to best practice and which measures are more
appropriate will be directed by the modelling objective. A number of commonly used
visual assessment techniques are outlined inTable 51. Statistical performance
measures and their relevance in various study types are listed below inTable 52.
Table 51 Plots for assessing model performance
Plot Assessment and Purpose
Daily and monthly plots (linear
and log)
Used to check the general size, shape and timing of hydrographs.
Linear plots will better show medium and high flows and log plots low
flows. Baseflow and recession characteristics can be reviewed. If
recessions are frequently too flat then this can indicate that the
interflow and baseflow are not represented correctly.
Scatter Plot Scatter plots show the ability of the model to match flows on actual
time steps. They show the flow ranges where the model is more
accurate. Linear and log plots will show the variability across the
various flow ranges. Often a line of best fit is shown to indicate the
bias of the model in estimating flows.
Ranked Plots Commonly referred to as frequency of excedence or flow durationgraphs, they show the percentage of time a flow is exceeded over the
modelled period. They show whether the modelled output can
replicate the observed flow regime. Flow duration curves are effective
diagnostics to ensure that both the variability and the seasonal pattern
are captured.
Cumulative mass or cumulative
residual mass curves
Scatter plots and flow duration curves do not examine the time
sequence of events. A model could appear to be replicating the flow
regime however the replication of regimes during wet and dry periods
may not be adequate. A cumulative residual mass curve is a
cumulative plot of residuals (flow value - mean of all values). A
residual, and therefore slope of the curve, will be positive during wet
periods as flows are higher than average and during dry periods theslope will be negative. If the curves diverge there may be a data
issue. If they diverge consistently in all wet or all dry periods it is likely
that model parameterisation for wet periods or dry periods may not be
appropriate.
Plotting average daily or monthly
flows (average of all Days,
average of all J anuaries)
A simple diagnostic to ensuring that the model can replicate
seasonality characteristics.
Page 26
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
29/50
eWater CRC
Table 52 Statistical performance measures (metrics) and their relevance in variousstudy types (Y Yes, N No)
Metric Purpose
Runoff
Yield
Climate
change
Landuse
change
Low
flow
Water
quality
Peak flow
/ floods
Difference in total runoff Y Y Y N N Y
Difference in total runoff over different
seasons of the year*Y
Y Y Y YY
Difference in total runoff contained
within high, medium and low parts of the
flow duration curve
Y
Y Y Y YY (high
flows)
Difference in proportion of time that
cease to flow occursN
Y Y Y YN
Difference in the slope of logarithm of
flow versus time for baseflow recessionperiods
N
N Y Y Y
N
Mean square error between observed
and modelled runoffY
Y Y N NY
Coefficient of determination (often
referred to as r)Y
Y Y N NY
Nash Sutcliffe coefficient of efficiency
on daily flowsY
Y Y N NY
Nash Sutcliffe coefficient of efficiency
on monthly accumulated flowsY
Y Y N NN
Nash Sutcliffe coefficient of efficiency
calculated using logarithm transformed
flows
N
Y Y Y Y
N
* Definition of seasons to be used will vary depending upon the climatic zone that the catchment is in. For
tropical areas, two seasons (a wet season from December-April and dry season from May-November)
may be appropriate. In Southern Australia, it may be appropriate to consider the four conventional
calendar seasons (Dec-Feb, Mar-May, J un-Aug and Sep-Nov).
** Definitions of high, medium and low flow ranges will depend upon the purpose of the study and the
catchment. Typical ranges might be High flows: days in observed data in the 0 to 20% probability of
exceedance range; Medium flows: days in observed data in the 20 to 80% probability of exceedance
range; Low flows: days in observed data with greater than 80% probability of exceedance and above the
cease to flow level at the gauge. Adjustment of the low and medium flow ranges may be required
particularly in response to the probability of cease to flow conditions at the gauge.
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
30/50
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
31/50
eWater CRC
recorded data should be checked using independent data sources (such as aerial
photography, satellite imagery, landuse, topographic or other spatial information).
A more sophisticated calibration approach can involve multiple calibration and
validation periods. As in the simple split sample approach, the model is calibrated to a
calibration period and then performance is tested over the validation period without
changing the model parameter values. This approach is then repeated multiple times,with each replicate using different start and end dates for the respective calibration and
validation periods. This allows a range of model performance statistics for calibration
and validation periods to be reported.
There will be some instances with this calibration and validation approach where the
calibrated parameters perform well against the calibration data set, but performs poorly
against the validation data set. In research type investigations, where the modeller may
be comparing different rainfall-runoff models, calibration methods, or objective
functions, the validation results can be used directly to help decide the best model or
method or objective function. However, in practical applications, a modeller may have
to decide either not to change the calibrated parameters and report the poor results, or
to recalibrate the model because the performance is unacceptable.
The modeller may choose the latter option, and may then recalibrate and compare
against the validation data set several times until the calibrated parameters perform
acceptably against both data sets. However, as the validation data set has now been
used to change the calibrated parameters, it is no longer an independent data set and
has in effect indirectly become part of the calibration data set.
This risk of having much poorer performance in validation than calibration may be
mitigated by ensuring as far as possible both data sets have similar flow distributions,An arbitrary approach to splitting the data, e.g., at the midpoint, may result in half of the
data being in a much wetter period. A model calibrated to these conditions would not
be expected to perform well under the drier conditions in the validation data set. More
alternate approaches should be considered on how to split the data set, perhaps into
non-contiguous periods, to ensure overall flow distributions are similar in each period.
Data is a valuable resource, and should be used to greatest effect. In most Australian
conditions, long data sets are needed to adequately represent climatic variability. An
alternative approach to having split samples is to use the complete data set to calibrate
the model, then to report its performance for different sub-periods, e.g., first half and
last half, or decadally, or driest X year period and wettest X year period. The objectivewould be to have a comparatively persistent performance across all these periods. This
does not necessarily give you an independent assessment of performance, but does
report on performance under different conditions.
Transposition of model parameter values from gauged to ungauged catchments may
be tested using a spatial variant on split sample validation. Under this approach,
component models from a gauged catchment with the calibrated parameter values for
that catchment can be applied to another gauged catchment to test the uncertainty and
bias introduced from transposition. Uncertainty ranges can be established by testing
contributions flow series produced by model outputs with parameter sets adopted from
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
32/50
Best Practice Guidelines for Rainfall-Runoff Modelling
several different gauged catchments. Examples of the performance of these
transposition approaches are discussed in Viney et al. (2009) and Chiew (2010).
Generally the same metrics used to assess the performance of the model during
calibration are also used to assess model performance during validation. The model
performance during validation is almost always poorer than during calibration because
model parameters are deliberately not specifically fitted to the data for the validationperiod.
6.3 Calibration and Validation of Models to Single Gauge
Sites, Mult iple Gauge Sites and Regionalisation of Model
Parameter Sets
It is a very common situation in a project that involves rainfall runoff modelling for flow
time series to be required for several catchments or subcatchments within the model
domain and for data to be available from two or more stream flow gauges to facilitate
calibration and validation. At locations where gauged flows are available and flowestimates are required, two options are available to the modeller:
The rainfall runoff models may be calibrated independently for each gauged
catchment. In this case, independent parameter sets will be derived for the
rainfall runoff models of each catchment; or
A joint calibration may be performed, with rainfall runoff models calibrated with
consistent parameters to fit to the gauge records from two or more gauges. In
this case, a single set of rainfall runoff model parameters will be produced for all
of the catchments that represents a compromise to fit the flows at all of the
gauges within that group.
The advantage of the joint calibration approach is that, assuming some degree of
homogeneity in the rainfall runoff response of the selected gauged catchments, the
parameter sets produced should be more robust when applied to other catchments
with similar response that were not used for the calibration.
If an automated calibration process is used for joint calibration of multiple catchments,
the objective function used for automated calibration to the gauged catchments will be
a weighted average of the objective function values produced at the individual gauges.
Options for selecting the weighting values are:
All gauged catchments contribute equally to the overall objective function;
Weights are assigned according to the length of available record (e.g. number
of days with data) at each site;
Weights are assigned according to the inverse of the correlation coefficient in
gauged flows between one gauge and one or more of the other gauges in the
set (i.e. gauges with strongly correlated recorded flows are assigned lower
weighting factors than gauges that have weaker correlations with other gauges);
Some combination of the above factors.
There are three main methods of developing flow data sets in residual ungauged
catchments between upstream and downstream gauges:
Page 30
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
33/50
eWater CRC
1. Calibrate a model to the difference in flow between the gauged upstream flows
routed to downstream (adjusted for known transmission losses) and
downstream gauges.
2. Adjust a flow data set from a nearby catchment using either recorded or
generated data,
3. Apply parameter values from other calibrated models and use catchment
appropriate climate data.
There are two main methods of developing flow data sets in ungauged catchments:
1. Develop a regression equation between flows for the ungauged catchment and
gauged catchments and apply this equation to transpose the flow, or
2. Apply parameter values from other calibrated models and use catchment
appropriate climate data.
Generally in the second case parameters for a neighbouring or nearby catchment are
used but climate data and catchment characterises of the catchment of interest are
applied in the model. Many studies have shown that selecting a donor catchmentbased on spatial proximity gives similar or better results than selecting a donor
catchment based on catchment attributes (Merz and Bloschl 2004, Oudin et al 2008;
Parajka et al. 2005; Zhang and Chiew 2009).
6.4 Automated, Manual and Hybrid Calibration Strategies
Calibration of hydrological models can be conducted using manual or automated
methods, or a combination of the two approaches (see Boyle et al, 2000 and Brdossy,
2007 for frameworks for combining manual and automated methods of model
calibration). Calibration involves the adjustment of model parameter values to improve
the fit of model output data to observations to a level that is acceptable.
In case of manual calibration, definition of goodness of fit is usually produced as a
combination of statistical indices and visual inspection of the observed and simulated
hydrographs. Whereas in case of automated calibration, definition of goodness of fit
is usually produced using an objective function. The objective function translates the
observed and modelled outputs into a single number, so that the results of successive
calibration iterations can be compared. Automated calibration routines use a defined
algorithm that runs the model multiple times, adjusting model parameter values
according to a strategy that is intended to improve the value of the objective function.
The sections that follow give information on the calibration methods available and their
relevance in various study types (shown in Table 3-1) which dealt with model choice
appropriate for intended purposes.
6.4.1 Manual Calibration
Manual calibration involves the modeller selecting a set of parameters for their model,
running the model once and then examining the output statistics from the model (from
the list discussed in Section 5). The modeller would then revise the values of one or
more parameters and repeat the above process. This may continue many times until
the model achieves the desired level of performance.
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
34/50
-
7/27/2019 eWater Guidelines RRM (v1 0 Interim Dec 2011)
35/50
eWater CRC
4. Repeatable. Different people will get same parameter values.
Weaknesses:
1. Is confined by the optimisation routine chosen and how the objectives are set.
2. Is dependent on the computer routine being set up accurately to reflect thechoices in 1.
3. Lack ability to check the relationships between the calibrated parameter values
produced as the calibration proceeds, which may cause investigation of sets of
parameters that are infeasible (unless appropriate checks are build within the
calibration algorithms).
4. Software is required to automate the optimisation process.
5. Parameter values commonly become trapped against the minimum and
maximum constraints of the allowable parameter ranges set by the user. If the
user does not check for this, the parameter set chosen may be sub-optimal as
the best parameter set may have parameter values that lie outside the
constraints set by the user at the time the optimisation is initiated.
There are two global optimisation methods included in Source: Shuffled Complex
Evolution (SCE) and Uniform random sampling. The analysis undertaken as part of the
testing with data from 200+ catchments in southeast Australia showed that there is an
advantage in following a global optimiser with a local optimiser to fine tune the
calibrated parameter values. The Rosenbrock method is included in the framework as
a local optimiser. The testing results suggest that the combination of SCE followed by
the Rosenbrock should be used (Vaze et al., 2011a,d).
6.4.3 Hybrid Calibration Str