calibration and uncertainty analysis of models with mcmc … · lake model code . mylake • model...
TRANSCRIPT
Calibration and uncertainty analysis of models with MCMC-simulation
Tuomo Saloranta Section for glaciers, snow and ice
Hydrology department
Norwegian Water Resources and Energy Directorate (NVE)
Norwegian Water Resources and Energy Directorate
How to increase the quality and benefits from models? ■ Apply techniques for model analysis
■ better transparency and understanding ■ optimized model performance ■ uncertainties analyzed and quantified
■ Automatic calibration and uncertainty analysis (MCMC)
■ more ”honest” results ■ probabilities (risk = probability x consequence) ■ scanning for all plausible parameter-alternatives
2
03.03.2012
Norwegian Water Resources and Energy Directorate
Background & different types of uncertainty
Norwegian Water Resources and Energy Directorate
Why uncertainty analysis?
■ Traditionally uncertainties are unwanted and often ”suppressed”.
■ However, sometimes they are an inherent part of the system, and cannot be reduced.
■ Uncertainty management especially important in: ■ policy-relevant science (many stakeholders) ■ when stakes are high
4
03.03.2012
Norwegian Water Resources and Energy Directorate
What is uncertainty analysis?
■ Point values are replaced by probability distributions. ■ What we know, and what we don’t know, is reflected in our
answers.
”It is better to be roughly right than precisely wrong” (Keynes)
5
03.03.2012
Norwegian Water Resources and Energy Directorate
Distributions, means, percentiles
2.5% percentil = 1
50% percentil = 2
97.5% percentil = 3
Unif (0.5, 2.5) Norm (2, 0.5)
Logn (ln2, 0.5)
Norwegian Water Resources and Energy Directorate
A typology of uncertainty
by Funtowicz & Ravetz (1990)
■ Technical uncertainty (inexactness)
■ Methodological uncertainty (unreliability)
■ Epistemological uncertainty (border with ignorance)
7
03.03.2012
Norwegian Water Resources and Energy Directorate
Model uncertainty – a simple example
8
03.03.2012
20
25
30
35
40
Y
0 2 4 6 8 10X
Norwegian Water Resources and Energy Directorate
Monte Carlo simulation
9
03.03.2012
Say, based on empirical data, we have estimed uncertainty of PEC and PNEC.
How about the uncertainty of the ratio PEC/PNEC?
Norwegian Water Resources and Energy Directorate
Monte Carlo simulation
10
03.03.2012
Sample parameters (p1, p2)
Save results from simulation round n
n=n+1 (until n=N)
Run model
Analyse the N saved results statistically
Start here
Norwegian Water Resources and Energy Directorate
Model uncertainty – a simple example
11
03.03.2012
Norwegian Water Resources and Energy Directorate
Model uncertainty – a simple example
12
03.03.2012
In Monte Carlo simulation the PEC/PNEC ratio was >1 in 21 of 1000 samples. Thus, by 97.9 % probability we are on the ”safe side”.
Norwegian Water Resources and Energy Directorate
What is MCMC simulation?
■ Markov chain Monte Carlo simulation ■ Based on Bayesian inference and Monte Carlo simulation ■ Automatic model calibration against observations
(statistical fit) ■ Reveals all possible parameter combinations that give a
proper model fit with observations ■ Previous knowledge can be utilized in estimating
parameters (prior distributions)
13
03.03.2012
Norwegian Water Resources and Energy Directorate
Monte Carlo simulation
14
03.03.2012
Sample parameters (p1, p2)
Save results from simulation round n
n=n+1 (until n=N)
Run model
Analyse the N saved results statistically
Start here
Norwegian Water Resources and Energy Directorate
Markov chain Monte Carlo simulation
15
03.03.2012
Sample parameters (p1, p2)
n=n+1 (until n=N)
Analyse the N saved results statistically
Start here
Save p1, p2
Reject/accept p1, p2 based on a comparison of simulations and observations
Run model
Norwegian Water Resources and Energy Directorate
16
03.03.2012
Monte Carlo simulation:
Parameter uncertainty estimates based on data or experts
• some simulation results from the uncertainty analysis may not correspond well to observations.
Markov Chain Monte Carlo simulation:
Parameter uncertainty estimates fitted to observations
• gives both a model calibration and an uncertainty analysis consistent with observations.
Simulations
Obser-vations +
Pr (accept)
recect
Norwegian Water Resources and Energy Directorate
Steps in MCMC simulation
p1
p2 o
tid
C x x
x
If better ”match” than at previous round, accept p
p1
p2 o
tid
C x x
x
If worse ”match” than at previous round, discard p with probability α o
o
p1
p2 o
o
p1
p2 o
o
o o
Norwegian Water Resources and Energy Directorate
Example #1
Norwegian Water Resources and Energy Directorate
Saloranta, T. M et al. 2009: Hydrology Research 40, 234-248.
AIMS:
• To simulate the impacts of the projected future climate on “lake water climate” between control (1961-1990) and A2/B2 (2071-2100) scenarios
• To take properly into account model parameter-related uncertainties in the simulation results
LAKE SITES IN FINLAND:
• Small and shallow Valkea-Kotinen and larger and deeper Pääjärvi
Lake modelling & uncertainty analysis
Norwegian Water Resources and Energy Directorate
METHODS:
• Lake model code MyLake
• Model calibration and uncertainty estimation by Markov chain Monte Carlo (MCMC) simulation method
Parameter uncertainties estimated in the calibration period (2000-2002), are used to express model uncertainties in the climate scenarios.
Lake modelling & uncertainty analysis
Norwegian Water Resources and Energy Directorate
MCMC simulation
Melting ice albedo Melting ice albedo
Prior distribution
3 separate chains
Norwegian Water Resources and Energy Directorate
Conclusions - future lake climate
Valkea-Kotinen
Norwegian Water Resources and Energy Directorate
Example #2
Norwegian Water Resources and Energy Directorate
Saloranta et al. 2008, Environmental Science and Technology 42, 200-206.
Updating of parameter distributions
posterior
prior
Fjord POP modelling & uncertainty analysis
Norwegian Water Resources and Energy Directorate
Saloranta et al. 2008, Environmental Science and Technology 42, 200-206.
Fjord POP modelling & uncertainty analysis
Norwegian Water Resources and Energy Directorate
Thanks!