uncertainty mapping for air quality harmonisation modelling ......, sam erik walker , sef van den...

Norwegian Institutefor Air Research

Harmo11 Conference, July 2007, Cambridge

Uncertainty mapping for air quality Uncertainty mapping for air quality modelling and data assimilationmodelling and data assimilation

Bruce Denby1, Ana Margarida Costa2, Alexandra Monteiro2, Agnes Dudek1, Sam Erik Walker1, Sef van den Elshout3 and

Bernard Fisher4

1 Norwegian Institute for Air Research (NILU), Kjeller, Norway2 CESAM & Department of Environment and Planning, University of Aveiro,

Portugal3 DCMR EPA Rijnmond, Schiedam, The Netherlands

4 Environmental Agency, Reading, UK

6th Framework Programme- Policy oriented Research Priority 8.1 Topic 1.5 Task 2

11th Harmonisation Conference

C

ambridge 2007


IntroductionIntroduction

IntroductionAim, background and applications

Sources of uncertaintyUncertainty parameters

Model errorProbability distribution functionsStandard deviation and bias

Uncertainty mappingSpatial distribution of model errorIndicative uncertaintyUncertainty related to input dataUsing variance from spatial statisticsUncertainty in exceedancesAccounting for temporal covariance, Monte Carlo methods, probability of exceedance, uncertainty in data assimilation methods, etc…

Conclusions


Sources of Sources of uncertaintyuncertainty

Uncertainty Uncertainty parametersparameters

Uncertainty Uncertainty mappingmapping

ConclusionsConclusions


C

ambridge 2007



AimRaise awareness and promote discussion on uncertainty mapping in air quality applications

BackgroundBased on discussions and developments during the FP6 project Air4EU (www.air4eu.nl)Discussions on uncertainty and its mapping can be found in a number of Air4EU case studies, recommendation documents and cross-cutting issue reportsUncertainty maps are presented on the Air4EU mapping tool (www.air4eumaps.info)







C

ambridge 2007



Why do we NOT use uncertainty maps?Decision makers do not want to know about uncertaintyThere is not enough information for the uncertainty assessmentIt may reflect unfavourably on the models

Why use uncertainty maps? It is at the heart of the scientific method to express uncertainty in any resultIt is honest and transparentProvides information on model quality Better basis for decision making







C

ambridge 2007


Sources of model uncertaintySources of model uncertainty

Model descriptionModel formulation

chemistry, dispersion, etc.

Numerical discretisationnumerical schemes, model resolution, etc.

Input dataEmissions MeteorologyBoundary conditions

Surface, horizontal, etc.

Monitoring data Data assimilation






RepresentativenessSpatial representativeness

model resolution, subgrid variability

Temporal representativenessstochastic processes


C

ambridge 2007


Uncertainty parametersUncertainty parameters

Range of parameters for assessing model error

Mean absolute error MAEMean square error MSENormalised mean square error NMSERoot mean square error RMSERelative percentile error RPE (EU directives)Bias BIASAverage normalised absolute bias ANBFractional bias FBCorrelation R2, rStandard deviation SD Normalised standard deviation NSDRegression coefficients slope, interceptIndex of agreement DFraction of predictions … FAC2

e.g. Chang and Hanna (2004), Borrego et al. (2007)

Model error is an example of poor spatial sampling of the PDF







C

ambridge 2007



Probability distribution functionsAre used in Bayesian approaches to uncertaintyDescribe the probable model result, given a ‘true’ or observed value (or visa versa)Contain all the uncertainty informationCan be used to derive other statistical parameters

EMEP model PDF. Daily mean PM10 concentrations on the regional scale

-1.5 -1 -0.5 0 0.5 1 1.50

0.2

0.4

0.6

0.8

1

1.2

1.4

Log 10(M/O)

Nor

mal

ised

PD

F

Mode l PDF (EMEP), daily me an PM10, all s tations






-1.5 -1 -0.5 0 0.5 1 1.50

0.2

0.4

0.6

0.8

1

1.2

1.4

Log 10 (M/O)

Nor

mal

ised

PD

F

Mode l PDF (AirQUIS), Hourly NO2 at Mangle rud, Os lo

AirQUIS model PDF. Hourly mean NO2 concentrations at a traffic station


C

ambridge 2007



The use of standard deviation (SD) and biasA PDF is not easy to show in map form, so …Use SD and bias to describe the normal PDFNOTE: PDFs in air quality tend to be log-normal rather than normal

Standard deviation is useful because…It can be statistically extracted from any sampleIt is similar to RMSEIt can by calculated from ensemble methodsIt can be derived from spatial statistical methods such as kriging

Bias must be includedSeparately or implicitly through, e.g., RMSEIf bias is known it should be removed







C

ambridge 2007


Uncertainty mappingUncertainty mappingExample maps of model error, legislative applications Spatial representation of RPE, Berlin O3 , REM-Calgrid model

RPE interpolated between points

using kriging11 11.5 12 12.5 13 13.5 14 14.5 15

51

51.5

52

52.5

53

53.5

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Spatial mapping of RPE for O3(Krigging interpolation method)

Spatial mapping of RPE for O3(no interpolation method)

11 11.5 12 12.5 13 13.5 14 14.5 1551

51.5

52

52.5

53

53.5

0.0-0.1

0.5-0.6

RPE (O3)

0.1-0.2

0.2-0.3

0.3-0.4

0.4-0.5RPE represented by dot sizes

P

PP

OMO

RPE−

=







C

ambridge 2007


Uncertainty mappingUncertainty mapping

Indicative uncertaintyPresenting maps that are indicative of the standard deviationGeneralised description of the standard deviation based on an absolute and a relative standard deviation

222 ),(),( yxMyx RAM σσσ +=

σR and σA can be calculated using:

Expert knowledge, e.g. σR = 25% ( )

M

OMn

n

iii

R

∑=

−= 1

21

σNormalised RMSE

∑=

⎟⎟⎠

⎞⎜⎜⎝

⎛−

−=n

i i

iR M

On 1

2

)1(11σSD of the PDF

( ) ( ) ( )

( )21 1

2

2

2

2,2/ −

′−

⎥⎥⎦

⎤

⎢⎢⎣

⎡ −+

+=

∑=

− n

MOMM

nnt

n

iii

MSDnM σ

σ αPrediction intervals

Fits to scatter plots







C

ambridge 2007



Example of an indicative uncertainty mapPrague, annual mean NO2 ,ATEM model

Model

Uncertainty

Based on normalised RMSEσR = 27%σA = 0







C

ambridge 2007


Uncertainty mappingUncertainty mappingApplication for scenario prediction

Predicted concentrations of PM10 for 2010 in Rotterdam, using the Urbis model

Model prediction

Uncertainty due to emissions

Expert assessment of emission uncertainty in the various source sectors

Spatial map of the emission scenario uncertainty







C

ambridge 2007


Uncertainty mappingUncertainty mappingSpatial statisticsMaps may also be made using kriging, residual kriging or other statistical interpolation methods

Uncertainty map

),(),( yxVaryxM =σ

SD is calculated in these methods using the kriging variance

Residual kriging of annual mean rural background PM10 using the

EMEP model and Airbase stations







C

ambridge 2007



Uncertainty in exceedancesUse daily mean uncertainty

probability of exceedanceUse the annual mean uncertainty

spatial representativeness biasCalculate the uncertainty in the number of exceedance days

Number of exceedances, and related uncertainty, of the daily mean EU limit

for PM10 rural background only

Large number of exceedance days (> 60 days)

Large uncertainty in the areas removed from

observations (SD > 35 days)







C

ambridge 2007



General commentsThere is currently no established methodology, parameter or presentation method for communicating uncertaintyIt will be necessary to have a common parameter(s) for representing uncertainty if it is to be useful for intercomparison purposes Different mapping methods use different methodologies for uncertainty assessment

RecommendationsSpatial interpolation of model error is not recommendedSome indication of the uncertainty must always be given, preferably as maps but otherwise as a single valuePresent assessment maps with a contour or colour selection that is indicative of the uncertaintyBias should be removed from maps when it is known







C

ambridge 2007



Future challengesConvincing air quality modellers to include uncertainty in their maps

- How many presentations of maps at this conference will include uncertainty?

Establishing homogenous and accepted methodologies for determining and presenting spatial uncertaintyIncrease cooperation between atmospheric modellers and spatial statistical groups Inclusion of uncertainty in the entire process, from emissions to risk assessment

- e.g. EU projects HEIMTSA and INTERESSEConvince decision makers that they should want to know about uncertaintyReformulation of directives and other legislation to properly include aspects of uncertainty







C

ambridge 2007

uncertainty mapping for air quality harmonisation modelling ......, sam erik walker , sef van den...

Documents