1

Isaac Newton Workshop on Probabilistic Climate Prediction

University of Exeter 20-23 Sep 2010

Professor David B. StephensonExeter Climate Systems

Mathematics Research Institute

2

Aims of the workshopNo universal agreement exists on what constitutes a reliable and robust framework for inferring and evaluating predictions of real-world climate.

This workshop aims- to debate the strengths and weaknesses of existing frameworks for inferring and evaluating predictions of real-world climate- identify and formulate pressing and potentially solvable problems in the mathematics of probabilistic climate prediction.

Workshop style: Interactive, constructive, stimulating.

Overview talks on the main issues and existing methods will set thescene and then these will be followed by small thematic breakout working groups. Working groups will reconvene to share ideas.

3

How should we use multi-model simulations ...

4

... to make inference about future observables?

Projections

Observations

Uncertainty around projections

sce

nar

ios

5

Forecast Assimilation

)y(p

)x(p)x|y(p)y|x(p

i

iiiii

Data AssimilationForecast Assimilation

)x(p

)y(p)y|x(p)x|y(p

f

fffff

Stephenson, D.B., Coelho, C.A.S., Balmaseda, M. and Doblas-Reyes, F.J. (2005): Forecast Assimilation: A unified framework for the combination of multi-model weather and climate predictions, Tellus A, 57 (3), pp 253-264

6

The Multi-Model Ensemble

A “fruit bowl of opportunity” {X1,X2,...,Xm}Note: Not a random sample from one homogeneous population(and it does not include all possible fruit!)

7

What does reality look like?

actual true climate Y – inferred from observations ZIt could not have been drawn out of my fruit bowl!

How can we infer properties of this from the fruit in the fruitbowl?

An inconvenient truth

8

Smoothies (multi-model means)

A smoothie is a weighted average of fruits. • It is not an item of real fruit! (important information has been lost by averaging)

• Non-unique choice of weights for making smoothies.

We require modelling frameworks for obtaining samples of real fruit from the posterior distribution p(Y|X) (not smoothies E(X) or E(X|Y)).

9

Should we use everything in the fruit bowl?

Should we select subsets? How should we weight the fruits?

“All fruit are wrong, but some are tasty” - Granny Smith

10

Homogeous samples

How to relate Y to X?• Are the {Xi} independent draws from a distribution centred on Y?• Are the {Xi} second-order exchangeable with each other and Y?• How best to model model discrepancy Y-Xi?

11

Breakout themesTheme A: Frameworks for quantifying uncertainty (Chair: Jonty Rougier in LT1)What frameworks are available for quantifying uncertainty in climate predictions (e.g. Bayesian probability, maximum likelihood, interval probabilities, random/fuzzy, etc.) and what are their respective strengths and limitations?

Theme B: Calibration of climate predictions (Chair: David Stephenson in room D)What grounds do we have for believing that our predictions are well-calibrated and how should we go about calibrating outputs from climate models (e.g. bias correction of extremes)?

Theme C: Evaluation of climate predictions (Chair: Chris Ferro in room E)How should we go about evaluating probabilistic climate predictions? How can we best determine the skill and reliability of probabilistic climate predictions at various space and time scales?

Theme D: Model processes and inadequacies (Chair: Mat Collins in room F)How should we represent and quantify known limitations in physical processes simulated by climate models? i.e. the ability to simulate long-lasting blocking events, the correct intensity of extreme storm events, etc.

Theme E: Problem area to be decided (Chair: Richard Chandler in Margaret Room)There are many other pressing and interesting questions in probabilistic climate prediction that require statistical research.

Each theme should have a rapporteur who will report back to the main sessions and record ideas on the wiki page http://empslocal.ex.ac.uk/iniw/Main/HomePage (user=newton pass=apple). Participants can move around themes.

12

Theme B: Calibration strategiesJohn Ho, Mat Collins, Simon Brown, Chris Ferro

How to infer distribution of Y’ from distributions of Y, X and X’?

1. No calibration Assume Y’ and X’ have identical distributions (i.e. no model biases!)

i.e. FY’ = FX’

2. Bias correction Assume Y’=B(X’) where B(.)=FY

-1 (FX(.))

3. Change factorAssume Y’=C(Y) where C(.)=FX’

-1 (FX(.))

4. Othere.g. Adjust parameters in parametric fits e.g.

X

Y Y’=?

X’

Y’=B(X’)

Y’=C(Y)

XXYY ''

13

Example: daily summer temperatures in London

Daily mean air temperatures

Y=Observations 1970-1999 from E-OBS gridded dataset (Haylock et al., 2008)

X=HadRM3 standard run (25 km resolution) forced by HadCM3; SRES A1B scenario.

n=30*120=3600 days

Black line = sample mean

Red line = 99th percentile

14

Probability density functions

Black line = pdf of obs data 1970-1999Blue line = pdf of climate data 1970-1999Red line = pdf of climate data 2070-2099

15

Linear calibration (constant shape)

ngmean warmi 6.3 i.e.

6.19)61.1565.19(35.3

01.395.15

:gives dataour for Which

)()'(

)'(

))'(()'('

)(

)(

Correction Bias

'

1

C

C

OE

G

GFFGBO

xFxF

xFxF

GGG

OO

GG

OO

GO

G

GG

O

OO

ngmean warmi 14 i.e.

1.20)61.1595.15(35.3

03.465.19

:gives dataour for Which

)()'(

)(

))(()('

)(

)(

factor Change

''

''

1'

'

''

C.

C

OE

O

OFFOCO

xFxF

xFxF

GOG

GG

GG

GG

GG

G

GG

G

GG

Two approaches give different future mean temperatures!

16

Change in 10-summer level 2040-69 from 1970-99

No calibrationTg’ - To

Bias correction Change factor

Substantial differences between different estimates!

17

18

Breakout themesTheme A: Frameworks for quantifying uncertainty (Chair: Jonty Rougier in LT1)What frameworks are available for quantifying uncertainty in climate predictions (e.g. Bayesian probability, maximum likelihood, interval probabilities, random/fuzzy, etc.) and what are their respective strengths and limitations?

Theme B: Calibration of climate predictions (Chair: David Stephenson in room D)What grounds do we have for believing that our predictions are well-calibrated and how should we go about calibrating outputs from climate models (e.g. bias correction of extremes)?

Theme C: Evaluation of climate predictions (Chair: Chris Ferro in room E)How should we go about evaluating probabilistic climate predictions? How can we best determine the skill and reliability of probabilistic climate predictions at various space and time scales?

Theme D: Model processes and inadequacies (Chair: Mat Collins in room F)How should we represent and quantify known limitations in physical processes simulated by climate models? i.e. the ability to simulate long-lasting blocking events, the correct intensity of extreme storm events, etc.

Theme E: Problem area to be decided (Chair: Richard Chandler in Margaret Room)There are many other pressing and interesting questions in probabilistic climate prediction that require statistical research.

Each theme should have a rapporteur who will report back to the main sessions and record ideas on the wiki page http://empslocal.ex.ac.uk/iniw/Main/HomePage (user=newton pass=apple). Participants can move around themes.

19

Ideas for Theme E? Theme E: Emergent problem area to be decided (Chair: Richard Chandler)There are many other pressing and interesting questions in probabilistic climateprediction that require statistical research.

Some such questions include:

• What is the purpose of introducing stochastic components into climate simulators?

• How can we determine what data (i.e. model simulations, observed data, paleoclimate data) are required to inform the development of improved climate projections?

• How should we best account for shared sources of uncertainty across a multi-model ensemble?

• What formal role is there for simpler simulators? • Do we believe the ergodic assumptions that are implicitly made in climate

science?• How best to visualise probability forecasts?• Others???

20

Tuesday 21 September: Overview talks and small group brainstorming

09:30-10:15 “Outstanding problems in probabilistic prediction of climate” David Stephenson10:15-11:00 “Model inadequacies and physical processes” Mat Collins11:00-11:30 Morning coffee/tea11:30-13:00 Small group breakout sessions on the 5 main themes13:00-14:00 Photo and Buffet lunch14:00-14:45 “Methodologies for probabilistic uncertainty assessment” Richard Chandler 14:45-15:30 “Probabilistic methodology used for UKCIP” David Sexton15:30-16:00 Afternoon tea/coffee16:00-16:15 "Probabilistic use of climate catastrophe multi-models" Gero Michel16:15-17:30 Small group breakout sessions17:30-19:00 Drinks reception in Holland Hall bar (kindly sponsored by Willis)19:00- Participants go for dinner at restaurants in Exeter

Wednesday 22 September: Overview talks and small group brainstorming

08:00-09:00 Breakfast in Holland Hall (for those staying there)Talks and discussions in the Queen’s building:09:30-10:15 “Non-probabilistic frameworks” Arthur Dempster10:15-11:00 “Probabilistic frameworks” Jonty Rougier11:00-11:30 Morning coffee/tea11:30-13:00 Small group breakout sessions on the 5 main themes13:00-14:00 Buffet lunch14:00-15:30 Small group breakout sessions15:30-16:00 Afternoon tea/coffee16:00-17:30 Small group breakout sessions17:30-19:00 Drinks reception in Holland Hall bar (kindly sponsored by Willis)19:00- Participants go for dinner at restaurants in Exeter

Thursday 23 September: Plenary summary and departure

08:00-09:00 Breakfast in Holland Hall (for those staying there)Summary discussions in the Queen’s building09:30-10:30 Summaries of breakout sessions (rapporteurs)10:30-11:00 Final discussion and future plans11:00-11:30 Morning coffee/tea