© Crown copyright Met Office

Uncertain ForecastPeter GibbsBroadcast meteorologist, BBC and Met Office

Obstacles and AssumptionsWhy are we not communicating it already?

• …the public don’t understand probabilities…

• …uncertainty is too difficult to communicate…

• …the Press don’t get it…

• …the Met Office (or BBC) is just covering itself…

• …you can’t say 50% chance - that is admitting defeat…

• …just tell me what will happen…I just need to make a decision…

• or is that “just make my decision for me” ?

© Crown copyright Met Office

Storm of 16th Dec 2011

© Crown copyright Met Office Ensemble Training 2012

5-day forecastBoth EPSs flag small risk exceptional low over UK

EC 944MOGREPS 938

© Crown copyright Met Office Ensemble Training 2012

Forecasters communicate low risk of very high impact

• BBC shows two alternative tracks:

• Track 1 is low probability but highest impact (severe gales and heavy snow).

• Track 2 has a higher probability but lower impact (some snow, but lighter winds).

© Crown copyright Met Office Montreal Workshop, March 2012

Max Temp uncertainty 1-5 Feb 2012

• Cold block over Europe

• Frontal system approaching from Atlantic

• Uncertainty about progression

BBC Graphic

presentation

© Crown copyright Met Office

Decision-making with Uncertainty

• There is always uncertainty in forecasts

• Recent study in US showed that where uncertainty is not shown, people make own assumptions (Joslyn et al, 2011)

• Often over-estimate uncertainty

• By explicitly providing uncertainty, people make better decisions

• …

© Crown copyright Met Office

Addressing assumptions:“the public don’t understand uncertainty”

“uncertainty is too difficult to communicate”

• Do ensemble forecasts cause confusion?

• Can players make better decisions using ensemble forecasts?

• How does this vary with presentation type?

• Demographic influences

Randomised design

Collected basic demographic data

Outcomes generated on-the-fly (based on probabilities)

Ability to separate 1st time players

© Crown copyright Met Office

Playing the game

How confident are you about each shift – tests ability to assess probability from presentation

Choose location or shift – tests ability to understand relative probability from presentation

© Crown copyright Met Office

Scoring

• Each shift is allocated an outcome randomly selected from the forecast distribution

• Element of luck!

• Amount of ice cream bought and sold depends on advice and outcome

• Total score at end of game:

© Crown copyright Met Office

Weather Game – “Engaging online users to gather feedback on the best way to represent forecast uncertainty”

-11,000 responses – Largest meteorological survey of its kind

-Collaboration with UK Universities – Bristol & Cambridge

-Showed that people make better decisions when presented with uncertainty

- Consistent across all age groups and educational levels

- People made best decisions with the most complex presentations

Courtesy of Liz Stephens