“probability forecast use” - initial findings - jan verkade edwin welles february 2012
TRANSCRIPT
“Probability forecast use”- initial findings -
Jan Verkade
Edwin Welles
February 2012
February 10, 2012
Outline
• Probability forecast study – overview• Initial findings• Summary and conclusions
But first: thanks to…
February 10, 2012
Thank-you to
• Diane Cooper, Service Hydrologist @ Chanhassen WFO• Reggina Cabrera, Chief, Hydrologic Services Division @ NWS
Eastern Region• John Blood, Senior Planner @ State of Minnesota Homeland
Security Emergency Management Services• Allen Glass, Bruce Elder, Tom Miller and Heather Winkleglass @
City of St Paul Emergency Management / Dept of Public Works• Scott Jutila and Patrick Foley @ USACE, St Paul District• Chris Franks, Meteorologist @ Chanhassen WFO
February 10, 2012
Thank-you to
• Greg Kruse, Chief of Hydrologic Monitoring Unit @ MN Dept of Natural Resources
• Marc Deutschmann, VP @ Houston Engineering• Greg Spoden, Climatologist @ State of Minnesota Climatology
Office, University of Minnesota• Walter Potts @ ESRI• Bill McAuliffe, Jim Kern, Ray Grumney and David Braunger @ Star
Tribune • Jim Bodensteiner and Jeff Berrington @ Xcel Energy
February 10, 2012
Last, but certainly not least, many thanks to…
Steve BuanService Coordination Hydrologist @ NWS NCRFC
February 10, 2012
Probability forecast study – purpose
• General move towards probabilistic forecasting, varying reasons• To capture perceived benefits, simply having the probability
forecast is not enough• Additional effort may be
required:
• visualisation
• communication
• decision-making
• verification
• “downstream” DSSs
• adaptation of businessprocesses / procedures
February 10, 2012
Probability forecast study – method
• Desk research
• Forecasting exercise with Dutch Waterboard
• Interviews with
• US NWS forecasters and forecast users
• Dutch Meuse River forecasters and forecast users
• Scottish Environment Protection Agency forecasters and forecast users
• Possible wrap up workshop with all participants
Findings
February 10, 2012
February 10, 2012
February 10, 2012
February 10, 2012
Preliminary findings: themes
• Visualisation and communication• Reflection of physical processes• Decision making• Training and education• Use of and rationale for probability forecasting
February 10, 2012
Why people use forecasts
• planning• planning• planning• awareness raising• decision making
February 10, 2012
Preliminary findings: visualisation
• Users may have issues with visualisation of probability forecasts• Main issue: probability forecasting adds a dimension to the
problem• In addition to:
• space (2 dimensions already)
• time
• a variable such as either flow or stage• A 5th dimension is added: probability
visualisation not trivial
February 10, 2012
Preliminary findings: visualisation• Suggested approach for visualisation:• As we can visualise only 2 (graphs) or 3 (maps) dimensions, we
need to “fix” the others• Choices for fixing vary
by user and use
this “dimensionality issue” requires somewhat sophisticated tools to visualise probability forecasts(e.g. GIS, programming languages)
February 10, 2012
Preliminary findings: communication
• Ties in with visualisation• Additional issue: “flow” and “stage” may not be meaningful to many
end users• Variables such as “inundated area”, “number of properties
affected”, “reference floods” may be effective to communicate risks
may require post-processing
February 10, 2012
Preliminary findings: physical processes
• Often heard: users compare forecasts with their knowledge of the physical system
• This is quite difficult when “abstract probabilities” come into play these cannot be confirmed with knowledge of hydrology
• some indication may be needed that the physics are okay
not an easy task; maybe solution is found inprovision of metadata
providing information on past performanceof probability forecasts will be helpful(for this: forecast verification is required)
February 10, 2012
Preliminary findings: decision-making
• Decisions are made based on a consequence, not on a hazard
• Must transform probabilities to make a binary decision
> Often times: intuition only
• Decision criteria can now include explicit expression of risk
• Risk = Probability x Consequence
• NWS concentrates on forecasting a possible hazard; decision support is needed to translate this into meaningful consequences
February 10, 2012
Preliminary findings: training and education
• Information overload is a real issue
• Trying to understand the forecast is not conducive to good decision making
• This should then be done prior to a “crisis situation”• Links between “hydrological variables” and real issues not always clear
for non-experts• Everyone found the NWS webinars extremely useful• Need to explain how to access the information in the forecasts by fixing
a dimension.
February 10, 2012
Preliminary findings: why probabilities
• Rationale for moving towards probability forecasting:
• “More realistic forecasts” widely accepted> Deterministic forecasts are “over confident”
• “Risk based” decision making easier said than done> Support systems not generally in place
• Benefits may not reside with both forecaster and end user> Forecasters provide more complete information> Users must do more interpretation and post-processing
Some ideas…
February 10, 2012
Decision support: from outlooks to planning (1)
Exc.probs Max stage[-] [ft]
98% 1395% 1490% 1580% 1750% 2520% 3210% 345% 352% 381% 40
February 10, 2012
Decision support: from outlooks to planning (2)
Determine potential consequences from inundation
maps
Do this for all relevant stages
February 10, 2012
Decision support: from outlooks to planning (2)
February 10, 2012
Decision support: from outlooks to planning (2)
Max stage #houses affected[ft] [-]13 014 015 017 025 032 534 2535 5738 10040 250
February 10, 2012
Max stage #houses affected[ft] [-]13 014 015 017 025 032 534 2535 5738 10040 250
Exc.probs #houses affected[-] [-]
98% 095% 090% 080% 050% 020% 510% 255% 572% 1001% 250
Decision support: from outlooks to planning (3)Exc.probs Max stage
[-] [ft]98% 1395% 1490% 1580% 1750% 2520% 3210% 345% 352% 381% 40
February 10, 2012
Decision support: from outlooks to planning (4)#houses affected [-]
0
50
100
150
200
250
300
0%10%20%30%40%50%60%70%80%90%100%
Exceedence probability [-]
Hou
ses
affec
ted
[ft]
Another idea…
February 10, 2012
36 Serious flood damage occurs at the University of Iowa campus.
34 Flood protection becomes necessary at the University of Iowa. Water affects several industrial businesses and warehouses along Commercial Drive.
32 Water floods county road bridge approaches along the river and affects streets and parking lots along Commercial Drive.
30 Flooding occurs in Coralville. Water inundates the Cedar Rapids and Iowa City Rail line near Coralville.
25.5 Water affects the north Wastewater Treatment Plant. Considerable flooding occurs at the University of Iowa.
25 Water floods Hancher Auditorium at the University of Iowa. Flooding problems occur elsewhere on the University of Iowa campus.
22 Urban flood damage occurs in Iowa City. Water enters homes along Edgewater Drive.
21 Flooding occurs in homes near Taft Speedway in Iowa City. Homes on east side of Quarry Road in Coralville require protection. Edgewater Drive becomes impassable.
20.5 Water affects the north Wastewater Treatment Plant.
19 Lowland flooding occurs in the Iowa City Park area.
Gather the impacts.
February 10, 2012
Assign probabilities to the stages.
21 ft 70%
25 ft 50%
30 ft 30%
34 ft 10%
February 10, 2012
36 Serious flood damage occurs at the University of Iowa campus.
34 Flood protection becomes necessary at the University of Iowa. Water affects several industrial businesses and warehouses along Commercial Drive.
32 Water floods county road bridge approaches along the river and affects streets and parking lots along Commercial Drive.
30 Flooding occurs in Coralville. Water inundates the Cedar Rapids and Iowa City Rail line near Coralville.
25.5 Water affects the north Wastewater Treatment Plant. Considerable flooding occurs at the University of Iowa.
25 Water floods Hancher Auditorium at the University of Iowa. Flooding problems occur elsewhere on the University of Iowa campus.
22 Urban flood damage occurs in Iowa City. Water enters homes along Edgewater Drive.
21 Flooding occurs in homes near Taft Speedway in Iowa City. Homes on east side of Quarry Road in Coralville require protection. Edgewater Drive becomes impassable.
20.5 Water affects the north Wastewater Treatment Plant.
19 Lowland flooding occurs in the Iowa City Park area.
70%
50%
30%
10%
Assign probabilities to the stages.
And some ideas about visualisation…
February 10, 2012
Recommendations: map-type visualisation
stage / flow
high/medium/lowexceedence prob
“Play” button fortime dimension
Fix any continuous variable;or define the event otherwise
February 10, 2012
February 10, 2012
Recommendations: graph-type visualisation
Fixed:•location•probability
Va
ria
ble
A
time
prob
abili
ty
timeFixed:•location•variable / event
variable = f(time) probability = f(time)
February 10, 2012
Conclusions• Multi-dimensional nature of probability forecasts makes them hard
to understand, (as opposed to the probabilities being too complicated).
• Everyone needs to fix at least one dimension to make sense of the probability forecasts
• Impacts/ consequences
• Planning Scenarios
• Analagous Events
• Background information on the
forecast origins helps. So does verification.
• Probabiilty forecasts are more informative! And useful!
February 10, 2012
Contact details
Edwin Welles
[email protected] , 301-642-2505
Jan Verkade
[email protected] , +31.88.335.8348
On the web:
http://www.deltares.nl/en
This presentation can be downloaded from:
http://publicwiki.deltares.nl/display/~verkade
Thank you for your attention