the science of risk: overview of risk analysis david moser, charles yoe, office of the chief...
TRANSCRIPT
The Science of Risk:Overview of Risk Analysis
David Moser, Charles Yoe,
Office of the Chief [email protected]
Institute for Water Resources2009
2
Objectives
• By the end of this training module you will be able to:– Identify and describe tasks of risk analysis– List some elements of a risk manager’s job– Identify several risk assessment tools– Describe what drives risk communication– Identify the 3 Ms of risk communication
3
Language and Models of Risk Analysis
4
Characteristics of Risk
• Risk is everywhere• Some risks are more serious than others• Zero risk is not an option• Risk is unavoidable• We need to. . .
– describe them (risk assessment)– talk about them (risk communication)– do something about them (risk
management)
Risk
• Risk is a measure of the probability and consequence of uncertain future events
• Risk includes – Potential for gain (opportunities) – Exposure to losses (hazards)
5
• We have been doing risk assessment for flood for a long time– EAD is a risk assessment metric
• Major rehab program requiring reliability analysis developed in 1992
• These focused on economic risks and expected values and deemphasize uncertainty
Risk Is Not New To Corps
Why Use Risk Analysis?
• Traditional standards based approaches are no longer enough—problems persist
• Identify uncertainties to be monitored and managed
• Science-based approaches have helped• Risk analysis is a new way of approaching
problems that integrates science and social values
• The global community is embracing risk analysis
7
Selected Corps Concerns
• Residual flood risk• Risk of infrastructure
malfunction• Risk to life, health and safety• Marine casualties• Navigation channel
sedimentation• Ice cover• Small boat harbor damages• Commercial fishery losses• Coastal Erosion
• Ecosystem restoration• Energy and power reliability• Dam safety• Loss of pool• Operation and maintenance
– Facility maintenance• Major rehabilitation• Risks to environment• Cost estimation• Project management• Budgeting Priorities• MilCon support
8
Science-Based Water Resource Analysis
Effective Modern
Water Resource Analysis
Water Resource Problems
Traditional Water Resource Analysis
Public Safety, EQ, NED and Other Benefits
ScienceInfrastructure
Risk ManagingOrientation
Risk Analysis
9
Risk Analysis
• Risk analysis is a decision-making framework that comprises three tasks: risk assessment, risk management, and risk communication.
Risk Management
• Policy and preference based
Risk Assessment
• Analytically based
Risk Communication
•Interactive exchange of information, opinions, and preferences concerning risks
Risk Analysis Is a Paradigm
• It is a way of thinking about things and organizing to solve problems– It is not an activity
• It is science based but not science
• It is the interface between science and values
• It is used for making decisions under uncertainty
12
Examples of Risk Analysis:Water Resources
• Expected annual damages
• Establishment of design levels
• Risk-informed cost estimation
• Dam safety program
• Levee safety program
• Establishment of product safety standards, performance standards, and specifications
• Scientific sampling protocols13
Why Do Risk Analysis?
• It improves the quality of our thinking before a decision is made—uncertainty is widespread
• It helps ensure a safe water resource infrastructure• It protects human, animal, and plant life and health• It provides a more reliable flow of outputs• It is better than the alternative• Because we have to-ERs, ECs, OMB, GAO
14
Context of Risk Analysis
• Risk analysis takes place within an organizational culture– COE structures and decision processes already exist– Goals, objectives and constraints are the
organization’s reality• There are decisions to be made• Risk analysis is a process and a paradigm, a way of
approaching problems and decisions that will look different in every organization that uses it
15
When To Do Risk Analysis
Extensive Risk AnalysisW/ Adaptive Management
Routine Risk Analysis
Modest Level of Risk Analysis
No Risk Analysis Required
Consequence of Being Wrong
UncertaintyMuch
Grave
Little
Minor
Risk Management-Informally
• What is the problem?• What questions do we
want risk assessment to answer?
• What can be done to reduce the impact of the risk described?
• What can be done to reduce the likelihood of the risk described?
• What are the trade-offs of available options?
• What is the best way to address the described risk?
• Is it working?
17
Working Definition
• Risk management is the process of problem finding and initiating action to identify, evaluate, select, implement, monitor and modify actions taken to alter levels of risk, as compared to taking no action.
18
Risk Management
19
Establish Decision ContextEstablish Decision Context
Identify RisksIdentify Risks
Analyze RisksAnalyze Risks
Evaluate RisksEvaluate Risks
Risk Management DecisionRisk Management Decision Mon
itor,
Eval
uate
, Mod
ifyM
onito
r, Ev
alua
te, M
odify
Com
mun
icat
e an
d Co
nsul
tCo
mm
unic
ate
and
Cons
ult
Risk
Ass
essm
ent
Risk Assessment—Informally
• What can go wrong?
• How can it happen?
• How likely is it?
• What are the consequences?
20
Working Definition
• Risk assessment is a systematic, evidence based approach for quantifying and describing the nature, likelihood, and magnitude of risk associated with the current condition and the same values resulting from a changed condition due to some action.
21
Risk Assessment StepsLook for the Hazard or Opportunity
Identify the hazards that can cause harm or the opportunities for gain that are uncertain.
Likelihood AssessmentAssess the likelihood of the various adverse and beneficial consequences. Characterize these likelihoods and their uncertainty qualitatively or quantitatively.
Consequence AssessmentDecide who or what may be harmed or benefited and in what ways. Gather and analyze the relevant data. Characterize the consequences and their uncertainty qualitatively or quantitatively.
Risk CharacterizationEstimate the probability of occurrence, the severity of adverse consequences, and the magnitude of potential gains, including attendant uncertainties, of the hazards and opportunities identified based on the evidence in the preceding steps. Characterize the risk qualitatively or quantitatively with appropriate attention to baseline and residual risks, risk reductions, transformations and transfers.
22
Good Risk Assessment
• Is science-based• Gets the right science and gets the science
right • Science provides basis for answer to risk
question(s)• Separates what we know from what we do not
know– Keeps track of uncertainties
• Assumptions and uncertainties are made clear
23
Risk Communication-Informally
• Why are we communicating?
• Who is our audience?
• What do our audiences want to know?
• What do we want to get across?
• How will we communicate?
• How will we listen?
• How will we respond?
24
Working Definition
• Risk communication is the open, two-way exchange of information and opinion about hazards and risks leading to a better understanding of the risks and better risk management decisions. Risk communication ensures that the decision makers, other stakeholders and affected parties understand and appreciate the process of risk assessment and in so doing can be fully engaged in and responsible for risk management.
25
Who Is Using Risk Analysis?
• DOD• EPA• FDA• USCG• USDA (APHIS, FSIS)• DOT• DOE
• OMB• GAO• WTO• Engineering
profession• Industry• International
organizations
26
Risk Management Overview
27
Risk Management
• Policy and preference based
Risk Assessment
• Analytically based
Risk Communication
•Interactive exchange of information, opinions, and preferences concerning risks
The Job!
• Risk managers are responsible for risk analysis; they identify or validate problems
• Risk managers need scientific information to make decisions– They ask questions that, when answered, will give them
the information they need to make decisions
• Risk assessors answer the questions and characterize the uncertainty in their answers
• Risk managers mitigate risks that are not acceptable• Risk managers make sure that risk communication takes
place
29
Test
• Which is the greater risk to human life in the U.S.?– Hazardous waste sites– Smoking
• Which is the U.S. spending about $30 billion on annually? – Cleaning hazardous waste sites– Programs to reduce smoking
• Which is the U.S. spending about $500 million on annually? – Cleaning hazardous waste sites– Programs to reduce smoking
30
Test
• Consider this scenario. A chemical solvent from a nearby plant is found in the air of one room in an elementary school. It is present at levels beneath 10 parts per billion. An advocate for the children says a fan in the window would make the school safe. Which risk management actions would you recommend?– Put a fan in the window– Require children who used to walk to school to
take the bus across snow-covered streets in rush hour to the high school
31
Test
• True or false. We could make decisions that are more rational and informed.
• True or false. We sometimes make policy based more on fear than fact.
• True or false. If we overspend on risks like pesticides or asbestos, which are real but relatively low-magnitude risks, we have less to spend on the prevention of foodborne illness, smoking, and crime.
• True or false. We often fear before we think.• True or false. Putting resources where the risks are
makes sense.
32
Can We Do Better?
• If we can improve our thinking before we make a decision under uncertainty, perhaps we can do better
• Risk analysis is designed for making decisions under uncertainty
• It all starts with risk management
33
Who Are the Risk Managers?
• In the Corps?
• Elsewhere?
34
COE Risk Management
Establish Decision ContextEstablish Decision Context
Identify RisksIdentify Risks
Analyze RisksAnalyze Risks
Evaluate RisksEvaluate Risks
Risk Management DecisionRisk Management Decision Mon
itor,
Eval
uate
, Mod
ifyM
onito
r, Ev
alua
te, M
odify
Com
mun
icat
e an
d Co
nsul
tCo
mm
unic
ate
and
Cons
ult
Risk
Ass
essm
ent
Risk Management Strategies
• Acceptance/Retention
• Avoidance
• Reduction
• Transfer
• Transformation
• Containment
• Constant monitoring
• Continuous research
• Development of substitutes
• Increasing resilience–surprises do less harm
• Precautionary principle
• Adaptive management
• Build confidence and trustworthiness
• Reduce uncertainties
• Clarify facts
• Involve affected people
• Deliberation
• Accountability
36
Principles for Choosing Risk Mitigation
• P&G– NED, PS, EQ
• Zero risk
• Weight-of-evidence
• Precautionary principle
• Sound science
• As low as reasonably achievable (ALARA) principle
• Appropriate level of protection (ALOP) principle
• Reasonable relationship
• Safety standards
37
Final Management Decision: The Best Option?
• Some possible criteria
– Prevent rather than control risk
– Offer stakeholders choice of options
– Based on best available scientific, economic, and technical information
– Economic feasibility
– Enforceable
– Consider tolerable level of risk (TLR)
38
Tolerable RiskIn
crea
sing
Ind
ivid
ual r
isks
and
soc
ieta
l con
cern
s
UnacceptableRegion
TolerableRegion
Broadly acceptableRegion
Risk cannot be justifiedexcept in extraordinary
circumstances
People and society are prepared to accept risk in
order to secure benefits
Risk regarded as insignificant, further effort
to reduce risk not required
Monitor/Evaluate/Modify
• Actively measure and evaluate success of decision, its implementation, and outcome to determine if changes are needed– See if desired risk reductions have been
achieved– Determine if new information is available
• Resources are required for these tasks
40
41
Historical Response to Risk• Risk presumed to be “managed” by engineering
design– Freeboard on levees and dams– Cost contingencies– Underkeel clearance design criteria– Channel width design criteria– Probable maximum flood spillways
• Or policy– Choose NED plan– Obligations and expenditures
Risk Assessment Tool Box
42
Risk Management
• Policy and preference based
Risk Assessment
• Analytically based
Risk Communication
•Interactive exchange of information, opinions, and preferences concerning risks
The Risk Assessment Model
• A scientifically based process consisting of the following steps:
1. Look for the hazard
2. Consequence assessment
3. Likelihood assessment
4. Risk characterization
• The definition includes quantitative risk assessment, which emphasizes reliance on numerical expressions of risk, and also qualitative expressions of risk, as well as an indication of the attendant uncertainties.
44
Look for the Hazard or Opportunity Identify the hazards that can cause harm or the opportunities for gain that are uncertain.
Likelihood AssessmentAssess the likelihood of the various adverse and beneficial consequences. Characterize these likelihoods and their uncertainty qualitatively or quantitatively.
Consequence AssessmentDecide who or what may be harmed or benefited and in what ways. Gather and analyze the relevant data. Characterize the consequences and their uncertainty qualitatively or quantitatively.
Risk CharacterizationEstimate the probability of occurrence, the severity of adverse consequences, and the magnitude of potential gains, including attendant uncertainties, of the hazards and opportunities identified based on the evidence in the preceding steps. Characterize the risk qualitatively or quantitatively with appropriate attention to baseline and residual risks, risk reductions, transformations and transfers.
Risk Characterizations
• Informal
• No set structure, minimal science (low end)
• Screening
• Separating elements into categories of interest, (e.g. significant vs. insignificant)
• Ranking• Put hazards and
opportunities in ordinal sequence
• Qualitative• Usually categorical or an
ordinal scoring system• Quantitative
• Deterministic • Probabilistic (high end)
46
Alternatives to Risk Assessment
• Ignorance
• Trial and error
• Professional judgment
• Expert opinion
• Precautionary principle
• Policy
47
Many Techniques
• These risk assessment models use many tools and techniques– Qualitative– Quantitative
• Science– Essential for all risk assessments
• Specific techniques, for example– Probability sampling– Monte Carlo process
48
Qualitative Methods Toolbox
• Increase or Decrease Risk• Risk Narratives• Evidence Mapping• Screening• Ratings• Rankings
• Enhanced Criteria Ranking
• Operational Risk Management (Risk Matrix)
• Develop a Generic Process
• Qualitative Assessment Models
• Multi-Criteria Decision Analysis
Evidence Maps• Evidence base or data• Pro and con
arguments, the warrants– Includes respective
supporting or attenuating arguments
• Conclusions of claim about existence of a hazard with remaining uncertainties
Enhanced Criteria Based Ranking
• Criteria• Ratings• All Possible Combinations of Ratings• Ranking• Evaluate Reasonableness of Ranking• Add Criteria• New Combinations of Ratings• New Ranking
ORM Risk Assessment Values
• Each risk you assess is placed in a cell and managed accordingly
Risk Levels
Risk Matrix
Quantitative Risk Assessment
• Safety Assessment
• Scenario Analysis– Deterministic Scenario Analysis– Probabilistic Scenario Analysis
• Sensitivity Analysis
• Uncertainty Analysis
• Modeling
• Vulnerability Assessment
Safety Assessment
• Identify Acceptable Risk (AR)
• Establish design standards to meet that risk
– Uncertainty factors
• Estimate Existing Risk (ER)
• ER/AR < 1 safety
56
ER/AR1.0
Safe Less Safe
Monte Carlo Simulation
0.00
0.08
0 10 20 30 40 0.0
0.4
5.0 8.8 12.5 16.3 20.0
X =
0.00
0.02
0 100 200 300 400
20 10
76.3% 0.000171615
0 0
0.8% Grounding?
1
48.1% 2.56255E-05
0 0
23.7% Allision?
1
1.0% 2.76973E-05
1 1
51.9% Collision
1
4.0% Casualty Occurs?
82
99.2% 0.029198219
81 81
72.7% Draft > Controlling Depth?
1818
96.0% 0.697278585
2502 1736 1736
Contain Oil?
74.9% 4.92731E-06
0 0
0.0% Grounding?
0
44.1% 7.29167E-07
0 0
25.1% Allision?
0
1.0% 9.23368E-07
0 0
55.9% Collision
0
4.9% Casualty Occurs?
39
100.0% 0.013376669
39 39
27.3% Draft > Controlling Depth?
684
95.1% 0.259915008
645 645
Vessels per Year
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
Yes
No
Yes
No
Yes
No
Yes
No
Yes
Probabilistic Scenario Analysis
Sensitivity Analysis
• Systematic investigation of uncertainties of key importance and/or concern to risk managers
• Creating scenarios to explore the range of possible outcomes
60
Vulnerability Assessment
CARVER is a DoD tool to assist in identifying vulnerable offensive enemy targets.
• Criticality—measure of public health and economic impacts of an attack
• Accessibility—ability to physically access and egress from target
• Recuperability—ability of system to recover from an attack • Vulnerability—ease of accomplishing attack • Effect—amount of direct loss from an attack as measured by
loss in production • Recognizability—ease of identifying target
62
Select a Process
• Identify a critical process or infrastructure and assess vulnerability across nation, or
• Assess vulnerability of components of a single process
Risk Communication Overview
64
Risk Management
• Policy and preference based
Risk Assessment
• Analytically based
Risk Communication
•Interactive exchange of information, opinions, and preferences concerning risks
66
Risk Communication Goals Tailor communication so it takes into account
the emotional response to an event. Empowers stakeholders and public to make
informed decisions. Prevent negative behavior and/or
encourage constructive responses to crisis or danger.
National Weather ServiceHurricane Ike Warning for GalvestonSeptember, 2008
"PERSONS NOT HEEDING
EVACUATION ORDERS IN
SINGLE FAMILY, ONE OR TWO
STORY HOMES WILL FACE
CERTAIN DEATH. ."
68
Communication Models
Basic Communication Model• Uni-directional or we tell “them” approach• Who says - what - when - to whom - through
what channel - with what effect
Risk Communication Model• Multi-directional• Actively involves the audience as an
information source
69
Risk Communication Outcomes
• Decrease illness, injury & deaths• Reduce property and economic losses• Build support for response plan• Assist in executing response plan• Prevent misallocation & wasting of resources• Keep decision-makers well informed• Counter or correct rumors• Foster informed decision-making concerning
risk
What Risk Communication is Not
• Spin• Public relations• Damage control• Crisis management• How to write a press release• How to give a media interview• Always intended to make people “feel
better” or reduce their fear
What Risk Communication IS
• Considers human perceptions of risk• Multi-directional communication among
communicators, publics and stakeholders• Activities before, during and after an event • An integral part of an emergency response
plan• Empowers people to make their own
informed decisions
It’s Risk Not Protection
Time in floodplain
Probability of 1 or more floods
10 years 65.132%25 years 92.821%30 years 95.761%75 years 99.963%100 years 99.997%
10-year floodplain occupant
73
Interpreting Risk• Communicating about risk is
difficult because of the way people interpret risk
• Involves competing perspectives: objective vs subjective
74
What Shapes Perceptions of Risk?
• Hazard – something that can go wrong• Probability – likelihood of it happening• Consequences – implications of hazard • Value – subjective evaluation of the
relative importance of what might be lost
FEELING
THINKING
75
Scientist - Consumer Disconnect
SCIENTISTEXPERT
knowsthinks
CONSUMER PUBLIC
feelsbelieves
Fact-based:
hazard, probabilityValue-based:
consequences, value
What Drives Risk Communication?
• Hazard—“the hazard,” the science– Experts’ concerns– Less concern to public
• Outrage—the broad context of the hazard– Public’s concern– Less concern to experts– Often cultivated by media on purpose and by
industry and government by ignorance
76
Outrage Factors Affecting Acceptability
• Catastrophic potential• Familiarity• Understanding• Controllability• Voluntary exposure• Effects on children• Manifestation of
effects• Victim identity
• Dread• Trust in institutions• Media attention• Accident history• Equity• Benefits• Reversibility• Origin
The Point
• Objective risk and perception of risk often diverge
• We are often managing fear more than we are risks
• Fear management can be found in the risk manager’s job description under “other duties as assigned”
78
79
A variety of risk comm approaches
Hazard (danger)
High
Low High
Outrage Management
Crisis /
Emergency Communication
Public
Relations
Precaution
Advocacy
Outrage(fear, anger)
80
Goal: Reduce outrage so people don’t take unnecessary precautions
Hazard (danger)
High
Low High
Crisis /Emergency
Communication
Public Relations
PrecautionAdvocacy
Outrage
Management
Outrage(fear,anger)
81
Goal: Increase concern for a real hazard to motivate preventive action
Hazard (danger)
High
Low High
Outrage Management
Crisis /Emergency
Communication
Public Relations
PrecautionAdvocacy
Outrage(fear,anger)
82
Hazard (danger)
High
Outrage(fear,anger)
Low High
Outrage Management
Public Relations
PrecautionAdvocacy
Crisis /
Emergency
Communication
Goal: Acknowledge hazard, validate concern, give people ways to act
3Ms of Risk Communication
• Message—what to say
• Messenger—who says it
• Media—how it should be presented
83
Message Development
• What are the three most important things for audience to know?
• What three things would audience most like to know?
• What three points is the audience most likely to get wrong unless we emphasize and explain them?
84
Subtle Changes
• Subtle changes in the way things are described can make a big difference.
• Imagine the U.S. is preparing for the outbreak of an unusual Mexican disease that is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume treatment programs with the following outcomes.
85
Choice One
• Program Options– Program A: will save 200 people– Program B: 1/3 probability of saving 600
people and 2/3 probability of saving no one
• Which do you prefer?
86
Choice Two
• Program Options– Program C: 400 people will die– Program D: 1/3 probability no one will die
and 2/3 probability 600 people will die
• Which do you prefer?
87
Principles of Risk Communication
• Know the audience
• Involve the scientific experts
• Establish expertise in communication
• Use a credible source to deliver information
• Share responsibility
• Differentiate between science and value judgment
• Assure transparency
• Put the risk in perspective
• We are all risk communicators
90
Take Away Points
• Risk analysis is a paradigm shift
• Risk analysis comprises three tasks—risk assessment, risk management, and risk communication
• Risk management is decision making that depends on– Science– Values important to stakeholders
91
Take Away Points (continued)
• Science is provided by risk assessment • Risk assessment has four steps:
– Identify the hazard – Consequence assessment– Likelihood assessment– Risk characterization
92
Take Away Points (continued)
• Risk communication is driven by hazard and outrage
• We need to communicate risk in ways people can understand and respond to effectively
93
To learn more about Actions for Change visit us on the web at
https://maps.crrel.usace.army.mil/AFC/
Questions?