seeing the real picture – strategies for dealing with ......slide 1 seeing the real picture –...
TRANSCRIPT
Slide 1
Seeing the real picture – strategies for dealing with uncertainties in risk assessment models Simon Firth, Firth Consultants Ltd Brownfield Risk & Remediation, 13th September 2017
Slide 2
Uncertainty in risk assessment Source
Pathways
Receptors
Cs CPOE
Normally conduct quantitative risk assessment to determine: • If risk is acceptable or not • Determine remediation criteria
Assess risk by: a. Compare concentration along pathway with guideline value; or b. Compare exposure with a toxicity reference value
How does uncertainty affect results?
Slide 3
The importance of uncertainty • How confident are we that we have made the right decision?
– If we decide that no further action is required, how certain are we that the receptors are protected?
– If we decide to remediate or mitigate the risks, how certain are we that we are not undertaking unnecessary work? Could we be doing more harm than good, spending resources and creating other risks for little benefit?
• Assessing and managing uncertainty is a key part of risk assessment and fundamental to good risk based management
US EPA Risk Assessment Guidance for Superfund Sites:
Slide 4
Types of uncertainty
How wrong
could we be?
Conceptual
Model
Parameter
Slide 5
Areas of uncertainty • Source
– Where? – source dimensions – What? – chemicals of concern – How much? - concentrations
• Pathway – What is(are) the pathway(s)? – how are contaminants likely to
get from A to B – Pathway properties – that affect contaminant fate and transport
• Receptor – Exposure characteristics – Sensitivity
Slide 6
Assessing uncertainty • Qualitative
– How conservative is each parameter/assumption? – How likely are we to have under- or over-estimated risks?
• Sensitivity analysis – Can help identify key uncertain parameters – Can give a feel for extent to which risks could be over- or under-
estimated
• Probabilistic modelling – Can help quantify likelihood of unacceptable risk occurring – But can also introduce new uncertainties
Slide 7
Qualitative analysis of uncertainties • C4SL project used a semi-quantitative approach to help
assess the overall level of conservatism • Each parameter was scored according to its likely level of
conservatism
Parameter Evaluation of uncertainty
Estimation of soil-gas concentration + / +++
Estimation of indoor air concentration using J&E ● / +++
Soil to plant concentration factors - / ++
Produce consumption rates ● / +
Homegrown fraction - / +
Overall evaluation of C4SL exposure modelling estimates for benzene for residential land-use
Highly conservative
• Scores combined to assess over-all level of conservatism
Slide 8
Sensitivity analysis
Min Max Justification
8970 11800 Range in reported concentrations
100 1000 Range reported in EA, 2002
25 100 0.5x and 2x value used for assessment
3 9 Reasonable range based on site data
6 14 Reasonable range based on site data
1.8 2.1 Range for Mercia Mudstone in ConSim
0.2 0.299 Range calculated using RTM porosity calculator
0.0018 0.012 Range in estimates from site
0.1 0.51 Range in estimates from falling head tests
37.5 150 0.5x and 2x value used for assessment
0.001 0.004 Range in measured values from site
Single Dual Both options tested
• Example using RTM model:
Slide 9
Probabilistic modelling
• Example using ConSim:
• But there will still be uncertainty in the results!
5 (black) 10 (red) 25 (yellow ) 50 (cyan)75 (grey) 90 (green) 95 (blue) WQS: 0.5 [mg/l]
Time [years]100.090.080.070.060.050.040.030.020.010.0D
ilute
d C
once
ntra
tion
Uns
atur
ated
Pat
hway
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
5 (black) 10 (red) 25 (yellow ) 50 (cyan)75 (grey) 90 (green) 95 (blue) WQS: 0.5 [mg/l]
Time [years]100.090.080.070.060.050.040.030.020.010.0D
ilute
d C
once
ntra
tion
Uns
atur
ated
Pat
hway
2.5
2.0
1.5
1.0
0.5
0.0
Using single (most likely) value for Kd
5 (black) 10 (red) 25 (yellow ) 50 (cyan)75 (grey) 90 (green) 95 (blue) WQS: 0.5 [mg/l]
Time [years]100.090.080.070.060.050.040.030.020.010.0D
ilute
d C
once
ntra
tion
Uns
atur
ated
Pat
hway
1.5
1.0
0.5
0.0
Using single (most likely) value for Kd + half-life
5 (black) 10 (red) 25 (yellow ) 50 (cyan)75 (grey) 90 (green) 95 (blue) WQS: 0.5 [mg/l]
Time [years]100.090.080.070.060.050.040.030.020.010.0D
ilute
d C
once
ntra
tion
Uns
atur
ated
Pat
hway
1.0
0.5
0.0
Using single (most likely) value for Kd + half-life + infiltration
• Monte Carlo modelling can help to quantify uncertainty
Slide 10
How certain do we need to be?
Pred
icte
d co
ncen
trat
ion
Scenario 1 Scenario 3 Scenario 5 Scenario 4 Scenario 2 Scenario 6
Range in estimates Most likely
Guideline value
Slide 11
Reducing uncertainty • Gather more data, e.g.
– More samples – Field testing of hydraulic conductivity – Bioaccessibility testing
• Measure rather than model, e.g. – Soil vapour concentrations – Concentrations in river
• Use a lines of evidence approach to constrain uncertainties, e.g. – Soil, groundwater and soil vapour concentrations to constrain source
dimensions – Calibrating model to observed concentrations
Slide 12
Source concentrations - soil
Mean = 496 mg/kg Confidence limits (using Chebychev) LCL95 = -382 mg/kg UCL95 = 1374 mg/kg
Slide 13
Source concentrations - soil
Slide 14
Composite samples
• Composite samples can be useful for reducing analytical costs
• But do lose information on spatial variability
Slide 15
Groundwater samples
Slide 16
Vapour samples
Dotted lines represent long-term geomean
Predicted concentration from groundwater concentrations
Slide 17
Communicating uncertainty • Very important to communicate level of
confidence/uncertainty in risk assessment results to decision makers/stakeholders
• Modelling reports should have a section on uncertainty – What are the key uncertainties – How could these affect model results
Slide 18
Key points • Uncertainty is integral to risk assessment process
– What are the key uncertainties? – How do these affect overall conclusions of the risk assessment?
• Measures to reduce uncertainty – Gather more data – Measure rather than model – Use all available lines of evidence
• Uncertainties and their impact on risk assessment conclusions should be communicated to decision makers/stakeholders
Slide 19
Thanks for listening!