expert elicitation process for isloca modeling –...
TRANSCRIPT
1
Expert Elicitation Process for ISLOCA Modeling – Process, Results, Lessons Learned PSA 2017 September 26, 2017
Steve Short, PE William Ivans, PE (formerly PNNL)
Outline
Project Scope
Elicitation Process Overview
Expert Panel Members
Representative Results
Lessons Learned
2
Scope of Elicitation Project
3
Develop expert elicitation process Develop input to NRC L3PRA ISLOCA model
Vogel Electric Generating Plant (VEGP) Units 1 and 2 ISLOCA pathways:
Residual Heat Removal (RHR) System Safety Injection (SI) System
PRA model parameters: Normally-closed isolation valves:
Failure rates (failures/hour) for check valves (CV) and motor operated valves (MOV) for internal large leak (ILL) failure mode Conditional probability of failure of the second down-stream valve given failure of the first valve (two valves in series)
Normally-open/closed MOVs that could be used to mitigate the ISLOCA: Failure-to-close (FTC) probability (failures/demand)
Probability of break by location outside containment
Elicitation Process
4
I. Prepare DetailedProblem Description
• Problem definition • Elicitation process and objectives• Level of detail expectations• System Notebooks
II. Select Expert Teamfor Elicitation
• PRA• Common Cause Failures• Piping and Valve Failure Experts• System Engineers
III. Expert Training• Initial Video Conference, Expert Elicitation Training• Second Conference – Review Training, Systems Familiarization
IV. IndividualElicitation Meetings
• Individual Elicitation with Each Team Member• PNNL Collects and Collates Results• Provide Draft Results to Team
V. Group ElicitationMeeting
•Review Results of Individual Elicitation Meetings•Group Elicitation with All Team Members•PNNL Collects and Collates Results
VI. Issue Draft Reportfor Review
• Review by NRC and Experts• Incorporate Comments
Adapted SSHAC process for ISLOCA
5
Preparation
Prepare detailed problem description and data
Select expert team for elicitation
Training
Understanding of problems, models & data Evaluate data Initial elicitation of judgment (center/body/range with justification ) on the stated problems
Review and discuss results of individual meetings Elicit and modify individual judgment
Integrate individual judgments and document the results Review the results by NRC and experts
Participatory peer review, transparency, and documentation
5
Individual meetings
Workshop – Group meeting
Summary of the adapted SSHAC process • The adapted process preserved SSHAC principles:
Structured process to facilitate elicitation and minimize biases Breadth of State-of-Knowledge – Evaluation of available data,
balance of expertise Independence – Judgment based on knowledge and individuals’
expertise Interaction in evaluating models/data and assessing uncertainties Integration (rather than consensus) of interpretations / judgment
• The three training sessions were effective and ensured that the experts understand the stated problems and expressed uncertainties with probability distribution.
• Using individual meetings in lieu of SSHAC Workshop #1 achieved the goals with some minor caveats that were fixed at the group meeting.
ISLOCA High/Low Pressure Interface Configurations on RHR/SI Systems
7
M M
MOV 1 MOV 2
OutsideContainment
(low pressure)
InsideContainment
(high pressure)
RCSRHR
• RHR Suction Pathways • Two normally closed MOVs in
series, both inside containment • MOVs leak tested each outage
M
MOV 3 CV 2
OutsideContainment
(low pressure)
InsideContainment
(high pressure)
RCSRHR/SI
CV 1
M
MOV 4 CV 4
OutsideContainment
(low pressure)
InsideContainment
(high pressure)
RCSRHR/SI
CV 3
• RHR/SI Hot Leg Injection Pathways
• Two CVs and one normally closed MOV in series; CVs inside containment, MOV outside containment
• RHR/SI Cold Leg Injection Pathways
• Two CVs and one normally open MOV in series; CVs inside containment, MOV outside containment
Expert Panel Members
Mr. Neal Estep – Kalsi Engineering, Inc. Nuclear power plant (NPP) valve engineer, 30+ years experience
Dr. Karl Fleming – KNF Consulting Services LLC NPP PRA/common cause failure, 40+ years experience
Mr. Michael Horton – Southern Co. System engineer, Sr. Reactor Operator license, 35 years experience
Mr. Paul Knittle – MPR Associates, Inc. NPP valve engineer, 30 years experience
Dr. Frederic Simonen – Lucius Pitkin, Inc. NPP piping fatigue/failure analysis, 40+ years experience
Dr. Michael Stamatelatos – Self (Retired – NASA) PRA/common cause failure, 40+ years experience
8
Internal Large Leak Failure Rate
Information provided to expert panel: VEGP RHR and SI System description
System notebooks Valve/component design, fabrication, and test specifications, including drawings
Current small internal leak failure rate assumption from NUREG/CR-6928 based on actual valve failure data:
MOVs – 2.02E-09 failures/hour (mean) CVs – 6.15E-09 failures/hour (mean)
INPO Equipment Performance and Information Exchange (EPIX) data on historical internal leaks in RHR/SI systems
9
Internal Large Leak Failure Rate
Elicitation approach: Use common Elicitation Form for all experts Each expert provides the change in failure rate relative to the failure rate for small internal leaks from NUREG/CR-6928
Information represents the best available data on valve ILL failures Best approach for experts not experienced with expressing failures in probabilistic terms Not funded to develop mechanistic failure models
Alternatively, expert can provide the component failure rate (failures/hour) Failure rate elicited for different leak size categories [effective diameter (inches) and leak rate (gpm)], depending on size of valve Failure rate elicited for lower bound (5%), best estimate (50%), and upper bound (95%) Elicited for each isolation MOV and CV
10
Internal Large Leak Failure Rate
Results Cumulative distribution functions developed for each “unique” valve design, each leak size category, and each expert Gamma distribution functions used based on industry practice; no technical reason to change The individual distributions developed for each expert were combined to estimate a single mixture distribution reflecting the uncertainty in the elicited results
Used same method as used in NUREG-1150 (arithmetic average of the individual probability distributions) Method results in higher mean and 95th percentile and lower 5th percentile results than other methods (because extremes dominate)
Two distributions reported: Actual mixture distribution represented by a lookup table of values Best fit parametric gamma distribution represented by the mean, alpha, and beta parameters
Total large internal leak failure rate Gamma distribution fit to the sum of the individual mixture distributions 11
12
Internal Large Leak Failure Rate – RHR Hot Leg Suction MOVs Leak Size: 0.8” to 10.75” (effective diameter)
Leak Rate: 370 to 67,200 gpm
Leak Size: 5” to 10.75” (effective diameter)
Leak Rate: 14,500 to 67,200 gpm
Leak Size (inches effective diameter) Leak Rate (gpm)
Mean ILL Failure Rate (failures/hour)
0.8 to 2 370 to 2,325 1.94E-09
2 to 5 2,325 to 14,500 3.04E-10
5 to 10.75 14,500 to 67,200 2.04E-10
Total (0.8 to 10.75) 370 to 67,200 2.35E-09
NUREG/CR-6928 (2010 Update) 2.02E-09
Expert Distributions
Mixture Distribution
Best Fit Gamma Distribution to Mixture Distribution
Best Fit Gamma Distributions for each Leak Size
Total Failure Rate Distribution
Elicitation approach: Use common Elicitation Form for all experts Changed from initial plan to elicit common cause failure probabilities (for input to common cause failure model) Each expert provides the conditional probability of failure of the 2nd valve in series with the 1st failed valve (as a consequence of the change in conditions due to the independent failure of the first valve, i.e., sudden pressure surge) Conditional failure probability elicited for different leak size categories [effective diameter (inches) and leak rate (gpm)], depending on size of valve Conditional failure probability elicited for lower bound (5%), best estimate (50%), and upper bound (95%) Elicited for each pair of isolation MOVs and CVs
13
Conditional Probability of Failure of 2nd Valve (MOV downstream and in series with 1st MOV)
Results Cumulative distribution functions developed for each “unique” valve design, each leak size category, and each expert Gamma distribution functions used (essentially same result as beta distribution) The individual distributions developed for each expert were combined to estimate a single mixture distribution reflecting the uncertainty in the elicited results
Again, used same method as used in NUREG-1150 (arithmetic average of the individual probability distributions) Lack of spread between 5%, 50%, and 95% (i.e., all same values) required some additional data handling in order to include this input in the development of the distributions
Two distributions reported: Actual mixture distribution represented by a lookup table of values Best fit parametric gamma distribution represented by the mean, alpha, and beta parameters
14
Conditional Probability of Failure of 2nd Valve (MOV downstream and in series with 1st MOV)
15
Conditional Probability of Failure of 2nd Valve (MOV downstream and in series with 1st MOV)
Leak Size: 5” to 10.75” (effective diameter)
Leak Rate: 14,500 to 67,200 (gpm)
Leak Size: 0.8” to 10.75” (effective diameter)
Leak Rate: 370 to 67,200 gpm
Expert Distributions
Mixture Distribution
Best Fit Gamma Distribution to Mixture Distribution
Best Fit Gamma Distributions for each Leak Size
Total Failure Rate Distribution
16
Conditional Probability of Failure of 2nd Valve (MOV downstream and in series with 1st MOV)
Leak Size (inches effective
diameter) Leak Rate (gpm)
Conditional Failure Probability
×2 for 2 pathways
0.8 to 2 370 to 2,325 1.03E-03 2.06E-03
2 to 5 2,325 to 14,500 5.19E-04 1.04E-03
5 to 10.75 14,500 to 67,200 1.37E-04 2.74E-04
Total (0.8 to 10.75) 370 to 67,200 1.52E-03 3.04E-03
Alpha Factor Model, group size of 4, staggered-testing, alpha factors from INL Database (α1 = 9.47E-01, α2 = 3.46E-02,
α3 = 1.41E-02, α4 = 4.70E-03)
1.13E-01 (conditional probability after summation of
frequency for all combinations/cutsets)
Failure-to-Close Probability (RHR MOV)
Information provided to expert panel: VEGP RHR and SI System description
System notebooks Valve/component design, fabrication, and test specifications, including drawings
Current failure-to-close (FTC) probability assumption from NUREG/CR-6928 based on actual valve failure data:
9.63E-04 failures/demand (mean)
17
Elicitation approach: Use common Elicitation Form for all experts Each expert provides the change in failure probability relative to the FTC probability from NUREG/CR-6928
Information represents the best available data on valve FTC probabilities Best approach for experts not experienced with expressing failures in probabilistic terms Not funded to develop mechanistic failure models
Failure probability elicited for different categories of differential pressure across the valve Failure probability elicited for lower bound (5%), best estimate (50%), and upper bound (95%) Elicited for each normally-open MOV that could potentially be used to isolate/mitigate an ISLOCA
18
Failure-to-Close Probability (RHR MOV)
Results Cumulative distribution functions developed for each “unique” valve design, each differential pressure category, and each expert Beta distribution functions used based on industry practice; no technical reason to change The individual distributions developed for each expert were combined to estimate a single mixture distribution reflecting the uncertainty in the elicited results
Used same method as used in NUREG-1150 (arithmetic average of the individual probability distributions)
Two distributions reported: Actual mixture distribution represented by a lookup table of values Best fit parametric beta distribution represented by the mean, alpha, and beta parameters
19
Failure-to-Close Probability (RHR MOV)
20
Failure-to-Close Probability (RHR MOV)
Differential Pressure (psig)
Mean FTC Probability
1000 2.19E-03
1500 5.67E-02
2000 5.93E-01
2695 5.93E-01
NUREG/CR-6928 (2010 Update)
9.63E-04
Expert Distributions
Mixture Distribution
Best Fit Beta Distribution to Mixture Distribution
Probability of External Break Location
Information provided to expert panel: VEGP RHR and SI System description
System notebooks Valve/component/piping design, fabrication, and test specifications, including drawings
21
Elicitation approach: Use common Elicitation Form for all experts Each expert provides the likelihood that the external break will occur at a specific location relative to a reference location (inside containment)
Assumes an external break will occur at some location
Eliciting probability of external break not within the project scope Best estimate probability elicited Elicited for each ISLOCA pathway
22
Probability of External Break Location
23
Probability of External Break Location
RHR ISLOCA Probability Break Location
0.0004 1
0.048 2
0.008 3
0.008 4
0.726 5
0.014 6
0.168 7
0.012 8
0.003 9
0.014 10
ISLOCA in the RHR system is initiated due to failure of the redundant MOVs on the suction lines from Hot Leg Loop 1
Mis-intrepretation of elicitation forms Provide definition/description of each Datasheet field Conduct internal test run(s) of Datasheets Reviewing Datasheets with the expert panel members as a group prior to elicitation sessions
Social pressure from observers The SSHAC expert elicitation process provides two mechanisms for addressing this:
allot some time at a specified time at the end of each day or each workshop to open the floor to questions and comments from observers an NRC subject matter expert could be included on the expert panel as a resource expert
Making preliminary elicitation results available Present results so that they cannot be correlated to specific panel members
Context and meaning for very low probabilities Conducting training exercises that are more representative of the magnitude of the probabilities being elicited 24
Selected Lessons Learned