implementation of a performance evaluation system for

Implementation of a Performance Evaluation System

for Nondestructive Testing Methods

SVTI Swiss Association for Technical InspectionsNuclear Inspectorate

NDE Laboratory

Daniel Algernon, Sascha Feistkorn and Michael Scherrer

Outline

www.svti.ch ICSC 2016 - "Making Safety Measurable" Dr.-Ing. Daniel Algernon

Nondestructive Testing (NDT)

Meaning for Safety(Making it Measurable)

Ensuring Reliability(e.g. in the Nuclear Industry) Transferring Concepts

between Industries

Conclusion

Motivation

• Highly safety relevant components & industries

• Assessment of structural integrity: Nondestructive Testing (NDT)


Nondestructive Testing (NDT)

NDT Methods• Ultrasonic Testing• Eddy Current Testing• Liquid Penetrant Testing• Magnetic Particle Testing• Radiographic Testing• Thermography• Visual Testing• …


Meaning of NDT for Safety Concepts

• NDT provides safety-relevant information

• «making safety measurable» requires quantitative information

• Reliable information!ICSC 2016 - "Making Safety Measurable"

Safety

Structural Integrity

Fracture Mechanics

Condition Assessment

Nondestructive Testing

Measures

www.svti.ch Dr.-Ing. Daniel Algernon

NDT in the Nuclear Industry


Operation

• Safe Operation of Nuclear Power Plants over design life time

• Long Term Operation based on actual condition

Struct.Integrity

• Structural Iitegrity of safety relevant components

• Significant aspect of technical condition

Testing & Evaluation

• Nondestructive Testing and Evaluation (NDE)

• Provide information for condition assessment

Reliability• Information provided must be reliable, since it is the basis for determining safety

Performance Evaluation

• Performance Evaluation of NDE inspection systems to determine their reliability

Demo & TJ

• Performance assessed in practical demonstrations as well as theoretical investigation (technical justification)

Elements of NDT

ICSC 2016 - "Making Safety Measurable"

Equipment

Test Object

Target

NDT Method

Procedure

Personnel

NDT = Information Provider

Value of NDT = Value of Information

Information must be relevant and reliable

Increase in reliability = increase in value

Information

www.svti.ch Dr.-Ing. Daniel Algernon

How to determine reliability?

• Theoretical Assessment– Model & Simulation

– Field Experience

• Practical Demonstration– «See how NDT System performs»

– Compare NDT results to true results• How to determine truth?

– Investigate in controlled conditions• Test blocks, simulating testing problem


Requirement for Practical Demonstr.

Contradiction:

Realistic and Controled (Simulation)


!

ENIQ-Methodology

European Network for Inspection and Qualification

• Qualification of Inspection Systems– Procedure– Equipment– Personnel

• Case Specific Practical Demonstration– Component and Flaw Specific– Postulated flaw mechanism– Realistic flaw simulation in test blocks

• Technical Justification– Theoretical Assessment of

• Inspection System• Suitability of Test Blocks and Test Flaws


Physically simulating field conditions

Physically simulating relevant flaw types (qualification target)

Mockups and test blocks

Example: Qualification on test blockExample: Field Testing

Practical Demonstrations


Component Full-Scale Mockup

Test Block

Weld

Test flaws

Component, Test Block, Mockup


Test Flaws

• Mandatory: realistic test flaws

• Example: representing intergranular stress corrosion cracking(IGSCC)


Evaluating Practical Demonstrations

Test Set = Test block or section containing test flaws

Test Sets are given to candidates

Several candidates per qualification project


Test set/Candidate Relation in IndEva


• IndEva: Evaluation System of Swiss Qualification Body

Per Test set:

As-Built Information

Indications reported by candidates

Graphic Representation of Testsets


• Evaluation 1: Singular relation Test Set / Candidate

• Evaluation 2: By candidate and related test sets

• Evaluation 3: Procedure (all candidates and test sets)

Evaluation Types in IndEva


Detection: What percentage of flaws has been detected

False Calls:

In absolute numbers

In % of actual number of test flaws

Per area

Determining Length and Thruwall extent: Statistics

RMS (Root Mean Square)

Mean + Standard deviation

Positioning: Tolerance window

Analysis Criteria


Evaluation of each individual candidate regarding qualification criteria fordetection, characterization, length and thruwall sizing

IndEva-Output per Candidate


Total evaluation taking into account the best, worst and averaged results ormedian

Identification of critical test flaws

IndEva-Results by Procedure


List of all individual test flaws and the respective results obtained

IndEva-Results per Test Flaw (Table)


Qualification

Qualification = Proof (Demonstration) of PerformanceQualification = Proof (Demonstration + Understanding) of Performance

Qualification = Proof (Demonstration + Understanding) of Performance

Technical Justification (TJ)


Technical Justification: Modeling

• Use of semi-analytical simulation software CIVA

• Powerful tool for parametric studies and technical justification

• Perspective: Reduction of testing effort

• Validation of model is essential, test blocks still needed!


Confidence

Engineering Understanding(Technical Justification)Practical Trials

(Statistics)

Technical Justification


Confidence

Practical Trials(Statistics)

Engineering Understanding(Technical Justification)

Since number of test flaws and practical trials is limited, technical justification is an important element

(e.g. in ENIQ Qualification system)

Technical Justification


Goal:

Combined statistical expression

of practical trials and technical justification

of results and confidence level

Approach: Bayesian Statistics

General Idea: Prior set of parameters adjusted according toinformation gained from technical justification andpractical trials, resulting in posterior set of parameters

Based on Binomial distribution

Quantification of Technical Justification


Bayesian Approach

• Gandossi & Simola, 2007


Transfering Concepts betweenIndustries

• Example: «Nuclear2Civil»

NDT in theNuclear Industry

NDT in newIndustry (CivilEngineering)

Transfer of Quality Concepts

Industry (CivilEngineering) Specific Adaption

Identification of IndustrySpecific Practices and

Needs

Adaption of quality-assurance (qualification) concepts to new industries (e.g. NDT in civil engineering).


Challenging Material vs. Qualification

• Inhomogeneousmaterial

• Individual structuraldesign

• Limited Budget…


• Needs Adaption of Concepts!

• Current Topic for DGZfP Quality Assurance Committee

Qualification of Concrete-NDT?

Concrete-NDE Validation Center

Laboratory Test Frame for automated application of nondestructive testing in civil engineering (NDT-CE)


Florida Department of Transportation

Conclusion


StructuralIntegrity

Quantify

NDT Quantitative Reliability

Making Safety Measurable

Process/Value Chain

Qualification

Industry SpecificAdaption

Thank you


Conclusion

• In many industries: NDT provides information used to rate component safety• reliability has direct effect on the operational safety of a component.• nuclear industry: qualification processes ensure NDT reliability (e.g. ENIQ)• Quantitative measures obtained from qualification target and statistical

distribution of trials• Qualification is combination of theoretical assessment (TJ) and practical

demonstration (PD)• Need: express theoretical assessment quantitatively and combine with PD• Approach: Bayesian statistics (Gandossi & Simola)

- capable, though challenging, since it requires adequate evaluation criteria.• Tool IndEva: statistical evaluation of the practical demonstration, identifying

strengths and weaknesses- Covers TJ quantification approach, currently evaluated

• Methodology Transfer: qualification process transferred to other industries, but needs adaption

- First example established: civil engineering (concrete structures).


• Qualification = Proof (Demonstration) of Performance

• Qualification process is an important element toensure reliable inspections systems

• Qualification is essential especially in case of safety-relevant compent evaluation (e.g. in nuclear power plants)

• Swiss Association for Technical Inspections (SVTI) serves as the NDE Qualification Body (QSt) in Switzerland

Qualification of Inspection Systems


As-Built Information

IndEva-Form for Testblocks and Flaws


Taking into account coordinate offsets or scaling:

(X + Offset ax) * Faktor bx + Offset cx

(Y + Offset ay) * Faktor by + Offset cy

IndEva-Candidate Indication Lists


IndEva-Results per Candidate


WCNDT 2016Implementation of Performance Evaluation Methodology in INDEVA

IndEva-Results per Test Flaw (Plot)


IndEva-Results per Test Flaw


implementation of a performance evaluation system for

Documents