creating pod practically - 9095 · inspection reliability short course, sept 2002 berlin. slide...

Inspection Reliability

Short Course, Sept 2002 Berlin. Slide Number 1

Creating PODPractically

Developing fundamental equationsLearning through case studies

Lloyd Schaefer - Honeywell [email protected]. Ripi Singh - Pratt & Whitney [email protected]

August, 2002

Introduction Round

Mission TodayGeneral Awareness

What ?

Why ?

How ?

Next ?

Knowledge

Information

Data

Action



What is NDT System Reliability?

What is the costif he is wrong?

What is the risk if he is wrong?

There is no flawThere is a flaw

Costly False call Risky miss



NDT Reliability

Degree that an NDT system is capable ofachieving its purpose regarding detection,characterization and false calls

– American European Workshop on NDT reliability 99

Quantitative measure of the efficiency ofthe NDT procedure in finding flaws ofspecific type and size

– Metals Handbook



Quantification

Ideally❖ Cracks larger than a

certain size can bedetected

Defect Size

Det

ect

No

Det

ect

Probability of DetectionReality

❖ There is a probabilityof detection for everycrack



NDT Reliability Measurement

POD - Probability of DetectionIs it an adequate representation ?

POFA - Probability of False AlarmEqually important from economic consideration

ROC - POD vs. POFAA measure of reliability

Coefficient of ContingencyA measure of individual performance



POD Curve (Safety)

Defect Size

POD

95% Confidence bound0.9

90/95 Crack size

POD = F(Finds)



ROC Curve (Economics)

PO False Call

PO T

rue

Call

> 80% Finds< 20% False calls

Pure

chan

ceROC = F(Finds, False Calls)



Coefficient of Contingency

Flawed Unflawed

Marked Finds (TP) False Calls (FP)

Not Marked Misses (FN) True no-calls (TN)

Coeff of Contingency = F (TP, TN, FP, FN)



Challenge

Human Factors Variables❖ Identification❖ Control❖ Quantification

Operator-Equipment-Environmentinteraction



What Do We Need to Understand?

NDT is not viewed as a friend of production

The program is in operational interest

Identify and eliminate deficiencies in NDTsystem

Ultimately operate ‘safer, cheaper, longer’



Popular Characteristics of POD

POD is Expensive

Certified inspectors do not need POD

Experienced and high salaried inspectorshave better POD

90/95 Crack size information is adequate

Imp to find small flaws

Mission Today

General Awareness

What ?

Why ?

How ?

Next ?



Damage Tolerance Concept

Service

Dam

age

size

Opportunity fordamage detection

Assumed detectableDamage size

InspectionInterval

Opportunity fordamage detection

Poorer than assumed

InspectionInterval

Safe?

Better than assumed

Damage growthEstimated tolerable damage size

Economic ?



Damage Tolerance Concept

Service

Dam

age

size

Tolerable damage

InspectionInterval

Detectable damage Assessed reliabilityDamage growth

Opportunity for damage detection

Improved Reliability

Mission Today

General Awareness

What ?

Why ?

How ?

Next ?



What Do We Need to Understand?

NDT is not viewed as a friend of production

The program is in operational interest

Identify and eliminate deficiencies in NDTsystem

Ultimately operate ‘safer, cheaper, longer’



NDT system performance

Knowledge on

reasons for the gap ?

Recommendactions

to bridge the gap

NDT system capability

Objective: Identify and Eliminate Deficiencies



Designed Experiment

Data base

ReliabilityInformation

Knowledge - a) System Capability, b) Improvement Avenues

ImprovedReliability

POD Analysis

RecommendedActions

Approach to Inspection Reliability

Classic POD Program

Advanced Reliability Program



What Constitutes an NDT System?

HumanApplication

Equipment

Environment



What Factors Influence the Most?

Human Factors

Application condition, access, …

Equipment sensitivity, resolution,complexity, …

Process, Materials, …

Interactions



Human Factors – still a challenge

Factors that impact inspector’sdiscrimination and decision-making ability❖ Organizational❖ Physical❖ Mental

Training and skill level is a major factor

Inspections with predictable outcome❖ Routine and monotonous

PODWe know what it is.

We know why we need itHow do we get it ?

Break One



What Constitutes an NDT System?

HumanApplication

Equipment

Environment



Typical NDT Assessment Program

Creation of specimens with defects

Visit to an NDT facility

Identification of a sample of inspectors

Conduct of NDT on set of specimens

Acquisition of inspection data

Data analysis and POD plots



NDT Assessment Program Elements

Facility SamplingInspector SamplingSpecimensSchedulingInspectionsData AcquisitionData AnalysisHuman factors



Specimens

Ideally, real parts with real cracks

Typically, synthetic parts or a combination of realand synthetic parts

Configuration as close to critical inspections aspossible

Presentation as close to real situation as possible

Special care in handling and maintenance



Specimens

Multiple identical specimens

Mounted on framework (racks) with quickinterchangeability feature

Multiple inspection sites per specimen

Uniquely numbered for tracking

Specimen inspection guideline similar to writtenprocedures

Routine surface cleaning process without damage



Specimen Defects

40-60 defects per setMost flaws in the zone of increasing POD❖ Preferred 10-90%❖ Typically 1-99% (Hard to judge)❖ Preferred size distribution linear on ‘log a’ scale

Flawed : Unflawed site :: 1 : 2.5-3Well characterized initially and regularlyFor details refer to MIL-HDBK-1823



Specimen Fabrication

Raw Specimen

Flaw growth

Final shape and size

Characterize and mark



Cracked Metal Specimens

Raw Specimen

SpecimenEDMThrough

Crack

Raw SpecimenSpecimen Crack

EDMRaw SpecimenSpecimen

EDM

Surface crack

Raw SpecimenSpecimen

EDMRaw SpecimenSpecimen

EDM

Corner crack

Raw Specimen

Specimen

Grip Area

Margin

Raw Specimen

Specimen

Grip Area

Margin



Corroded Metal Specimens

ASTM Standards

SpecimenCorrosionMachined

SpecimenRaw Specimen

CorrosionMachined

Specimen

Grip Area

Margin

Raw Specimen

Specimen

Grip Area

Margin

Environment

Painted

ExposedSpecimen

Grip Area

Margin

Raw Specimen

Specimen

Grip Area

Margin

Environment

Painted

Exposed



Inspections

Briefings to Management and InspectorsNDT in actual work environment, routineequipment, materials and processAvoid excessive detail in inspection guidelineIsolate results from one inspection to another❖ Swapping of specimens on framework, and❖ Thorough cleaning after each inspection

Onsite monitors to answer questions, without biasObservation of facility characteristics



Confidentiality

Inspector must be assured that❖ His identity will remain confidential❖ His performance will lead to a data point only❖ Interest is in overall average performance❖ There is no penalty for any individual❖ Aim at system level improvement

Human factor associated with being observed stillplays a role

You cannot observe without altering the observation



Response Matrix

Marked❖ Presence or absence❖ Size quantified

True no-callMissNot marked

False CallFindMarked

No FlawFlaw



Signal/noise Discrimination

Decision

Misses False Calls

SignalNoise

Signal amplitude

Prob

abili

ty

dens

ity

func

tion



Signal/noise Discrimination

Poor process/setupPoor DiscriminationPoor reliability

SignalNoise

Good procedure, equipment, …Inspector dependent reliability

Good

Poor Poor

SignalNoise



Data Analysis

Demonstration of capability at one cracklength

Determination of POD function throughsingle inspection of cracks covering arange of lengths

Estimation of POD function andconfidence bounds through multipleinspections of cracks covering a range oflengths

BinomialDistributionTheory(Grouping)

RegressionAnalysis(Curve fitting)



Data Analysis

Two StepsGenerate a point estimate ofdetection probability forvarious crack lengths over arange of interestFit an appropriate curve thatoffers minimum deviation ormaximum likelihood to thescattered data



Log-odds Model

−=−=

−−=

caxcapy

caeca

ln ; ˆlnln

;1

)( PODβα



v

Indi

cate

d cr

ack

leng

th (a

)

True crack length (a)

a vs. a Analysis

Consider a lognormal scatterin indicated crack length forvarious cracks lengthsPOD is the probability ofindicated crack lengthexceeding the threshold ofdetection Requires quantification ofsignal leading to detect call

Threshold

True crack length (a)

POD



Design of NDT Reliability Experiments

Methods of controlling all controllable factorsMethods of tracking all uncontrollable factorsMethods of accounting any unobservable factorsSpecimen design and flaw size distributionFlaw characterization and maintenanceSize of experiment for meaningful conclusionFacility sampling and inspector samplingProtocols for consistent conduct



Design of Experiments

DOE is a precise test methodology foranalyzing cause and effect relationship

Well suited for NDI reliability study

Tool for Human Factors quantification

L32 for 15 variables



Protocol Requirements

Program❖ Schedule, Travel, Inspection, Analysis, QA

Hardware❖ Specimens, Packaging, Cleaning materials,

Software❖ Databases, Data acquisition & analysis tools, Data

archiving, Multi-media briefings

Personnel❖ Dedicated office, Field personnel cooperation, expertise

and experience



Alternate Approaches

Modeling and Simulation

Existing Data Analysis

Reliability Formula



Modeling and Simulation

Reduce the cost of the program

Reduce the program duration

Simulate geometries and defects that aredifficult to create in a hardware form

Quickly change defect characteristics

Isolate NDT and human decision process



Existing Data Analysis

Field data of flaw sizes found

Back extrapolation to flaw sizes missed

Conventional data analysis



American-European Workshops

R = f(IC) – g(AP) – h(HF)

Debated in 1999Next debate 2002

Proposed in 1997

Reliab

ility

Intri

nsic C

apab

ility

Applic

ation

Param

eter

Human

Facto

r


We know how to get it.Can we get examples ?

Break Two

Practical ApplicationsLearning through example

Step 1

Understanding POD curves



First things first

What is a POD?

What is a POD curve?

What is a POD “point estimate”

How does NDE system performance drivethe shape of these curves?

Lets take a graphical walk through the process



Relation of system response to POD curve

50% POD “point”for an 83 unit decisionthreshold

50% POD “point”for an 129 unit decisionthreshold

Regression equation from A-hat.exe program

Lets try one out!

Practical ApplicationsExample One

Considerations in FPI - POD



FPI - Reliability Formula ElementsIC - Chemistry, fluid mechanics

AP - Material, surface condition, location,specific process

HF - Contrast, spatial perception, behavior



FPI - Process parameters



FPI - A-hat vs a, or Pass/Fail?

Pass/fail model-program considerations❖ Process and behavior must conform to model

assumptions…. or no result/non-sense result❖ Asymptotic signal to noise

A-hat vs. a - Preferred❖ Given

Sufficient quantity of data - as few as 20 pointsMeasurable strength of response

❖ At minimum, mean performance can be calculated



FPI - Thresholds (noise, decision, saturation)

Effect of decision thresholds; 5-30milsPenetrant POD performance as a function of decision threshold

0.00E+00

2.00E-01

4.00E-01

6.00E-01

8.00E-01

1.00E+00

1.20E+00

0.00E+00 5.00E-02 1.00E-01 1.50E-01 2.00E-01 2.50E-01 3.00E-01 3.50E-01 4.00E-01

flaw length (in.)

POD



FPI - Source data from flat plates

ra001 0.023 0.025ra001 0.115 0.125ra001 0.069 0.07ra003 0.086 0.13ra003 0.065 0.1ra003 0.036 0.03ra003 0.095 0.1ra004 0.086 0.11ra004 0.11 0.115ra004 0.096 0.105ra004 0.05 0.055ra004 0.02 0.015ra004 0.03 0.03ra005 0.026 0.025ra005 0.072 0.085ra005 0.036 0.03ra005 0.102 0.1ra005 0.094 0.1ra006 0.088 0.1ra006 0.076 0.08ra006 0.044 0.05ra006 0.118 0.115ra007 0.067 0.065ra007 0.075 0.07ra007 0.106 0.115ra007 0.082 0.09ra007 0.016 0.002ra008 0.097 0.09ra008 0.017 0.02ra008 0.041 0.035ra008 0.057 0.04ra008 0.119 0.13ra009 0.074 0.075ra009 0.118 0.115ra009 0.027 0.02ra011 0.042 0.055ra011 0.026 0.03ra011 0.028 0.025ra011 0.07 0.075ra012 0.048 0.045ra012 0.04 0.035ra012 0.111 0.115ra012 0.071 0.08ra012 0.088 0.09ra013 0.036 0.03ra013 0.032 0.025ra013 0.019 0.002ra013 0.036 0.002ra014 0.082 0.085ra014 0.03 0.025ra015 0.07 0.07ra015 0.016 0.002ra015 0.034 0.025ra015 0.068 0.075ra015 0.089 0.075

T. P he lps 7/27/93 P e ne tra nt S ca tte r (ZL-66A & S KDNF)

00.020.040.060.08

0.10.120.14

0 0.05 0.1 0.15

Actua l fla w le ng th (in.)

T. P he lps 7/27/93 P e ne tra nt P OD (ZL-66A & S KD-NF)

00.20.40.60.8

11.2

0 0.01 0.02 0.03 0.04 0.05

Fla w Le ngth (in.)

90/95 = .028"



Isolating chemical (App) parameters -Influences of FPI parameters using flat panels

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 2 4 6 8 10 12 14 16

Set "A" of NASA-SSME panels used for all exams. Panels include 56 cracks on 15 4"x15" panels (both sides). Size .016" to .118 (.361

extraneous, non-verified flaw)

A-h

at 9

0/95

CIF

(nde

) len

gth

(inch

es)

L 3 W-W A-NQ

L3 W-W AB-NQ

L 4 PE AB-NQ

TL-KN

L4 WW AB-NQ

L3 WW DP

Linear (L3 WW DP)

Linear (L3 W-W AB-NQ)

Linear (L 3 W-W A-NQ)

Linear (L4 WW AB-NQ)

Inspector

-& Group performance...



FPI - Defining group performance expectations

Influences of FPI parameters using flat panels

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0 2 4 6 8 10 12 14 16

Inspector Group

A-h

at 9

0/95

CIF

(nde

) len

gth

(inch

es)

Example Two

Laser Methods - POD

Shearography



Application - Bonded Structures

Traditionally acoustic inspections❖ UT, resonance, “tap”…

Lasers can sense unbond, mapping the related displ.

Bonded ThermalProtection

Close-outstructures

Braze bonds& Honeycomb

Narrow faying surface

braze bonds

SupportStructures

Honeycomb is A-286, .032facesheets over 1.4” thick.003-.004” core, .5x.2~” cellsize. Core is perforated.

facesheet

Liner/core



Shearography - Designing for a-hat vs a



POD from X--LS-01Base Area Width Length

Flaw ID1 .25x.5 0.125 0.25 0.52 .25x.75 0.187 0.25 0.753 .25x1 0.25 0.25 14 .25x1.25 0.3125 0.25 1.255 .25x1.5 0.375 0.25 1.56 .25x1.5 0.375 0.25 1.57 .25x1.25 0.3125 0.25 1.258 .25x1 0.25 0.25 19 .25x.75 0.187 0.25 0.75

10 .25x.5 0.125 0.25 0.511 .2x.25 0.05 0.2 0.2512 .25x.25 0.0625 0.25 0.2513 .2x.25 0.05 0.2 0.2514 .25x.25 0.0625 0.25 0.2515 .2x.5 0.1 0.2 0.516 .2x.75 0.15 0.2 0.7517 .2x1 0.2 0.2 118 .2x1.25 0.25 0.2 1.2519 .2x1.5 0.3 0.2 1.520 .25x.25 0.0625 0.25 0.2521 .5x.5 0.25 0.5 0.522 .25x.5 0.125 0.25 0.523 .25x.75 0.1875 0.25 0.7524 .25x1 0.25 0.25 1

Shearography - A-hat vs a analysis

1st Cycle Braze Liner/Closeout Shearography POD(Valid for unbonds .2" and larger)

0.00.1

0.20.3

0.40.50.6

0.70.8

0.91.0

0.0000 0.0200 0.0400 0.0600 0.0800 0.1000 0.1200 0.1400 0.1600 0.1800

Unbond Area (.2"x.2" Decision threshold)

POD

90/95=.104 sq. in.

1st Cycle Braze A-hat Scatter(168 responses)

y = 1.03xR2 = 0.7562

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4Area (sq. in.) - .2" min dim fwd>aft

Shea

rogr

aphy

resp

onse

ar

ea(s

q.in

.)

POD/MTD/Prod. Data Linear (POD/MTD/Prod. Data)



Types of analysis -Comparing between candidate methods

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.1 0.2 0.3 0.4 0.5 0.6

Thre s ho ld (inc he s )



Conduct Ultrasonic A-hat vs a for HC panel1 2 3

ap7 0.375 0.28 0.28 0.28ap8 0.25 0.16 0.16 0.12ap9 0.375 0.28 0.28 0.28ap10 0.25 0.12 0.12 0.12ap11 0.375 0.28 0.28 0.28ap12 0.25 0.2 0.16 0.16ap13 0.375 0.32 0.28 0.24ap14 0.25 0.08 0.04 0.08ap15 0.375 0.28 0.28 0.28ap16 0.25 0.04 0.04 0.08ap17 0.5 0.4 0.44 0.44ap18 0.5 0.4 0.36 0.44ap19 0.5 0.4 0.4 0.4ap20 0.5 0.4 0.4 0.4ap21 0.5 0.4 0.36 0.4at01 0.1875 0.025 0.025 0.025at02 0.1875 0.025 0.04 0.04at03 0.1875 0.025 0.025 0.025at04 0.1875 0.025 0.025 0.025at05 0.1875 0.025 0.025 0.025at06 0.25 0.04 0.04 0.05at07 0.25 0.05 0.08 0.04at08 0.25 0.04 0.04 0.04at09 0.25 0.04 0.04 0.04at10 0.25 0.04 0.04 0.04at11 0.375 0.24 0.24 0.2at12 0.375 0.2 0.2 0.24at13 0.375 0.24 0.24 0.24at14 0.375 0.24 0.24 0.28at15 0.375 0.2 0.24 0.24at16 0.5 0.36 0.36 0.4at17 0.5 0.4 0.4 0.44at18 0.5 0.36 0.36 0.36at19 0.5 0.36 0.36 0.36at20 0.5 0.36 0.36 0.36ab11 0.375 0.04 0.04 0.04ab12 0.375 0.08 0.04 0.08ab13 0.375 0.12 0.12 0.12ab14 0.375 0.16 0.12 0.12ab15 0.375 0.16 0.12 0.12ab16 0.5 0.28 0.28 0.32ab17 0.5 0.2 0.24 0.2ab18 0.5 0.28 0.36 0.32ab19 0.5 0.28 0.24 0.28ab20 0.5 0.4 0.32 0.4bp01 0.25 0.04 0.04 0.08bp02 0.25 0.04 0.025 0.025bp03 0.375 0.16 0.24 0.2bp04 0.375 0.16 0.16 0.2bp05 0.5 0.32 0.36 0.4bp06 0.5 0.36 0.4 0.44bp07 0.375 0.24 0.24 0.2bp08 0.25 0.025 0.025 0.025bp09 0.375 0.2 0.24 0.28bp10 0.25 0.04 0.04 0.04bp11 0.375 0.28 0.28 0.2bp12 0.25 0.12 0.08 0.04bp13 0.375 0.28 0.28 0.28bp14 0.25 0.08 0.12 0.08bp15 0.375 0.24 0.24 0.24

U ltra s o nic s c a tte r d a ta fo r .016"s kin, .1875" c e ll p a ne l

0

0.1

0.2

0.3

0.4

0.5

0.6

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

P ro g ra mme d U nb o nd D ime ns io n (inc he s )

Me a s ure d Unb o nd D ime ns io n (in.)

Thre s ho ld : 5x5 p ixe ls >= 6d b



Conduct Shearography A-hat vs a for HC panel1 2 3

ap07 0.375 0.4942 0.4942 0.5295ap08 0.25 0.3353 0.3 0.3177ap09 0.375 0.4589 0.4589 0.4589ap10 0.25 0.3 0.3177 0.3177ap11 0.375 0.4589 0.4589 0.4942ap12 0.25 0.353 0.3 0.3353ap13 0.375 0.4412 0.4236 0.4412ap14 0.25 0.3177 0.3 0.3177ap15 0.375 0.3883 0.4236 0.4236ap16 0.25 0.353 0.353 0.3706ap17 0.5 0.5295 0.5471 0.5648ap18 0.5 0.6883 0.6883 0.653ap19 0.5 0.6177 0.5824 0.6001ap20 0.5 0.653 0.6883 0.6707ap21 0.5 0.5648 0.6001 0.5824at01 0.188 0 0 0at02 0.188 0 0 0at03 0.188 0 0 0at04 0.188 0 0 0at05 0.188 0 0 0at06 0.25 0 0 0at07 0.25 0 0.2824 0.2647at08 0.25 0 0 0at09 0.25 0 0 0at10 0.25 0 0 0.1765at11 0.375 0.3883 0.3883 0.353at12 0.375 0.4236 0.4589 0.4412at13 0.375 0.353 0.3883 0.3883at14 0.375 0.4942 0.5295 0.4942at15 0.375 0.4236 0.4765 0.4765at16 0.5 0.5824 0.5824 0.5824at17 0.5 0.6177 0.6177 0.6177at18 0.5 0.6177 0.6001 0.6001at19 0.5 0.5824 0.6354 0.6001at20 0.5 0.6001 0.653 0.6001ab11 0.375 0.3177 0.3353 0.2824ab12 0.375 0.3 0.2824 0.3ab13 0.375 0.3177 0.3177 0.3ab14 0.375 0.3706 0.353 0.353ab15 0.375 0.3353 0.353 0.353ab16 0.5 0.4589 0.4944 0.4589ab17 0.5 0.353 0.3706 0.353ab18 0.5 0.5648 0.5648 0.5471ab19 0.5 0.5118 0.5648 0.5648ab20 0.5 0.6707 0.653 0.653bp01 0.25 0 0 0bp02 0.25 0 0 0bp03 0.375 0.4236 0.4059 0.4236bp04 0.375 0.4589 0.3883 0.4059bp05 0.5 0.6001 0.5824 0.5824bp06 0.5 0.5824 0.6001 0.5824bp07 0.375 0.4942 0.4765 0.4942bp08 0.25 0 0 0bp09 0.375 0.4765 0.4765 0.4765bp10 0.25 0.2824 0.3 0.3bp11 0.375 0.5118 0.4942 0.4942bp12 0.25 0.2647 0.3117 0.2647bp13 0.375 0.4765 0.4765 0.4765bp14 0.25 0.2647 0.3 0.3bp15 0.375 0.4765 0.4942 0.4942

S he a rogra phy re s pons e s ca tte r fo r .016" s kin, .1875 ce ll pa ne ls

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.1 0.2 0.3 0.4 0.5 0.6

P rogra mme d unbond d ime ns ion (inche s )

Me a s ure d unbond (in.)

Example Three

Pressure Vessel POD

Ultrasonic & Eddy Current



PV-POD Using transfer functions to fill gaps

Often we are asked to develop POD…

But we can not create perfect knowledge❖ Exact material❖ Exact geometry❖ The precise flaws expected in the design

All orientationsAll morphologies

How can we approximate what we do not know?



Pressure vessel POD - Transfer functions

Target application; 120mm welded vessel❖ Internals are expensive, can’t afford false calls❖ Thin - 1mm wall

Situation - Resources to assess with❖ LCF cracks in flat plate - welded to spec

Non-welded plate 5mm❖ Automated UT & EC - Acquisition & Analysis❖ EDM artifacts for all critical locations

Must estimate differences from lab to field



Transfer process

•Compare family of EDMs across thicknesses

•Compare family of EDMs across geometries & PM vs weld



PV - POD; Some results... ET for the surface

P V ET S ca tte r o f Me a n Re s pons e s fo r 26 Cra ck S a mple

010000200003000040000

0 0.05 0.1 0.15

Cra ck Le ngth (in.)

A/D

P V ET P OD for Ba tte ry P a re nt Me ta l

0

0.5

1

1.5

0 0.01 0.02 0.03 0.04 0.05

Fla w Le ngth (in.)

P OD

(from 26pa re nt.pod)



PV - POD; UT for the weld volume

Me a n ultra s onic re s pons e ve rs us le ng th

0

200

400

600

800

1000

1200

1400

0 0.02 0.04 0.06 0.08 0.1 0.12

Cra ck Le ng th (inche s )

Amplitude (mv)

0

0.2

0.4

0.6

0.8

1

1.2

0 0.02 0.04 0.06

Fla w Le ngth (in.)

90/95 = .037"x.008"

Discipline in calibration will assure estimate holds in practice!!

Example Four

Radiographic POD considerations



Radiography - HF dependantDespite advances in image processing mostapplications rely on human interpretation

Detection targets include much beyond simplecracks of length and depth❖ Pores, voids, cast shrink, honeycomb damage

WeldPM



How to achieve valid POD data for RT- without destructive sectioning of natural flaws?

Solution - Consensus evaluation of testset

Inspectors differ on which are real flaws

24A/6R 21A/9R 16A/14R



Using baseline consensus to reduce variance

Intersection of inspector agreement foundvalid in Metallurgical assessment

21A/9R

24A/6R 16A/14R

Example Five

Variation AnalysisMeasuring known unknowns



All Solutions and POD analyses imperfectControl known knowns

Measure known unknowns and account for inanalysis



Example - EC inspection of aircraft lap splices-Per print fastners not all in a row

Off axis model

Parse new peaks based onoffset

Take the data and effort to understand what you can see varying…There will be plenty which you can not!


We know how to get it.Is it worth the effort ?

Break Time

Is the effort worth it ?

What is the cost of notdoing it ?


We know how to get it.It is worth the effort.How to use POD Info ?

Mission Today

General Awareness

What ?

Why ?

How ?

Next ?



Risk

Comes from❖ A developing defect (initial, accidental), and❖ A missed defect during an inspection

Assessment based on❖ Probability of “initial defect ”❖ Estimated defect growth rate❖ Probability of “missing a defect”



Risk Without Inspections

Probability of failureat time ‘T’ after manufacture

Equals

Probability of havingan initial defect of size ‘ao’

that grows tocritical size ‘ac’ in time ‘T’

Usage

Def

ect

ac

ao

T

POF(T) = P(initial ao)



Risk With Inspections

Probability of failureat time ‘T’ after manufacture

Equals

Probability of havingan initial defect of size ‘ao’

that grows tocritical size ‘ac’ in time ‘T’

TimesProbability(ies) of missing the

defect(s) during inspection(s)

Usage

Def

ect

ac

ao

T

a1

a2

POF = P(initial ao)* P(missing a1)* P(missing a2)



Usage0

Def

ect

ac

Usage

Service0

Pm

Man

ufac

ture

Prob

. of F

ailu

re

Defect

POD

ac0Multiplier0

1.0

Pm0

Prob. of ao Pm0

ac

ao

T

Insp

ectio

n

(1 –

PO

D)

Insp

ectio

n

90/95 Crack Size ?


Short Course, Sept 2002 Berlin. Slide Number 92Service

0

Pm

Man

ufac

ture

Prob

. of F

ailu

re

Defect

POD

ac0

Service

Def

ect

ac

Multiplier0

1.0

Pm0

Prob. of ao Pm0

ac

ao

T

Insp

ectio

n

(1 –

PO

D)

0

0.9



Popular Characteristics of PODPOD is Expensive

Certified inspectors donot need POD

Experienced and highsalaried inspectors havebetter POD

90/95 Crack sizeinformation is adequate

Imp to find small flaws

MYTHS

High return on investment;Cost of not doing POD ?

POD addresses the NDEsystem

POD cannot be empiricallyrelated to experience orsalary

90/95 is not a systemparameter. Use completePOD curve for reliability

Imp not to miss big ones

FACTS



POD is in some sense “NDT of NDT”

Managers and Engineers tend to think theydon’t need NDT, There systems are good

We NDT fellows tend to think the sameway, we think we don’t need POD orreliability assessment.

If NDT is important, then POD is important

Mission Today

General Awareness

What ?

Why ?

How ?

Next ?



Further Information

MIL-HDBK-1823

AGARD LS 190

FAA Protocol, Sandia Labs

AF Protocol, Karta Tech

NDE Capabilities Handbook, NTIAC

Materials Evaluation July 2001 Issue



4 Hour Course – Feed back please

Please provide course feedback

Feel free to contact for additional info

Ripi Singh, [email protected] Schaefer, [email protected] Mueller, [email protected] www.pnde.net (October 2002 workshop host)

creating pod practically - 9095 · inspection reliability short course, sept 2002 berlin. slide...

Documents