faa william j. hughes technical center probability of detection studies to quantify flaw detection...
TRANSCRIPT
FAA William J. HughesTechnical Center
Probability of Detection Studies to
Quantify Flaw Detection inComposite Laminate Structures
Probability of Detection Studies to
Quantify Flaw Detection inComposite Laminate Structures
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
Dennis Roach & Tom RiceSandia National Labs
FAA Airworthiness Assurance Center
A340 HTP SkinA340 HTP Skin
FAA William J. HughesTechnical Center
ITG Team ParticipantsITG Team ParticipantsInspection Task Group Team ParticipantsInspection Task Group Team Participants
CACRC Inspection Task Group Members:
Wolfgang Bisle – AirbusChris Dragan – Polish Air Force Institute of TechnologyDon Duncan – US AirwaysJim Hofer – BoeingQuincy Howard – BoeingJeff Kollgaard – BoeingFrancois Landry – Bell HelicopterRobert Luiten – KLM AirlinesAlex Melton – Delta Air AirlinesEric Mitchell – American AirlinesStephen Neidigk – Sandia Labs AANCKeith Phillips – Airbus Tom Rice – Sandia Labs AANCDennis Roach – Sandia Labs AANC (Chair)Vilmar da Silva do Vale – EmbraerDennis von Seelen - Lufthansa Technik Darrell Thornton – UPSSam Tucker – United AirlinesRoy Wong – Bombardier
Rusty Jones, Larry Ilcewicz, Dave Galella, Paul Swindell – FAA
FAA William J. HughesTechnical Center
Composite Structures on Boeing 787 Aircraft
Carbon laminate
Carbon sandwich
Fiberglass
Aluminum
Aluminum/steel/titanium pylons
A380 Pressure Bulkhead
Composite Center Wing Box
Program Motivation - Extensive/increasing use of composites on commercial aircraft and increasing use of NDI to inspect them
Program Goals: Assess & Improve Flaw Detection Performance in Composite Aircraft Structure
FAA William J. HughesTechnical Center
One airline reports 8 composite damage events per aircraft (on avg.) with 87% from impact; cost = $200K/aircraft
Sources of Damage in Composite StructureSources of Damage in Composite Structure
Bird StrikeTowing Damage
Lightning Strike on
Thrust Reverser
Disbonding at skin-to-
honeycomb interface
FAA William J. HughesTechnical Center
Significant Internal Damage
Source: Carlos Bloom (Lufthansa) & S. Waite (EASA)
Inspection Challenge – Hidden Impact DamageInspection Challenge – Hidden Impact Damage
Backside fiber failure from ice impact
Visible Impact Damage – external skin fracture
Backside Damage – internal skin fracture & core crush
Extent of Visible Damage from Outside
Damage from ground vehicle
FAA William J. HughesTechnical Center
Cumulative POD Curves for the Woodpecker on All Panel Types
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
Flaw Size (Dia. in Inches)
Pro
bab
ility
of
Det
ecti
on
3 Ply Carbon 3 Ply Fiberglass 6 Ply Carbon 6 Ply Fiberglass 9 Ply Carbon 9 Ply Fiberglass
Experiment to Assess Flaw
Detection Performance
Composite Flaw Detection Experiment
Participation from over 25 airlines and maintenance depots
Industry-wide performance curves generated to quantify:
• How well current inspection techniques are able to reliably find flaws in composite honeycomb structure
• The degree of improvements possible through integrating more advanced NDI techniques and procedures.
Composite HoneycombFlaw Detection Experiment
Composite HoneycombFlaw Detection Experiment
Blind application of techniques to study hits, misses, false calls, and flaw sizing
FAA William J. HughesTechnical Center
Purpose• Determine in-service flaw detection capabilities: 1) conventional NDT
methods vs. 2) improvements through use of advanced NDT. • Optimize laminate inspection procedures.• Compare results from hand-held devices with results from scanning
systems (focus on A-scan vs. C-scan and human factors issues in large area coverage).
• Provide additional information on laminate inspections for the “Composite Repair NDT/NDI Handbook” (ARP 5089).
An Experiment to Assess Flaw Detection Performance in Composite Laminate Structures
An Experiment to Assess Flaw Detection Performance in Composite Laminate Structures
737 Composite Horiz. Stabilizer
A380 Fuselage Section 19
FAA William J. HughesTechnical Center
Approach
• Statistical design of flaws and other variables affecting NDI - range of types, sizes & depths of flaws
• Study factors influencing inspections including composite materials, flaw profiles, substructures, complex shapes, fasteners, secondary bonds, and environmental conditions
• POD and signal-to-noise data gathering
• NDI Ref. Stds. prepared to aid experiment
Flaw Detection in Solid Laminate CompositesFlaw Detection in Solid Laminate Composites
Expected Results - evaluate performance attributes 1) accuracy & sensitivity (hits, misses, false calls, sizing) 2) versatility, portability, complexity, inspection time (human factors) 3) produce guideline documents to improve inspections 4) introduce advanced NDI where warranted
FAA William J. HughesTechnical Center
Specimen Types – Solid laminate carbon (12 to 64 plies)• Contoured and tapered surfaces• Substructures – stringers, ribs, spars; honeycomb impediment• Bonded & sealed joints; fasteners• Large enough to warrant scanners; complex geometry to challenge
scanners• Carbon, uniaxial tape
Low Energy Impact
0
0
0
00
-45
-45
0
9090
+45
+45
Specimen Design - Flaw Detection in Solid Laminate Composites
Specimen Design - Flaw Detection in Solid Laminate Composites
Flaw Types - statistically relevant flaw distribution with range of sizes & depths (near front & back surfaces; in taper regions)
1) interply delaminations (“kissing” and air gap)2) substructure damage3) skin-to-stiffener disbonds4) simulated impact damage
FAA William J. HughesTechnical Center
Thick Laminate With Complex Taper - FabricationThick Laminate With Complex Taper - Fabrication
Flaw templates - ensure proper location of flaws
Flawsinserted
FAA William J. HughesTechnical Center
Thick Laminate With Complex Taper - FabricationThick Laminate With Complex Taper - Fabrication
FAA William J. HughesTechnical Center
Thick Laminate With Complex TaperThick Laminate With Complex Taper
20 PLIES
12 PLIES3.50"
4.75"
14.00".25" STEP PER PLY
STRINGER 2.00" X 2.00" X .125" THK
20 PLIES
12 PLIES
.25" STEP PER PLY
.50" STEPPER PLY
4.75"
38.00"8.25"
7.25"
.50" STEPPER PLY
19.00"
7.25"
7.25"
8.00"
2.00"
1.00"
COMPLEX TAPER NO. 1 FLAW LAYOUT10-14-04
DELAM 1 (4PL-PI) COMPLEX GEOMETRY
.50".75"1.00" .25"
DELAM 2 (GR-I)COMPLEX GEOMETRY
.50".75"1.00" .25"
1.50"
1.50"
.50".75"1.00" .25"1.50"
IMPACT (IM-T)TAPER
2.00"
2.00"
DELAM 1 (4PL-PI)FULL THK
DELAM 2 (GR-I)FULL THK
IMPACT (IM)FULL THK
.50".75"1.00" .25"1.50"2.00"
.50".75"1.00" .25"1.50"2.00"
.50".75"1.00" .25"1.50"2.00"
.50".75"1.00" .25"
DELAM 1 (FBH) SUBSTRUCTURE
.75"1.00"1.50"2.00"
DISBOND 1 (PT)SUBSTRUCTURE
1.00" .75" .50" .25"
DISBOND 2 (GR-I)SUBSTRUCTURE
A B C D E F
G H I J K L
M N O P Q R
M-2
P-1
R-2B-3
E-1
L-2
I-1
O-3
S T U V W X
Y Z AA BB CC DD
EE FF GG HH II
JJ KK LL MM
NN OOPP QQ
RR SS TT UU
FF-3
W-2
Y-2
BB-2
U-3
BB-3
Z-3
HH-3
X-2
T-2
GG-2
PP-2
TT-2
F-2
L-1
LL-3
38.00"
19.00"
Type I Specimen
FAA William J. HughesTechnical Center
0.75"
PI GR-I FBH
PI GR-I FBH
32 PLY
2.75" 10.50"
(B/W PLIES 16 & 17)
(B/W PLIES16 & 17)
(IN SUBSTR.) (B/W LAMINATE &SUBSTR.)
16.00"
1.50"
1.51"
1.50"
3.19"
ALL FLAWS NOT IN OR UNDER STRINGER ARE 1.00" IN DIAMETERAND FLAWS IN OR UNDER STRINGER (IN LAMINATE) ARE .75" IN DIAMETER
PI GR-I FBH PT(B/W PLIES 16 & 17)
(B/W PLIES16 & 17)
(IN SUBSTR.) (B/W LAMINATE &SUBSTR.)PI
GR-I
FBH
PI
GR-I
FBHPI GR-I FBH PT
1.375"
10-26-06
2.50" 2.75" 2.75"
STRINGER TYPE "A" (TYP)
STRINGER TYPE "B" (TYP)
= 1.00" DIAMETER FLAW
= 0.750" DIAMETER FLAW
STRUCTURE THICKNESS NO. OF PLIES
LAMINATE # 3 0.208" 32LAMINATE # 3 & STRINGER "A" 0.433" 67
0.283" 44LAMINATE # 3 & STRINGER "B"
TYP
0.75"TYP
STRINGER TYPE "A" = 0.225" THK (APPROX. 35 PLIES)
STRINGER TYPE "B" = 0.075" THK (APPROX. 12 PLIES)
1.25"
0.25"
2.50"
1.50"
2.50"
2.50"
3.25"
2.50"
2.50"
0.76"
0.60"
5.00"
4.50"
6.50"
13.25"
8.00"
1.00" 2.00" 2.00" 2.00"
1.25"
20 Ply Ref Std
32 Ply Ref Std
Reference Standards – Feedback PanelsReference Standards – Feedback Panels
FAA William J. HughesTechnical Center
Specimen Set - Flaw Detection in Solid Laminate Composites
Specimen Set - Flaw Detection in Solid Laminate Composites
Thickness Range:12 – 64 plies
Inspection Area:46 ft.2
Number of Flaws:202
FAA William J. HughesTechnical Center
Specimen Set - Flaw Detection in Solid Laminate Composites
Specimen Set - Flaw Detection in Solid Laminate Composites
Thickness Range:12 – 64 plies
Simple Tapers
Complex tapers
Substructure Flaws
Curved Surfaces
Array of flaw types
NDI Ref. Stds.
FAA William J. HughesTechnical Center
Solid Laminate Experiment Participants Solid Laminate Experiment Participants
FAA William J. HughesTechnical Center
Solid Laminate Flaw Detection Experiment Implementation
Solid Laminate Flaw Detection Experiment Implementation
PODs calculated for overall laminate, by thickness family, by substructure effects, by complex geometry effects, by flaw types, etc. 57 inspectors
FAA William J. HughesTechnical Center
POD Curves for 12-20 Ply Solid Laminate Family
POD Curves for 12-20 Ply Solid Laminate Family
False Calls: Constant thickness = 0.4/inspectorComplex Geometry = 4.0/inspector
34 ft.2 inspection area
Individual and Cumulative Comparisons
Flaw Size (Diameter in Inches)
Pro
bab
ilit
y o
f D
etec
tio
n
Overall: POD[90/95] = 1.29” dia.
Constant Thickness(12, 20, 28 plies): POD[90/95] = 0.86” dia.
Complex Geometry(tapered, curved, substructure, fasteners, honeycomb): POD[90/95] = 1.49” dia.
FAA William J. HughesTechnical Center
POD Improvements from Use of Methods to Ensure Proper Coverage
POD Improvements from Use of Methods to Ensure Proper Coverage
Flaw Size (Diameter in Inches)
Pro
bab
ilit
y o
f D
etec
tio
n
FAA William J. HughesTechnical Center
POD Curves for 20-32 Ply Solid Laminate Family
POD Curves for 20-32 Ply Solid Laminate Family
False Calls: Constant thickness = 0.8/inspectorComplex Geometry = 0.3/inspector
12 ft.2 inspection area
Overall: POD[90/95] = 0.82” dia.
Constant Thickness(32 plies): POD[90/95] = 0.74” dia.
Complex Geometry(tapered, curved, substructure, fasteners, honeycomb): POD[90/95] = 0.93” dia.
Substructure: POD[90/95] = 1.50” dia.
Individual and Cumulative Comparisons
Flaw Size (Diameter in Inches)
Pro
bab
ilit
y o
f D
etec
tio
n
FAA William J. HughesTechnical Center
Desire to Transition Inspectors from “Average” to “Good” to “Outstanding”Desire to Transition Inspectors from
“Average” to “Good” to “Outstanding”
Flaw Size (Diameter in Inches)
Pro
bab
ilit
y o
f D
etec
tio
n
FAA William J. HughesTechnical Center
Overall Performance of Pulse-Echo UT forFlaw Detection in Composite Laminates
Overall Performance of Pulse-Echo UT forFlaw Detection in Composite Laminates
Flaw Size (Diameter in Inches)
Pro
bab
ilit
y o
f D
etec
tio
n
Overall: POD[90/95] = 1.13” dia.
Constant Thickness(32 plies): POD[90/95] = 0.80” dia.
Complex Geometry(tapered, curved, substructure, fasteners, honeycomb): POD[90/95] = 1.33” dia.
FAA William J. HughesTechnical Center
Ramp Damage Check Experiment (RDCE) – “Spin Off” Subset of Solid Laminate Experiment
Ramp Damage Check Experiment (RDCE) – “Spin Off” Subset of Solid Laminate Experiment
Purpose: to assess new, ultrasonic-based “Go”/“No-Go” equipment that OEM’s plan to deploy at airports and other
non-scheduled maintenance depotsusing non-NDI personnel (e.g. A&Ps).
Usage Scenario: the equipment will be deployed whenever visual clues or other events occur which warrant closer scrutiny
of a composite laminate structure; ground personnel, with appropriate training on such equipment, will set up the
equipment in accordance with OEM-supplied procedures, and then make an assessment of the region in question.
Limitations: such “Go”/“No-Go” UT equipment is intended to be used to assess local indications or regions only, not
for wide-area inspections; equipment operators are directed to very distinct locations.
FAA William J. HughesTechnical Center
Ramp Damage Check Experiment – Ultrasonic Devices with “Go”/“No-Go” Capabilities
Ramp Damage Check Experiment – Ultrasonic Devices with “Go”/“No-Go” Capabilities
Olympus –“Ramp Damage Checker”
General Electric –“Bondtracer”
FAA William J. HughesTechnical Center
Ramp Damage Check Experiment – POD Curves for Composite LaminatesRamp Damage Check Experiment –
POD Curves for Composite Laminates
Flaw Size (Diameter in Inches)
Pro
bab
ilit
y o
f D
etec
tio
n
• Overall: POD[90/95] = 0.78” dia.
• Similar results for GE Bondtracer and Olympus Ramp Damage Checker
• Similar performance from both inspectors and A&P mechanics
• Less spray in individual performances
False Calls = 0.6/inspector14 ft.2 inspection area
FAA William J. HughesTechnical Center
Thermography
MAUSSystem
SAM System
Shearography
Wide Area and C-Scan Inspection MethodsWide Area and C-Scan Inspection Methods
PE Phased Array UT UT Wheel Array
UltraImage Scanner
FAA William J. HughesTechnical Center
Solid Laminate Experiment – Advanced NDI Testing Evaluations
Solid Laminate Experiment – Advanced NDI Testing Evaluations
• Pulsed Thermography – Thermal Wave Imaging• Phased Array UT – Olympus Omniscan • Phased Array UT – Toshiba Matrix Eye • Phased Array UT – Sonatest RapidScan• Phased Array UT – Boeing MAUS V• Phased Array UT – GE RotoArray & Phasor• Microwave – Evisive• Digital Acoustic Video – Imperium Acoustocam• Ultrasonic Video - DolphiCam• Line Infrared – Mistras• Shearography – LTI• Shearography – Dantec• Backscatter X-ray – Scannex• Acousto Ultrasonics – Physical Acoustic Corp. T-SCOUT• Locked-In Infrared – MovieTherm• Laser UT – iPhoton
FAA William J. HughesTechnical Center
PA-U
T, Toshiba M
atrixEye
PA-U
T, Olympus O
mniScan
Laser U
T, iPhoton
PA-U
T - NDTS M
AUS V
DAV, I
mperium A
coustoCAM
PE-U
T, Industry
Flash IR
- TWI
Shearo
graphy - Dantec
Line IR
- Mistra
s
Lock-In
IR, M
oviTHERM
POD[90/95] 0.69 0.72 0.85 0.88 1.12 1.13 2.3 >3 >3 >3False Calls 1 0 0 0 0 5.5 0 1 NA 26
Sample of POD Results for Composite Flaw Detection Performance of Advanced NDI
Sample of POD Results for Composite Flaw Detection Performance of Advanced NDI
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Pro
bab
ility
of
Det
ecti
on
Flaw Size (Diameter in Inches)
Individual & Cumulative POD[90] Curve Comparison for the Combined 12-20 and 20-32 Ply Specimen Sets for All Flaws in the Constant Thickness & Complex Geometry Regions
9 - Advanced NDI Methods
PA-UT, Toshiba MatrixEye
PA-UT, Olympus OmniScan
Laser UT, iPhoton
PA-UT, NDTS MAUS V
DAV, Imperium AcoustoCAM
PE-UT, Industry Cumulative
Flash IR, TWI
Shearography, Dantec
Line IR, Mistras
Lock-In IR, MoviTHERM
Flaw Size (Diameter in Inches)
Pro
bab
ilit
y o
f D
etec
tio
n
FAA William J. HughesTechnical Center
Survey of Industry Composite NDI TrainingSurvey of Industry Composite NDI Training
Only 25% of responders currently have special composite NDI training in
place
FAA William J. HughesTechnical Center
Question 5 – Do inspectors also receive general composite training to
understand composite materials, plies, lay-ups, scarfed repairs, composite
design, composite processing, etc.?
Question 15 - If experience level is a factor in determining qualification to perform certain
inspections, do you use some sort of apprentice program to expose newer inspectors to such
inspections?
FAA William J. HughesTechnical Center
• Results can be used by OEMs and airlines to: 1) define detectability for various flaws/damage, 2) guide NDI deployment & training; used by FAA to produce guidance documents.
• Flaw Detection - Conventional, manually-deployed PE-UT: POD[90/95] = 1.12” dia.; skin flaw detection is higher, flaw detection in substructure is more challenging (POD[90/95] = 1.34” dia.)
• False Call rates were extremely low: 1 false call per 17 ft.2 (flaws ≥ 0.25 in.2)
• Optimum inspection rates = 2 ft.2 /hour• NDI Performance Obstacles – attenuation, complex signal reflections,
confounding presence of signal harmonics, rapid variations produced by changing/complex geometry, optimum deployment and difficulty with inspecting large areas
• Controlled and Proper Use of Ramp Damage Check “Go”/”No-Go” equipment can produce good performance (POD[90/95] = 0.77” dia.) Caution – proper calibration area must be used; requires good
schematics (available?) Cannot be used in taper or changing geometry regions
Conclusions – Inspection of Solid Laminate StructuresConclusions – Inspection of Solid Laminate Structures
FAA William J. HughesTechnical Center
Recommendations – How to move inspections from “average” to “good” to “outstanding”
Recommendations – How to move inspections from “average” to “good” to “outstanding”
• Increased exposure to representative composite inspections – common industry NDI Feedback Specimens
• Increased, focused composite NDI training (ARP covering 1st two bullets)
• Use of NDI and composite shop apprenticeships (OJT, awareness training, formal/uniform use of this tool)
• Enhanced NDI procedures – deployment, signal interpretation, clearer schematics showing structural configuration
• Use of inspection coverage aids should be required
• Divide large area inspections into a number of smaller regions
• Add audible alarms & detection lights to probe in “Go”/”No-Go” devices
• Prepare additional industry guidance to address training, use of NDI Reference and Proficiency specimens, procedures, composite construction awareness – produced by joint effort of OEMs, Airlines, FAA & industry groups
FAA William J. HughesTechnical Center
2013 Airlines for America NDT Forum
POD Studies to Quantify Flaw Detection in Composite Laminate Structures
Dennis RoachTom Rice
FAA Airworthiness Assurance CenterSandia National Laboratories
Composites have many advantages for use as aircraft structural materials including their high specific strength and stiffness, resistance to damage by fatigue loading and resistance to corrosion. The aircraft industry continues to increase its use of composite materials, most noteworthy in the arena of principle structural elements. This expanded use, coupled with difficulties associated with damage tolerance analysis of composites, has placed greater emphasis on the application of accurate nondestructive inspection (NDI) methods. Traditionally, a few ultrasonic-based inspection methods have been used to inspect solid laminate structures. Recent developments in more advanced NDI techniques have produced a number of new inspection options. Many of these methods can be categorized as wide area techniques that produce two-dimensional flaw maps of the structure. An experiment was developed to assess the ability of both conventional and advanced NDI techniques to detect voids, disbonds, delaminations, and impact damage in adhesively bonded composite aircraft structures. A series of solid laminate, carbon composite specimens with statistically relevant flaw profiles were inspected using conventional, hand-held pulse echo UT and resonance, as well as, new NDI methods that have recently been introduced to improve sensitivity and repeatability of inspections. The primary factors affecting flaw detection in laminates were included in this study: material type, flaw profiles, presence of complex geometries like taper and substructure elements, presence of fasteners, secondarily bonded joints, and environmental conditions. This experiment utilized airline personnel to study Probability of Detection (POD) in the field and to formulate improvements to existing inspection techniques. In addition, advanced NDI methods for laminate inspections – such as thermography, shearography, scanning pulse-echo UT, ultrasonic spectroscopy, laminography, microwave and phased array UT – were applied to quantify the improvements achievable through the use of more sophisticated NDI. This paper presents the experiment design used to evaluate applicable inspection techniques and the key results from the POD testing conducted at multiple airline facilities.