title of presentation - imdr-+roussel... · 2019-02-27 · ecam : cond lav+galley fan fault …...

Aircraft maintenance logbooks analysis

David ROUSSEL, Airbus Group Innovations, Computational Intelligence & Services

Fabrice LATCHURIE, Airbus, In-Service Data Management

Des méthodes aux applications du TAL dans le retour d’expérience

Journée IMdR du 16 mai 2017

Summary

• Overview of Aircraft In-Service data management

• What are Aircraft Logbook coding tasks ?

• Logbook auto-coding investigations with Machine Learning

• Assisted coding industrial implementation with dictionaries and rules

• Lesson learnt and Perspectives

- 16/05/2017 Des méthodes aux applications du TAL dans le retour d’expérience 2

In service data management & coding

Benchmark, identify

weaknesses

solutions Improve the

fleet

In-service data

Change considered

Text Mining and Statistical Modules plugged into the in-service

data coding processes

Impact

Accuracy of codes and results

Proposed solutions

Benefits

Reduce time to process the in-service data

in the context of Airbus fleet growth

Increase quality of the in-service data processing

Airlines

logbooks

coding

End-to-end approach to address multi program

in-service issues

Objective is to efficiently drive the Fleet Performance Improvement, based on:

250 operators reporting data for

22,000 flights per day for an average of

9000 interruptions per month (delays, flight cancellation, in-flight turn back, diversion)


http://www.iconarchive.com/show/icons8-metro-style-icons-by-visualpharm/Ecommerce-Idea-icon.html

identify, develop and validate data modules for modifications

Deliver contextualized documentation

Manage/maintain documentation & preconditions

Maintenance

Planning

Maintenance

Execution

Maintenance Monitor and

Control

Technical

requests

Instant Monitoring

and advices

In Service Technical Event

Management Post Design Services

Technical

Publications

Maintenance logbooks (from

MPD, Pirep, Carep, Marep…)

Reliability Monitoring

Unscheduled

Maintenance

Aircraft reports

(from CMS, ACMS, …)

Processes overview

What are Aircraft Logbook coding tasks ?

• Manual coding of these feedbacks has been performed by inspecting

textual fields which are providing a “description” and “work performed”.

• These text fields are unstructured, can contain abbreviations, spelling and

grammatical mistakes (of non native speakers), etc.

After data collection, “coding” means enriching the raw text data by fields

like:

ATA chapters (based on ATA-100 standard)

Custom codes (Maintenance action, Finding, Symptoms/Faults,

Operational Phase, Effects)

Related events (e.g. ECAM message)

LAVATORY GALLEY FAN FAULT.AFT

GALLEY FAN RPLCD.

From unstructured data (free text)

To structured data

ATA 6D: 21-23-51

ACTION: Replaced

ECAM : COND LAV+GALLEY FAN FAULT

… Other TAGS

Coding

Enrichment with related event

Categorization (hierarchical)

Unique or multiple tagging


Auto-coding investigations with Machine Learning

Textual pre-

processing

Auto-Classification

models learning Historical coded Data

Coding

(with confidence)

Assisted-Coding for

confident cases

(rules)

Cross-evaluation

- 16/05/2017 6

New data

Most representative

contributors

ATA codes

(e.g. one year A320)

Fault complex codes

representation

(e.g. small airline data)

MAIN CHALLENGES:

• Code values are unbalanced

• Non-native written Technical English

• Complex codes (combinations) are under-represented

• Coding practices may vary between coders, are subject to errors

• Empty code values are difficult to interpret (missing codes 3.3 %)

Des méthodes aux applications du TAL dans le retour d’expérience

ATA-4D by Airline

Action-Finding

Effect

Fault

Rule-based fusion

Classifiers

Effect-Phase

Coupled codes

Machine Learning approaches tested


Coded field ATA–4D

by Airline

(660

codes)

Action

(15)

Finding

(38 )

Fault

(30)

Phase

(15)

Effects

(16)

% Success Overall 82%

85% 73% 70% 70% 62%

% Success with high

confidence

55%

13 % 36% 19%

No improvement expected

Good hopes for improvement

% Success Overall – refers to the % of correctly coded

cases in the sample

(but where the errors are is unknown)

% success with high confidence means the sub-set of

coded cases which are closed to expert coding

(success > 95 %)

CONCLUSION was to investigate dictionary-based approach + business rules to provide suggestions


First results obtained

Defining a dictionary and elucidating business rules

A dictionary is intuitively more relevant for multiple tagging (combinations)

and ATA chapters. It gives rise to better description of the codes.

It enables refinement of coding schemes (e.g. ATA 4 digits ATA 6 digits)

by supporting IHM improvements for codes selection.

Unique code selection is performed by explicit prioritization rules

(e.g. replacement action > inspection, 291151 Ata chapter > 291100, …)

Contextual rules are used to detect positive or negative textual contexts

(e.g. for disambiguation, syntactic variants, negation, mentions of postponed action...)

To ease dictionary creation, substitutions are used to normalize the raw text

(e.g. RPLCD REPLACED).

Fuzzy matching is used for compound terms and mentions to Aircraft messages.


Illustration of lexical ambiguity between actions

Adjusted and Serviced Current status: 6500 substitutions, 9500 rules

290000 HYDRAULIC + LEAK

291100 GREEN + EMP

291100 GREEN + ENGINE 1

291100 GREEN + HYDRAULIC + SYSTEM

291100 GREEN + PRESSURE

291100 ENGINE 1 + HYDRAULIC

291114 GREEN + MANIFOLD

291114 1011GM

291114 1111GM

291115 GREEN + PTU + MANIFOLD

291115 1013GM

291115 1113GM

291117 GREEN + PRESSURE + SWITCH

291117 SWITCH + PUMP

291117 1074GK

291117 1074GK

291121 GREEN + AIR + ACCUMULATOR

291121 1072GM

291122 GREEN + ACCUMULATOR + CHARGE + VALVE

291122 1071GM

291132 GREEN + PRESSURE + RELIEF + VALVE

291132 1063GM

291133 GREEN + PRIORITY + VALVE

291133 1064GM

291134 GREEN + SAMPLING + VALVE

291134 1187GM

291135 GREEN + CHECK + VALVE + REVERSER

291135 3008KM1

291135 3008KM1

291135 3009KM1

291135 3010KM1

291136 GREEN + CHECK + VALVE + PUMP + DELIVERY

291136 1050GM

291136 1060GM

291137 GREEN + CASE + DRAIN

291137 1041GM

291138 GREEN + COUPLING + ACCUMULATOR

291138 1672GM

291139 GREEN + HALF + COUPLING

291139 1038GM

291139 1700GM

291139 1700GM

291141 GREEN + LOW + AIR + PRESSURE

291141 GREEN + RESERVOIR

291141 1000GQ

291142 GREEN + ACCUMULATOR

291142 1070GM

291143 EDP + FILTER + ENGINE 1

291143 EDP + FILTER + LEFT

291143 EDP + FILTER + GREEN

291143 1084GM

291143 1086GM

291144 GREEN + FILTER + LOW + PRESSURE

291144 1002GM

291144 1030GM

291145 GREEN + FILTER + HIGH + PRESSURE

291145 1048GM

291146 GREEN + DAMPENER

291146 1085GM

291147 GREEN + SLIDE + COMPENSATOR

291147 1010GM

291148 GREEN + CHECK + VALVE + WTB

291148 1170GM

291148 1410GM

291148 1411GM

291149 GREEN + HYDRAULIC + ENGINE + TUBE

291149 HYDRAULIC + ENGINE 1 + TUBE

291151 EDP + ENGINE 1

291151 EDP + LEFT

291151 ENGINE 1 + PUMP

291151 GREEN + EDP

291151 GREEN + PUMP

291151 1030GK

291152 GREEN + ENGINE + FIRE + VALVE

291152 1046GK

291153 GREEN + DAMPER + EDP

291153 1600GM

291163 GREEN + GROUND + MANIFOLD

291200 BLUE + HYDRAULIC + SYSTEM

291200 BLUE + PRESSURE

291214 BLUE + MANIFOLD

291214 2011GM

291215 BLUE + ELEC + PUMP

291215 2706GJ

291217 BLUE + PRESSURE SWITCH

HYDRAULIC LK ON ENG#1 EDP,

RPLCD #1 HYD PMP OPS AND LK CK

GOOD

Text normalization Matching Ranked suggestions

From

Abbreviation and most current

misspelling errors

AVIONCS

AVCS AVIONIC

AVNCS AVNX

AVNXS

TO

Normalized text

AVIONICS

HYDRAULIC LEAK ON ENGINE 1 EDP

REPLACED 1 HYDRAULIC PUMP OPS

AND LEAK CHECK GOOD



AND LEAK CHECK GOOD



AND LEAK CHECK GOOD



AND LEAK CHECK GOOD



AND LEAK CHECK GOOD


REPLACED #1 HYDRAULIC PUMP OPS

AND LEAK CHECK GOOD

290000 HYDRAULIC + LEAK

291100 ENGINE 1 + HYDRAULIC

EDP + ENGINE 1

ENGINE 1 + PUMP

Human decision making

Assisted coding process

291151

- 16/05/2017 10

IHM of assisted coding

Related warnings are

ranked by fuzzy

matching score

Text is highlighted regarding

the selected coding field


Dictionary + rules modeling and test

match

3A - HYDRAULIC LK ON ENGINE 1

EDP TEST.

REPLACED #1 HYD PUMP OPS AND

LK CK GOOD

291151

Text to be analysed Principle:

If a keyword or a combination

of keywords is discriminative

in several coded examples,

then a lexical item or rule is

added for the corresponding

code.

Dictionary creation is

compliant with PROVALIS

exchange format to benefit

from the rich set of Text

Mining methods implemented

in WordStat tool, for case

studies.

ATA Code candidates

291100

@ENG1&HYD [##ENGINE_1 NEAR HYD* /Y /C5]

@GREEN&EMP [##GREEN NEAR ##EMP /Y /C5]

@GREEN&ENG1 [##GREEN NEAR ##ENGINE_1 /Y /C5]

@GREEN&HYD [HYD* NEAR ##GREEN /Y /C5]

With

##ENGINE_1 = [ENGINE BEFORE 1 /Y /C2]

##GREEN = G or GREEN

##EDP = EDP or [ENG* NEAR DRIV* NEAR PUMP* /A]

291100

291151

@EDP&LEFT [##EDP NEAR LEFT /Y /C7]

@GREEN&EDP [##GREEN AND ##EDP /Y /C]

@EDP&ENG1 [##EDP AND ##ENGINE_1 /Y /C]

@ENG1&PMP [##ENGINE_1 AND ##PUMP /Y /S]

1030GK

match

6D

4D


Fast Dictionary builder

Rules and keywords

(already defined) Existing coding selection

(here ATA)

REQUIREMENT #1:

ease observation

and rules creation

Relevant keywords related

to ATA 291151 based on

TF-IDF TF-ICF analysis

Terms co-occurrence


Lesson learnt & Perspectives

• First experiments to analyze logbooks through automated coding have shown that Machine Learning scores are

difficult to exploit (mainly due to existing coding errors, inconsistencies and code representativeness in the training datasets).

• Machine learning models also learn “bad” coding habits.

• Rules based on a lexical approach have been implemented to provide suggestions to “human” coders and improve quality.

Rules are managed to implement relevancy, redundancy, accuracy principles.

Based on coding activity traces, individually, suggestions are 85 to 95% accurate with good hopes for improvement.

• Next improvements would be:

• To link and visualize logbooks data in a context of past and next maintenance events, to deliver better results.

• To learn scoring models to adapt ranking mechanisms according human decision makings

• To model the codes through analogy with their prototypical examples (human categorization)

It is now possible to generate positive/negative pairs of a priori semantically similar texts to test Deep Learning disambiguation

(e.g. Siamese Recurrent Architectures).


© Airbus 2017. All rights reserved. Proprietary document. This document and all information contained herein is the sole property of AIRBUS. No intellectual property rights are granted by the

delivery of this document or the disclosure of its content. This document shall not be reproduced or disclosed to a third party without the express written consent of AIRBUS. This document and

its content shall not be used for any purpose other than that for which it is supplied. The statements made herein do not constitute an offer. They are based on the mentioned assumptions and

are expressed in good faith. Where the supporting grounds for these statements are not shown, AIRBUS will be pleased to explain the basis thereof.

AIRBUS, its logo, A300, A310, A318, A319, A320, A321, A330, A340, A350, A380, A400M are registered trademarks.

Appendix – Past Coding HMI

Use Tab 'Insert - Header & Footer' for Presentation Title - Siglum - Reference

Appendix – Coding detail

Name Description

AC Aircraft NA Not Aircraft NM Maintenance

NO Operation

NS Lack of spares


Chargeability

SYMPTOM/FAULT codes

Effect codes

Name Description

AA Aborted approach

AB Abandoned aircraft AT Aborted take off BL Belly landing, ditching

CH Aircraft changed

CN Cancellation

DP Depressurisation

DV Diversion

DY Delay

ED Emergency descent ES Engine shut down in flight ET Etops incident GI Ground interruption

IF Interrupted flight JT Jettison

TL Tail strike

Name Description

AS APU auto shut down without crew action

BA Bomb alert BF Buffering

BS Bird strike

DF Handling difficulties, refuel inop

DI Disconnecting, disengaging

EI Erractic indication

EM Emergency system failure

EX Parameter limit exceedance

FE Fire extinguisher activated

FF Fatigue failure

FI Fire (real or warning) FO Foreign object damage

FU Fuel system failure

FW False warning

HA Hail HL Hard landing

JM Jamming

LK Leaks

LL Low level LP Low pressure

LS Lightning strike

MF Multiple failure

OD Odour in cabin

OV Overheat PL Partial power loss

RO Runway overshoot SA Engine slow to accelerate

SD Significant primary structure failure or damage

SE Significant speed exceedance

SF System failure, emergency procedure, abnormal oper SG Engine stall SK Smoke (real or warning) TB Turbulance

UF Uncontained failure

UN Unlocked

VB Vibrations

Appendix – Coding detail

Name Description

AD Adjusted

CL Cleaned

DI De-iced

FD FOD

IN Inspected, checked

LB Lubricated

ME MEL applied

PR Purged, bleed

RK Reracked

RP Replaced

RR Repaired

RS Reset SR Serviced

SW Swapped

TG Tightened, secured


Name Description

AD Wrong Adjustment AW Abnormal wear BR Broken

BU Heat Damage

CH Chaffed

CK Crack

CL Clogged

CT Contaminated fluid

DB Debonding

DC Discard

DE Deformation

DI Disconnected

DL Delamination

DT Dirty

ER Erosion

FL Fluid Level Out of limit FN Furnishing Damaged

IC Ice Frozen

IN Fluid Ingress

JM Jammed

LB Lightbulb Burnt LE Leak

LP Pressure Out of limit LT Tension Out of limit MI Missing

NI Nil finding

OT Other PL Protection Damage

RT Corrosion

SC Scratches

SE Seal Damage

SP Separated

SV Incorrect servicing

TF Test Failure

TP TPS Damage

TW Tire Worn

UN Unlocked

UR Unreadable

Finding codes Action codes

Classification errors analyzis


From rules to regex patterns + metadescriptions

Understanding of a rule is easier through metadescriptions however rules are suffering from exceptions that require to

redefine them at low level with regular expressions

EX1 (leaking|leaks?|leakage|fluid)(?!.+((ck|chk|check|

(checked|chk'?d?) (normal|ok|good)))) => LK

EX2 (?<!(not|no|eng)[-:\s\w]{1,10})(apu)[-:\s\w]{1,40}(auto(matic)?)? ?s/?(hut)?(-|s |

)?(down?)? => AS

Both metadescription and reg expressions are relevant to support management of the system

e.g. LEAK* NOT NEAR #NEGATION => LK

APU_SHUTDOWN expressions NOT JUST BEFORE NEGATION => AS


LAV GALLEY zone


title of presentation - imdr-+roussel... · 2019-02-27 · ecam : cond lav+galley fan fault …...

Documents