identiﬁcation of statistical patterns in complex systems ...statistical patterns in such systems,...

A

ppsTIvd

K

1

pneTrlidgfta

ISA Transactions®Volume 45, Number 4, October 2006, pages 477–490

0

Identification of statistical patterns in complex systems viasymbolic time series analysis

Shalabh Gupta,* Amol Khatkhate,† Asok Ray,‡ Eric Keller§

The Pennsylvania State University, University Park, PA 16802, USA

�Received 27 August 2005; accepted 26 February 2006�

bstract

Identification of statistical patterns from observed time series of spatially distributed sensor data is critical forerformance monitoring and decision making in human-engineered complex systems, such as electric power generation,etrochemical, and networked transportation. This paper presents an information-theoretic approach to identification oftatistical patterns in such systems, where the main objective is to enhance structural integrity and operation reliability.he core concept of pattern identification is built upon the principles of Symbolic Dynamics, Automata Theory, and

nformation Theory. To this end, a symbolic time series analysis method has been formulated and experimentallyalidated on a special-purpose test apparatus that is designed for data acquisition and real-time analysis of fatigueamage in polycrystalline alloys. © 2006 ISA—The Instrumentation, Systems, and Automation Society.

eywords: Fault/analysis; Forcasting; Goodness monitoring

ttpccddocpttoolivtte

. Introduction

Given a subsystem of a human-engineered com-lex system, the critical issue is whether its dy-amics can be adequately described by a math-matically and computationally tractable model.he presence of uncertainties and chaos may often

estrict routine applications of the fundamentalaws of physics to model such systems �1�. Thisssue has motivated the study of complex systemsuring the last few decades, which is continuallyaining importance from the perspectives of bothundamental sciences and technological applica-ions. Specifically, sole reliance on model-basednalysis for pattern identification in complex sys-

*E-mail address: [email protected]†E-mail address: [email protected]‡Corresponding author. E-mail address: [email protected]§
E-mail address: [email protected]
019-0578/2006/$ - see front matter © 2006 ISA—The Instrumentat

ems has been found to be infeasible because ofhe difficulties to achieve requisite accuracy andrecision of the nonlinear spatial-temporal sto-hastic models. For example, no existing modelan capture the dynamical behavior of fatigueamage at the grain level solely based on the fun-amental principles of molecular physics. More-ver, small deviations in the initial conditions andritical parameters of the system may eventuallyroduce large bifurcations and chaotic outputs inhe expected dynamical behavior �2�. Despitehese difficulties, the key problem—identificationf statistical patterns—can be formulated in termsf how the inherent dynamics of a complex non-inear �and possibly nonstationary� process can benferred from the data generated from sensing de-ices and ancillary instrumentation. In essence,ime series analysis of observed data is needed forracking the changes in statistical patterns of thevolving system dynamics in real time �3,4�.
Human-engineered complex systems, such as
ion, Systems, and Automation Society.

epmhtttidsTeiadvagTatp

paanbaAaacttn

asfmTpdtmaa�c

tfscdaiSalw

2

csrppcas

•

•

pestvtsn

478 Gupta, Khatkhate, Ray, and Keller / ISA Transactions 45, (2006) 477–490

lectric power generation plants, petrochemicallants, and networked transportation systems,ust be designed to be information-intensive and

ave capabilities for early detection and identifica-ion of any anomalous behavior to ensure struc-ural integrity and operation reliability. This objec-ive can be achieved via provision of diversenformation sources through a network of sensingevices and ancillary instrumentation, installed atelected spatial locations of the complex system.he information provided by the sensor networknhances the capacity of measuring small changesn the dynamical behavior of the system. For ex-mple, the information derived from time seriesata of sensors at different spatial locations pro-ides the capability to capture small parametricnd nonparametric perturbations at localized re-ions and their global effect on the entire system.hese sensors must provide online informationnd quantitative estimates of damage precursorshat cause the observed deviations in the statisticalattern.Various signal processing tools have been em-

loyed to extract useful information from thevailable time-series data. Technical literaturebounds with diverse techniques of pattern recog-ition �for example, see citations in Ref. �5��; arief survey of pattern recognition tools fornomaly detection is reported in Ref. �6�.nomaly detection using symbolic time series

nalysis �STSA� �7� has been recently reported �8�nd a comparative evaluation of this novel analyti-al method shows its superior performance rela-ive to other existing pattern recognition tools inerms of early detection of small changes in dy-amical systems �9,10�.This paper presents the theoretical framework ofnovel STSA-based method for identification of

tatistical patterns and its experimental validationor early detection of fatigue damage in structuralaterials of human-engineered complex systems.he experimental platform consists of a special-urpose electromechanical test apparatus that isesigned for data acquisition and analysis of fa-igue damage in polycrystalline alloys. The infor-

ation, needed for anomaly detection and damagenalysis, is derived from a network of distributednd heterogeneous fatigue damage sensing devicese.g., ultrasonic, mechanical displacement, loadell, accelerometer, and optical microscope�.
The paper is organized in six sections including o
he present one. Section 2 outlines the procedureor identification of statistical patterns in complexystems. Section 3 introduces the underlying con-epts of symbolic time series analysis for anomalyetection �8�. Section 4 describes the experimentalpparatus on which the anomaly detection methods validated for early detection fatigue damage.ection 5 discusses the results of early detectionnd identification of fatigue cracks under cyclicoading. The paper is concluded in Section 6 alongith recommendations for future research.

. The procedure for pattern identification

This section outlines the procedure for identifi-ation of statistical patterns in complex dynamicalystems. The procedure is based on the systemesponse, excited by selected input stimuli �e.g.,ersistent excitation�, or due to self excitation. Theattern identification of the quasistationary pro-ess is recognized as a two-time scale �i.e., fastnd slow time scale� problem in the followingense:

Fast time scale refers to the local behavior ofthe system, where changes in the patterns of theprocess dynamics are assumed to be insignifi-cant. It is assumed that the statistical distribu-tion of the system dynamics is stationary at thefast time scale �11�, i.e., no statistical changesoccur during this period.Slow time scale refers to the long-term behav-ior of the system, where the patterns of the pro-cess dynamics might deviate from those underthe nominal conditions. It is assumed that anyobservable nonstationary behavior pattern is as-sociated with changes occurring on the slowtime scale. The pattern changes, if they occur,develop gradually on the slow time scale andmay lead to accumulation of anomalies andfaults.

The notion of fast and slow time scales is de-endent on the specific application and operatingnvironment. In general, the time span in the fastcale, over which data series are collected, is ainy �i.e., several orders of magnitude small� inter-al in the slow time scale. While the statistics ofhe process dynamics are assumed to be locallytationary in these time intervals, it may exhibitonstationary statistics at different slow-time ep-
chs.

t

••

taepsaiq

•

•

eiwc

aagofTspl

3

STustT

•

•

•

3d

bii1ge

wcei�sFs

479Gupta, Khatkhate, Ray, and Keller / ISA Transactions 45, (2006) 477–490

Statistical pattern identification in the above set-ing is categorized by two inter-related problems:

the forward �or analysis� problem andthe inverse �or synthesis� problem.

The primary objective of the forward problem iso identify the patterns in the process dynamicsnd to track statistical changes occurring over thentire span of slow time. Specifically, the forwardroblem aims at detecting the deviations in thetatistical patterns in the time series data, gener-ted at slow-time epochs, from the nominal behav-or pattern. Solutions of the forward problem re-uire the following steps:

generation of multiple sensor time series datasets under self excitation, or under externalstimuli, spanning the system behavior underdifferent operational conditions andanalysis of the data sets to characterize the sys-tem’s behavioral pattern based on certain fea-tures at different slow-time epochs as the pro-cess evolves.

This paper addresses the forward problem andxperimentally validates the STSA-based patterndentification method on mechanical structures,here the source of possible anomalies is fatigue

rack damage.The inverse �or synthesis� problem infers the

nomalies based on the observed time series datand the information on anomaly characterization,enerated in the forward problem. The major rolef the inverse problem is to provide informationor monitoring and control of the system behavior.he inverse problem is currently under active re-earch and details on the solution method is re-orted in Ref. �8�; the results on the inverse prob-em will be reported in a forthcoming publication.

. STSA

This section briefly describes the technique ofTSA �7� for anomaly detection in real time �8�.he STSA method of anomaly detection makesse of the vector information on �finitely many�tates of an automaton generated by partitioninghe space over which the time-series data evolve.he steps for STSA are as follows:

transformation of time-series data from the s

continuous domain to the symbolic domain;construction of the finite state machine struc-ture based on symbolic sequences and calcula-tion of its state probability vector at varioustime epochs on the slow time scale; andstatistical pattern identification based on the de-viation of this vector information from thenominal condition.

.1. Transformation from continuous to symbolicomain

The time-series data are transformed into a sym-ol sequence by partitioning a compact region �n the phase space, over which the data evolves,nto finitely many discrete blocks as shown in Fig.. Let ��1 ,�2 , . . .�m� be a partitioning of the re-ion �, such that it is exhaustive and mutuallyxclusive set, i.e.,

�j=1

m

� j = � and � j � �k = � ∀ j � k

�1�

Each block � j is labelled as the symbol � j � �,here the symbol set � is called the alphabet set

onsisting of m different symbols. As the systemvolves in time, it travels through various blocksn its phase space and the corresponding symbol

j � � is assigned to it, thus converting a dataequence to a symbol sequence . . .�i1

�i2. . .�ik

. . ..ig. 1 exemplifies the partitioning of the phasepace where each block is assigned a particular

Fig. 1. An example of space partitioning.

ymbol such that a symbol sequence is generated

fTgOsag1i

3

p�nteoactntttesswivisssboracapb

wsft

aiwb

3

stswslolscfntlbrcantvdst

Fd


rom the phase space at a given slow time epoch.hus, the symbol sequences represent coarseraining of the trajectories’ time evolution �11�.nce the symbol sequence is obtained, the next

tep is the construction of the finite state machinend calculation of the state visiting probabilities toenerate state probability vector as shown in Fig.. The details of these steps are explained in com-ng sections.

.2. Wavelet space partitioning

Several partitioning techniques have been re-orted in literature for symbol generation3,12,13�, primarily based on symbolic falseeighbors. These techniques rely on partitioninghe phase space and may become cumbersome andxtremely computation intensive if the dimensionf the phase space is large. This paper has adopted

wavelet-based partitioning approach �14� be-ause wavelet transform �15� largely alleviateshese shortcomings and is particularly effective foroisy data from high-dimensional dynamical sys-ems. In this method, called wavelet space parti-ioning �8�, the time series data are first convertedo the wavelet transform data, where wavelet co-fficients are generated at different scales and timehifts. The graphs of wavelet coefficients versuscale, at selected time shifts, are stacked startingith the smallest value of scale and ending with

ts largest value and then back from the largestalue to the smallest value of the scale at the nextnstant of time shift. The arrangement of the re-ulting scale series data in the wavelet space isimilar to that of the time series data in the phasepace. The wavelet space is partitioned with alpha-et size �� into segments of coefficients on therdinate separated by horizontal lines such that theegions with more information are partitioned finernd those with sparse information are partitionedoarser. In this approach, the maximum entropy ischieved by the partition that induces uniformrobability distribution of the symbols in the sym-ol alphabet. Shannon entropy �16� is defined as

S = − �i=1

��

pi log�pi� , �2�

here pi is the probability of the ith state andummation is taken over all possible states. Uni-orm probability distribution is a consequence of
he maximum entropy partitioning. Fig. 2 shows s
n example of the maximum entropy partitioningn the wavelet space for alphabet size �� =6,here the partitioned regions are marked by sym-ols ranging from 0 to 5.

.3. State machine construction

The partitioning as described in the previousubsection is performed at �slow-time� epoch t0 ofhe nominal condition having zero anomaly mea-ure. A finite state machine is then constructed,here the states of the machine are defined corre-

ponding to a given alphabet � and windowength D. The alphabet size �� is the total numberf partitions while the window length D is theength of consecutive symbol words forming thetates of the machine �8�. The states of the ma-hine are chosen as all possible words of length Drom the symbol sequence, thereby making theumber n of states to be equal to the total permu-ations of the alphabet symbols within word ofength D, �i.e., n� ��D� where some states maye forbidden and have zero probability of occur-ence. The choice of �� and D depends on spe-ific experiments, noise level and also the avail-ble computation power. A large alphabet may beoise sensitive while a small alphabet could misshe details of signal dynamics. Similarly, a highalue of D is extremely sensitive to small signalistortions but would lead to larger number oftates requiring more computation power. Usinghe symbol sequence generated from the time-

ig. 2. Maximum entropy partitioning in the waveletomain.

eries data, the state machine is constructed on the

pb

qostmscM

csi�

Doppin=dsdo

l��ttsaqmpc

ri

fsa

spmpssOgsmsrNtopc

w

tstawsSptdttt


rinciple of sliding block codes �17� as explainedelow.The window of length D on the symbol se-

uence . . .�i1�i2

. . .�ik. . . is shifted to the right by

ne symbol, such that it retains the last �D−1�ymbols of the previous state and appends it withhe new symbol �i�

at the end. The symbolic per-utation in the current window gives rise to a new

tate. The machine constructed in this fashion isalled a D-Markov machine �8� because of itsarkov properties.Definition 3.1: A symbolic stationary process is

alled D-Markov if the probability of the nextymbol depends only on the previous D symbols,.e., P��i0

/�i−1. . .�i−D

�i−D−1. . . �= P��i0

/�i−1. . .

�i−D�.

The finite state machine constructed above has-Markov properties because the probability of

ccurrence of symbol �i�on a particular state de-

ends only on the configuration of that state, i.e.,revious D symbols. For example, if �= �0,1�,.e., �� =2 and D=2, then the number of states is� ��D=4; and the possible states are Q�00,01,10,11�, some of which may be forbid-en. Fig. 3 shows the construction of the finitetate machine for the above example where forbid-en states, if any, will have zero probability ofccurrence.Once the partitioning alphabet � and word

ength D are determined at the nominal conditiontime epoch t0�, they are kept constant for allslow time� epochs �t1 , t2 , . . . tk . . . �, i.e., the struc-ure of the machine is fixed at the nominal condi-ion. That is, the partitioning and the state machinetructure, generated at the nominal condition serves the reference frame for data analysis at subse-uent time epochs. The states of the machine arearked with the corresponding symbolic word

ermutation and the edges joining the states indi-

Fig. 3. Finite state automaton with D=2 and �= �0,1�.

ate the occurrence of an event �i�. The occur- a

ence of an event at a state may keep the machinen the same state or move it to a new state.

Definition 3.2: The probability of transitionsrom state qj to state qk belonging to the set Q oftates under a transition � :Q��→Q is defineds

jk = P�� qj,�� → qk�;�k

jk = 1. �3�

Thus, for a D-Markov machine, the irreducibletochastic matrix �ij� describes all transitionrobabilities between states such that it has atost ��D+1 nonzero entries. The left eigenvectorcorresponding to the unit eigenvalue of is the

tate probability vector under the �fast time scale�tationary condition of the dynamical system �8�.n a given symbol sequence . . .�i1

�i2. . .�il

. . .enerated from the time series data collected atlow time epoch tk, a window of length �D� isoved by keeping a count of occurrences of word

equences �i1. . .�iD

�iD+1and �i1

. . .�iDwhich are,

espectively, denoted by N��i1. . .�iD

�iD+1� and

��i1. . .�iD

�. Note that if N��i1. . .�iD

�=0, thenhe state q�i1

. . .�iD�Q has zero probability of

ccurrence. For N��i1. . .�iD

��0, the transitionsrobabilities are then obtained by these frequencyounts as follows:

jk P�qk�qj� =P�qk,qj�

P�qj�=

P��i1. . . �iD

��

P��i1. . . �iD

�

⇒ jk N��i1

. . . �iD��

N��i1. . . �iD

��4�

here the corresponding states are denoted by qj�i1

�i2. . .�iD

and qk�i2. . .�iD

�.The time series data under the nominal condi-

ion �set as a benchmark� generates the state tran-ition matrix nom that, in turn, is used to obtainhe state probability vector pnom whose elementsre the stationary probabilities of the state vector,here pnom is the left eigenvector of nom corre-

ponding to the �unique� unit eigenvalue �18�.ubsequently, state probability vectors1 ,p2 , . . .pk. . . are obtained at slow-time epochs

1 , t2 , . . . tk. . . based on the respective time seriesata. Machine structure and partitioning should behe same at all slow-time epochs. A deviation fromhe nominal behavior �e.g., the statistical distribu-ion at the nominal condition� is called an anomaly
nd these anomalies are characterized by a scalar

cfta

•

•

Adtcrastafi

wsowtitcata

4

b

diffbcsttt

4

assasT

t


alled anomaly measure �M�. In the context ofatigue crack phenomena in mechanical structures,he anomaly measure is based on the followingssumptions:

Assumption 1: The evolution of damage M�t�is an irreversible process, i.e., with zero prob-ability of self healing. This assumption impliesthe following conditions for all time t�0,

�i� M�0,�ii� dM /dt�0.

Assumption 2: The damage accumulation at aslow time epoch t, when the dynamical systemhas reached a quasisteady state equilibrium, isa function of the entire path taken to reach thatstate.

Let us digress in the context of fatigue damage.lthough the crack length has been traditionallyefined by a straight line joining the starting pointo the tip of the crack, the actual crack follows aomplicated path, possibly fractal in ductile mate-ials, to reach a particular point. Therefore, thebove assumption 2 implies that the anomaly mea-ure should be determined from the actual pathraversed and not just the end points. Accordingly,nomaly measure at a slow-time epoch tk is de-ned as:

Mk �l=1

k

d�pl,pl−1��;

�5�

d�x,y� = ��j=1

n

�xj − yj��1/�

,

here the exponent �� 1, � depends on the de-ired sensitivity to small deviations. Large valuesf � suppress small changes in the signal profilehich might be due to noise and spurious oscilla-

ions in the signal. Therefore, the choice of alphas a trade-off between suppression of small fluc-uations due to noise and those due to the actualhanges in the signal profile resulting from dam-ge growth. In this paper, the exponent is choseno be �=2 implying energy equivalence of thenomaly measure.

. Experimental validation on a test apparatus

This section validates the concept of STSA-
ased pattern identification for detection of fatigue o
amage in mechanical structures. With the aim ofnvestigating changes in statistical patterns due toatigue damage and consequent decision makingor damage reduction, a laboratory apparatus haseen designed to introduce fatigue damage in criti-al components �19�. These components are de-igned to break in a reasonably short period ofime to enhance the speed of experiments. Fromhis perspective, the design requirements of theest apparatus include:

�i� operability under cyclic loading with mul-tiple sources of input excitation;

�ii� damage accumulation in test specimens �atselected locations� within a reasonable pe-riod of time with negligible damage inother components of the test apparatus;and

�iii� accommodation of multiple failure sites forcomparative evaluation of structural dam-age at various spacial locations.

.1. Description of the test apparatus

The test apparatus is designed and fabricated asthree-degree of freedom �DOF� mass-beam

tructure excited by oscillatory motion of twohakers. A schematic diagram of the test apparatusnd the instrumentation is shown in Fig. 4; dimen-ions of the pertinent components are listed inable 1.The test apparatus is logically partitioned into

wo subsystems: �i� the plant subsystem consisting

Fig. 4. Schematic diagram for the test apparatus.

f the mechanical structure including the test

stmccst�m

tsammmeca

tticjsigpiscT

laifismsde

4s

ptdcstoiacRvea

macv�iDgtep

TS

C

MM

MBB

S


pecimens to undergo fatigue crack damage, ac-uators and multiple sensors and �ii� the instru-entation and control subsystem consisting of

omputers, data acquisition and processing, andommunications hardware and software. The sen-ors include: two peizoelectric accelerometers,wo linear variable displacement transducersLVDT�, and two load cells for force measure-ent.Two of the three major DOFs are directly con-

rolled by the two actuators, shaker No. 1 andhaker No. 2, and the remaining DOF is observ-ble via displacement and acceleration measure-ents of the three vibrating masses: mass No. 1,ass No. 2, and mass No. 3. The inputs to theultivariable mechanical structure are the forces

xerted by the two actuators; and the outputs to beontrolled are the displacements of mass No. 2nd mass No. 3.The three test specimens in Fig. 4 are represen-

atives of plant components, which are subjectedo fatigue crack damage. The mechanical structures excited at one or more of the resonant frequen-ies so that the critical component�s� can be sub-ected to different levels of cyclic stresses with noignificant change in the external power injectionnto the actuators. The excitation force vector,enerated by the two actuators, serves as the in-uts to the multi-DOF mechanical structure to sat-sfy the requirement No. 1. The failure site in eachpecimen, attached to the respective mass is a cir-ular hole of diameter 8.5 mm as shown in Fig. 5.

able 1tructural dimensions of the test apparatus.

omponent Material

Length �mm�Mass �kg� �length �

width �thickness�

ass 1 Mild steel 1.0ass 2 Aluminium

6061-T60.615

ass 3 Mild steel 2.2eam 1 Mild steel 800 � 25.4 � 12.7eam 2 Aluminium

6061-T6711.2 � 22.2 � 11.1

pecimens Aluminium6061-T6

203.2 � 22.2 � 11.1

he test specimens are thus excited by different b

evels of cyclic stresses as two of them are directlyffected by the vibratory inputs while the remain-ng one is subjected to resulting stresses, thusunctioning as a coupling between the two vibrat-ng systems. In the present configuration, three testpecimens are identically manufactured and theiraterial is 6061-T6 aluminum alloy. In future re-

earch, different materials will be selected for in-ividual specimens that may also undergo differ-nt manufacturing procedures.

.2. Hardware implementation andoftware structure

The electromechanical fatigue damage test ap-aratus is interfaced with Keithley Data Acquisi-ion Boards �DAS 17/18 STDA� having 6 analog/igital �A/D� channels and 4 digital/analog �D/A�hannels. Data acquisition is carried out with aampling rate at 500 Hz for monitoring and con-rol. The time-series data for statistical pattern rec-gnition are decimated as needed. The real-timenstrumentation and control subsystem of this testpparatus is implemented on a Pentium personalomputer �PC� platform. The software runs on theeal-Time Linux Operating System and is pro-ided with A/D and D/A interfaces to the amplifi-rs serving the sensors and actuators of the testpparatus.The control software performs real-time com-unication tasks, in addition to data acquisition

nd built-in tests �e.g., limit checks and ratehecks�. The data acquisition is an interrupt ser-ice routine �ISR� for the direct memory accessDMA� completion interrupt. The A/D board isnitialized to take 12 readings per frame. TheMA controller on the PC motherboard is pro-rammed to read 12 single 16-bit words and storehem sequentially in a given memory location forach transfer. When a reading is taken, the result isut into a first in, first out �FIFO� on the A/D

Fig. 5. Side view of failure site on the beam specimen.

oard and a DMA request is issued. The DMA

caiDbpctm

smtuoFRtpFsotidt

4

ttttsTfiolcdfictusfifsprpm


ontroller on the motherboard retrieves the datand stores it in the system random access memoryn sequence. After every 12th reading is stored, theMA controller asserts a signal that is loopedack to an interrupt line by the A/D board. At thisoint, control is given to the ISR. As the DMAontroller and A/D board are initialized to workogether, no periodic programming of either ele-

ent is necessary for subsequent data acquisition.Additional hardware devices consist of ultra-

onic sensors that provide information about theicrostructural changes that take place as the fa-

igue crack damage evolves. The user space forltrasonic data acquisition and processing consistsf a C program that connects through the DataIFO with the data acquisition kernel of theTLinux operating system and also stores the data

o a text file for off-line analysis. There is anotherrogram that connects through the CommandIFO and instructs the kernel to start or stop theystem, increase or decrease the gain, and increaser decrease the excitation frequency. Other addi-ional features of the software are a graphical usernterface as shown in Fig. 6 both for real-time dataisplay as well as for connecting to the stage mo-

Fig. 6. Graphical user

ion of the optical microscope. d

.3. Real time implementation

This subsection presents real-time implementa-ion of STSA-based fatigue damage detection inhe laboratory environment. The nominal condi-ion is chosen after the start of the experiment at aime epoch t0 when the system attains the steadytate and is assumed to be in a healthy condition.he states of the automaton �see Figs. 1 and 3� arexed in advance using a priori determined valuesf the parameters: alphabet size �� and windowength D. The algorithms for partitioning and ma-hine construction are based on the time-seriesata at the nominal condition t0. The resulting in-ormation �i.e., the partition and the state probabil-ty vector at this nominal condition� is stored foromputation of anomaly measures at future slow-ime epochs, t1 , t2 , . . . , tk , . . . that are separated byniform or nonuniform intervals of time. The timeeries data of multiple sensors are written on textles so that the STSA algorithm can read the datarom the text files to calculate the anomaly mea-ure at those time epochs. The algorithm is com-utationally fast for real-time execution and theesults can be plotted on the screen such that thelot updates itself with the most recent anomalyeasure at that particular time epoch. This proce-

ce of control software.
interfa
ure allows on-line condition monitoring at any

t

4

eliflolciifeteoisfdracwiapttttatTsi

4

ftdstaott

fluttspmwao

acroacdsfcwmtthdul


ime and is capable of issuing early warnings.

.4. Generation of fatigue crack damage

The mechanical system in Fig. 4 is persistentlyxcited near resonance so as to induce a stressevel that causes fatigue damage, ultimately lead-ng to failure. The applied stress is dominantlyexural �i.e., bending� in nature and the amplitudef oscillations is symmetrical about the zero meanevel �i.e., a reversed stress cycle �20��. Under cy-lic loading, the specimens undergo fatigue crack-ng where the far-field stress is elastic and plastic-ty is only localized near the crack site. Theatigue damage occurs at a time scale that is �sev-ral order of magnitude� slow relative to the fastime scale dynamics of the vibratory motion andventually leads to a catastrophic failure. Closebservation indicates that fatigue failure developsn the following sequence: �i� the repeated cyclictress causes incremental crystallographic slip andormation of persistent slip bands; �ii� gradual re-uction of ductility in the strain-hardened areasesults in the formation of submicroscopic cracks;nd �iii� the notch effect of the submicroscopicracks concentrates stresses to form a large crackhich grows till complete fracture occurs. Crack

nitiation may occur at a microscopic inclusion ort site�s� of local stress concentration. In this ex-erimental apparatus, the sites of stress concentra-ion are localized by creating a hole in each of thehree specimens. Since the mechanical structure ofhe test apparatus consists of beams and masses,he underlying dynamics can be approximated byfinite set of first order coupled differential equa-

ions with parameters of damping and stiffness.he damping coefficients are very small and thetiffness constants slowly change due to the evolv-ng fatigue crack.

.5. Sensors for damage detection

The apparatus is equipped with multiple sensorsor damage detection including ultrasonic flaw de-ectors and an optical microscope for localizedamage sensing. The advantages of using ultra-onic transducers over optical microscope includehe ease of installation at the desired damage sitend detection of early anomalies before the onsetf widespread fatigue crack propagation. Never-heless, the optical microscope serves to calibrate
he ultrasonic flaw detectors. d
(a) Ultrasonic flaw detector—The ultrasonicaw detector functions by emitting high frequencyltrasonic pulses that travel through the specimeno the receiver transducers �21�. A piezoelectricransducer is used to inject ultrasonic waves in thepecimen and a single receiver transducer islaced on the other side of the circular notch toeasure the transmitted signal. The ultrasonicaves produced were 5 MHz sine wave signals

nd they were emitted during a very short portionf every load cycle.The sender and receiver ultrasonic transducers

re placed on two positions, above and below theentral notch, so as to send the signal through theegion of crack propagation and receive it on thether side, as shown in Fig. 7. Since material char-cteristics �e.g., voids, dislocations, and shortracks� influence the ultrasonic impedance, minuteamage in the specimen is likely to change theignature of the signal at the receiver end. There-ore, the signal can be used to capture smallhanges during the early stages of fatigue damage,hich may not be possible to detect by an opticalicroscope. Prior to the appearance of a crack on

he specimen surface, deformations �e.g., disloca-ions and short cracks� inside the specimen mayave already caused detectable attenuation and/oristortion of the ultrasonic waves. Therefore, theltrasonic sensors are utilized for generating theocalized information of the growth of fatigue

Fig. 7. Ultrasonic flaw detection scheme.

amage.

m6ofc

tststtctfomsadstra

scodfiitsafc

5a

m

aicisteasFcatfuasailswfifiwdsnac

TS

S

LAL

U


(b) Optical microscope—The traveling opticalicroscope, shown as part of the schematic in Fig.

, provides direct measurements of the visible partf a surface crack. The microscope can be shiftedrom left to right side and vice versa to track therack tip on the specimen.(c) Accelerometers, load cells, and displacement

ransducers — As the body of the accelerometer isubjected to vibrations, the embedded piezoelec-ric crystal is continually compressed andtretched. The fluctuating force, which is propor-ional to the instantaneous acceleration, generatesime-dependent electric charge that, in turn, isonverted to a voltage signal. The load cell func-ions on a similar principle and is primarily usedor measuring the time-dependent forces exertedn the vibrating structure. LVDT’s are alsoounted on the test apparatus to measure the in-

tantaneous positions of the masses. These sensorsre useful for analyzing the effects of local fatigueamage on the global performance of the vibratoryystem. The locations of various sensors on theest apparatus are listed in Table 2. Raw time se-ies data plots for four sensors �ultrasonic, LVDT,ccelerometer, and load cell� are shown in Fig. 8.It is observed that the crack always starts at the

tress-concentrated region on the surface near theenter of the circular notch but the exact locationf the origin of the crack can be treated as a ran-om event. Formation of very small cracks is dif-cult to detect and model due to large material

rregularities. This paper focuses on integratinghe information generated from both the ultrasonicensors and the sensors measuring the global vari-bles like accelerometers, LVDTs, and load cellsor characterization of fatigue damage in the smallrack regime.

. Results and discussion onnomaly detection

STSA method has demonstrated superior perfor-

able 2ensor mountings.

ensor Location

VDT Mass 1, mass 3ccelerometer Mass 1, mass 3oad cell Shaker 1, shaker 2

ltrasonic transducersSpecimen between mass 1

and mass 3

ance �in terms of early detection of evolving a

nomalies, and robustness� relative to other exist-ng pattern recognition tools such as principalomponent analysis and artificial neural networks,n both electronic systems �10� and mechanicalystems �9�. This section describes the experimen-al validation of the symbolic dynamic tools forarly detection and quantification of fatigue dam-ge based on time-series data, generated from theensors mounted on the test apparatus as shown inig. 4. In the experiments, both shakers are ex-ited by a sinusoidal input of amplitude 0.85 Vnd frequency 14.09 Hz �89 rad/s� throughouthe run of each experiment. The time-series datarom displacement, accelerometer, load cell andltrasonic sensors, mounted on the experimentalpparatus �see Table 2�, were collected from thetart of the experiments after the system responsettained the stationary behavior. The results shownn Fig. 9 are based on the analysis of data col-ected from sensors mounted on mass No. 1 andhaker No. 1. The time-series data for each sensorere saved for a total period of 90 kc in 110les. 1 min of time-series data was stored in eachle for all sensors. The total life of the specimenas 110 min corresponding to 90 kc. Theata set at the beginning stage of experimentserved as the reference point representing theominal behavior of the dynamical system. Thenomaly measure at the nominal condition washosen to be zero and was subsequently updated at

Fig. 8. Raw time series data for four different sensors.

pproximately 3 min intervals.

wSs�sdtatvovmewregwcn

odscdgs�C

tptc9vs

stc2hzptrmapcrtcsg

siv

Fa F

s


The alphabet size, window length, and theavelet basis were chosen as �� =8, D=1 �seeec. 3� and the wavelet basis “gaus2” �22�, re-pectively, for time-series data sets of all sensorsdisplacement, accelerometer, load cell, and ultra-onic�. Absolute values of the wavelet scale seriesata �see Sec. 3.2� were used to generate the par-ition because of the symmetry of the data setsbout their mean. Increasing the value of �� fur-her did not improve the results and increasing thealue of depth D created a large number of statesf the finite state machine, many of them havingery small or zero probabilities. The finite stateachine constructed with the choice of the param-

ters �� =8 and D=1 has only eight states and itas able to capture early anomalies. The algo-

ithm is computationally very fast relative to thevolution of fatigue damage. The wavelet basis ofaus2 provided better results than many otheravelets of the Daubechies family �15� because it

losely matches the shape of the sinusoidal sig-als.The three plots in Fig. 9 represent the evolution

f anomaly measure derived from the time-seriesata of accelerometer, load cell and displacementensors, respectively. These sensors measure thehanges in the global performance of the systemue to evolving fatigue damage in a localized re-ion of the system. Fatigue crack growth in thepecimen connecting mass No. 1 and mass No. 3see Table 2� changes the stiffness of the material.

ig. 9. Normalized anomaly measure plots derived fromccelerometer, load cell, and displacement sensors.

onsequently, the dynamics of the system are al-

ered because of coupling between different com-onents of the apparatus �see Fig. 4�. As a result,he time-series data of these sensors recordhanges in the statistical patterns as shown in Fig.. The three plots in Fig. 9 are normalized by di-iding the anomaly measure values with the re-pective maximum for comparative evaluation.Fig. 10 exhibits the normalized anomaly mea-

ure plot derived from the time-series data of ul-rasonic sensors that are mounted on the specimenonnecting mass No. 1 and mass No. 3 �see Table�. This specimen is subjected to a relativelyigher load because of resonance and synchroni-ation of two shakers and therefore has higherrobability of failure. Ultrasonic sensors capturehe effects of small fatigue growth in the localizedegion of the mechanical structure. The growth ofultiple small cracks inside the specimen causes

ttenuation of the ultrasonic waves. The crackropagation stage starts as multiple small cracksoalesce into a single large crack. At this point, aapid change is observed in the profile of the ul-rasonic signal. Finally, the specimen breakageauses complete attenuation of the received ultra-onic signal because of high impedance of the airap.As seen in Fig. 10, ultrasonic sensors capture

low progression of fatigue damage in the local-zed region of the mechanical structure from theery beginning. The sharp change in the slope at

ig. 10. Normalized anomaly measure derived from ultra-onic sensors.

58 min indicate the transition from the crack

irospdtpdncctsmasedpitgomaslm

rdsgfssmtf

6

tot�btga

pfTitpips

tdsl

•

•

•

rosantsmrgm

A

UA0

R


nitiation phase to crack propagation phase. Theesults from displacement, load cell, and acceler-meter sensors, as seen in the three plots of Fig. 9how similar trends for all three sensors. Theselots show little change in the anomaly measureuring the crack initiation phase because small fa-igue cracks do not significantly affect the globalerformance; subsequently, the effects of fatigueamage on the overall system behavior becomeoticeable. The three plots in Fig. 9 exhibit a slopehange at 45 min. At this point, the stiffnesshange in the specimen caused major change inhe dynamics of the system as recorded by theseensors. A change in the stiffness constant of theaterial alters the natural frequency of vibrations

nd thereby affecting the global dynamics of theystem. These results for localized damage and itsffect on the global performance of the system in-icate that a control strategy can be built to takeredictive actions, based on the derived damagenformation, for fault mitigation and control. Inhis experiment, the STSA method captured theradual progression of faults much earlier than theccurrence of catastrophic failure. This is of para-ount practical importance as it can provide

mple time for the hierarchical supervisory controlystem to execute decision and control laws forife extension without significant loss of perfor-

ance �19�. This is an area of future research.It is emphasized that the anomaly measure is

elative to the nominal condition of that particularata set and may not represent damage in the ab-olute sense. Any value of anomaly measurereater than zero just indicates some deviationrom the nominal condition and it signifies thatome small faults might have occurred inside thepecimen. However, inferring anomalies and for-ulation of a decision and control law for life ex-

ension is an inverse problem and is a topic ofuture work.

. Summary, conclusions, and future work

This paper presents the concept and experimen-al validation of a novel method for identificationf statistical patterns in complex dynamical sys-ems. The underlying principle is based on STSA7,8� of observed process variable�s�, which isuilt upon the concepts of symbolic dynamics, au-omata theory, and information theory. The histo-rams on state probability distribution are gener-
ted from the observed time-series data to serve as
atterns of the evolving behavior change resultingrom stiffness reduction due to fatigue damage.he capability of the proposed STSA method for

dentification of behavior patterns is experimen-ally validated on a special-purpose laboratory ap-aratus by a demonstration of how fatigue damagenformation can be extracted in real time from dis-lacement, accelerometer, load cell, and ultrasonicensor signals.The proposed method of statistical pattern iden-

ification is useful for performance monitoring andecision making in human-engineered complexystems. The main features are summarized be-ow:

Information extraction from observed time se-ries data in real time;detection and identification of gradually evolv-ing anomalous behavior due to progression offaults; andearly warnings on incipient faults to avert cata-strophic failures.

The reported work is a step towards building aeliable instrumentation system for early detectionf faults in human-engineered complex systems,uch as electric power generation, petrochemical,nd networked transportation. Further research isecessary before its usage in industry. The utiliza-ion of the information provided by anomaly mea-ure for appropriate control action for damageitigation is an area of future work and would

equire stochastic analysis of multiple data setsenerated under identical loading and environ-ental conditions.

cknowledgments

This work has been supported in part by the.S. Army Research Laboratory and the U.S.rmy Research Office under Grant No. DAAD19-1-1-0646.

eferences

�1� Badii, R., and Politi, A., Complexity HierarchicalStructures and Scaling in Physics. Cambridge Univer-sity Press, Cambridge, 1997.

�2� Ott, E., Chaos in Dynamical Systems. Cambridge Uni-versity Press, Cambridge, 1993.

�3� Abarbanel, H. D. I., The Analysis of Observed ChaoticData. Springer-Verlag, New York, 1996.

�4� Kantz, H., and Schreiber, T., Nonlinear Time Series
Analysis. 2nd ed., Cambridge University Press, Cam-

�

�

�

�

�

�

�

�

�

�

�

�

�

scst

simtS

A4rrAatafcb


bridge, 2004.�5� Duda, R., Hart, P., and Stork, D., Pattern Classifica-

tion. Wiley, New York, 2001.�6� Markou, M., and Singh, S., Novelty detection: a

review—Parts 1 and 2, Signal Process. 83, 2481–2521�2003�.

�7� Daw, C. S., Finney, C. E. A., and Tracy, E.R., A re-view of symbolic analysis of experimental data, Rev.Sci. Instrum. 74 �2�, 915–930 �2003�.

�8� Ray, A., Symbolic dynamic analysis of complex sys-tems for anomaly detection, Signal Process. 84 �7�,1115–1130 �2004�.

�9� Gupta, S., Ray, A., and Keller, E., Symbolic time se-ries analysis of ultrasonic data for early detection offatigue damage, Mech. Syst. Signal Process. �in press�.

10� Chin, S., Ray, A., and Rajagopalan, V., Symbolic timeseries analysis for anomaly detection: A comparativeevolution, Signal Process. 85 �9�, 1859–1868 �2005�.

11� Beck, C., and Schlogl, F., Thermodynamics of ChaoticSystems: an Introduction. Cambridge University Press,Cambridge 1993.

12� Kennel, M. B., and Buhl, M., Estimating good discretepartitions form observed data: Symbolic false nearestneighbors, Phys. Rev. E 91�8�, 084102 �2003�.

13� Davidchack, R. L., Lai, Y. C., Bolt, E. M., and Dha-mala, H., Estimating generating partitions of chaoticsystems by unstable periodic orbits. Phys. Rev. E 61,1353–1356 �2000�.

14� Rajagopalan, V., and Ray, A., Wavelet-based spacepartitioning for symbolic time series analysis, Pro-ceedings of 44th IEEE Conference on Decision andControl and European Control Conference (Seville,Spain), December 2005.

15� Mallat, S., A Wavelet Tour of Signal Processing 2/e.Academic, New York, 1998.

16� Cover, T. M., and Thomas, J. A., Elements of Infor-mation Theory. Wiley, New York, 1991.

17� Lind, D., and Marcus, M., An Introduction to SymbolicDynamics and Coding. Cambridge University Press,Cambridge, 1995.

18� Plemmons, R. J., and Berman, A., Nonnegative Matri-ces in the Mathematical Science. Academic, NewYork, 1979.

19� Zhang, H., Ray, A., and Phoha, S., Hybrid life extend-ing control of mechanical systems: Experimental vali-dation of the concept. Automatica 36 �1�, 23–36�2000�.

20� Klesni, M., and Lukas, P., Fatigue of Metallic Materi-als. Material Science Monographs 71, Elsevier, NewYork, 1992.

21� Keller, E. E., and Ray, A., Real-time health monitor-ing of mechanical structures, Struct. Health Monit.2�3�, 191–203 �2003�.

22� Wavelet Toolbox, Matlab, Mathworks Inc.

Shalabh Gupta received aB.Tech �honors� in mechanicalengineering from Indian Instituteof Technology, Roorkee, India inthe year 2001. He has two M.Sdegrees, one in mechanical engi-neering and the other in electricalengineering, both from Penn StateUniversity. He is currently en-rolled in the doctoral program inthe Mechanical Engineering De-partment under the supervision ofDr. Ray. His general research in-terests include: control theory,

ignal processing, pattern recognition, fault diagnosis, and statisti-al mechanics. He is currently working as a graduate research as-istant at the Electromechanical Systems Laboratory �EMSL� inhe Mechanical Engineering Department of Penn State University.

Amol Khatkhate received a B.E.in mechanical engineering fromVeer Mata Jijabai TechnologicalInstitute �VJTI�, Mumbai, India inthe year 2001. He currently hastwo M.S degrees, one in mechani-cal engineering and the other inelectrical engineering, both fromPenn State University. The topicof his dissertation is “AnomalyDetection and Life ExtendingControl in ElectromechanicalSystems.” His general researchinterests include: control theory,

ignal processing, and pattern recognition. His specific areas ofnterest include diagnostics, prognostics and structural health

onitoring. He is currently working as a graduate research assis-ant at the Electromechanical Systems Laboratory �EMSL�, Penntate.

Asok Ray received a Ph.D. de-gree in mechanical engineeringfrom Northeastern University,Boston, MA, and also graduatedegrees in each discipline of elec-trical engineering, mathematics,and computer science. Dr. Rayjoined the Pennsylvania StateUniversity in July 1985, and iscurrently a distinguished profes-sor of mechanical engineering.Dr. Ray had been a senior re-search fellow at NASA Glenn Re-search Center under a National

cademy of Sciences award. Dr. Ray has authored or coauthored00 research publications including about 180 scholarly articles inefereed journals such as transactions of ASME, IEEE, AIAA, andesearch monographs. Dr. Ray is a fellow of IEEE, a fellow ofSME, a fellow of World Innovation foundation �WIF�, and an

ssociate fellow of AIAA. Dr. Ray’s research experience and in-erests include: control and optimization of continuously varyingnd discrete-event dynamical systems, intelligent instrumentationor real-time distributed systems, and modeling and analysis ofomplex dynamical systems from thermodynamic perspectives inoth deterministic and stochastic settings.

olgB


Eric Keller has been a senior re-search associate at the AppliedResearch Laboratory at PennState University since June 2001.His research interests are in theareas of sensor networks, robot-ics, signal processing, and healthmonitoring systems. Dr. Keller re-ceived his doctoral degree in me-chanical engineering from PennState University in 2001, wherehis research focused on online de-tection of fatigue damage. From1983 to 1994, Dr. Keller was an

fficer in the U.S. Air Force where he worked in acquisition andogistics engineering. Dr. Keller received a M.S. in aerospace en-ineering from the Air Force Institute of Technology in 1985 and a.S. degree in mechanical engineering from Virginia Tech in 1982.

identiﬁcation of statistical patterns in complex systems ...statistical patterns in such systems,...

Documents