automated quality control of geophysical time series

AUTOMATED QUALITY CONTROL OF GEOPHYSICAL TIME SERIES

The algorithmic systems developed at Geophysical Center of Russian Academy of Sciences are intended for recognition of disturbances with defined morphology on time series. These algorithms were applied to 1-minute and 1-second INTERMAGNET data for recognition of artificial disturbances on the magnetograms. INTERMAGNET network is the basis for geomagnetic field monitoring so requirements for reliability of collected data are very high. Therefore, an important task is an objective and formalized recognition and further elimination of possible anthropogenic anomalies in data records.

Fuzzy comparisonson positive numbers

Nearness in finitemetrical space

Limit in finitemetrical space

Densityas measureof limitness

Smoothingtime series

Monotonoustime series

Fuzzy logicand geometry on

time series:geometricmeasuresPredication

of time series.Forecast

Anomalies ontime series.

DRAS. FLARS. FCARS

Extremumson time series

Convex time series

Separation of denseSubset.

Crystal. Monolith

Clusterization.Rodin

Search of linearStructure.

Tracing

Morphological time series

analysis

Morphological correlation

Stochasticcorridors

Multidimensial time series

analysis

1999-2008

2009

2010-

Discrete Mathematical Analysis (DMA) Scheme

A spike is a chain of interrelated singular record fragments representing disturbances that are substantial vertically and insignificant horizontally and that do not lead to a shift of the recording level [Bogoutdinov et al., 2010].

Artificial disturbances on geomagnetic records

Example of spike recognition on 1-minute data (FRD, X, 2005)

Jump recognition on 1-minute data

Jump recognition on satellite magnetic data (GOES, 2 Hz)

The results of training and testing show that SP, SPs and JM algorithms are efficient enough to recognize almost all artificial spikes and jumps detected by data experts manually. This also provides the possibility to carry out retrospective analysis and quality control of the magnetograms available at ICSU World Data Centers.

SP algorithm block scheme

( , ) max | ( ) ( ) |: , [ , ]yO t y t y t t t t t

Brute-force search of free parameter values: 4 600 sets of values

Spike recognition on 1-minute INTERMAGNET magnetograms

A.A. Soloviev1, A. Chulliat2, R.V. Sidorov1, Sh.R. Bogoutdinov1

1-Geophysical Center RAS, Moscow, Russia; 2 – Insitiut de Physique du Globe de Paris, France

SP algorithm recognition results

Jump recognition on INTERMAGNET magnetograms

SPs recognition statistics

Sh.R. Bogoutdinov, A.D. Gvishiani, S.M. Agayan, A.A. Solovyev, E. Kihn, Recognition of Disturbances with Specified Morphology in Time Series. Part 1: Spikes on Magnetograms of the Worldwide INTERMAGNET Network, Izvestiya, Physics of the Solid Earth, 2010, Vol. 46, No. 11, pp. 1004–1016 A. Soloviev, A. Chulliat, S. Bogoutdinov, A. Gvishiani, S. Agayan, A. Peltier, B. Heumez (2012), Automated recognition of spikes in 1 Hz data recorded at the Easter Island magnetic observatory, Earth Planets Space, Vol. 64 (No. 9), pp. 743-752, 2012, doi:10.5047/eps.2012.03.004

1( , )

a b

b an a b

, where 0

A fuzzy comparison ( , )n a b of non-negative numbers a and b measures in the scale segment [ 1,1] the rate of superiority of “b ” over “a ”: segment [ 1,1] the rate of superiority of “b ” over “a ”:

If 1 20 NA a a a is a finite set and 0b , then

( , ) ( ) [ 1,1]n a b es a b .

( , )( ) ( , ) [ 1,1]i in a b

es A b n A bN

Example of spike recognition on 1-second data

Comparison with classical methods

SPs algorithm block schemeSpike recognition on 1-second magnetograms

Comparison with F methodComparison with statistical

algorithms

Components XYZ and intensity F

PeriodNumber

of obser-vatories

Events recognize

d

Target miss

False alarm

Learning 2007 7 290 0% 5.2%Exam 1 2008 5 110 1.0% 8.2%Exam 2 2003* 7 1032 0% 15.4%Exam 3 2005* 7 535 0.2% 14.6%

* Increased magnetic activity

Jump is an anomaly on a record leading to its baseline shift.

(rinf [ , ] lsup [ , ], rsup [ , ] linf [ , ])[ , ]

(linf [ , ] rsup [ , ],lsup [ , ] rinf [ , ])

n y a b y a b y a b y a bjmesA a b

n y a b y a b y a b y a b

1 2[ , ] ( )( ) [ , ] arg min [ , , ]

def

a b T Tj A a b jmes y a b

[ , ] 0.5jmesA a b

JM Algorithm: Calculating measures of jumpiness using fuzzy bounds

where rinf, linf are the fuzzy lower bounds, rsup, lsup are the fuzzyupper bounds,

Potential jump (red), wings (green), fuzzy bounds (black)

References:

[Soloviev et al., 2012]

~ 140 days

Natural geomagnetic pulsations on 1-second data