multivariate extensions of emd applications in data fusion ...mandic/d_mandic_hht_qingdao_2011.pdfc...

Multivariate Extensions of EMDApplications in Data Fusion and BCI

Danilo P. Mandic

Imperial College London, UKAckn: Naveed Ur Rehman, David Looney, Cheolsoo Park

[email protected], URL: www.commsp.ee.ic.ac.uk/∼mandic

c© D. P. Mandic The Third HHT Conference, Qingdao, China 1

About Imperial College


Outline

2 Concept of Data Fusion Via Fission

2 Empirical Mode Decomposition (EMD)

2 Hilbert-Huang reconstruction and examples

2 Complex EMD

2 Applications: Image Restoration and Image Fusion

2 Trivariate EMD: Methodology and Applications

2 Multivariate EMD (MEMD)

2 Filter bank property of MEMD

2 Noise-assisted MEMD

2 Conclusions


Problem Statement

Classic spectrum estimation: From a finite record of stationary datasequence, estimate how the total power is distributed over frequency.

Has found a tremendous number of applications:-

◦ Seismology – oil exploration, earthquake, Radar and sonar – location ofsources, Speech and audio – recognition, Astronomy – periodicities,Economy – seasonal and periodic components, Medicine – EEG, ECG

Modern view: From a finite or infinite record of non-stationary andnon-liner data sequence, estimate how the total power is distributed overtime-frequency.

What are the basis for this analysis

◦ Parametric: stochastic models (AR, MA, ARMA), ...

◦ Non-parametric: Fourier analysis, wavelets, Vigner-Ville, ...

◦ Data driven: adaptivity in the derivation of the bases


Bases for Signal Decomposition

“Linear methods” - signal analysed based on inner products with apredefined family of basis functions.

◦ Fourier: bases are sin and cos which are orthonormal and in generalrequire an infinite number of terms in the expansion

F{x[n]} =N−1∑

n=0

x[n]e−ωn

= < x, e > x = [x0, . . . , xN−1]T , e = [1, e2ω, . . . , e(N−1)ω]T

◦ Wavelet: assumes projecting on a pre-defined “mother” wavelet, whichshrinks and expands, thus bypassing some of the problems of Fourieranalysis, but not entirely

The bases (template functions) determine the properties of representationand influences the physical reading

e.g. “frequency bins” instead of “instantaneous frequency”


Real World Signals – ‘Nonlinearity’ and ‘Stochasticity’

a) Periodic oscillations b) Small nonlinearity c) Route to chaos

d) Route to chaos e) small noise f) HMM and others

Hence: we need to look into nonparametric representations ofnonlinear and nonstationary signals


Speech Example – saying “Matlab” – ‘specgramdemo‘

Frequency

time

M aaa t l aaa b

For every time instant “t”, the PSD is plotted along the vertical axis

Darker areas:- higher magnitude of PSD


STFFT of a speech signal

(wide band spectrogram) (narrow band spectrogram)

dB

Data=[4001x1], Fs=7.418 kHz

-50

-40

-30

-20

-10

0

10

20

30

20

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Fre

qu

en

cy,

kH

z

dB

50 100 150 200 250 300 350 400 450 500-5

0

5

Time, ms

Am

pl

515.5028 ms

0.0000 Hz

29.2416 dB

dB

Data=[4001x1], Fs=7.418 kHz

-40

-30

-20

-10

0

10

20

30

20

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Fre

qu

en

cy,

kH

z

dB

50 100 150 200 250 300 350 400 450-5

0

5

Time, ms

Am

pl

241.5745 ms

1.8545 kHz

3.2925 dB

(win-len=256, overlap=200, ftt-len=32) (win-len=512, overlap=200, ftt-len=256)


What is the Right Basis for Real World Data?

Consider

• Amplitude modulated signal x(t) = m(t) cos(ω0t) → m(t) - envelope

• Phase modulated signal x(t) = a cos(Φ(t)) → Φ(t) - phase

Problem: there is an infinite number of pairs [a(t),Φ(t)] s.t.m(t) cos(ω0t) = a cos(Φ(t))

Solution: an analytic transform z(t) = x(t) + H(

x(t))

Remark#1: z(t) cannot be real, as F(

z(t))

= 0 for ω < 0

Remark#2: Hilbert transform (analytic signal) makes it possible toassociate a unique pair

[

a(t),Φ(t)]

to any real x(t) = ℜ{

a(t)eΦ(t)}

Remark#3: For x(t) = a(t) cosΦ(t) ⇒ H{

x(t)}

= a(t) sinΦ(t)

Remark#4: From instantaneous phase Φ(t) → instantaneousfrequency

f(t) = dΦ(t)/dt

so we have an excellent resolution and do not depend on stationarity


MEMD vs STFFT for a speech signal

(STFFT spectrogram) (Hilbert-Huang spectra)

dB

Data=[4001x1], Fs=7.418 kHz

-60

-50

-40

-30

-20

-10

0

10

20

30

-6020

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

Fre

qu

en

cy,

kH

z

dB

50 100 150 200 250 300 350 400 450 500-5

0

5

Time, ms

Am

pl

515.5028 ms

0.0000 Hz

29.2416 dB

Fre

qu

en

cy

Time

MEMD

500 1000 1500 2000 2500 3000 35000

500

1000

1500

2000

2500

3000

3500

(win-len=256, overlap=200, ftt-len=256) (no. of directions=64)


Empirical Mode Decomposition (EMD): Introduction

2 Empirical Mode Decomposition has been recently introduced for thetime-frequency analysis of nonstationary and nonlinear signals

2 Projection-based techniques assume stationarity and/or linearity in theinput signal (Fourier and Wavelet approach), and hence, are notsuitable for non-linear, non-stationary data

2 The Empirical mode decomposition (EMD) algorithm is a fullydata-driven method which extracts the basis functions adaptively fromthe input data, through the so called sifting process

2 The adaptive nature of EMD also facilitates accurate time-frequencyrepresentation of the signal at the level of instantaneous frequency(through the Hilbert-Huang spectrum)

2 Standard EMD is limited to the analysis of single data channels -modern applications require its multichannel extensions

2 For data fusion ⇔ same number of IMFs for all data channels


Hilbert-Huang Spectrum: Results

Spectrogram and Hilbert–Huang Spectrum for a sum of two frequency modulated signals

and a tone. The Hilbert-Huang spectrum shown in (right) clearly shows better resolution

than the spectrogram shown on (left).

1000 2000 3000 4000 5000 6000 70000

500

1000

1500

2000

2500

3000

3500

4000

Spectrogram

Time Index

Fre

qu

en

cy (

Hz)

Fre

qu

en

cy (

Hz)

Time Index

Hilbert−Huang Spectrum

1000 2000 3000 4000 5000 6000 70000

500

1000

1500

2000

2500

3000

3500

4000


Vector sensors - Data Fusion - Synchrony

Renewable Energy Body motion sensor Wearable techologies

2D and 3D anemometers 3D - position, gyroscope, speed Biomechanics

control of wind turbine gait, biometrics virtual reality


Vector sensors - 3D anemometer


Applications: How do we Decompose - Fuse RGB

Computer Games Medical Applications Avionics

Rotation of polygons 3D time-space. Trajectory tracking of

to form 3D graphics. Data are naturally recorded in angular properties (eg. velocity).

2D and 3D electromagnetic field.


A General Data Fusion Problem

How do we make a mental ”image” of a meal

• taste bitter, rotten

• smell pleasant, spices

• vision colour, presentation, rare, medium, well done

• touch bread, toast

• hearing, temperature, toughness


Data Fusion via Fission

Input

FusionFission

IMFN

IMF2

IMF1

...

frequency

time

Output

2 Fission : Decomposion into “particles”

2 Fusion: Recombination of particles into the desired signal


Automated Data Fusion via Fission (MLEMD)

Standard EMD

1

maskbinary

IMFn

IMF3IMF2IMF1

EMDrecombinationsignal

input

...

0

11

00

EMD for Data Fusion via Fission

x(k)

M

c (k)1

Bivariate

Machine Learning

EMD

W(k)

y(k)

c (k)

Incorporating the scale and temporal information

Most Machine Learning or Adaptive Filtering algorithms can beincorporated (Looney and Mandic, ICASSP 2009).


Benefits of the Data Fusion Approach

The synergy of information fragments offers some advantages overstandard algorithms, such as:-

• Improved confidence due to complementary and redundant information;

• Robustness and reliability in adverse conditions (smoke, noise,occlusion);

• Increased coverage in space and time; dimensionality of the data space;

• Better discrimination between hypotheses due to more completeinformation;

• System being operational even if one or several sensors aremalfunctioning;

• Possible solution to the vast amount available information.


Some Literature


Image Enhancement and Fusion Using EMD

2 Image features (object texture or unwanted noise) can be attributed tolocal variations in spatial frequencies

2 Therefore, the behaviour of the extracted image modes can reflect thesefeatures

2 Correct fusion of the “relevant” IMFs can be used to highlight (orremove) specific image attributes

We consider the fusion capabilities of EMD under the following headings:

• Image Denoising

• Image Restoration (Illumination Removal)

• Image Fusion (of Images From Multiple Image Modalities)


Image Denoising

Noise contamination is a common problem when acquiring real worldimages and consequently image denoising is an important element ofimage processing. Many existing methods, however, are sub-optimal forthe following reasons:

2 They make unrealistic assumptions about the data (ICA - unrealisticindependence conditions, PCA - noise and original image can beseparated by linear projection)

2 They are not optimised for enhancing higher order (nonlinear) statistics,that are commonly associated with the perceptual quality of an image,and do not cater for other real world data characteristics such asnonstationarity (block based Weiner filtering)

2 They are computationally complex (Bayesian and particle models)


Image Denoising

Consider an original image corrupted by white Gaussian noise.

Original Contaminated (SR 12.3 dB)


Image Denoising

Decomposing the contaminated image by EMD, we obtain the following:

Note how each of ‘Image Modes’ represents the frequency scales within theimage. The higher index IMFs contain high frequency detail such as theimage edges while slowly oscillating effects such as illumination arecontained within the low index IMFs.


Image Denoising

Noisy image, SNR = 13 dB EMD of the image

0 100 200 300 400 5000

200400

imag

e

50 100 150 200 250 300 350 400 450 500−20

020

c 1

50 100 150 200 250 300 350 400 450 500−20

020

c 2

50 100 150 200 250 300 350 400 450 500−20

020

c 3

50 100 150 200 250 300 350 400 450 500−20

020

c 4

50 100 150 200 250 300 350 400 450 500−20

020

c 5

50 100 150 200 250 300 350 400 450 500−50

050

c 6

Time Index

Denoising using PREMD, SNR=17.5 dB Denoising using MLEMD, SNR=21 dB


The Method Naturally Deals With Texture

Cracked varnish on wood: Left – original; Midle – cracks; Right – wood pattern

Carpet: Left – original; Midle – texture; Right – carpet patternThe texture is separated naturally as higher frequency T-F components


Image Restoration (Illumination Removal)

2 A key problem for a machine vision system is image changes that occurdue to scene illumination

2 Incident light on a surface produces complex artifacts, making itdifficult for the system to separate changes caused by local variations inillumination intensity and colour

• It can be assumed that shade in images creates low valued regions withlarge extrema that change slowly

• It is therefore likely that the effects of the shade will be isolated in thelower index IMFs and a shade free image can be achieved by combiningthe relevant IMFs


Illumination Removal – Real World Objects

Image with shade Shade only Original image

Shade removal: the shading is now uniform across the image surfaceImage with shade Shade only Original image


Human Visual System – Importance of Phase

Information

Surrogate images. Top: Original images I1 and I2; Bottom: Images I1 and I2 generated

by exchanging the amplitude and phase spectra of the original images.


Image Fusion (From Multiple Image Modalities)

2 Image fusion is becoming an important area of research, particularly asdifferent methods of image acquisition become available

2 The fused image retains all “relevant information” from the differentsources while disregarding unwanted artifacts

• Given the unique “fission” properties of EMD, it has a strong potentialfor fusion

• We propose the use of complex EMD with the input images as real andimaginary components respectively

• The instantaneous amplitude of the extracted IMFs indicates, for eachfrequency level at each pixel, which of the components contains thesalient information. Fusion can be achieved by combining only IMFcomponents with the largest instanteous amplitudes.


Obstacles to Automatic Heterogeneous EMD Fusion

2 The fully adaptive and empirical nature of the algorithm compromisesthe uniqueness of the decomposition.

2 Signals with similar statistics often yield different IMFs (in both numberand frequency) - difficult to compare sources in T-F

Consider sinusoid corrupted by different realisations of AWGN. Note thethe difference in the IMFs.

0 100 200 300 400 500 600−1

01

U

0 100 200 300 400 500 600−1

01

C1+

C2

0 100 200 300 400 500 600−1

01

C3

100 200 300 400 500 600−1

01

C4

0 100 200 300 400 500 600−1

01

C5:

r

Time Index

0 100 200 300 400 500 600−1

01

U

0 100 200 300 400 500 600−1

01

C1+

C2

0 100 200 300 400 500 600−1

01

C3

100 200 300 400 500 600−1

01

C4

0 100 200 300 400 500 600−1

01

Time Index

C5:

r

Seven IMFs Eight IMFs


Obstacles to Automatic Heterogeneous EMD Fusion

Automatic fusion algorithms are necessary for widespread use!

But this is not often possible using standard EMD because

3 Uniqueness of the scales cannot be guaranteed;

3 Comparison of IMFs from different sources is meaningless!

Thus, automatic fusion of heterogeneous sources using EMD is onlypossible if their IMFs are

2 equal in number;

2 matched in properties (frequency).

[Rotation Invariant Complex EMD, Altaf, Mandic et al., 2007, Complex

EMD, Tanaka and Mandic, 2007, Bivariate EMD, Rilling, Flandrin andGoncalves, 2007]


Empirical Mode Decomposition: Underlying Idea

• The basic idea behind EMD is to consider an input signal as fastoscillations superimposed on slow oscillations.

• The fast oscillations are repeatedly sifted from the input signal until amonotonic signal (residue) is obtained.


Complex EMD - Local Mean Estimation

−200 −150 −100 −50 0 50 100 150 200 250−100

−50

0

50

100

150

200

X

Y

Original dataenvelope 1envelope 2Mean

(e) RIEMD

−200 −150 −100 −50 0 50 100 150 200 250−100

−50

0

50

100

150

200

X

Y

Original Dataenvelope 1envelope 2envelope 3envelope 4Mean

(f) BEMD

1

2

3

4

5

30

210

60

240

90

270

120

300

150

330

180 0

(g) A complex wind signal

1

2

3

4

5

30

210

60

240

90

270

120

300

150

330

180 0

(h) IMF6 + IMF7


Heterogeneous EMD Fusion

3 It was proposed [Looney and Mandic] to use the complex extensions ofthe algorithm to decompose heterogeneous sources simultaneously.

3 The approach may be used to find “common scales” within differentdata sets, thus addressing the problem of uniqueness.

Observe how common frequency scales are found in different signals (U1and U2) by applying complex extensions of EMD to (U1 + jU2).

20 40 60 80 100−4

04

U1

20 40 60 80 100−4

04

C2

0 20 40 60 80 100−4

04

C1

20 40 60 80 100−4

04

C3

20 40 60 80 100−4

04

Time Index

C4:

r

20 40 60 80 100−4

04

U2

20 40 60 80 100−4

04

C1

20 40 60 80 100−4

04

C2

20 40 60 80 100−40 4

C3

20 40 60 80 100−4

04

Time Index

C4:

r

Real Imaginary


Fusion Results [Looney and Mandic ICDSC’08)

Visual Thermal Pixel Average Fusion

PCA Fusion Wavelet Fusion Complex EMD Fusion


Out of focus image fusion using complex EMD

A (original) B (original) Out of focus fusion

A (original) B (original) Fusion

Looney and Mandic, IEEE Transactions on Signal Processing, April 2009.


Complex EMD vs Wavelets

EMD fusion Wavelet fusion

The wavelets produce artifacts - around the text visible as shaded “boxes”


Fusion of Exposure Images (gray scale) using Complex

EMD: Methodology

2 Two input gray scale images are converted into vectors by concatenating their rows, to

form a complex signal.

2 The complex signal is processed using the bivariate EMD algorithm, resulting in

multiple complex IMFs.

2 Scale images corresponding to each source are then combined locally using the local

fusion algorithm to give a fused image.

BIVARIATE EMD

Image A

Image B

B1

B2

BM

Fusion Based on Local

Variance

Fused Image

A1

A2

AM

(Gray-scale image fusion methodology using Bivariate EMD.)


Fusion of Exposure Images (gray scale) using Complex

EMD: Results

(Input image 1) (Input image 2)

(Wavelet based image fusion) (EMD based image fusion)


Fusion of Exposure Images (colored RGB) using

Complex EMD: Methodology

2 The three channels (red, green, and blue) of a color image are processed by separate

applications of bivariate EMD algorithm.

2 Three sets of complex-valued IMFs are obtained which correspond to the red, green,

and blue channels of the input images.

2 Each set is processed separately by the fusion algorithm to yield the fused red, green

and blue channel, which are combined to yield a fused RGB image.

a R

b R

a G a B

b G b B

BIVARIATE EMD

BIVARIATE EMD

BIVARIATE EMD

Image A

Image B

R1

R2

RL

G1

G2

GM

B1

B2

BN

Fusion based on Local Variance



R

G

B

Fused RGB

Image

b B

(RGB color image fusion methodology using Bivariate EMD.)


Complex EMD-based Colored (RGB) Image Fusion:

Results

(Input image 1) (Input image 2)

(‘Local’ Image Fusion) (EMD based image fusion)


Other Possibilities – “Enviromental Dimension”

Visual Thermal Image Fusion

From “Image Fusion and Enhancement via Empirical Mode Decomposition”, H. Hariharan

et al., Journal of Pattern Recognition Research, 2006

Here, the fusion was performed manually, without using any machinelearning or extensions of EMD.


Trivariate EMD (TEMD): Underlying Idea

2 Designed to extend EMD to process trivariate signals.

2 Basic Idea: A trivariate signal is considered as a combination of a signal with fast 3D

rotating component superimposed on a slowly rotating component.

2 Local mean signal is considered as a slowly rotating component.

2 Huangs sifting algorithm is used to extract 3D rotating modes in a trivariate signal.

D. Looney and D. P. Mandic, IEEE Transactions on Signal Processing, 2009


TEMD: Local Mean Estimation

2 To calculate the local mean, multiple projections of the input signal are taken, with

each corresponding to a particular direction in 3D space.

2 The extrema of the projected signal are interpolated to yield quaternion-valued

envelopes, which are then averaged to yield an estimate of the local mean:

m(t) =1

KN

K∑

k=1

N∑

n=1

pφnθk

(1)

where pφnθk

denotes the projections along the direction, represented by {θk, φn}, in

3-dimensional space; K and N represent the total number of directions taken along

directions θ and φ respectively.

−200 −150 −100 −50 0 50 100 150 200 250−100

−50

0

50

100

150

200

X

Y

Original Dataenvelope 1envelope 2envelope 3envelope 4Mean

(Multiple envelopes along a multivariate signal.)


TEMD: Choice of direction vectors

2 The choice of the set of direction vectors for generating multiple envelopes is a crucial

step.

2 The direction vectors are chosen along multiple longitudinal lines on a sphere, to

encompass the whole 3D space; a direction vector is represented by a point on the

sphere.

−1

0

1

−1

0

1 −1

0

1

XY

Z

O

A

(a) A direction vector in 3D space


TEMD: Tai-Chi data analysis

−100 0 100−100−50050−100

−50

0

50

XY

Z

0 100 200 300 400−100

0

100

Inpu

t

−50 0 50−50050−50

0

50

XY

Z

0 100 200 300 400−50

0

50

IMF1

−5 0 5−505

−2

0

2

XY

Z

0 100 200 300 400−5

0

5

IMF2

−5 0 5−10010−5

0

5

XY

Z

0 100 200 300 400−10

0

10

IMF3

−100 0 100−100−50050−100

−50

0

50

XY

Z

0 100 200 300 400−100

0

100

Time (Samples)

Resid

ue

(a) 3D plots of decomposed components (b) Decomposition of individual components of the input trivariate signal


Fusion of light exposure images (colored RGB) using

TEMD: Methodology

2 The three channels (red, green, and blue) of a color image are processed by separate

applications of TEMD algorithm.

2 Three sets of quaternion-valued IMFs are obtained which correspond to the red, green,

and blue channels of the input images.

2 Each set is processed separately by the fusion algorithm to yield the fused red, green

and blue channel, which are combined to yield a fused RGB image.

aR

bR

cR

aG

aB

bG

bB

cG

cB

TRIVARIATE

EMD

TRIVARIATE

EMD

TRIVARIATE

EMDImage A

Image B

Image

C

R1

R2

RL

G1

G2

GM

B1

B2

BN

Fusion based

on local

Variance

Fusion based

on local

Variance

Fusion based

on local

Variance

R

G

B

Fused

RGB

Image

(RGB color image fusion methodology using TEMD.)


Multiple Color Image fusion using TEMD: Example

(Input Image 1) (Input Image 2) (Input Image 3)

(Fused Image)


Multivariate EMD: Direction vectors on a 3D sphere

2 To generate a more uniform pointset in multidimensional spaces, the set of direction

vectors generated via Hammersley sequence is used.

2 Direction vectors for taking projections of trivariate signals on a sphere are shown for

(a) spherical coordinate system; (b) a low-discrepancy Hammersley sequence are

shown.

−1

0

1

−1

0

1−1

0

1

−1

0

1

−1

0

1−1

0

1

(a) (b)


Multivariate EMD: Direction vectors on a 4D

hypersphere

Direction vectors on a hypersphere WXYZ generated by using low discrepancy

Hammersley sequence (a, b and c), and equiangular coordinate system (d, e and f), are

shown.‘

−1

0

1

−1

0

1−1

0

1

WX

Y

(a)−1

0

1

−1

0

1−1

0

1

XY

Z

(b)−1

0

1

−1

0

1−1

0

1

WY

Z

(c)

−1

0

1

−1

0

1−1

0

1

WX

Y

(d)−1

0

1

−1

0

1−1

0

1

WY

Z

(e)−1

0

1

−1

0

1−1

0

1

XY

Z

(f)


Multivariate EMD: Illustration on a Hexavariate Signal

Original

signal

IMF1

IMF2

IMF3

IMF4

0 500 1000−5

0

5

U0 500 1000

−5

0

5

V

0 500 1000−5

0

5

W

0 500 1000−5

0

5

X

0 500 1000−5

0

5

Y

0 500 1000−5

0

5

Z

0 500 1000−2

0

2U 1

0 500 1000−2

0

2

V 1

0 500 1000−2

0

2

W1

0 500 1000−2

0

2

X 1

0 500 1000−5

0

5

Y 1

0 500 1000−5

0

5

Z 1

0 500 1000−2

0

2

U 2

0 500 1000−2

0

2V 2

0 500 1000−2

0

2

W2

0 500 1000−2

0

2

X 2

0 500 1000−2

0

2

Y 2

0 500 1000−2

0

2

Z 2

0 500 1000−2

0

2

U 3

0 500 1000−2

0

2

V 3

0 500 1000−2

0

2W

3

0 500 1000−2

0

2

X 3

0 500 1000−2

0

2

Y 3

0 500 1000−2

0

2

Z 3

0 500 1000−2

0

2

Time Index

U 4

0 500 1000−2

0

2

Time Index

V 4

0 500 1000−2

0

2

Time Index

W4

0 500 1000−2

0

2

Time Index

X 4

0 500 1000−5

0

5

Time Index

Y 4

0 500 1000−2

0

2

Time Index

Z 4


MEMD: Decomposition of a Hexavariate Tai Chi signal

The proposed algorithm is applied to the body motion data recorded in a Tai Chi

sequence. The data was captured using two inertial 3D MTx sensors attached to the left

hand and the left ankle of an athlete; these were combined to form a single hexavariate

signal.


Rotational modes for 3D+3D bodysensor Tai-Chi data

−1000

100

−1000

100−200

0

200

X

Original (left hand)

YZ

−2000

200

050

100−200

0

200

X

Original (left ankle)

Y

Z

−40 −20 0 20 40

−500

50−100

0

100

X

IMF3

Y

Z

−500

50−50

050

−100

0

100

X

IMF3

Y

Z

−1000

100

−500

50−100

0

100

X

IMF4

Y

Z

−50 0 50−20

020

−100

0

100

X

IMF4

Y

Z

−200

20

−20−10

010

−40−20

02040

X

IMF5

Y

Z

−200

20−20

020

−20

0

20

X

IMF5

Y

Z

−80−60

−40−20

0

−200

2040

−500

50100

X

Residue

Y

Z

−100 −80 −60 −40

6065

7075

−50

0

50

X

Residue

Y

Z


MEMD as a Filter Bank

Spectra of IMFs (IMF1-IMF9) obtained for a single realization of an 8-channel white

Gaussian noise via MEMD (top) and the standard EMD (bottom). Overlapping of the

frequency bands corresponding to the same-index IMFs is more prominent in the case of

MEMD based filters.

100

101

102

Sp

ectr

um

Spectra of a single realization of white noise from MEMD

100

101

102

Frequency (Log)

Sp

ectr

um

Spectra of single realization of white noise from standard EMD

100

101

102

Sp

ectr

um

Averaged spectra of white noise realizations from MEMD

100

101

102

Frequency (log)

Sp

ectr

um

Averaged spectra of white noise realizations from standard EMD

All realisations Averaged spectra


Multivariate EMD: Noise-assisted MEMD to reduce

mode-mixing (a single channel case)

2 Extra noise channels are added as extra dimensions to the original signal.

2 The resulting multidimensional IMFs are aligned according to the filter bank structure

of MEMD reducing mode mixing within the signal IMFs.

(Standard EMD) (NA-MEMD: Signal channel with two extra noise channels)

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−2

0

2

IMF

1

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−2

0

2

IMF

2

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−2

0

2

IMF

3

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−2

0

2

Time Index

IMF

4−

en

d

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000−2

0

2

Orig

ina

l

0 5000 10000−2

0

2

Orig

inal

0 5000 10000−2

0

2

Noi

se

0 5000 10000−2

0

2

Noi

se

0 5000 10000−2

0

2

IMF

1−5

0 5000 10000−2

0

2

0 5000 10000−2

0

2

0 5000 10000−0.5

0

0.5

IMF6

0 5000 10000−2

0

2

0 5000 10000−2

0

2

0 5000 10000−2

0

2

0 5000 10000−2

0

2

0 5000 10000−2

0

2

Time Index

IMF8

0 5000 10000−2

0

2

Time Index0 5000 10000

−2

0

2

Time Index

0 5000 10000−0.5

0

0.5

IMF7

⇒ An alternative to EEMD without mixing signal and noise!

NA-MEMD= process your data channel and several noise channels with MEMD.


A 2-channel Case: NA-MEMD to reduce mode-mixing

Decomposition without noise channels Decomposition with two extra noise channels

0 700 1400−2

0

2

Orig

ina

l

0 700 1400−2

0

2

0 700 1400−2

0

2

IMF

1

0 700 1400−2

0

2

0 700 1400−2

0

2

IMF

2

0 700 1400−2

0

2

0 700 1400−2

0

2

IMF

3

0 700 1400−2

0

2

0 700 1400−2

0

2

Time Index

IMF

4

0 700 1400−2

0

2

Time Index

700 1400−2

0

2

Orig

ina

l

700 1400−2

0

2

700 1400−2

0

2

IMF

3

700 1400−2

0

2

700 1400−2

0

2

IMF

4

700 1400−2

0

2

700 1400−2

0

2

Time Index

IMF

5

700 1400−2

0

2

Time Index

◦ The exta noise channels help

◦ So with NA-EMD we have a unified framework for the decomposition of both

single-channel data and M-channel data


Operation of NA-MEMD and EEMD

Ensemble EMD (EEMD)

EMD

respect.

IMFs

all

over

Average

N_i − WGN realisation IMFs of EEMD

S − useful signal

IMF_3

IMF_2

IMF_m

IMF_1

IMF_nr

IMF_n1

IMF_2q

IMF_21

IMF_1p

IMF_11

S+N_n

S+N_2

S+N_1

......

......

...

EMD

EMD

Noise-assisted MEMD (NA-MEMD)Signal

IMF space

m x (n+1) dimensionalEMD

multivariate

using

Process

Noise_n

Noise_1

In NA-MEMD signal is

NOT added to noise,

but instead the signal

channel and noise channels

are processed in a

multidimensional space

using MEMD. This way

we guarantee the same

number of IMFs and the

same frequency contents

at every level of such

multidimensional IMFs.

Usually 2-4 noise channels

are sufficient for good

decomposition (see the

simulations)


NA-MEMD vs Ensemble EMD

Ensemble EMD (EEMD)

◦ Performs EMD over an ensembleof signal plus white noise, andaverages the ensemble IMFs

◦ Uses the filter bank property ofEMD on white noise by populatingthe time-frequency space

◦ This way it reduces mode mixing

◦ EMD is applied separately overan ensemble of signal plus noise

◦ ⇒ each realization may havedifferent number of modes (IMFs)and output contains residual noise

◦ Only valid for univariatesignals

Noise–Assisted MEMD

◦ Makes use of the filter bankproperty of MEMD on white noiseto populate entire T–F space

◦ Unlike EEMD, it does notdirectly interfere with the originalsignal (as noise is added toseparate channels)

◦ ⇒ the output contains minimalresidual noise due to leakagewhich is negligible

◦ Since a single MEMD is appliedto the multivariate signal → samenumber of modes (IMFs)

◦ Is valid for both univariateand multivariate signals


NA–MEMD vs EEMD: Some Examples

0 200 400 600 800 1000 1200 1400 1600 1800 2000−2

0

2

XDecomposition of X via EEMD

0 200 400 600 800 1000 1200 1400 1600 1800 2000−0.5

0

0.5

IMF1

0 200 400 600 800 1000 1200 1400 1600 1800 2000−1

0

1

IMF2

0 200 400 600 800 1000 1200 1400 1600 1800 2000−2

0

2

IMF3

0 200 400 600 800 1000 1200 1400 1600 1800 2000−2

0

2

IMF4

0 200 400 600 800 1000 1200 1400 1600 1800 2000−2

0

2

Time Index

IMF5

:end

0 200 400 600 800 1000 1200 1400 1600 1800 2000−2

0

2

X

Decomposition of X via N−A MEMD

0 200 400 600 800 1000 1200 1400 1600 1800 2000−0.1

0

0.1

IMF1

0 200 400 600 800 1000 1200 1400 1600 1800 2000−1

0

1

IMF2

0 200 400 600 800 1000 1200 1400 1600 1800 2000−0.5

0

0.5

IMF3

0 200 400 600 800 1000 1200 1400 1600 1800 2000−2

0

2

IMF4

0 200 400 600 800 1000 1200 1400 1600 1800 2000−2

0

2

Time Index

IMF5

:end

0 500 1000 1500 20000

0.002

0.004

0.006

0.008

0.01

0.012

0.014

|e(t

)|

Time Index

Error Function (e(t)) for EEMD

0 500 1000 1500 20000

1

2

3

4

5

6x 10

−16

|e(t

)|

Time Index

Error function (e(t)) for N−A MEMD

EEMD for 500 realisations of WGN NA-MEMD with two noise channels


Sensitivity to Noise Power

Ensemble EMD Noise Assisted MEMD

0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

2

2.5

3

3.5

4x 10

−3

PWGN

/Psignal

Pe

(t)

Average Power of Error vs Noise Power for EEMD

0 0.2 0.4 0.6 0.8 10.8

1

1.2

1.4

1.6

1.8

2x 10

−32

PWGN

/Psignal

Pe

(t)

Average Power of Error vs Noise Power for NA−MEMD

EEMD:

◦ The decomposition error increases

with an increase in noise power

◦ This is because noise is added

to the signal and interferes with the

decomposition

NA-MEMD:

◦ The error does not increase with an

increase in noise power

◦ This independence of noise power is

because the noise is in separate channels

and does not interfere with the signal


BCI Application: Estimating cleaned EEG using MEMD

Filter bank structure of MEMD helps to clean the EEG signal by separating the brain

electrical activity from unwanted artefacts, such as the electrooculogram (EOG) and

electromyogram (EMG).

500 1000 1500−100

0

100Fp

1Recorded EEG

500 1000 1500−100

0

100Estimated EOG artefact

500 1000 1500−50

0

50

Cleaned EEG

500 1000 1500−100

0

100

Fp2

500 1000 1500−100

0

100

500 1000 1500−50

0

50

500 1000 1500−50

0

50

C3

500 1000 1500−40

−20

0

20

500 1000 1500−50

0

50

500 1000 1500−100

0

100

samples

C4

500 1000 1500−100

0

100

samples500 1000 1500

−50

0

50

samples


Imaginary Motion BCI: Mu Rhythm (8-12Hz)

• Occupies in the alpha range (8-12Hz)

• Strongly suppressed during the performance of contralateral motor acts

• Reflects the electrical output of the synchronisation of large portions of pyramidal

neurons of the motor cortex which control hand and arm movement

• Active movements/observed actions/imagined actions

(a)

(b)

(c)

(a) power of mu band on scalp during motor imagery(b) average time-frequency representation

(c) a typical single trial showing imagery-related modulation


Spectrogram Comparison

2 Motor imagery : A dynamic state during which an individual mentally simulates a

given action

2 Experiment : A subject imagined herself/himself moving left arm, right arm or foot

2 Event-related synchronization (ERS) over ipsilateral area during the motor imagery

task can be expected around 10Hz (mu rhythm)

Right hemisphere (C4+C6) during the imagination of right arm movement

time (s)

fre

qu

en

cy (

Hz)

0.5 1 1.5 2 2.5 30

5

10

15

20

25

30

time (s)

fre

qu

en

cy (

Hz)

0 0.5 1 1.5 2 2.5 30

5

10

15

20

25

30

time (s)

fre

qu

en

cy (

Hz)

0 0.5 1 1.5 2 2.5 30

5

10

15

20

25

30

time (s)

fre

qu

en

cy (

Hz)

0 0.5 1 1.5 2 2.5 30

5

10

15

20

25

30

(a) STFT (b) EMD (c) EEMD (d) MEMD


Classification Performances

Subject Algorithm IMFs Classification rate [%]

DFT(8-30Hz) 81.057 ± 4.291A EMD 1-3 65.710 ± 5.426

EEMD 1-3 79.727 ± 4.492MEMD 2-3 84.747 ± 4.762

DFT(8-30Hz) 57.370 ± 5.564B EMD 1-4 60.607 ± 5.882

EEMD 1-2 67.387 ± 5.824MEMD 2-3 70.167 ± 5.363

DFT(8-30Hz) 64.860 ± 6.020C EMD 1-4 60.653 ± 6.020

EEMD 1-3 75.250 ± 5.117MEMD 2-3 78.483 ± 4.362

DFT(8-30Hz) 86.700 ± 4.081D EMD 1-3 79.470 ± 5.447

EEMD 2 87.433 ± 3.917MEMD 2-3 89.133 ± 3.235

2 200 trials of motor imagery for left

hand, right hand and foot movements

2 Channels : ‘FC3’, ‘FC4’, ‘Cz’, ‘C3’,

‘C4’, ‘C5’, ‘C6’, ‘T7’, ‘T8’, ‘CCP3’,

‘CCP4’

2 Common spatial patterns (CSP) were

used to extract features

2 Average classification rates of 100

repetitions while mixing sample order

(Support vector machine was used)

2 MEMD produced the best

classification results using 2nd

and 3rd IMFs for all subjects


Multivariate extensions of EMD

2 For robust modelling, it is crucial that similar oscillatory modes frommultiple channels are aligned

2 Extensions of EMD for multivariate time series:– Original EMD algorithms [Huang et al, Proc. Roy. Soc. A, 1998]– Complex EMD [Tanaka and Mandic, IEEE SPL, 2007]– Rotation Invariant EMD [Mandic et al., Proc ICASSP, 2007]– Bivariate EMD [Rilling, Flandrin, Goncalves, IEEE SPL, 2007]– Trivariate EMD [Rehman and Mandic, IEEE TSP, 2009]– Quadrivariate EMD [Rehman and Mandic, Proc. IJCNN, 2010]– Multivariability via ’dendrograms’ [Rutkowski et al, Jour. Circ., Sys.,

and Comp, 2010]– Multivariate EMD [Rehman and Mandic, Proc. Roy. Soc. A, 2010]– Filterbank property of Multivariate EMD and Noise-Assisted MEMD

[Rehman and Mandic, IEEE Tr. Sig. Proc., 2011]

◦ Matlab code at www.commsp.ee.ic.ac.uk/∼mandic


Conclusions

2 EMD is non-parametric and self adaptive which is advantageous whendecomposing real world data into its natural frequency modes

2 It is a powerful tool for the purposes of “data fusion via fission”

2 Multivariate extensions of EMD have been proposed which takemultiple projections of a signal by sampling hyperspheres usingequi-angular coordinate system and low discrepancy quasi-Monte Carlobased Hammersley sequences

2 The proposed method extracts common rotational modes across thesignal components, making it suitable for e.g. fusion of informationfrom multiple sources, and follows a filter bank structure

2 Making use of the quasi-dyadic filter bank property of multivariateextension of EMD, noise-assisted MEMD method (NA-MEMD) hasbeen presented which has been shown to reduce mode mixing

2 NA-EMD is a viable alternative to EEMD


Thank you

十分感谢！


multivariate extensions of emd applications in data fusion ...mandic/d_mandic_hht_qingdao_2011.pdfc...

Documents