Transcript
Page 1: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

國立雲林科技大學National Yunlin University of Science and Technology

1

SOM time series clustering and prediction with recurrent neural networks

Aymen Cherif , Hubert Cardot , Romuald Bone2011, Necurocomputing

Presented by Chien-Hao Kung2011/11/3

Page 2: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

2

Outlines· Motivation· Objectives· Methodology· Experiments· Conclusions· Comments

Page 3: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

3

Motivation

· Local models for regression have been the focus of a great deal of attention in the recent years.

· Many models have been proposed to cluster time series and they have been combined with several predictors

Page 4: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

4

Objectives

· This paper presents an extension for recurrent neural networks applied to local models and a discussion about the obtained results.

Page 5: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology

· From global models to local models─ Step1: The time series is embedded into M-dimensional

space vectors

─ Step2:The time series is clustered into sublearning sets.

─ Step3:Local predictions are performed on each subset.

5

Page 6: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology

· Multi-layer perceptron─ A multilayer perceptron (MLP) is a feedforward

artificial neural network model

Step1: The time series is embedded into M-dimensional space vectors

6

Page 7: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology· VQ

─ VQ was a method used for reducing a large volume of vectors to a smaller number of distribution.

· Self-Organizing Maps(SOM)─ The SOM has the advantage of being easy to use─ However, since the original Self-Organizing Maps

algorithm does not take into account the temporal sequence processing.

7

Step2: The time series is clustered into sublearning sets

Page 8: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology

8

· Alternative clustering way

- Type1:the use of recurrent processing of time signal with recurrent BMU computation Temporal Kohonen Map(TKM) Recursive Self-Organizing Maps(RSOM)

- Type2:consists in mapping the temporal dependencies to spatial correlation. Mege Self-Organizing Maps(MSOM) The SOM with Temporal Activity Diffusion(SOMTAD)

Step2: The time series is clustered into sublearning sets

Page 9: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology

9

Step3:Local predictions are performed on each subset.

· MLP as local predictors─ The use of a temporal windows which is precisely the

same as the one used in the clustering step.─ The feedforward nature of the MLP network─ The output calculation and the weights modification

are done at the same time step as the learning process.

Page 10: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology

10

Step3:Local predictions are performed on each subset.

· RNN as local predictors─ Original RNN

─ Back-Propagation Through Time(BPTT)

Page 11: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology

11

Step3:Local predictions are performed on each subset.

· RNN as local predictors

Page 12: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Methodology

12

Step3:Local predictions are performed on each subset.

· RNN as local predictors

Page 13: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

· Time series─ Sunspots time series

─ Laser time series

─ The Mackey-Glass(MG)-17

Experiments

13

Page 14: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

· Sunspots time series

Experiments

14

Page 15: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments

15

· Laser time series

Page 16: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments

16

· MG-17 time series

Page 17: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments

17

· Experiments on sunspot

Page 18: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments

18

· Experiments on Laser time series

Page 19: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.Experiments

19

· MG-17 time series

Page 20: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

20

Conclusions· This paper preferred to use the original SOM

algorithm in order to demonstrate the contribution of RNN as a local model.

· However, this paper saw that the performance of the model depends on the clustering and also on the nature of the time series.

Page 21: SOM time series clustering and prediction  with recurrent neural networks

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

21

Comments

· Drawback─ The paper is useful for time series

Application─ Time sereis


Top Related