september 2007 iw-smi2007, kyoto 1 a quantum-statistical-mechanical extension of gaussian mixture...

September 2007September 2007 IW-SMI2007, KyotoIW-SMI2007, Kyoto 11

A Quantum-Statistical-Mechanical A Quantum-Statistical-Mechanical Extension of Gaussian Mixture ModelExtension of Gaussian Mixture Model

Kazuyuki TanakaKazuyuki TanakaGraduate School of Information Sciences,Graduate School of Information Sciences,

Tohoku University, Sendai, JapanTohoku University, Sendai, Japanhttp://www.smapip.is.tohoku.ac.jp/~kazu/http://www.smapip.is.tohoku.ac.jp/~kazu/

In collaboration withIn collaboration withKoji TsudaKoji TsudaMax Planck Institute for Biological Cybernetics,Max Planck Institute for Biological Cybernetics, 　　 GermanyGermany


ContentsContents

1.1. IntroductionIntroduction

2.2. Conventional Gaussian Mixture ModelConventional Gaussian Mixture Model

3.3. Quantum Mechanical Extension of Quantum Mechanical Extension of Gaussian Mixture ModelGaussian Mixture Model

4.4. Quantum Belief PropagationQuantum Belief Propagation

5.5. Concluding RemarksConcluding Remarks


Information Processing Information Processing by using Quantum Statistical-Mechanicsby using Quantum Statistical-Mechanics

Quantum Annealing in OptimizationsQuantum Annealing in Optimizations

Quantum Error Correcting CodesQuantum Error Correcting Codes

etc...etc...

Massive Information Processing Massive Information Processing by means of Density Matrixby means of Density Matrix


MotivationsMotivations

How can we construct the quantum Gaussian mixture model?How can we construct a data-classification algorithm by using the quantum Gaussian mixture model?


ContentsContents






September 2007 IW-SMI2007, Kyoto 6

Prior of Gauss Mixture Model

1)1( iXP

2)2( iXP

3)3( iXP

N

iii xXPxXP

1

)()(

1 32

1

23

Histogram

Label xi is generated randomly and independently of each node.

3 labels

xi =1 xi =2 xi =3

One of three labels 1,2 and 3 is assigned to each node.


Date Generating Process

Data yi are generated randomly and independently of each node.

22

)(2

1exp

2

1

)|(

kikk

iii

y

kXyYP

10,60 11

30,150 11

20,200 33

xi =1

xi =2

xi =3


Gauss Mixture Models

ixii xXP )(

N

i kikk

N

i kiiii

yg

kXPkXyYPyYP

1

3

1

1

3

1

)(

)()|(),,|(

2

2)(

2

1exp

2

1)()|( ki

kkikiii yygkXyYP

Prior Probability

Data Generating Process

),,|(maxarg)ˆ,ˆ,ˆ(),,(

yP

Marginal Likelihoodfor Hyperparameters, and


Conventional Gauss Mixture Models

1

0

1

0

)(

)(

N

iik

N

iiki

k

y

yy

k kk

kki μkg

μkgy

),,|(

),,|()(

ρ

1

0

1

0

2k

2

)(

)()-(

N

iik

N

iiki

k

y

yy

N

iikk y

N 1

)(1

, ,

(yi)

Data :2

1

Ny

y

y

y

Parameters :2

1

Nx

x

x

x


yP

Labels


ContentsContents







Quantum Gauss Mixture Models

N

i kikk ygyYP

1

3

1

)(),,|(

)(ln00

0)(ln0

00)(ln

ln00

0ln0

00ln

expTr

)(ln00

0)(ln0

00)(ln

expTr

)(ln00

0)(ln0

00)(

Tr

)(

3

2

1

3

2

1

33

22

11

33

22

11

3

1

i

i

i

i

i

i

i

i

i

kikk

yg

yg

yg

yg

yg

yg

yg

yg

yg

yg

2

2)(

2

1exp

2

1)( ki

kkik yyg



3

2

1

ln00

0ln0

00ln

F

)(ln00

0)(ln0

00)(ln

)(

3

2

1

i

i

i

i

yg

yg

yg

yG

N

i

iyyP1 )exp(Tr

))(exp(Tr),,|(

F

GF

3

2

1

ln

ln

ln

F

N

i kikk ygyYP

1

3

1

)(),,|(

Quantum Representation



3

1

3

1

)(

33

22

11

)(ln

)(ln

)(ln

)(

k lkl

ikl

i

i

i

i

B

yg

yg

yg

y

X

H

N

i

iyyP1 )exp(Tr

))(exp(Tr),,|(

F

H

)(

)()(ln)(

lk

lkygB ikkikl

100

000

000

000

000

010

000

000

001

331211 XXX



yP

)(

)(

)(

1

0)()()1(

)(

Tr

Tr

eTr

Tr)eTrln(

i

i

i

ii

i

yH

yHkl

y

yHkl

yHy

kl

e

eX

deXe

B

HH

and,forConditionExtremun

Linear Response Formulas

N

i

iyyP1 )exp(Tr

))(exp(Tr),,|(

F

H

Maxmum Likelihood Estimation in Quantum Gauss Mixture Model



N

iikk

N

iikki

k

y

yy

1

1

)(Tr

)(Tr

ρX

ρX

)(

)(

Tr)(

i

i

y

y

ie

ey

H

Hρ

N

iikk

N

iikki

k

y

yy

1

1

2k

2

)(Tr

)(Tr)-(

ρX

ρX

N

iikkk y

N 1

)(1

lnTrexp ρX

, ,

(yi)

Data :2

1

Ny

y

y

y

Parameters :2

1

Nx

x

x

x


yP


Image SegmentationImage Segmentation

Original Image

Histogram

ConventionalGauss Mixture

Model Quantum Gauss Mixture Model

= 0.2 = 0.4

0 2550 255 0 255

)exp(Tr

))(exp(Tr

),,|(

F

GF

i

i

y

yP



Original Image

Histogram

ConventionalGauss Mixture

Model Quantum Gauss Mixture Model

= 0.5 = 1.0

0 255 0 255 0 255)exp(Tr

))(exp(Tr

),,|(

F

GF

i

i

y

yP


ContentsContents







Image Segmentation by Combining Image Segmentation by Combining Gauss Mixture Model with Potts Model Gauss Mixture Model with Potts Model

104321

,1924 ,1923

,1272 ,641

Belief PropagationBelief Propagation

NeighbourNearest :,

1

0

exp

)(

ijxx

N

ix

ji

i

J

xXP

== >

Potts Model

4 labels



Original Image Histogram Gauss Mixture

Model

Gauss Mixture

Model and

Potts Model

Belief Belief

PropagationPropagation


Density Matrix and Reduced Density Matrix

Bij

ijHH ˆ Hρ exp

1

Z

ρρ ii \tr ρρ ijij \tr

ijii ρρ \tr

Reduced Density Matrix

Reducibility Condition

ji


Reduced Density Matrix and Effective Fields

iBiki λρ

kiZexp

1

i\Bljl

j\Bkikijij

ji

λIIλHρ exp1

ijZ

i

jiAll effective field

are matrices

All effective field

are matrices

iB

jBi \iB j \


Belief Propagation for Quantum Statistical Systems

i\Bljl

j\Bkikij

j\Bkikij

ji

i

λIIλH

λλ

exptrlog \iij

i

Z

Z

Propagating Rule of Effective Fields

ijii ρρ \trji

Output


ContentsContents







SummarySummaryAn Extension to Quantum Statistical Mechanical An Extension to Quantum Statistical Mechanical Gaussian Mixture ModelGaussian Mixture ModelPractical Algorithm Practical Algorithm Linear Response Formula Linear Response Formula

Application of Potts Model and Application of Potts Model and Quantum Belief PropagationQuantum Belief Propagation

Applications to Data MiningApplications to Data MiningExtension to Quantum Deterministic Annealing

Future ProblemFuture Problem

september 2007 iw-smi2007, kyoto 1 a quantum-statistical-mechanical extension of gaussian mixture...

Documents