23.02.03 1 successive bayesian estimation alexey pomerantsev semenov institute of chemical physics...

46
23.02.03 1 Successive Bayesian Successive Bayesian Estimation Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

Upload: domenic-byrd

Post on 12-Jan-2016

228 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 1

Successive Bayesian EstimationSuccessive Bayesian Estimation

Alexey PomerantsevSemenov Institute of Chemical Physics

Russian Chemometrics Society

Page 2: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 2

AgendaAgenda

1. Introduction. Bayes Theorem

2. Successive Bayesian Estimation

3. Fitter Add-In

4. Spectral Kinetics Example

5. New Idea (Method ?)

6. More Applications of SBE

7. Conclusions

Page 3: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 3

1. Introduction1. Introduction

Page 4: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 4

The Bayes Theorem, 1763The Bayes Theorem, 1763

Thomas Bayes (1702-1761)

Posterior Probability Prior Probabilities

L(a,2)=h(a,2)L0(a,2)

Likelihood Function

Where to takethe prior

probabilities?

Page 5: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 5

Jam Sampling & Blending TheoryJam Sampling & Blending Theory

0.20 0.30 0.50

0.50 0.20 0.05

Now we know the origin ofa worm in the jam!

Page 6: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 6

2.Successive Bayesian Estimation (SBE)2.Successive Bayesian Estimation (SBE)

Page 7: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 7

SBE ConceptSBE Concept

y 1 X 1 y 2 X 2 . . . y k X k

. . .

. . .

Whole data set

f 1 (X 1 , a 0 , a 1 ) f 2 (X 2 , a 0 , a 2 ) f k (X k , a 0 , a k )

Data subset 1 Data subset 2 Data subset k

Post

a 0 , a 1

s 12 N 1

Prior

a 0

s 12 N 1

Post

a 0 , a 2

s 22 N 2

Prior

a 0

s k2 N k

Post

a 0 , a k

s k2 N k

Resulta 0 , a 1 ,…, a k

s 2 N

SBE principles

1) Split up whole data set

2) Process each subset alone

3) Make posterior information

4) Build prior information

5) Use it for the next subset

How to eat away

an elephant?Slice by slice!

Page 8: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 8

OLS & SBE Methods for Two SubsetsOLS & SBE Methods for Two Subsets

OLS

SBE

Quadraticapproximation

near theminimum!

Page 9: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 9

Posterior & Prior InformationPosterior & Prior Information

Subset 1. Posterior Information

Rebuilding (common & partial parameters)

Subset 2. Prior Information

Make Posterior,rebuild it and apply as Prior!

Page 10: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 10

Prior Information of Type IPrior Information of Type I

Posterior Information Prior Information

Parameter estimates Prior parameter values b

Matrix A Recalculated matrix H

Variance estimate s2 Prior variance value s02

NDF Nf Prior NDF N0

Objective Function

The same errorvariance in theeach subset

of data!

Page 11: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 11

Prior Information of Type IIPrior Information of Type II

Posterior Information Prior Information

Parameter estimates Prior parameter values b

Matrix A Recalculated matrix H

Objective Function

aDifferent errorvariances in the

each subsetof data!

Page 12: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 12

SBE Main TheoremSBE Main Theorem

Different order of subsets processing

Theorem (Pomerantsev & Maksimova , 1995)

SBEagree with

OLS!

Page 13: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 13

3. Fitter3. Fitter Add-InAdd-In

Page 14: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 14

A B C D E F G H I J K L M N O P Q R S T1

2 Data

3 x t y w f A B C

4 13 0 0.047 1 0.047 1 0 0

5 13 2 0.553 1 0.56 0.125 0.448 0.4266

6 13 4 0.412 1 0.403 0.016 0.209 0.775

7 13 6 0.304 1 0.308 0.002 0.079 0.9194

8 13 8 0.275 1 0.27 2E-04 0.028 0.9729 13 10 0.253 1 0.257 3E-05 0.01 0.990410

11 Bayesian Information

12 Name Value Matrix Exclude

13 k1 1.07 265.3 146.5 0 0 0

14 k2 0.554 146.5 1117 0 0 0

15 0 0 0 0 0 0

16 0 0 0 0 0 0

17 0 0 0 0 0 0

18

19

2021

22

23

24

25

2627

28

29

30

FitterFitter Workspace WorkspaceA B C D E F G H I J K L M N O P Q R S T1

2 Data General

3 x t y w f A B C Date 01.08.01 19:07

4 13 0 0.047 1 0.047 1 0 0 Data Bayes!rData

5 13 2 0.553 1 0.56 0.125 0.448 0.4266 Model Bayes!ABCbayes

6 13 4 0.412 1 0.403 0.016 0.209 0.775 ParametersBayes!rParam

7 13 6 0.304 1 0.308 0.002 0.079 0.9194 Bayes Bayes!rBayes

8 13 8 0.275 1 0.27 2E-04 0.028 0.972 Precision1E-11

9 13 10 0.253 1 0.257 3E-05 0.01 0.9904 Convergence0.00110 Error typeRelative11 Bayesian Information Significance0.05

12 Name Value Matrix Exclude Confidence0.95

13 k1 1.07 265.3 146.5 0 0 0 PredictionLinearization

14 k2 0.554 146.5 1117 0 0 0

15 0 0 0 0 0 0 Parameters estimation

16 0 0 0 0 0 0 Name Initial Final Deviation

17 0 0 0 0 0 0 k1 1.06969 1.03907 0.060509

18 k2 0.55366 0.53661 0.027954

19 p -3.05769 -3.05769 0.019371

20 q -0.002 -0.002 0.039994

21 r -1.38757 -1.38757 0.017332

22

23 Parameters Search Progress

24 k1 1.03907 Objective value 0.0139

25 k2 0.53661 Completeness 100%

26 p -3.05769 Objective change -1E-07

27 q -0.002 Iteration 2

28 r -1.38757

29

30

x =

y

A

B

C

13

0.0

0.5

1.0

0 2 4 6 8 10 t

y

y=exp(p)*A+exp(q)*B+exp(r)*CA=A0*exp(-k1*t)B=k1*A0/(k1-k2)*[exp(-k2*t)-exp(-k1*t)]+B0*exp(-k2*t)C=A0+B0+C0+A0/(k1-k2)*[k2*exp(-k1*t)-k1*exp(-k2*t)]-B0*exp(-k2*t) A0="cA0" B0="cB0" C0="cC0"

k1=?

k2=?

p=?

q=? r=?

Fitter is atool for SBE!

Page 15: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 15

A B C D E F G H I J K L M1

2

3

4

5

6

7

8

9

10

11

A B C D E F G H I J K L M1

2 BoxBod Data3 x y w f4 0 05 1 109 16 2 149 17 3 149 18 5 191 19 7 213 110 10 224 111

Data & Model Prepared for FitterData & Model Prepared for Fitter

A B C D E F G H I J K L M1

2 BoxBod Data Parameters3 x y w f a 1004 0 0 b 0.45 1 109 16 2 149 17 3 149 18 5 191 19 7 213 110 10 224 111

'BoxBOD modely=a*[1-exp(-b*x)] a=? b=?

A B C D E F G H I J K L M1

2 BoxBod Data Parameters3 x y w f a 213.809414 0 0 0.00 b 0.54723755 1 109 1 90.116 2 149 1 142.247 3 149 1 172.418 5 191 1 199.959 7 213 1 209.1710 10 224 1 212.9111

0

100

200

0 4 8 x

y

'BoxBOD modely=a*[1-exp(-b*x)] a=? b=?

ResponseWeight

Fitting

Predictor

ParametersEquation

CommentValues

Apply Fitter!

Page 16: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 16

Model Model ff((xx,,aa))

Different shapes of the same model

Explicit model y = a + (b – a)*exp(–c*x)

Implicit model 0 = a + (b – a)*exp(–c*x) – y

Diff. equation d[y]/d[x] = – c*(y –a); y(0) = b

Presentation at worksheet

'Цикл "увлажнение-сушка"

M=Sor*hev(t1-t)+Des*[hev(t-t1)+imp(t-t1)]'Кинетика "увлажнения"

Sor=Sor1*hev(USESor1)+Sor2*[hev(-USESor1)+imp(-USESor1)]'Кинетика "сушки"

Des=Des1*hev(USEDes1)+Des2*[hev(-USEDes1)+imp(-USEDes1)]'Условие применимости асимптотик

USESor1=Sor2-Sor1 USEDes1=Des1-Des2'константы и промежуточные величины

t3=(t-t1)*hev(t-t1) t4=t*hev(t1-t)+t1*[hev(t-t1)+imp(t-t1)] P2=PI*PI P12=(PI)^(-0.5) R=r*(M1-M0)*exp(-r*t4) K=M1+(M0-M1)*exp(-r*t4) V0=M0-C0 V1=M1-C0'асимптотика сорбции при 0<t<tau

Sor1=C0+4*P12*(d*t)^0.5*[M0-C0+(M1-M0)*beta] beta=1-exp(-z) x=r*t z=(a1*x+a2*x*x+a3*x*x*x)/(1+b1*x+b2*x*x+b3*x*x*x) a1=0.6666539250029 a2=0.0121051017749 a3=0.0099225322428 b1=0.0848006232519 b2=0.0246634591223 b3=0.0017549947958'кинетика сорбции при tau<t<t1

Sor2=K-8*S1 S1=U01/n0+U11/n1+U21/n2+U31/n3+U41/n4 n0=P2 U01=[(V0*n0*d-V1*r)*exp(-n0*d*t4)+R]/(n0*d-r) n1=P2*9 U11=[(V0*n1*d-V1*r)*exp(-n1*d*t4)+R]/(n1*d-r) n2=P2*25 U21=[(V0*n2*d-V1*r)*exp(-n2*d*t4)+R]/(n2*d-r) n3=P2*49 U31=[(V0*n3*d-V1*r)*exp(-n3*d*t4)+R]/(n3*d-r) n4=P2*81 U41=[(V0*n4*d-V1*r)*exp(-n4*d*t4)+R]/(n4*d-r)'асимптотика десорбции при t1<t<t1+tau

Des1=K*[1-4*P12*(d*t3)^0.5]-8*S1'кинетика десорбции при t1+tau<t

Des2=8*S2 S2=U02/n0+U12/n1+U22/n2+U32/n3+U42/n4 U02=(K-U01)*exp(-n0*d*t3) U12=(K-U11)*exp(-n1*d*t3) U22=(K-U21)*exp(-n2*d*t3) U32=(K-U31)*exp(-n3*d*t3) U42=(K-U41)*exp(-n4*d*t3)'неизвестные параметры

d=?

M0=?

M1=?

C0=?

r=?

t1=?

Rathercomplexmodel!

Page 17: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 17

4. Spectral Kinetics Modeling4. Spectral Kinetics Modeling

17

1319

2531

3743

49 0 2 4 6 8 10

Page 18: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 18

Spectral Kinetic DataSpectral Kinetic Data

wavelengths wavelengths wavelengths

= +sp

ec

ies

tim

e

spectral signal conc. pure spectra errors

tim

e

species

t

imeY C P E

Y(t,x,k)=C(t,k)P(x)+E

Y is the (NL) known data matrix

C is the (NM) known matrix depending on unknown parameters k

P is the (ML) unknown matrix of pure component spectra

E is the (NL) unknown error matrix

K constants L wavelengths M species N time points

This is largenon-linearregressionproblem!

Page 19: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 19

How to Find Parameters k?How to Find Parameters k?

Method Idea Dimension Problem

Full OLS(hard)

K+ML >> 1Large

dimension

Short OLS(hard)

K+MS 10

Smallprecision

WCR(hard&soft)

K 10Matrix

degradation

GRAM(soft)

K+MA 100

Just onemodel

)(ln

stk

kt

e

e

s

1k

This is a challenge!

Page 20: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 20

Simulated Example Goals Simulated Example Goals

Compare SBE estimates with ‘true’ values

Compare SBE estimates for different order

Compare SBE estimates with OLS estimates

Page 21: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 21

Model. Two Step KineticsModel. Two Step Kinetics

0C0CBkdt

dC

0B0BBkAkdt

dB

1A0AAkdt

dA

02

021

01

)(;

)(;

)(;

Ak

Bk

C1 2

‘True’ parameter values

k1=1 k2=0.5

Standard‘training’

model

Page 22: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 22

Data SimulationData Simulation

C1(t) = [A](t)

C2(t) = [B](t)

C3(t) = [C](t)

P1(x) = pA (x)

P2(x) = pB (x)

P3(x) = pC (x)

Simulated concentration profiles Simulated pure component spectra

B

CA

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10time

con

cen

trat

ion

s

A B C

0.0

0.2

0.4

0.6

0.8

1.0

0 5 10 15 20 25 30 35 40 45 50conventional wavelengths

spec

tral

sig

nal

Y(t,x)=C(t)P(x)(I+E)

STDEV(E)=0.03

Usual way ofdata simulation

Page 23: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 23

t=0

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50

conventional wavelengths

sp

ec

tra

l sig

na

lt=0

t=2

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50

conventional wavelengths

sp

ec

tra

l sig

na

lt=0

t=2

t=4

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50

conventional wavelengths

sp

ec

tra

l sig

na

l

Simulated Data. Spectral ViewSimulated Data. Spectral View

t=0

t=2

t=4

t=6

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50

conventional wavelengths

sp

ec

tra

l sig

na

lt=0

t=2

t=4

t=6

t=8

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50

conventional wavelengths

sp

ec

tra

l sig

na

lt=0

t=2

t=4

t=6

t=8t=10

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 10 20 30 40 50

conventional wavelengths

sp

ec

tra

l sig

na

l Spectralview of data

Page 24: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 24

Simulated Data. Kinetic ViewSimulated Data. Kinetic View

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

time

sp

ec

tra

l sig

na

l

Kinetic viewof data

Page 25: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 25

One Wavelength EstimatesOne Wavelength Estimates Conventional wavelength 3

Estimates

30.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 time

y

k 1

k 2

30.0

1.0

2.0

3.0

4.0

14

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 time

y

Conventional wavelength 14

k 1

k 2

1430.0

1.0

2.0

3.0

4.0

Conventional wavelength 51

510.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 time

y

k 1

k 2

14 5130.0

1.0

2.0

3.0

4.0

k 1

k 2

14 51 O30.0

1.0

2.0

3.0

4.0Bad accuracy!

Page 26: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 26

1234

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 time

y

k 1

k 2

D0.0

0.5

1.0

1.5

Direct order

Estimates

Four Wavelengths EstimatesFour Wavelengths Estimates

53525150

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 time

y

k 1

k 2

D I0.0

0.5

1.0

1.5

Inverse order

165

29

8

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 time

y

k 1

k 2

D RI0.0

0.5

1.0

1.5

Random order

k 1

k 2

D ORI0.0

0.5

1.0

1.5

Bad accuracy,again!

Page 27: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 27

SBE Estimates at the Different OrderSBE Estimates at the Different OrderDirect 1, 2, 3, ….

Random 16, 5, 29, ….

k 2

k 1

0.25

0.50

0.75

1.00

1.25

1.50

53 49 45 41 37 33 29 25 21 17 13 9 5 1

conventional wavelengths

k 2

k 1

0.25

0.50

0.75

1.00

1.25

1.50

1 8 15 22 29 36 43 50conventional wavelengths

k 2

k 1

0.25

0.50

0.75

1.00

1.25

1.50

16 41 27 33 19 2 15 51 21 9 24 50 12 22

conventional wavelengths

Inverse 53, 52, 51, ….

0.95 Confidence Ellipses

Random

Direct

Inverse

'True'

0.85

0.95

1.05

1.15

1.25

0.42 0.44 0.46 0.48 0.5 0.52 0.54 0.56

k 2

k 1

Random

Direct

Inverse

'True'

0.85

0.95

1.05

1.15

1.25

0.42 0.44 0.46 0.48 0.5 0.52 0.54 0.56

k 2

k 1

SBE (practically)doesn’t depend onthe subsets order!

Page 28: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 28

SBE Estimates and OLS EstimatesSBE Estimates and OLS Estimates

OLS

SBE

'True'

0.85

0.90

0.95

1.00

1.05

1.10

1.15

1.20

1.25

0.42 0.44 0.46 0.48 0.5 0.52 0.54 0.56

k 2

k 1

SBE estimatesare close to

OLS estimates!

Page 29: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 29

Spectrum A

0

0.2

0.4

0.6

0.8

1

1.2

1 11 21 31 41 51conventional wavelength

sp

ec

tra

l sig

na

l

-0.2

0

0.2

0.4

0.6

ac

cu

rac

y

Spectrum A

0

0.2

0.4

0.6

0.8

1

1.2

1 11 21 31 41 51conventional wavelength

sp

ec

tra

l sig

na

l

-0.2

0

0.2

0.4

0.6

ac

cu

rac

y

Pure Spectra EstimatingPure Spectra Estimating

Spectrum B

0

0.2

0.4

0.6

0.8

1

1.2

1 11 21 31 41 51conventional wavelength

sp

ec

tra

l sig

na

l

-0.2

0

0.2

0.4

0.6

ac

cu

rac

y

Spectrum C

0

0.2

0.4

0.6

0.8

1

1.2

1 11 21 31 41 51conventional wavelength

sp

ec

tra

l sig

na

l

-0.2

0

0.2

0.4

0.6

ac

cu

rac

ySBE givesgood spectraestimates!

Page 30: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 30

Real World Example Goals Real World Example Goals

Apply SBE for real world data

Compare SBE with other known methods

Page 31: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 31

DataData

Bijlsma S, Smilde AK. J.Chemometrics 2000; 14: 541-560

Epoxidation of 2,5-di-tert-butyl-1,4-benzoquinone

SW-NIR spectra

-8

-6

-4

-2

0

2

4

6

8

860 865 870 875 880

wavelength

sp

ec

tra

l sig

na

l

240 spectra

1200 time points

21 wavelengths

Preprocessing:

Savitzky-Golay filter

-8

-6

-4

-2

0

2

4

6

8

860 865 870 875 880

wavelength

sp

ec

tra

l sig

na

l

PreprocessedData

Page 32: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 32

Progress in SBE EstimatesProgress in SBE Estimates

k 1

0.0

0.1

0.2

0.3

0.4

860 862 864 866 868 870 872 874 876 878 880

wavelength (nm)

k 2

0.0

0.1

0.2

0.3

0.4

860 862 864 866 868 870 872 874 876 878 880

wavelength (nm)

k 2

k 1

0.0

0.1

0.2

0.3

0.4

860 862 864 866 868 870 872 874 876 878 880

wavelength (nm)

SBE workswith the realworld data!

Page 33: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 33

0.10

0.15

0.20

0.25

0.30

0.35

0.40

-0.05 0.00 0.05 0.10 0.15 0.20 0.25

k 2

k 1

WCR

0.10

0.15

0.20

0.25

0.30

0.35

0.40

-0.05 0.00 0.05 0.10 0.15 0.20 0.25

k 2

k 1

SBE and the Other MethodsSBE and the Other Methods

WCR

LM-PAR0.10

0.15

0.20

0.25

0.30

0.35

0.40

-0.05 0.00 0.05 0.10 0.15 0.20 0.25

k 2

k 1

WCR

GRAM

LM-PAR0.10

0.15

0.20

0.25

0.30

0.35

0.40

-0.05 0.00 0.05 0.10 0.15 0.20 0.25

k 2

k 1

SBE

WCR

GRAM

LM-PAR0.10

0.15

0.20

0.25

0.30

0.35

0.40

-0.05 0.00 0.05 0.10 0.15 0.20 0.25

k 2

k 1

SBE gives thelowest deviationsand correlation!

Page 34: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 34

5. New Idea5. New Idea

Page 35: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 35

y=a1x1+a2x2+a3x3

Bayesian Step Wise Regression Bayesian Step Wise Regression Ordinarily Step Wise Regression Bayesian Step Wise Regression

Objective function

BSWR accountscorrelations of

variables in step wise estimation

Page 36: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 36

BSW Regression & Ridge RegressionBSW Regression & Ridge Regression

BSWR is RR witha moving center

and non-Euclideanmetric

Page 37: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 37

Example. RMSEC & RMSEPExample. RMSEC & RMSEP

BSWR givestypical U-shape ofthe RMSEP curve

Page 38: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 38

Linear Model. RMSEC & RMSEPLinear Model. RMSEC & RMSEP

y=a1x1+a2x2+a3x3+a4x4+a5x5

0.0

0.1

0.2

0.3

0.4

0.5

PLS PCR OLS SWR BSWR

RMSEC

RMSEP

BSWR is notworse then PLS or PCR and betterthen SWR

Page 39: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 39

0.0

0.2

0.4

0.6

0.8

1.0

1.2

PLS PCR OLS SWR BSWR

RMSEC

RMSEP

Non-Linear Model. RMSEC & RMSEPNon-Linear Model. RMSEC & RMSEP

0.0

0.2

0.4

0.6

0.8

1.0

1.2

PLS PCR OLS SWR BSWR

RMSEC

RMSEP

5544332211 xk5

xk4

xk3

xk2

xk1 aaaaay eeeee

For non-linearmodel BSWR is

better then PLS or PCR

Page 40: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 40

Variable selectionVariable selection

BSWR is just an idea, not

the method soany criticism is welcomed now!

Page 41: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 41

6. More Practical Applications of SBE6. More Practical Applications of SBE

Page 42: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 42

Antioxidants Activity by DSCAntioxidants Activity by DSCDSC Data Oxidation Initial Temperature (OIT)

C=0.1

C=0.05

C=0.025

470

490

510

530

550

570

0 5 10 15 20Heating rate v , grad/min

OIT

T,K

20

1510

5

2

-5

-4

-3

-2

-1

0

1

2

3

4

460 470 480 490 500 510Temperature, K

DS

C s

ign

al

To testantioxidants!

Page 43: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 43

Network Density of Shrinkable PE by TMANetwork Density of Shrinkable PE by TMA

4

2

1

3

5

1.1

1.2

1.3

1.4

1.5

0 10 20 30 40 50 60 70 80 90

Time, min

Elo

ng

ati

on

L/L

o

AB

C

D 2D 1 D 30

2

4

6

8

10

12

14

0 5 10 15 20 25Dose, MRad

Ch

em

ica

l mo

du

lus

, gm

m2

TMA Data Network density

To solvetechnological

problem!

Page 44: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 44

PVC Isolation Service Life by TGAPVC Isolation Service Life by TGA

Critical Level

0.0

0.1

0.2

0.3

0 5 10 15 20 25Time, yr

Co

nc

en

tra

tio

n

T=20C, F=2.0, P=0.95 T=30C, F=1.5, P=0.95

Service Life 2110

0.90

0.92

0.94

0.96

0.98

1.00

0 10 20 30 40 50

Time t , min

Ma

ss

ch

am

ge

, y

370

410

450

490

Te

mp

era

ture

T, K

TGA Data Service life prediction

To predictdurability!

Page 45: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 45

Tire Rubber StorageTire Rubber StorageElongation at break Tensile strength

T=140 C T=125 C T=110 C

T=20 CCritical

level

26

0

1

2

3

4

5

6

0 20 40 60Time, hr

Elo

ng

ati

on

@ b

rea

k

0 15 30 45Time, yr

T=140 C T=125 C T=110 C

T=20 CCritical

level

23

0

5

10

15

20

25

30

0 20 40 60Time, hr

Te

ns

ile, K

Pa

0 15 30 45Time, yr

To predictreliability!

Page 46: 23.02.03 1 Successive Bayesian Estimation Alexey Pomerantsev Semenov Institute of Chemical Physics Russian Chemometrics Society

23.02.03 46

7. Conclusions7. Conclusions

1 SBE is of general nature and it can be used for any model

2 SBE agrees with OLS

3 SBE gives small deviations and correlations

4 SBE uses no subjective a priori information

5 SBE may be useful for non-linear modeling (BWSR?)

Thanks!