13 multivariate calibration

7/28/2019 13 Multivariate Calibration

1/14

Regression methodsLinear regression

Y = m X + b

A linear relationship is assumed to exist

between to factors.

This was already discussed in an earlier unit.

Regression methodsMultiple linear regression

Y = m1 X1 + m2 X2 + ... mn Xn + e

This is a linear regression fit that is extended toseveral variables.

It is useful when several factors contribute to theoverall observed response.

Multivariate calibrationTypically, a multivariate method implies thatyou have multiple X (independent) andmultiple Y (dependent) variable.

We will outline three multivariateapproaches to creating a calibration curve.

Ordinary Least Squares (OLS)

Principal component regression (PCR)

Partial least squares regression (PLS)

While each optimizes the fit of your datadifferently, method evaluation, optimizationand the results are often the same.

With traditional linear and multiple linear regressiowere limited to a single Y (dependent) variable.

OLS (also called a general linear model - GLM) cabe seen as an extension of this approach. You hava Y matrix instead of a Y vector.

Mathematically, the matrix formulations for MLS anOLS (GLM) are the identical - except for allowing fa Y matrix. Basically a combination of MLS andsimultaneous equations.

XLStat will handle either approach - based on the

number of Y variables you give it.

One limit with OLS is that you need moreobservations than X variables and more Xvariables than Y variables.

Results can be irratic if you have variablesthat are: Collinear (ones with a high degree of

linear correlation.)

Invariate (ones that dont vary much.) Can try to remove all invariate and all but

one collinear (in a block) and hope for thebest (XLStat will do this.)

For these reasons, OLS is not as commonlyfor multi-Y type problems (compared to PCRand PLS).

Principal component regression

This is a simple extension of OLS.

It is assumed that each member of your set canbe assigned a quantitative class value.

First, generate a PCA model for your data.

Using the PCA scores, conduct a multiplelinear regression where your Y values are thequantitative class values.


2/14

Principal componentregression

OLS

Principal component regressio

Partial least squares regressionPLS modeling relies on a simultaneous fitof both an independent and dependentmatrix.

The objective is to derive latent variablesthat are similar to principal components.

Major difference is the attempt tominimize the variance of both arrays.

Called PLS1 for a Y vector and PLS2where there is a Y matrix.

Partial least squares regressionWith PLS, the goal it to extract the latent variables byusing the X array to properly align the Y array (or vector)and then reversing the process.

Y

X

q

w

t

u

Partial least squares regressionEach factor to be determined should end upwith a different PC set.

It may require a different number ofcomponents to adequately model eachquantitative variable.

The approach insures that the best fit is obtainfor all variables - which is both good and bad.

Considered best approach when the number ofvariables is high and correlated variables arelikely (basically the opposite of OLS).

Validation

All methods require validation.

Again, cross validation is one of the bestapproach. (leave one out method)

It permits you to determine a predictionerror sum of squares (PRESS) or the root-mean-square value of prediction error(RMSPE)

Tracking the PRESS value will tell you theoptimum number of components to use.


3/14

PRESS

PRESS= (yi- yVi/ )2

RMSPE= nTPRESSb l2

Goodness of fit.

There are several measures of modelquality beyond the use of PRESS.

Whats used is based on both the type ofmodel (OLS/PCR or PLS) and the softwareused.

XLStat can produce a huge amount ofinformation to evaluate.

Well look at only a few of the modelquality indexes.

These two methods have similar measures of modelquality since both rely on an OLS fit.

Coefficient of determination of the model (R2)

Values between 0 and 1 (1 is best).

Interpreted as the proportion of variability of thedependent variable(s) explained by the model.

R2= 1-wi(yi- yi

i=1

n

/ ) 2

wi(yi- yVii=1

n

/ ) 2

where yi= n1 wiyii=1

n

/

Adjusted R2

Takes into account the number of variables used inthe model.

Can be a negative value if R is small.

W = sum of the wi weights

p = number of x variables

adjR2= 1- (1-R2)W- p-W- 1

Root Mean Square of Errors (RMSE)

PRESS RMSE

is the prediction of the ith obsevation when itsnot included when building the model. A largedifference between RMSE and PRESS RMCEindicates that the model is sensitive to the presence(or absence) of some observations - outliers.

RMSE=W-p

1wi(yi-

i=1

n

/ yVi)2

yVi(-1)

PRESSRMSE= W-p

wi(yi- y^

i(-1)i=1

n

/ ) 2

Besides the regression information, you haveadditional measures that can be used.

Q2cum(h) index.

Measures global contribution of the h firstcomponents as to predictive quality of the model.

It involves the cross-validation PRESS and the modelSum of Squares of Errors with one less component.The max Q2cum index represents the most stablemodel.

Qcum(h)2= 1-

SCEk(j-1)k=1

q

/

PRESSkjk=1

q

/

j=1

h

/


4/14

QR2 Ycum index.Sum of the coefficients of determination (R2)between the dependent variables and the h firstcomponents for the dependent variables.

QR2 Xcum index.

Sum of the coefficients of determination (R2

)between the independent variables and the hfirst components for the independent variables.

These are similar to the Q2cum(h) index - but onlyfor one of the blocks of data.

Note: other programs will either a) call thesedifferent things or b) use different measures.

Rating Octane of Gasoline using Near IR.

ASTM method is complex and expensive. A simple method would be more desirable.

Experimental

Unleaded gasoline samples were assayed

by the ASTM method. NIR spectra (900-1600 nm) were obtained.

OLR, PCR and PLS models were studied.

X matrix - spectra at 20 nm intervals.

ASTM octane number by Research methodwas used as the Y matrix (vector).

A 915 nm, CH2 stretch

B 1021 nm, CH2/CH3combination band

C 1151 nm, aromaticand CH3 stretch

D 1194 nm, CH3 stretch

E 1394 nm, CH2

combination bands

F 1412 nm aromatic &CH2 combination bands

G 1435 nm aromatic &CH2 combination bands

A B

C

D

E

F

G

High R2 value and

RMSE and Press

RMSE are similar.

Only 9 variables ended up being used in

building the OLS model.

!"

!#

!$

!%

%&

%"

!" !# !$ !% %& %!"#$%&'#$()&'*+#(,

)&'*+#,

'()*+,

-./*0.)*12


5/14

!"#$%&'(')'*#$%+$,+-.&+',&/-+0$1/

!"#$

!"

!%#$

!%

!$

&

$

%

%#$

"

'( ') '$ '* '+ '' ', ,& ,% ," ,(

!"#$%&'(

*#$%+$,+-.&+

',&/-+0$1/

PCR

OLS

Note that you get a simprovement in the fiwith PCR. Might haveproblem with outliers

Also, you have a largenumber of degrees offreedom since all of t

original variables werused. With OLS, moswere discarded.

!

"

#!

#"

$!

$"

%!

%"

&$ &% &' &" &(

!"#$"%&%'

()*&%+,-.&

!

$!

'!

(!

)!

#!!

!.#.-,')+&+,0),1)-)'2345

Almost 90% of the

variance is captured inthe first component.

Over 99% in the firstthree.

!"#$

!"

!"#% !"!'#&

!'#"

!$

!"#(

!'#)

&%#"

!"#%

!"#&

!$#"

!$#"

!"#)

&%#"

&%#*

!'

&%#"

!'

!"#%

!"#$

!$#(

!"#%

&%#"

!'#*

&*#)

!"#)

&%#'

!"#%

&%#&

&%#!

!"#(

!(#!

!(#$

!(#&

!(#&!"#%

!$#"

!"

!(#!

!"#'

&%#!

&%#&

!'#)

!$#&

!$#&

!"#$

&*#%

&%#"

!"#$

!(#&

&*#*

!"#!

!$#"!$#&

!"#*

!"#'

&%#&

+)

,

)

+%, +) , ) %, %)

!"#$%&'()#*+

!($""'),*+

- ./0 1 2 3 4 50 6 4/ 0 78

!"

!#

!$

!%

%&

%"

!" !# !$ !% %& %"

!"#$%&'#$()&'*+#(,

)&'*+#

,

' ()* +, -. /* 0.) * 12 !"#$%&'(')'*#$%+$,+-.&+',&/-+0$1/

!"

!#

!$

!%

&

%

$

#

"

'# '( ') '* *% *#

!"#$%&'(

*#$%+$,+-.&+',

&/-+0$1/

!"#$%&'()%*+,&-,&.(/-$0&"1&2"/3".$.+4

!

!"#

!"$

!"%

!"&

'

' # ( $ )

5"/3".$.+4

6.#$7


6/14

PCR

OLS

PLS

!""

!#"

!$"

!%"!&"

'"""'"#"

'"$"

'"%"

'"&"''""

''#"

''$"

''%"

''&"

'#""

'##"

'#$"

'#%"

'#&"

'(""

'(#"'($"

'(%"

'(&"

'$""

'$#"

'$$"

'$%"

'$&"

')""

')#"

')$"

')%"

')&"

'%""

*+,-./01

2'

2"34)

2"3)

2"3#)

"

"3#)

"3)

"34)

'

2' 2"34) 2"3) 2"3#) " "3#) "3) "34) '

!"

!#

5

6

Similar

a scor

plot

!"

!#

!$

!%

%&

%"

!" !# !$ !% %& %"

!"#$%&'#$()&'*+#(,

)&'*+#,

'()*+,

-./*0.)*12

!"#$%&'(')'*#$%+$,+-.&+',&/-+0$1/

!"#$

!"

!%#$

!%

!$

&

$

%

%#$

"

'( '$ ') '* *% *(

!"#$%&(

*#$%+$,+-.&+',

&/-+0$1/

+,-./0 123.42-.56

For this data set, there is no significantdifference for OLR, PCR and PLS calibrationmodels.

Each produces a comparable fit of the dataand has about the same level of residualerror.

Lets look at another example - this time withmultiple dependent (Y) variables.

A series of nickel alloys were assayed by X-ray fluorescence. Four of the elements are

known to have specific spectral features thatallow prediction. There are a total of 15samples.

! Elements present! ! Si, Mn, Ni, Cr, Mo, Ti and Fe.

(bold = with spectral features)

OLS, PCR and PLS models were built andcompared.

4

0 50 100 150 200 250 3000

1

2

3

4

5

6x 10

Variable

Absorbance


7/14

Due to the limited number of samples,

there are too many X points/sample.

By creating replicate copies of the samples(in triplicate), an initial OLS was conducted

to determine which X points would have

been automatically eliminated.

This left 8 X points/sample.

Pred(% Si) / % Si

0

0.2

0.4

0.6

0.8

1

1.2

1.4

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Pred(% Si)

%S

i

% Si / Standardized residuals

-1.5

-1

-0.5

0

0.5

1

1.5

0 0.5 1

% Si

Standardizedr

esiduals

Pred(% Mn) / % Mn

0

0.5

1

1.5

2

2.5

0 0.5 1 1.5 2 2.5

Pred(% Mn)

%Mn

% Mn / Standardized residuals

-1.5

-1

-0.5

0

0.5

1

0 0.5 1 1.5 2 2.5

% Mn

Standardized

residuals

Pred(% Ni) / % Ni

0

5

10

15

20

0 5 10 15 20

Pred(% Ni)

%N

i

% Ni / Standardized residuals

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 5 10 15

% Ni

Standardizedr

esiduals


8/14

Pred(% Cr) / % Cr

5

7

9

11

13

15

17

19

21

5 7 9 11 13 15 17 19 21

Pred(% Cr)

%C

r

% Cr / Standardized residuals

-1.5

-1

-0.5

0

0.5

1

5 10 15 20

% Cr

Standardized

residuals

Pred(% Mo) / % Mo

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

Pred(% Mo)

%Mo

% Mo / Standardized residuals

-1.5

-1

-0.5

0

0.5

1

1.5

0 1 2 3 4

% Mo

Standardize

d

residuals

Pred(% Ti) / % Ti

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-0.4 -0.2 0 0.2 0.4 0.6 0.8 1

Pred(% Ti)

%T

i

% Ti / Standardized residuals

-1

-0.5

0

0.5

1

1.5

0 0.2 0.4 0.6 0.8 1

% Ti

Standardized

residuals

60

65

70

75

80

85

90

60 65 70 75 80 85 90

Pred(% Fe)

%Fe

-1.5

-1

-0.5

0

0.5

1

1.5

60 65 70 75 80 85

% Fe

Standardizedr

esiduals

PCR gives higher R2 values but look at the Press RMSE

OLS results

!

"

#

$

%

&!

&"

' & ' " ' ( '# ' ) ' $ ' * '% ' + '& ! '& & ' &" ' &( '& #

!"#$"%&%'

()*&%+,-.&

!

"!

#!

$!

%!

&!!

!.#.-,')+&+,0),1)-

)'2345


9/14

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

"#)

" "#& "#$ "#) "#* ' '#& '#$

+,-.

-/012032.452,3

56.27086

!

!"#

!"$

!"%

!"&

'

'"#

'"$

! !"# !"$ !"% !"& ' '"# '"$

()*+,-.*+/0/1,

0/1

,

!

!"#

$

$"#

%

%"#

! !"# $ $"# %

&'()*+,()-.-/0

./0

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

"#)

" "#( ' '#( & (

*+,-

./0-10213451+2

56317086

!

"

#

$

%

&!

&"

&$

! " # $ % &! &" &$

'()*+,-)*./.0+

/.0

+

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

"#)

" ( '" '(

*+,-

./012032-452+3

56-27086

!

"

##

#$

#%

#!

#"

! " ## #$ #% #! #"

'()*+,-)*./.0(

/.0

(

!"#$

!"#%

!"#&

!"#'

!"#(

!"#)

"

"#)

"#(

"#'

"#&

* + )) )' )% )* )+

,-./

012342/45674-/

785492:8

!"#$

"#$

%#$

$

'#$

(#$

$#$

!"#$ "#$ %#$ $ '#$ (#$ $#$

)*+,-./+,01023

123

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

"#)

" ' & % $

*+,-

./0120324562

+367428097

!"#$

"

"#$

"#%

"#&

"#'

(

(#$

!"#$ " "#$ "#% "#& "#' ( (#$

)*+,-./+,0102-

102

-

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

"#)

" "#& "#$ "#) "#*

+,-.

/0123143.563,

467.38197


10/14

!"

!#

$"

$#

%"

%#

&"

!" !# $" $# %" %# &"

'()*+,-)*./.0)

/.0

)

!"#$

!"#%

!"#&

!"#'

!"#(

!"#)

"

"#)

"#(

"#'

"#&

$" $% *" *% +" +% ,"

-./0

123453657805.

60975:3;9

!"#$%&'()%*+,&-,&.(/-$0&"1&2"/3".$.+4

5

567

568

569

56:

56;

56

56?

7

7 8 9 : ; < = > ? 75 77 78 79

@"/3".$.+4

A.#$B

The Q2 pattern indicates that most of the information if brought out in the first

few components but then noise is brought out to finally find a way to fit thenon-spectral species.

PLS gives an even better fit - but not ahuge improvement compared to PCR.

!

!"#

!"$

!"%

!"&

'

'"#

'"$

! !"# !"$ !"% !"& ' '"# '"$

()*+,-.*+/0/1,

0/1

,

!"#$

!"#%

!"#&

"#&

"#%

"#$

" "#' "#( "#) "#* & '

+,-.

-

/012032.452,3

56.27086

!

!"#

$

$"#

%

%"#

! !"# $ $"# % %"#

&'()*+,()-.-/0

./0

!"#$

!"#%

!"#&

"#&

"#%

"#$

" "#$ & $ ' '#$

()*+

,-.+/.0/123/

)0341/5.64

!

"

#

$

%

&!

&"

&$

! " # $ % &! & " & # & $

'()*+,-)*./.0+

/.0

+

!"#$

!"#%

!"#&

"#&

"#%

"#$

" $ &"

'()*

+,-./-0/*12/(0

23*/4-53


11/14

!

"

##

#$

#%

#!

#"

! " ## #$ #% #! #"

'()*+,-)*./.0(

/.0

(

!"#$

!"#%

!"#&

"

"#&

"#%

' ( )) )* )+ )' )( &)

,-./

012342/45674-/

785492:8

!"#$

"

"#$

"#%

"#&

"#'

(

(#$

!"#$ " "#$ "#% "#& "#' ( (#$

)*+,-./+,0102-

102

-

!"#$

"#$

%#$

$

'#$

(#$

$#$

!"#$ "#$ %#$ $ '#$ (#$ $#$

)*+,-./+,01023

123

!"#$

!"#%

!"#&

"#&

"#%

"#$

" & ' % (

)*+,

-./01/213451

*256317/86

!"#$

!"#%

!"#&

"#&

"#%

"#$

" "#' "#( "#) "#*

+-.

/0123143.563

,467.38197

Fantastic fitsconsideringthat there is

NO data tosupportthem.

!"

!#

$"

$#

%"

%#

&"

!" !# $" $# %" %# &"

'()*+,-)*./.0)

/.0

)

!"#$

!"#%

!"#&

"

"#&

"#%

$" $' (" (' )" )' *"

+-.

/012314356.3,4

.7538197

Although we were able to develop modelsthat appeared to be able to predict theamounts of all seven species, there is actuallyonly real information about four of them.

The PCR and PLS modes will produce a fitregardless of noise, lack of a positiveresponse, .

Care must be taken to ensure that your data

set contains real information about all of thecomponents.

30 different hydrocarbon blends wereassayed by UV/Vis-NIR.

Each blend had known levels of isooctane,toluene and decane but also containedother hydrocarbons at unknown levels.

Each blend was measured using twodifferent UV/Vis-NIR instruments.

Because we have more X variables thansamples, we cant use OLS.

490

510

530

550

570

590

610

630

650

670

690

710

730

750

770

790

810

830

850

870

890

910

930

950

970

990

1010

1030

1050

1 0 7 0

The data


12/14

!"#$$%&'()

!

"!

#!

$!

%!

&!

'!

(!

) " ) % ) ( ) "! ) "$ ) "' )" * ) ## ) #& ) #+ ) $" ) $% ) $( ) %! )% $ ) %' ) %* ) ) && )& +

*+,-

.,/

$01*'2$

!

#!

%!

'!

+!

"!!

3242'*),1

$1*#,*5,',)6789

Almost all of the

variance was in thefirst component.

PCR PCR

Near perfect fits but the Press RMSE indicatthat the model is subject to outliers.

!

"

! "

! "

!"

!

"

! "

! "

!"

! "

!"

!

"

!

"

!"

!

"

!

"

!

"

!

"

!

"

!

"

!

"

!

"

!

"!

"

!

"

!

"

!

"

!"

! "

!

"!

"

#!$

#!$$ #%$ $ %$ !$$

!"#$%&'%"#()

!&$*'&+()

The first PC contains mostly information aboutinstrument type. PCR discarded that PC whenbuilding the models.

Model parameters show that the first component wagiven very little weight.

!"#$%&'()**+,-.#/'0'&'()**+,-.#

!

"

#!

#"

$!

$"

%!

%"

! " #! #" $! $" %! %"

!"#$%&'()**+,-.#/

&'(

)**+,-.#

!"#$%%&'()*"+",'()-(.-/0*-".*$/-1(2$

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

" ( '" '( &" &( %" %(

!"#$%%&'()*

,'()-(.-/0*-".

*$/-1(2$

!"#$%&'()*+#,#-'.'&'()*+#,#

!

"!

#!

$!

%!

&!

'!

! "! #! $! %! &! '!

!"#$%&'()*+#,#-

&'(

)*+#

,#

!"#$%&'('")"*+,(-,.-/0'-".'1/-&,%1

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

" '" &" %" $" ("

!"#$%&'('

*+,(-,.-/0'-

".'1/-&,%1


13/14

!"#$%&'(#)*+#,'-'&'(#)*+#

!

"

#!

#"

$!

$"

%!

%"

! " #! #" $! $" %! %"

!"#$%&'(#)*+#,

&'(

#)*+#

!"#$%&'$"(")*&'+&,+-.$+",$/-+0&1/

!"#$

!"#%

!"#&

!"#'

"

"#'

"#&

"#%

"#$

"#(

" ( '" '( &" &( %" %(

!"#$%&'$

)*&'+&,+-.$+",

$/-+0&1/

!"#$%&'()%*+,&-,&.(/-$0&"1&2"/3".$.+4

!

!"#

!"$

!"%

!"&

!"'

!"(

!")

!"*

!"+

#

# $ %

5"/3".$.+4

6.#$7

PLS has a harder time with this dataset. Basicallybecause of the variations due to having two

instruments.

PLS insists on using that as latent variable.

!"

#"

!"

#"

!"

#"

!"

#"

!"#"

!"

#"

!"

#"

!"

#"

!"

#" !"

#"

!"

#"!"

#"

!"

#"!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"

#"

!"#"

!"

#"

!"

#"

!"

#"

$%"

&"

%"

$!&" $%" &" %"

!"

#

!$#

%&'()*+,-.'#-.#+/('#!$#+.0#!"#

Plot of first and second latent variables.

!"#$%&'()**+,-.#/'0'&'()**+,-.#

!

"

#!

#"

$!

$"

%!

%"

! " #! #" $! $" %! %"

!"#$%&'()**+,-.#/

&'(

)**+,-.#

!"#$%%&'()*"+",'()-(.-/0*-".*$/-1(2$

!"#$

!"

!%#$

!%

!$

&

$

%

%#$

"

& $ %& %$ "& "$ '& '$

!"#$%%&'()*

,'()-(.-/0*-

".*$/-1(2$

!"#$%&'()*+#,#-'.'&'()*+#,#

!

"!

#!

$!

%!

&!

'!

! "! #! $! %! &! '!

!"#$%&'()*+#,#-

&'(

)*+#

,#

!"#$%&'('")"*+,(-,.-/0'-".'1/-&,%1

!"

!#$%

!#

!&$%

&

&$%

#

#$%

"

& #& "& '& (& %&

!"#$%&'('

*+,(-,.-/0'-".

'1/-&,%1


14/14

!"#$%&'(#)*+#,'-'&'(#)*+#

!"

#

"

$#

$"

%#

%"

&"

!" # " $# $" %# %" &"

!"#$%&'(#)*+#,

&'(

#)*+#

!"#$%&'$"(")*&'+&,+-.$+",$/-+0&1/

!"#$

!"

!%#$

!%

!$

&

$

%

%#$

"

& $ %& %$ "& "$ '& '$

!"#$%&'$

)*&'+&,+-.$+",

$/-+0&1/

!"#$%&'()*+*$,(%(

-./0123).4'%#-2567897:5

!

!"#

$

$"#

%

%"#

$ & # ' ( $$ $& $# $' $( %$ %& %# %' %( &$ && &' &( )$ )& )# )' )( #$ #& ## #' #(

!;(&'

13 multivariate calibration

Documents