LATENT VARIABLES

AND

CORRELATION

Data and Information

Obs. no. x1 x2

1 1 -1 2 1/2 -1/2 3 -1/2 1/2 4 -1 1

1. Is there any common information in {x1, x2}?

2. If yes, can we express such common information quantitatively?

Physical Sciences

x1 = - x2

x2 = - x1

Empirical Sciences

x1

x2

1

2

3

4

e1

e2

Statistical formulation

Cor (x1,x2) = 1

||||||||21

21 xx

xxt

Correlation is the statistician's solution to expressing the strength in correspondence between variables.This reduces the graph into a number!And statisticians like numbers.

Latent variable (LV)

x1 - x2

Obs no. x1 x2 x1 - x2 1 1 -1 2 2 1/2 -1/2 1 3 -1/2 1/2 -1 4 -1 1 -2

x1

x1-x2(LV)

1

2

3

4

cor (x1, x1-x2) = 1

x2

x1-x2(LV)

1

2

3

4

cor (x2, x1-x2) = -1

The linear combination x1-x2

contains the same information as x1and x2 separately

Correlation is symmetrical in the involved variables

),(2/1

1)( 2

2/1

1)( 2

1))((

),( ij

kikjij

kikjij

ji xxcorN

ixx

N

ixx

N

ixxxx

xxcor

Direction independent

Latent variables are also symmetrical in the variables

In addition, they can handle an unlimited number of variables simultaneously!

x1 = f (LV) = f (x1 - x2)

x2 = g (LV) = g (x1 - x2)

x2

x1

PC1

e1

e2

p2

p1

PC1 = p1e1 + p2e2

Principal Component (PC)

Information and variation

Latent variable (LV)

Shows common (shared) variance (information!)in a suite of variables !

LV = w1e1 + w2e2+..+ wMeM

Key steps in the development of latent variable analysis (LVA)

i) Correlation (Galton)ii) Factors, expressions of partial correlations (Spearman, Pearson, Thurstone)

Psychometrics

Francis Galton, Proc. Royal Society of London, XL & XLV.

C.Spearman, “General Intelligence”, objectively determined and measured, American Journal of Psychology, 15 (1904) 201-293.

L.L.Thurstone, Multiple factor analysis, Psycological Review, 38 (1931) 406-427.

“the length of the arm is said to be correlated with that of the leg, because a person with a long arm has usually a long leg and conversely.”

Sir Francis Galton, Proc. Royal Society of London, XL & XLV. (1886)

“The hidden underlying causes of associations”

C. Spearman, The proof and measurement of association between two things, American Journal of Psycology, 15 (1904) 72-101.

Spearman (1904):

General factor (intelligence)

“Common source of variation”

“hidden underlying cause of the variations.”

LATENT INFORMATION!

Partial correlation between measured variables and latent variables (factors)

LVa = , a = 1, 2, ...,A

where A is the dimension of the model

eim

iwia

1

“we can arrive at estimating the correspondence of whatever may be common to the first pair of faculties with whatever may be common to the second pair. By combining such correlations of higher order, it is feasible to execute any required amount of elimination and selection, so that eventually a dissociation and exactness may be introduced into psychology such as can only be compared with quantitative chemical analysis”

Spearman (1904), American J. Psychology, p. 259

“Some time ago a psychologist told me that factors where meaningless abstractions and that people who found them where foolish to do so. The chief point of this article is to show that factors are meaningful abstractions, but that people are foolish if they don’t find them”.

K.J.Holzinger, Why do people factor?, Psychometrika, 7 (1942) 147-156.

“Let us not forget a simple principle that every scientist takes for granted, namely, that he would rather measure something significant without any sampling distribution than to measure something trivial or irrelevant because its sampling distribution is known.”

L.L.Thurstone, Multiple-factor Analysis, The University of Chicago Press, Chicago and London, 1947