factor analysis.ppt

7/24/2019 Factor Analysis.ppt

1/28

2007 Prentice Hall 19-1

Factor Analysis

2007 Prentice Hall

Advanced Statistical Modeling

Adnan ButtAssistant Professor!ra "niveristy# $arac%i


2/28


Factor Analysis Factor analysisis a general na&e denoting a class of

'rocedures 'ri&arily used for data reduction andsu&&ari(ation)

Factor analysis is an interdependence techniquein t%at anentire set of interde'endent relations%i's is e*a&ined +it%out&a,ing t%e distinction et+een de'endent and inde'endent

variales) Factor analysis is used in t%e follo+ing circu&stances.

/o identify underlying di&ensions# or factors# t%at e*'laint%e correlations a&ong a set of variales)

/o identify a ne+# s&aller# set of uncorrelated variales tore'lace t%e original set of correlated variales in suse!uent&ultivariate analysis regression or discri&inant analysis)

/o identify a s&aller set of salient variales fro& a larger setfor use in suse!uent &ultivariate analysis)


3/28

2007 Prentice Hall 19-

Factors UnderlyingSelected Psychographics

and Lifestyles

Factor 2

Footall Baseall

3vening at %o&e

Factor 1

Ho&e is est 'lace4o to a 'arty

Plays Movies


4/28


Statistics Associated with FactorAnalysis

Bartlett's test of sphericity)Bartlett6s test of s'%ericityis a test statistic used to e*a&ine t%e %y'ot%esis t%at t%evariales are uncorrelated in t%e 'o'ulation) n ot%er

+ords# t%e 'o'ulation correlation &atri* is an identity

&atri* eac% variale correlates 'erfectly +it% itself r8 1ut %as no correlation +it% t%e ot%er variales r8 0)

Correlation matrix)A correlation &atri* is a lo+ertriangle &atri* s%o+ing t%e si&'le correlations#r#

et+een all 'ossile 'airs of variales included in t%eanalysis) /%e diagonal ele&ents# +%ic% are all 1# are

usually o&itted)


5/28

2007 Prentice Hall 19-

Communality):o&&unality is t%e a&ount of variance

a variale s%ares +it% all t%e ot%er variales eingconsidered) /%is is also t%e 'ro'ortion of variancee*'lained y t%e co&&on factors)

igen!alue)/%e eigenvalue re'resents t%e total

variance e*'lained y eac% factor)

Factor loadings)Factor loadings are si&'lecorrelations et+een t%e variales and t%e factors)

Factor loading plot)A factor loading 'lot is a 'lot of

t%e original variales using t%e factor loadings ascoordinates)

Factor matrix) A factor &atri* contains t%e factorloadings of all t%e variales on all t%e factors e*tracted)



6/28

2007 Prentice Hall 19-;

Factor scores) Factor scores are co&'osite scoresesti&ated for eac% res'ondent on t%e derived factors)

"aiser#$eyer#%l&in "$%( measure of samplingadequacy)/%e $aiser-Meyer-erences et+een t%e oserved

correlations# as given in t%e in'ut correlation &atri*# and t%ere'roduced correlations# as esti&ated fro& t%e factor &atri*)

Scree plot)A scree 'lot is a 'lot of t%e 3igenvalues againstt%e nu&er of factors in order of e*traction)



7/28


Conducting Factor Analysis)SP%*+*,

*U$B) -. -/ -0 -1 -2 -3

. 4566 0566 3566 1566 /566 1566

/ .566 0566 /566 1566 2566 1566

0 3566 /566 4566 1566 .566 0566

1 1566 2566 1566 3566 /566 2566

2 .566 /566 /566 0566 3566 /566

3 3566 0566 3566 1566 /566 1566

4 2566 0566 3566 0566 1566 0566

7 3566 1566 4566 1566 .566 1566

8 0566 1566 /566 0566 3566 0566

.6 /566 3566 /566 3566 4566 3566

.. 3566 1566 4566 0566 /566 0566

./ /566 0566 .566 1566 2566 1566

.0 4566 /566 3566 1566 .566 0566

.1 1566 3566 1566 2566 0566 3566

.2 .566 0566 /566 /566 3566 1566

.3 3566 1566 3566 0566 0566 1566

.4 2566 0566 3566 0566 0566 1566

.7 4566 0566 4566 1566 .566 1566

.8 /566 1566 0566 0566 3566 0566

/6 0566 2566 0566 3566 1566 3566

/. .566 0566 /566 0566 2566 0566

// 2566 1566 2566 1566 /566 1566

/0 /566 /566 .566 2566 1566 1566

/1 1566 3566 1566 3566 1566 4566

/2 3566 2566 1566 /566 .566 1566

/3 0566 2566 1566 3566 1566 4566

/4 1566 1566 4566 /566 /566 2566

/7 0566 4566 /566 3566 1566 0566

/8 1566 3566 0566 4566 /566 4566

06 /566 0566 /566 1566 4566 /566


8/28

2007 Prentice Hall 19-?

Conducting Factor Analysis

:onstruction of t%e :orrelation Matri*

Met%od of Factor Analysis

@eter&ination of u&er of Factors

@eter&ination of Model Fit

Prole& for&ulation

:alculation ofFactor Scores

nter'retation of Factors

otation of Factors

Selection ofSurrogate =ariales


9/28


Conducting FactorAnalysisFormulate the Pro9lem /%e oCectives of factor analysis s%ould e identiDed)

/%e variales to e included in t%e factor analysis

s%ould e s'eciDed ased on 'ast researc%# t%eory#and Cudg&ent of t%e researc%er) t is i&'ortant t%att%e variales e a''ro'riately &easured on aninterval or ratio scale)

An a''ro'riate sa&'le si(e s%ould e used) As aroug% guideline# t%ere s%ould e at least four or Dveti&es as &any oservations sa&'le si(e as t%ereare variales)


10/28


Correlation$atrix

/ale 19)2


11/28


/%e analytical 'rocess is ased on a &atri* ofcorrelations et+een t%e variales)

Bartlett6s test of s'%ericity can e used to test t%e null%y'ot%esis t%at t%e variales are uncorrelated in t%e

'o'ulation. in ot%er +ords# t%e 'o'ulation correlation&atri* is an identity &atri*) f t%is %y'ot%esis cannot ereCected# t%en t%e a''ro'riateness of factor analysiss%ould e !uestioned)

Anot%er useful statistic is t%e $aiser-Meyer-


12/28


n principal components analysis#t%e total variance int%e data is considered) /%e diagonal of t%e correlation&atri* consists of unities# and full variance is roug%t intot%e factor &atri*) Princi'al co&'onents analysis isreco&&ended +%en t%e 'ri&ary concern is to deter&ine t%e&ini&u& nu&er of factors t%at +ill account for &a*i&u&

variance in t%e data for use in suse!uent &ultivariateanalysis) /%e factors are calledprincipal components)

ncommon factor analysis#t%e factors are esti&ated

ased only on t%e co&&on variance) :o&&unalities areinserted in t%e diagonal of t%e correlation &atri*) /%is&et%od is a''ro'riate +%en t%e 'ri&ary concern is toidentify t%e underlying di&ensions and t%e co&&onvariance is of interest) /%is &et%od is also ,no+n as

principal axis factoring)

Conducting Factor Analysis+etermine the $ethod of Factor

Analysis


13/28


)esults of PrincipalComponents Analysis

/ale 19)


14/28



/ale 19)#cont)


15/28



/ale 19)#cont)


16/28

2007 Prentice Hall 19-1;

/%e lo+er-left triangle contains t%e re'roducedcorrelation &atri* t%e diagonal# t%e co&&unalitiest%e u''er-rig%t triangle# t%e residuals et+een t%e

oserved correlations and t%e re'roducedcorrelations)


/ale 19)#cont)


17/28


A Priori +etermination5 So&eti&es# ecause of 'rior,no+ledge# t%e researc%er ,no+s %o+ &any factors toe*'ect and t%us can s'ecify t%e nu&er of factors to ee*tracted efore%and)

+etermination Based on igen!alues5 n t%isa''roac%# only factors +it% 3igenvalues greater t%an 1)0are retained) An 3igenvalue re'resents t%e a&ount ofvariance associated +it% t%e factor) Hence# only factors

+it% a variance greater t%an 1)0 are included) Factors +it%variance less t%an 1)0 are no etter t%an a single variale#since# due to standardi(ation# eac% variale %as a varianceof 1)0) f t%e nu&er of variales is less t%an 20# t%isa''roac% +ill result in a conservative nu&er of factors)

Conducting Factor Analysis+etermine the *um9er of

Factors


18/28

2007 Prentice Hall 19-1?

+etermination Based on Scree Plot5 A scree 'lot isa 'lot of t%e 3igenvalues against t%e nu&er of factorsin order of e*traction) 3*'eri&ental evidence indicatest%at t%e 'oint at +%ic% t%e scree egins denotes t%etrue nu&er of factors) 4enerally# t%e nu&er offactors deter&ined y a scree 'lot +ill e one or a fe+&ore t%an t%at deter&ined y t%e 3igenvalue criterion)

+etermination Based on Percentage of -ariance5n t%is a''roac% t%e nu&er of factors e*tracted isdeter&ined so t%at t%e cu&ulative 'ercentage ofvariance e*tracted y t%e factors reac%es a satisfactorylevel) t is reco&&ended t%at t%e factors e*tracteds%ould account for at least ;0 of t%e variance)

Conducting Factor Analysis+etermine the *um9er of

Factors


19/28


Scree Plot

0)

2 5

;

:o&'onent u&er

0)0

2)0

)0

3igenva

lue

1)0

1)

2)

1

Fig) 19)


20/28


+etermination Based on Split#;alf )elia9ility5/%e sa&'le is s'lit in %alf and factor analysis is

'erfor&ed on eac% %alf)


21/28


Alt%oug% t%e initial or unrotated factor &atri* indicatest%e relations%i' et+een t%e factors and individualvariales# it seldo& results in factors t%at can einter'reted# ecause t%e factors are correlated +it%

&any variales) /%erefore# t%roug% rotation t%e factor&atri* is transfor&ed into a si&'ler one t%at is easier tointer'ret)

n rotating t%e factors# +e +ould li,e eac% factor to %avenon(ero# or signiDcant# loadings or coeGcients for only

so&e of t%e variales) i,e+ise# +e +ould li,e eac%variale to %ave non(ero or signiDcant loadings +it%only a fe+ factors# if 'ossile +it% only one)

/%e rotation is called orthogonal rotationif t%e a*esare &aintained at rig%t angles)

Conducting FactorAnalysis

)otate Factors


22/28


/%e &ost co&&only used &et%od for rotation is t%e!arimax procedure) /%is is an ort%ogonal &et%odof rotation t%at &ini&i(es t%e nu&er of variales

+it% %ig% loadings on a factor# t%erey en%ancing t%einter'retaility of t%e factors)


23/28


Factor $atrix Before and After)otation

Factors

a

Hig% oadingsBefore otation

Fig) 19)5

Hig% oadingsAfter otation

Factors

=ariales

1

2

5

;

1

I

II

I

I

2

I

I

I

I

1

I

I

I

2

I

I

I

=ariales

1

2

5

;


24/28


A factor can t%en e inter'reted in ter&s oft%e variales t%at load %ig% on it)

Anot%er useful aid in inter'retation is to 'lott%e variales# using t%e factor loadings as

coordinates) =ariales at t%e end of an a*isare t%ose t%at %ave %ig% loadings on onlyt%at factor# and %ence descrie t%e factor)

Conducting FactorAnalysis=nterpret Factors


25/28


Factor LoadingPlot

Fig) 19)

.56

652

656

#652

#.56

Component.

Component Plotin)otated Space

.56 652 656 #652 #.56

-.

-0

-3

-/

-2

-1

Component-aria9le . /

-. 6583/ #/533#

6/

-/ #254/#6/ 65717

-0 65801 #65.13

-1 #8570#6/ 65721

-2 #65800 #7516#6/

-3 75004#6/ 65772

)otated Component $atrix

Component /


26/28

2007 Prentice Hall 19-2;

/%e factor scoresfor t%e it% factor &aye esti&ated

as follo+s.

Fi= Wi1X1+ Wi2X2+ Wi3X3+ . . . + WikXk

Conducting FactorAnalysisCalculate Factor Scores


27/28


By e*a&ining t%e factor &atri*# one could selectfor eac% factor t%e variale +it% t%e %ig%est

loading on t%at factor) /%at variale could t%en eused as a surrogate variale for t%e associatedfactor)

Ho+ever# t%e c%oice is not as easy if t+o or &ore

variales %ave si&ilarly %ig% loadings) n suc% acase# t%e c%oice et+een t%ese variales s%oulde ased on t%eoretical and &easure&entconsiderations)

Conducting Factor

AnalysisSelect Surrogate-aria9les


28/28

2007 Prentice Hall 19-2?

/%e correlations et+een t%e variales can e

deduced or re'roduced fro& t%e esti&atedcorrelations et+een t%e variales and t%efactors)

/%e di>erences et+een t%e oserved correlationsas given in t%e in'ut correlation &atri* and t%ere'roduced correlations as esti&ated fro& t%efactor &atri* can e e*a&ined to deter&ine&odel Dt) /%ese di>erences are called residuals)

Conducting FactorAnalysis+etermine the $odel Fit

factor analysis.ppt

Documents