Factor Analysis This tutorial will show you how to carry out a factor analysis in SPSS. Factor analysis allows you to look at the relationship between a large number of variables (for example, questions on a questionnaire), and see whether they can be grouped and summarised using a smaller number of factors (or latent variables). Latent variables are not measured directly, but are hidden underneath your data, influencing the scores on variables that you do have. The key concept here is that groups of variables may be related to one another, because they are all associated with the same underlying factor. Factor analysis enables you to group variables together to identify and interpret what those factors might be; which in turn helps you to better understand the variance in your data. There are different types of factor analysis, and different methods for carrying it out. This tutorial will focus on exploratory factor analysis using principal components analysis (PCA). Exploratory factor analysis is used when you do not have a pre-defined idea of the structure or number of factors there might be in a set of data. As such it tends to be used to explore newly developed questionnaires. PCA aims to reduce a set of variables into a smaller, more meaningful set of factors by looking for clusters of variables that appear to be related to one another (and therefore may be tapping into the same underlying factor). PCA is primarily concerned with identifying variables that share variance with one another. Worked Example In this tutorial, we will look at how to use factor analysis to analyse a newly developed questionnaire designed to measure teachers’ beliefs about different approaches to learning, and their attitudes towards signing and inclusivity in the classroom. In this example, 600 participants completed a questionnaire examining views on teaching styles by agreeing or disagreeing with a series of statements about how children learn. The questionnaire had 15 items (variables) and answers were given on a scale of 1 to 5, where: 1 = Strongly agree 2 = Agree 3 = Neither agree nor disagree 4 = Disagree 5 = Strongly disagree

This is what the data looks like in SPSS (you can download this yourself, from datafile: ‘Week 7 FA Data.sav’:

Remember, in SPSS all data from one participant needs to go in one row. On a questionnaire, each question represents a different variable… so we need one column for each question. As questionnaires consist of multiple questions, you will have a dataset with a large number of columns. The aim of factor analysis is to reduce large sets of variables, to a smaller number of latent factors. To start the analysis, CLICK on Analyze, then Dimension Reduction and Factor.

This opens the Factor Analysis dialog box. Here we need to tell SPSS which variables we want to include in the analysis. As we want to run the factor analysis on the whole questionnaire, we need to select all of the variables, as shown here. Then move them across to the Variables box by CLICKING on the top blue arrow.

Now, CLICK on the Descriptives… option to tell SPSS what output you want it to produce.

As factor analysis looks for relationships between variables, we first need to establish that these relationships do, in fact, exist. To do this, we need to produce a Correlation Matrix. To do this, SELECT the Coefficients option. To ensure accurate correlations (and factor analysis), it’s important that we have a fairly large sample size. And for factor analysis to be meaningful, we have to have a suitable number of correlations between variables. The Kaiser-Meyer-Olkin (KMO) test of sampling adequacy can tell us whether our sample size is sufficient. And Bartlett’s test of sphericity indicates whether we have enough correlations.

Once you have selected these options, CLICK Continue to return to the main Factor Analysis Dialog box. From here, we want to tell SPSS how we want it to extract our factors. To do this, CLICK on the Extraction option.

There are a number of different methods that can be used to extract factors from our data.

In this case, we are using Principal Components Analysis, so we want to make sure this

option is selected (which it is).

Factor analysis will always

extract as many factors as

there are variables. In this case

we have 15 questions on the

questionnaire, so 15 factors will

be extracted.

BUT – a lot of these factors will

be meaningless. As such, we

want to extract the smallest

number of factors that we can,

that best explains the patterns

in our data.

There are a number of different methods you can use to decide how many factors you want

to keep, and which make the most sense. One method commonly used is the Scree Plot, so

we need to SELECT this option first (as shown above).

As this factor analysis is exploratory, we don’t know how many factors we want the analysis

to extract. As such, we keep the default option that extracts factors with eigenvalues over

1. However, if you have an idea of how many factors you are expecting (for example,

because you have already run your analysis and tried to interpret the results), you can ask

SPSS to produce that specific number of factors by selecting the ‘Fixed number of factors’

option.

For future reference, you may need to increase the number in the ‘Maximum Iterations for

Convergence’ box if you have tried to run your analysis and the output tells you that a

solution could not be found in 25 iterations.

Once you have chosen your Extraction options, CLICK Continue to return to the main Factor

Analysis Dialog box. From here, we need to tell SPSS what type of rotation we want it to use

when finding a factor solution. To do this, CLICK the Rotation button.

Rotation maximises the loading of each of your variables onto one factor, while minimizing it’s loading on the others. This should help you when it comes to interpreting what the factors represent.

In most instances, like this one, you will use Varimax, which is appropriate when you think your factors are likely to be independent of one another. However, if research suggests your factors are likely to be related (i.e. highly correlated), then Direct Oblimin should be your choice here. Again, if you have tried to run your analysis

and the output tells you that a solution could

not be found in 25 iterations, you may need to

increase the number in the final box.

Once you have chosen your Rotation options, CLICK Continue to return to the main Factor

Analysis Dialog box. If you plan to run some form of analysis on how your participants

scored on the different factors you extracted… you can ask SPSS to save these scores. To do

this, CLICK on the Scores… button.

To create and save participants scores for each factor that you produce, SELECT the Save as variables option. CLICK continue

Once back at the Factor Analysis dialog box, CLICK on the final Options button.

To help us interpret the SPSS output, we want to ask SPSS to list the variables that load onto each factor in order – so question items that are most strongly related to the factor appear first. This will help us interpret what these factors are likely to represent.

To do this, SELECT the Sorted by size option. To further help with interpretation, we want to filter out any variables that are only weakly related to each factor. To do this SELECT the Suppress small coefficients option. For even more help, we can change the Absolute value below option to .40, so that any extremely weak loadings below this cut-off will not be displayed. CLICK continue to return to the Factor Analysis dialog box. And CLICK OK to run the factor analysis.

SPSS produces the results of the Factor Analysis in the Output window.

This tutorial will now go through the relevant output box-by-box. Correlation Matrix

The first box displays the correlation matrix for your data. As SPSS produced a huge correlation table, it has been cropped here for the tutorial. As Factor Analysis looks for relationships between the variables, there need to be at least some moderate-to-high correlations in your data (i.e. correlations above the value of r=0.3). There are plenty of moderate correlations here (see above) suggesting the analysis is appropriate… but in factor analysis we also want to avoid multicollinearity (i.e. any overly high correlations of r>0.9). If this is a problem, you may want to consider removing one of the questions with the high correlation. As this isn’t a problem in the data set, we can continue with the output.

KMO and Bartlett’s Test The next box in our output displays two statistics we need to look at to confirm that Factor Analysis is appropriate for our data set.

First, the Kaiser-Meyer-Olkin test of sampling adequacy assesses whether or not our sample size is sufficient for factor analysis. A value of less than 0.5 indicates the sample is too small, but ideally we are aiming for 0.7 or above. In this case the value is KMO = .87, which means our sample size is sufficient. The second statistic is Bartlett’s test of sphericity which tells us whether we have an adequate number of correlations between our variables for factor analysis. In this case we are looking for a significance value of less than your alpha level (i.e. p<0.05), just like ANOVA. In this case the value is p < .001, which means that we have enough correlations for factor analysis. Communalities

We don’t need the next table to interpret our output. Skip to the next output box

Total Variance Explained There are three key components to this table, which are highlighted here.

Initial Eigenvalues: The first three columns list all of the factors that can be found within the data set. As factor analysis always extracts as many factors as there are variables, in this case there are 15 factors in total.

The % of Variance column tells you how much of the variance in the dataset can be explained by each factor. The first few factors account for relatively large proportions of the variance compared to the latter factors. We are really only interested in extracting factors that account for a meaningful amount of variance.

Extraction Sums of Squared Loadings: The middle set of columns is almost identical to the first, except it only displays the factors that account for a significant amount of variance in our data.

As we asked SPSS to use a criterion of eigenvalues over 1 for extraction, this section only displays the factors that meet this criteria. The eigenvalue for each factor (before rotation) can be seen in the Total column. In this example, SPSS has extracted four factors as a result of the factor analysis.

Rotation Sums of Squared Loadings: The final set of columns gives the eigenvalues of the extracted factors after rotation has taken place.

Rotation maximises the loading of each of your variables onto one factor, while minimising its loading on the others. This optimises the factor loadings which also brings the eigenvalues more into line with one another.

When reporting how much variance each factor accounts for, you want to use this set of columns.

Scree Plot So far SPSS has extracted four factors. But how many factors you actually end up extracting is up to you - not SPSS… so you need to consider other options. This graph plots all 15 eigenvalues for your factors. This can help visualise which factors to keep. These plots often show a point in the curve (or 'elbow') where the eigenvalues drop off and level out. Eigenvalues above this point may be important enough to retain, whereas the others may not.

Scree plot curves can often be difficult to interpret. For example, in this case the graph appears to tail off after 2 factors… but there is also another drop after 4. So using this method of extraction, you may be able to justify either 2 or 4 factors here. To determine exactly how many factors to retain, you may want to run your analysis a few times exploring the different factor options and see which one makes the most sense. Unlike many statistics which are black and white, Factor Analysis is more grey… it is an exploratory tool and should only be used as a guide to you, the researcher.

Component Matrix This table tells you how each variable loads onto each of four factors before rotation. We are not really interested unrotated factor solution, so scroll down to the (very similar looking) next table. Rotated Component Matrix This is the most important table in your output. It tells you how each variable loads onto each of four factors after rotation, and to what extent. This allows you interpret what each of your extracted factors might represent. Factor Analysis only tells you which variables group together mathematically - it’s up to you, the researcher, to interpret what this means. To establish what your factors might be, you need to look at all of the variables that load onto them and try to establish a common theme. For example, looking at the 7 variables that load onto the Factor 1, they all seem to be about aspects of social interaction in the learning process. As such, we might want to name this factor ‘Social Learning’.

The variables that load most strongly onto Factor 2 seem to refer to opinions surrounding the inclusion of children with special needs in the classroom and in teaching practices. In this case, we might want to name this factor ‘Inclusivity’. For Factor 3, items seem to refer to the importance of student-directed vs teacher-directed learning. Negative loadings simply have the opposite relationship with the factor than positive loadings... although the factor name should reflect the positive relationships. So in this case, we could call this factor: ‘Self-directed learning’. Finally, Factor 4 seems to comprise items referring to the importance of clear and well explained solutions in learning. As such, we could call this factor: ‘Clarity’.

In some cases, variables load onto more than one factor. In which case, you have to decide which one is most appropriate in terms of interpretability. In this case, the 6th and 13th variables refer directly to the student's or teacher's role in learning. As such, it seems that both of these variables belong with Factor 3 (rather than Factors 1 or 4).

Component Transformation Matrix

This final table is not needed for interpreting the Factor Analysis.

How do we write up our results?

When writing up the results of your factor analysis you need to include all of the relevant information covered in this tutorial:

1. State how many questions you analysed and the type of analysis used:

Fifteen questions relating to teaching style and attitude were factor analysed using principal components analysis with varimax rotation.

2. Confirm that factor analysis was appropriate by reporting the KMO measure of sampling adequacy and the Bartlett’s test of sphericity:

Kaiser-Meyer-Olkin measure of sampling adequacy was .87, above the commonly recommended value of .6, and Bartlett’s test of sphericity was significant (χ2 (105) = 2983.77, p < .001).

3. Say how many factors you have found, how you extracted them, and how much variance is explained overall by these factors:

Using both the scree plot and eigenvalues > 1 to determine the underlying components, the analysis yielded four factors explaining a total of 60.66 per cent of the variance in the data.

4. Explain what each of your factors represent, with examples of the questions that

load on to them, and how much variance each explains. For example, for Factor 1 you could say:

Factor 1 was labelled ‘social learning’ because of the high loadings by the following items: children learn best through collaborative activities; helping children to talk to one another in class productively is a good way of teaching; meaningful learning takes place when individuals are engaged in social activities. This first factor explained 22.53 per cent of the variance after rotation.

You need to include similar descriptions for each of the other three Factors: Inclusivity, Self-directed learning, Clarity… along with your rotated components matrix table. You may also want to include a copy of your scree plot in the report. This brings us to the end of this tutorial. Why not download the data file from this tutorial and see if you can run the analysis yourself.