1

Stanovich and West’s (2007) Actively Open-Minded Thinking Scale: An Examination of Factor

Structure

Kimberly R. Marsh

Dena A. Pastor

James Madison University

Paper presented at the annual meeting of the Association for Psychological Science, Washington,

D.C., May 2011

2

Abstract

Research examining the divergence of human thinking from what is considered to be the ideal

standard has indicated that human thinking is often prone to error and everyday thinking

processes, judgments, and decision making are negatively affected by our own biased

viewpoints. These cognitive shortcomings have inspired researchers to examine characteristics of

what they deem to be “good thinking” such as actively open-minded thinking. In order to

examine the relationship between characteristics of good thinking and performance on critical

thinking tasks, Stanovich and West (2007) developed their own Actively Open-Minded Thinking

(AOT) scale. The current study examined the factor structure of Stanovich and West’s (2007)

AOT scale by examining multiple factor solutions via exploratory factor analysis. Results

indicated that AOT items did not load on the six subscales originally developed by Stanovich

and West (2007) nor were they unidimensional; instead, a 4-factor solution emerged. Theoretical

and practical implications of these findings are discussed.

3

Stanovich and West’s (2007) Actively Open-Minded Thinking Scale: An Examination of Factor

Structure

There is an extensive body of research examining the divergence of human thinking from

what is considered to be the ideal standard. Perhaps unsurprisingly, this research has indicated

that human thinking is often prone to error and everyday thinking processes, judgments, and

decision making are negatively affected by our own biased viewpoints. Knowledge of these

common cognitive shortcomings has inspired research that seeks to identify aspects of good

thinking practices. Ideally, in identifying aspects of good thinking, researchers in this domain

may identify some individual skills or characteristics that could be cultivated and lend

themselves to the improvement of reasoning.

According to Baron (1991), “good thinking” is a complex process that incorporates,

“optimal search for possibilities, evidence, and goals, and fairness in the search for evidence and

in inference” (p. 172). Conversely, Baron characterizes aspects of “poor thinking” as inadequate

search for evidence and the general tendency of people to allow their own biases to affect their

reasoning (Baron, 1991, p. 172). Baron further delineates good and poor thinking by identifying

what he believes to be the defining aspect of good thinking; that it is actively open-minded. This

actively open-minded thinking component of good thinking incorporates a search for evidence,

the extent to which is determined by the importance of the question, and an objectivity in one’s

consideration and review of the evidence (Baron, 1991).

There has been some research conducted in the area of cognitive psychology that

suggests the distinct nature of cognitive ability (i.e., general intelligence) and individual thinking

dispositions, such as actively open-minded thinking (Baron, 1985, 1988, 1991; Ennis, 1987;

Moshman, 1994; Norris, 1992; Perkins, Faraday, & Bushey, 1991). More specifically, there is

4

some evidence to suggest that each of these variables is able to account for unique variance in

performance on critical thinking tasks, suggesting that cognitive ability and individual thinking

dispositions may address different aspects of what is considered to be good thinking (Stanovich

& West, 1997, 1998, 2007). Stanovich and West have devoted much of their research to

examining the nature of the relationship between cognitive ability, individual thinking

dispositions, such as actively open-minded thinking, and whether these constructs can uniquely

predict performance on a number of cognitive tasks. For instance, Stanovich and West (1997)

evaluated individual performance on an instrument they refer to as the Argument Evaluation

Task. Participants were instructed to objectively evaluate the quality of a number of arguments

crafted by a fictitious individual after previously indicating their own opinions and thoughts on

the subject matter of these arguments (e.g., social and political issues, such as, crime, gun

control). Stanovich and West (1997) found that after controlling for cognitive ability (e.g., self-

reported SAT scores), individual thinking disposition variables, such as actively open-minded

thinking accounted for a significant amount of variance in performance on the Argument

Evaluation Task; ultimately providing more support for the argument that cognitive ability and

individual thinking disposition variables may both contribute to characteristics of critical

thinking.

Stanovich and West (1997, 1998, 2007) have been instrumental in the development and

utilization of multiple versions of their own Actively Open-minded Thinking scale (AOT).

Stanovich and West (1997) conceptualized actively open-minded thinking as the tendency of

people to engage in flexible or open-minded thinking and avoid dogmatic or categorical thinking.

Stemming from this definition of actively open-minded thinking, six subscales were

conceptualized for the current Stanovich and West (2007) 41 item version of the AOT scale:

5

Flexible thinking (Stanovich & West, 1997), Openness-Values (Costa & McCrae, 1992),

Dogmatism (Paulhus & Reid, 1991; Robinson, Shaver, & Wrightsman, 1991; Troldahl & Powell,

1965), Categorical thinking (Epstein & Meier, 1989), Belief identification (Sá et al., 1999), and

Counterfactual thinking (Stanovich & West, 1997). Although there have been multiple versions

of the AOT scale (Sá, West, & Stanovich, 1999; Stanovich & West, 1997) consisting of different

combinations of items and subscales leading up to the most current, 41 item version of Stanovich

and West’s (2007) AOT scale, the rationale behind scale modifications and revisions is unclear.

Purpose of the Current Study

To date, no published factor analysis studies have been conducted on the most current, 41

item version of Stanovich and West’s (2007) AOT scale. Stanovich and West (2007) currently

suggest summing participant item responses for the six subscales, indicating that they believe the

actively open-minded construct to be unidimensional. The purpose of the current study is to

perform exploratory factor analysis (EFA) on the most current version of the Stanovich and West

(2007) AOT scale in order to examine the emergent factor structure. An EFA will inform the

validity of the AOT scores by indicating the extent to which the relationships among items

conform to Stanovich and West’s conceptualization of the construct, aid in identifying items not

functioning as intended, and inform understanding of the actively open-minded thinking

construct itself.

Methods

Participants and Procedures

Undergraduate students from a mid-sized university participated in the current study.

Participants registered to participate through the university subject pool and were awarded class

credit for an introductory psychology course in exchange for their participation in the study.

6

Undergraduate research assistants conducted study sessions in an on-campus computer lab,

running four participants at a time. Participants completed a series of measures in addition to the

Stanovich and West (2007) AOT scale via Qualtrics Survey Software.

A total of 452 students initially completed the survey, however, seven participants were

listwise deleted due to missing data. The remaining sample (N = 445) was screened for both

univariate and multivariate normality. Upon data screening, variable distributions were deemed

univariate normal (skewness and kurtosis values falling within the acceptable range of |2| and |7|,

respectively; Kline, 2005). Using DeCarlo’s (1997) macro to screen for multivariate outliers, 15

cases were flagged as outliers at the .05 significance level. These cases were removed from the

data set, resulting in the final sample used for the subsequent EFA (N = 430). Of the 430

undergraduate students who were included within the analyses for the current study, the majority

were first-years (70.2%), female (70.2%), and Caucasian (85.1%). Mean participant age was

19.00 years (SD = 1.43).

Actively Open-minded Thinking (AOT) Scale. Participants completed Stanovich and

West’s (2007) 41 item version of the AOT. This version of the AOT consists of six subscales

including: Flexible thinking (Stanovich & West, 1997), Openness-Values (Costa & McCrae,

1992), Dogmatism (Paulhus & Reid, 1991; Robinson et al., 1991; Troldahl & Powell, 1965),

Categorical thinking (Epstein & Meier, 1989), Belief identification (Sá et al., 1999), and

Counterfactual thinking (Stanovich & West, 1997). Participants indicated their agreement with

each of the 41 statements on a Likert scale consisting of 6 response options: Disagree Strongly

(1), Disagree Moderately (2), Disagree Slightly (3), Agree Slightly (4), Agree Moderately (5),

and Agree Strongly (6). Items were reflected in such a way that higher AOT scores indicated a

7

higher level of agreement with items that endorsed actively open-minded thinking. Cronbach’s

alpha for the Stanovich and West (2007) AOT scale was .83.

Results

Data Screening

Descriptive statistics for each of the Stanovich and West (2007) AOT scale items are

presented in Table 1. Referring to the estimates provided in Table 1, we can see that, overall, the

distribution of the variables are univariate normal with skewness and kurtosis values falling

within the acceptable range of |2| and |7|, respectively (Kline, 2005). It should be noted that these

descriptive statistics are based upon the reduced sample (N = 430) utilized within the analyses

after the data were screened and cleaned.

AOT inter-item Pearson correlation coefficients ranged from -.21 to .48, indicating that

multicollinearity was not an issue within the current sample. It should be noted that these inter-

item correlations are rather low; suggesting that AOT items are not substantially related to one

another and thus factor analysis may be inappropriate (Nunnally & Bernstein, 1994). However,

Bartlett’s test of sphericity, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO)

index, and the Measure of Sampling Adequacy (MSA) indices were still examined in order to

determine whether it was appropriate move forward with factor analysis.

Bartlett’s test of sphericity (2(820) = 4088.23, p < .001) indicated that we should reject

the null hypothesis that the AOT variables are completely uncorrelated, thus suggesting that the

variables are inter-correlated and it is appropriate to run a parallel analysis procedure and

conduct factor analysis (Gorsuch, 1983). The KMO index (KMO = .85) approached one,

indicating that the correlation matrix was amenable to factoring. This KMO index value is

considered to be “meritorious” by Kaiser’s (1974) standards, indicating that the AOT variables

8

are likely to share common factors. Based upon the values in the anti-image correlation matrix,

we can conclude that the amenability of the variables to factoring is acceptable. Overall, the

Measure of Sampling Adequacy (MSA) values along the diagonal in the anti-image correlation

matrix were fairly high (approaching a value of one), indicating that each of the AOT variables

share an adequate amount of variance with the remaining AOT variables. Furthermore, the

absolute values on the off-diagonal of the anti-image correlation matrix were rather small (close

to 0). That is, the negative partial correlations amongst the AOT variables were quite small,

indicating that any combination of two AOT variables do not relate to one another after

controlling for how these two variables relate to the remaining AOT variables.

Exploratory Factor Analysis

Both a scree plot of eigenvalues based on the reduced correlation matrix and the results

of a parallel analysis (Horn, 1965) were examined in order to determine the number of factor

solutions for review (see Figure 1). All possible AOT factor eigenvalues were plotted on the

scree plot. Upon visual examination of the scree plot, it appears that plotted eigenvalues begin to

“level off” after the first seven plotted eigenvalues, suggesting that these subsequent factors

account for insignificant amounts of variance in AOT scores above and beyond that accounted

for by the first seven factors. The results of the parallel analysis procedure mirror those of the

visual examination of the scree plot suggesting the extraction of seven factors based upon the

fact that the first seven eigenvalues extracted from the AOT data set are greater than those

eigenvalues calculated based upon randomly generated data sets that have similar characteristics.

Because of the exploratory nature of the analyses, all seven factor solutions (i.e., 1-factor, 2-

factor, 3-factor, etc.) were subsequently examined. However, only the results for the 4-factor

9

solution are presented and discussed in detail due to the fact that this emergent factor solution

displayed what most closely resembles simple structure and was most interpretable.

Table 2 displays the resulting pattern coefficients for the 4-factor EFA solution using

Direct Oblimin rotation with delta set to zero. Pattern coefficients provide the most interpretable

factor loading estimates as they reflect the correlation between each item and the factor after

controlling for the inter-correlations amongst other items. Item loading criteria employed within

the current study was |.40|, thus items were deemed to load on a factor if their pattern coefficient

was greater than or equal to |.40|.

Results indicated that AOT item functioning was problematic. There were nine items that

failed to load on any of the factors across the seven factor solutions examined. Failure of these

items to load on any factor may indicate that these items are confusing for respondents or they

fail to measure similar aspects of the actively open-minded thinking construct. There were nine

additional items that failed to load on any of the four factors in the 4-factor solution. This may

also suggest that these items are problematic for respondents.

There were 13 items that loaded solely on the first factor of the 4-factor solution and

accounted for 15.84% of the total variance in AOT scores. Nine of the 13 items that loaded on

the first factor in the 4-factor solution consistently loaded on the first factor across multiple

solutions. Overall, these items appeared to represent participant proclivity to engage in a kind of

rigid, black and white thinking style that is characterized by dogmatic and categorical thinking.

This suggests that these previously hypothesized aspects of actively open-minded thinking

merged together to form one factor.

There were five items that loaded solely on the second factor of the 4-factor solution and

accounted for 4.33% of the total variance in AOT scores. Overall, items that loaded on this

10

second emergent factor appeared to reflect aspects of flexible, open-minded thinking. Similar to

those items that loaded on factor 1, the most salient item loadings on factor 2 were items that

were originally identified as two aspects of the actively open-minded thinking construct, flexible

thinking and openness- values; thus, it appears as though these two previously hypothesized

aspects of actively open-minded thinking have merged to form the second emergent factor in the

4-factor solution. Items that loaded on factors 3 and 4 were relatively minor, or secondary, in

respect to the first two emergent factors of the 4-factor solution. There were three items that

loaded on factor 3 and accounted for 3.36% of the total variance in AOT scores. Unlike the first

two emergent factors in the 4-factor solution, these items did not appear to be the result of

previously specified aspects of the actively open-minded thinking construct merging to form one

factor. Moreover, these items appear to reflect a “perseverance in your beliefs” factor not

originally hypothesized.

Finally, there were only two items that loaded on factor 4 and accounted for 2.63% of the

total variance in AOT scores. These two items were originally specified as the counterfactual

thinking subscale by Stanovich and West (1997, 2007). Although these items load on their

originally hypothesized factor, they do not account for much variance in AOT scores suggesting

that counterfactual thinking may not be a central component of actively open-minded thinking.

Inter-factor correlations for the 4-factor solution were rather low, ranging from .05 to .23;

providing further evidence to suggest the distinct nature of AOT items statements that load on

each of the 4 emergent factors. AOT item communalities reported in Table 2 range from .06 to

.48. Item communality estimates indicate the amount of common variance in the item explained

by the factor solution. These item communality estimates are quite low. We would hope that

these item communality estimates would be higher, indicating that the set of four emergent

11

factors explains a substantial amount of common variance within a specific item; however, this is

not the case.

The total variance accounted for in AOT score by the four emergent factors combined

was 26.15%. It should be noted that this estimate of total variance reflects overlap amongst the

factors as the rotation method allowed them to inter-correlate, thus it reflects a slight

overestimate of the variance accounted for by the factors. The total variance estimate is rather

low, leaving approximately 74% of the variance in AOT scale scores unaccounted for by the four

emergent factors. The large amount of variance left unaccounted for in AOT scores could have

several implications for the current version of the AOT scale. First, item wording could be such

that it confuses participants, causing them to respond in a contradictory manner to items meant to

address similar aspects of the construct. Secondly, item statements could be measuring distinct

concepts that do not covary, or relate in such a way, to adequately represent Stanovich and

West’s (1997, 2007) original conceptualization of actively open-minded thinking.

Discussion

The purpose of the current study was to examine the emergent factor structure of

Stanovich and West’s (2007) AOT scale in an effort to determine whether scores on the AOT

accurately represent actively open-minded thinking as conceptualized by Stanovich and West

(1997, 2007). Stanovich and West (2007) suggest calculating an AOT scale total score by

summing participant responses across item statements originally developed to address six facets

of the actively open-minded thinking construct, including, dogmatic thinking, categorical

thinking, flexible thinking, openness to values, belief identification, and counterfactual thinking.

The suggested calculation of an AOT total score implies that Stanovich and West believe the

scale to be unidimensional. However, this does not appear to be the case based upon the findings

12

of the current study. AOT items did not load on the six subscales originally developed by

Stanovich and West (2007) nor were they unidimensional; instead, a 4-factor solution emerged.

EFA results not only indicated that calculation of an AOT total score is inappropriate, but

that the current version of the Stanovich and West (2007) AOT scale needs to be revised and

conceptualization of the actively open-minded thinking construct revisited. Examination of the

AOT inter-item correlation matrix revealed that inter-item correlations were quite low, indicating

that the 41 items currently on the AOT scale may be measuring distinct, or seemingly unrelated

concepts. Given the low inter-item correlations, it is not surprising that although the EFA

resulted in an interpretable 4-factor solution, there were a series of problematic items identified

in need of substantial revision or removal (e.g., items that failed to load across factor solutions,

items that failed to load on any of the emergent four factors). In addition, the emergent factors

failed to adequately represent theorized aspects of actively open-minded thinking (e.g., factor 3

which did not appear to represent a theorized aspect of actively open-minded thinking, factor 4

which only consisted of two items). Moreover, estimates regarding the percentage of variance

accounted for in AOT scores by each of the four factors, item communalities, and total variance

accounted for in AOT scores by all four factors further support the distinct nature of AOT item

statements.

Confusing AOT scale item content and/or wording may contribute to the distinctness of

the item statements and, ultimately, the failure of items to form factors, particularly factors that

are able to explain a large percentage of variance in scores. For example, one of the nine items

that failed to load on any of the factors across multiple solutions reads, “Even though freedom of

speech for all groups is a worthwhile goal, it is unfortunately necessary to restrict the freedom of

certain political groups”. The wording for this particular item is confusing and it may be difficult

13

for respondents to decipher the meaning of the statement. Another item statement that failed to

load on any factor solution reads, “My blood boils over whenever a person stubbornly refuses to

admit he’s wrong”. This particular item does not seem to relate to the other items on the scale in

any logical or intuitive way. There appear to be other aspects of AOT items that pose additional

threats to comprehension, such as wording that encourages respondents to misconstrue subject

matter. For example, there are several items that inquire about one’s “beliefs” or opinions about

“lifestyles” (e.g., “It is important to persevere in your beliefs even when evidence is brought to

bear against them”, “I consider myself broad-minded and tolerant of people’s lifestyles”).

Wording such as this may cause the respondent to think of religious beliefs or about one’s

“lifestyle” in regards to sexual orientation, thus causing participants to respond to the items in

differential manners based upon their own interpretation of the item content. There is evidence to

suggest that respondents are interpreting items in different ways. Zilberberg and Pastor (2009)

conducted a study in which they asked respondents to “think aloud” as there were responding to

a subset of items statements from the AOT scale. The think aloud exercise revealed that item

wording was ambiguous and participants had differential conceptualizations of item subject

matter (Zilberberg & Pastor, 2009).

The emergent 4-factor solution offered insight regarding how AOT scores reflect

Stanovich and West’s (2007) conceptualization of actively open-minded thinking. As mentioned

previously, Stanovich and West (1997, 2007) originally drafted items to reflect six facets of

actively open-minded thinking, but two of the four factors within the emergent 4-factor solution

appeared to be the result of a merging of originally hypothesized subscales. More specifically,

factor 1 seemed to represent a merging of Stanovich and West’s (2007) originally specified

dogmatic and categorical thinking subscales, with items from each subscale loading on the

14

factor. These items reflected a similar, concrete or dichotomizing thinking style. Similar to factor

1, factor 2 appeared to be the result of a merging of previously specified subscales. Items loading

on factor 2 appeared to represent the merging of the flexible thinking and openness to values

subscales originally specified by Stanovich and West (2007), with items from both subscales

loading on the factor to represent an emergent factor that reflected a flexible/open-minded

thinking style.

One could make the argument that none of the items loading on the four factors in the

emergent 4-factor solution address central components of actively open-minded thinking

previously detailed by researchers who have defined the construct. However, items loading on

factors 3 and 4 in the 4-factor solution seem to be addressing thinking characteristics particularly

tangential to the actively open-minded thinking construct. More specifically, items loading on

factor 3 appeared to represent a previously unspecified aspect of Stanovich and West’s (2007)

concept of actively open-minded thinking construct, with these items reflecting perseverance in

one’s beliefs. Factor 4 was defined by only two items with salient factor loadings that reflected

the propensity of individuals to engage in counterfactual thinking. Although these two items that

loaded on factor 4 were the two items originally developed to represent Stanovich and West’s

(2007) counterfactual thinking subscale, it is unclear as to whether they should represent core

aspects of actively open-minded thinking. The presence of these two factors (factors 3 and 4) in

particular raises the question of what are the essential components of actively open-minded

thinking.

Stanovich and West (1997) provide one way of thinking about actively open-minded

thinking when they described it as the tendency of people to engage in flexible or open-minded

thinking and avoid dogmatic or categorical thinking. It was this conceptualization of actively

15

open-minded thinking that inspired and informed the development of their AOT scale. Other

researchers, such as Baron, conceptualize actively open-minded thinking differently. Baron

(1991) characterizes actively open-minded thinking as a search for evidence, the extent to which

is determined by the importance of the question, and an objectivity in one’s consideration and

review of the evidence. An instrument measuring actively open-minded thinking as described by

Baron would look quite different than the Stanovich and West (2007) AOT scale. As mentioned

previously, it could be argued that the components of Stanovich and West’s (2007) instrument

might be related to actively open-minded thinking, but are not necessarily essential features of

the construct.

The Stanovich and West (2007) AOT scale has been used in many research endeavors,

however, development and purpose of the scale is relatively unclear. Did Stanovich and West set

out to develop a psychometrically sound (i.e., valid and reliable) measure of respondents’

propensity to engage in actively open-minded thinking? Or was their intent to develop a measure

that evaluates tangentially related aspects of actively open-minded thinking that relates well to

other measures of similar characteristics (e.g., cognitive ability, need for cognition, argument

evaluation)? One way of viewing the current Stanovich and West (2007) AOT instrument,

consistent with the latter question, is that it is comprised of features related to actively open-

minded thinking, but not necessarily essential to the definition of the construct. The development

of the instrument may have been guided by the goal of measuring things related to good

thinking, not necessarily by the intention to reliably and validity measure actively open-minded

thinking itself. It is therefore not surprising that scores on the AOT scale relates to other

constructs in expected ways; however these relationships are between aspects related to actively

open-mined thinking and other constructs, not between the construct and actively open-minded

16

thinking itself. If the AOT scale was developed with the first purpose in mind, we recommend

that researchers review the literature and revisit the conceptualization of the actively open-

minded thinking construct in order to develop a more valid and reliable scale that reflects core,

theoretical components of actively open-minded thinking. This process might begin with

revisiting Baron’s conceptualization and initiating the development of a measure of actively

open-minded thinking that directly reflects this characteristic of good thinking.

17

Table 1

Item Descriptive Statistics

Note. N = 430.

Item M SD Skewness Kurtosis

1 4.07 1.37 -0.20 -0.93

2 3.44 1.19 0.03 -0.43

3 4.67 1.19 -0.80 0.04

4 5.40 0.80 -1.65 3.79

5 4.54 1.28 -0.76 0.04

6 5.26 0.98 -1.56 2.45

7 3.86 1.50 -0.07 -1.10

8 4.82 1.17 -1.04 0.75

9 3.71 1.41 0.10 -0.93

10 4.77 0.94 -0.82 1.14

11 4.30 1.41 -0.44 -0.82

12 3.17 1.08 0.09 -0.13

13 3.16 1.37 0.09 -0.86

14 4.97 1.20 -1.08 0.38

15 3.27 1.27 0.10 -0.61

16 4.15 1.19 -0.15 -0.72

17 4.67 1.26 -0.79 -0.01

18 4.91 0.98 -1.22 2.21

19 3.08 1.28 0.18 -0.51

20 3.94 1.19 -0.18 -0.41

21 3.80 1.15 0.11 -0.50

22 4.66 1.01 -0.51 0.03

23 4.17 1.23 -0.15 -0.58

24 4.87 1.25 -1.24 1.18

25 3.93 1.46 -0.34 -0.84

26 4.43 1.01 -0.30 -0.25

27 4.32 1.02 -0.49 0.40

28 3.78 1.41 -0.29 -0.76

29 4.52 0.95 -0.16 -0.43

30 4.56 1.01 -0.40 0.16

31 2.60 1.13 0.55 0.12

32 3.64 0.95 -0.03 0.82

33 4.65 1.06 -0.68 0.50

34 3.96 1.24 -0.34 -0.33

35 3.76 1.19 0.08 -0.54

36 3.36 1.01 0.30 0.04

37 3.96 1.06 -0.08 0.12

38 4.96 1.15 -1.00 0.33

39 4.89 1.06 -0.86 0.37

40 3.22 1.05 0.49 0.34

41 4.51 0.93 -0.37 0.60

18 Table 2

4-Factor Solution Pattern Coefficients, Item Communalities, and Variance Accounted for by Each Factor

Note. N = 430. (R) indicates that item is reflected.

1 2 3 4

1 Even though freedom of speech for all groups is a worthwhile goal, it is unfortunately necessary

to restrict the freedom of certain political groups. (R)0.22 0.05 -0.04 0.05 0.06

2 What beliefs you hold have more to do with your own personal character than the experiences

that may have given rise to them. (R)0.20 -0.03 0.13 0.18 0.11

3 I tend to classify people as either for me or against me. (R) 0.53 -0.02 -0.03 -0.05 0.27

4 A person should always consider new possibilities. 0.36 0.27 -0.26 -0.02 0.27

5 There are two kinds of people in this world: those who are for the truth and those who are against

the truth. (R)0.55 0.02 0.12 0.06 0.36

6 Changing your mind is a sign of weakness. (R) 0.57 0.07 -0.20 -0.01 0.33

7 I believe we should look to our religious authorities for decisions on moral issues. (R) 0.16 0.28 0.31 0.00 0.25

8 I think there are many wrong ways, but only one right way, to almost anything. (R) 0.50 0.11 0.19 0.04 0.37

9 It makes me happy and proud when someone famous holds the same beliefs that I do. (R) 0.35 -0.01 0.21 -0.06 0.20

10 Difficulties can usually be overcome by thinking about the problem, rather than through waiting

for good fortune.0.07 0.32 -0.19 0.03 0.15

11 There are a number of people I have come to hate because of the things they stand for. (R) 0.53 -0.12 -0.08 -0.23 0.29

12 Abandoning a previous belief is a sign of strong character. -0.32 0.37 0.22 -0.07 0.21

13 No one can talk me out of something I know is right. (R) 0.11 -0.06 0.40 0.00 0.19

14 Basically, I know everything I need to know about the important things in life. (R) 0.45 0.01 0.12 0.15 0.27

15 It is important to persevere in your beliefs even when evidence is brought to bear against them.

(R)0.04 0.21 0.54 0.05 0.38

16 Considering too many different opinions often leads to bad decisions. (R) 0.44 0.07 0.17 0.08 0.29

17 There are basically two kinds of people in this world, good and bad. (R) 0.52 0.07 0.14 -0.08 0.33

18 I consider myself broad-minded and tolerant of other people's lifestyles. 0.24 0.34 -0.15 -0.11 0.23

19 Certain beliefs are just too important to abandon no matter how good a case can be made against

them. (R)0.08 0.12 0.56 -0.07 0.36

20 Most people just don't know what's good for them. (R) 0.49 -0.06 0.17 -0.02 0.29

21 It is a noble thing when someone holds the same beliefs as their parents. (R) 0.36 0.07 0.28 -0.02 0.26

22 Coming to decisions quickly is a sign of wisdom. (R) 0.59 0.02 -0.02 0.08 0.36

23 I believe that loyalty to one's ideals and principles is more important than "open-mindedness." (R)0.42 0.35 0.25 0.02 0.48

24 Of all the different philosophies which exist in the world there is probably only one which is

correct. (R)0.36 0.28 0.22 0.10 0.37

25 My beliefs would not have been very different if I had been raised by a different set of parents. (R)0.03 -0.15 -0.05 0.68 0.47

26 If I think longer about a problem I will be more likely to solve it. 0.03 0.28 -0.08 -0.06 0.09

27 I believe that the different ideas of right and wrong that people in other societies have may be

valid for them.0.09 0.49 0.04 0.07 0.28

28 Even if my environment (family, neighborhood, schools) had been different, I probably would

have the same religious views. (R)0.06 -0.06 0.02 0.54 0.30

29 There is nothing wrong with being undecided about many issues. 0.19 0.26 -0.12 -0.06 0.13

30 I believe that laws and social policies should change to reflect the needs of a changing world. 0.01 0.53 0.05 0.05 0.30

31 My blood boils over whenever a person stubbornly refuses to admit he's wrong. (R) 0.15 -0.17 0.14 -0.26 0.13

32 I believe that the "new morality" of permissiveness is no morality at all. (R) 0.29 0.05 0.16 0.07 0.15

33 One should disregard evidence that conflicts with your established beliefs. (R) 0.35 0.26 0.15 0.23 0.37

34 Someone who attacks my beliefs is not insulting me personally. 0.04 0.42 0.05 -0.09 0.20

35 A group which tolerates too much difference of opinion among its members cannot exist for long.

(R)0.37 0.13 0.09 -0.01 0.20

36 Often, when people criticize me, they don't have their facts straight. (R) 0.25 -0.12 0.12 -0.05 0.09

37 Beliefs should always be revised in response to new information or evidence. -0.15 0.51 0.15 0.05 0.27

38 I think that if people don't know what they believe in by the time they're 25, there's something

wrong with them. (R)0.56 0.00 -0.15 0.04 0.30

39 I believe letting students hear controversial speakers can only confuse and mislead them. (R) 0.50 0.19 -0.04 0.25 0.41

40 Intuition is the best guide in making decisions. (R) 0.08 -0.11 0.25 0.15 0.12

41 People should always take into consideration evidence that goes against their beliefs. 0.06 0.48 0.08 0.05 0.27

Percent of Variance Accounted for by each Factor 15.84 4.33 3.36 2.63

Factor

Item textItem

Item

communalities

19

Figure 1. AOT Scale EFA Parallel Analysis Results

Figure 1. AOT scale EFA parallel analysis results. N = 430.

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Raw Data Eigenvalues

Generated Data Eigenvalues

20

References

Baron, J. (1985). Rationality and intelligence. New York: Cambridge University Press.

Baron, J. (1988). Thinking and deciding. New York: Cambridge University Press.

Baron, J. (1991). Beliefs about thinking. In J. F. Voss, D. N. Perkins, & J. W. Segal (Eds.),

Informal reasoning and education, pp. 169-186. Hillsdale, N. J. : Erlbaum.

Costa, P. T., & McCrae, R. R. (1992). Revised NEO personality inventory. Odessa, FL:

Psychological Assessment Resources.

DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychological Methods, 3(2), 292-

307.

Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. Baron & R.

Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26) New York:

Freeman.

Epstein, S., & Meier, P. (1989). Constructive thinking: A broad coping variable with specific

components. Journal of Personality and Social Psychology, 57, 332 – 350.

Gorsuch, R.L. (1983). Factor Analysis, 2nd Edition, Hillsdale, N.J.: Erlbaum.

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis.

Psychometrika, 30, 179-185.

Kaiser, H.F., 1974. Index of factorial simplicity. Psychometrika 39,

31–36.

Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling (2nd

Edition). New

York: Guilford.

21

Moshman, D. (1994). Reasoning, metareasoning, and the promotion of rationality. In A.

Demetriou & A. Efklides (Eds.), Intelligence, mind, and reasoning: Structure and

development (pp. 135-150), Amsterdam: Elsevier.

Norris, S. P. (1992). Testing for the disposition to think critically. Informal Logic, 14, 157-164.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-

Hill.

Paulhus, D. L., & Reid, D. B. (1991). Enhancement and denial in socially desirable responding.

Journal of Personality and Social Psychology, 60, 307 – 317.

Perkins, D. N., Faraday, M., & Bushey, B. (1991). Everyday reasoning and the roots of

intelligence. In J. Voss, D. Perkins, & J. Segal (Eds.), Informal reasoning and education

(pp. 83-105). Hillsdale, NJ: Erlbaum.

Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Measures of Personality and Social

Psychological Attitudes (Vol. 1, pp. 560 – 564). San Diego, CA: Academic Press.

Sá, W. C., West, R. F., & Stanovich K. E. (1999). The domain specificity and generality of belief

bias: Searching for generalizable critical thinking skill. Journal of Educational

Psychology, 91, 497-510.

Stanovich, K. E., & West, R. F. (1997). Reasoning independently of prior belief and individual

differences in actively open-minded thinking. Journal of Educational Psychology, 89(2),

342-357.

Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of

Experimental Psychology, 127 (2), 161-188.

Stanovich, K. E. & West, R. F. (2007). Natural myside bias is independent of cognitive ability.

Thinking & Reasoning, 13, 225-247.

22

Troldahl, V., & Powell, F. (1965). A short-form dogmatism scale for use in field studies. Social

Forces, 44, 211 – 215.

Zilberberg, A. & Pastor D. A. (2009). Willingness to consider contradictory evidence scale: A

mixed methods approach. Paper presented at the Northeastern Educational Research

Association, Rocky Hill, CT.