ARTICLE

Explicitly Teaching Critical Thinking Skillsin a History Course

Anne Collins McLaughlin1 & Alicia Ebbitt McGill2

Published online: 20 March 2017# Springer Science+Business Media Dordrecht 2017

Abstract Critical thinking skills are often assessed via student beliefs in non-scientific waysof thinking, (e.g, pseudoscience). Courses aimed at reducing such beliefs have been studied inthe STEM fields with the most successful focusing on skeptical thinking. However, criticalthinking is not unique to the sciences; it is crucial in the humanities and to historical thinkingand analysis. We investigated the effects of a history course on epistemically unwarrantedbeliefs in two class sections. Beliefs were measured pre- and post-semester. Beliefs declinedfor history students compared to a control class and the effect was strongest for the honorssection. This study provides evidence that a humanities education engenders critical thinking.Further, there may be individual differences in ability or preparedness in developing suchskills, suggesting different foci for critical thinking coursework.

1 Introduction

Many people in the USA and elsewhere hold beliefs for which there is no or can be noempirical evidence. Such beliefs range from astrology to psychic powers to not scientificallytested alternative medicines to conspiracy theories. Lobato et al. (2014) referred to these beliefsas epistemically unwarranted (henceforth Bunwarranted^), meaning that they do not considerall of the evidence available to separate justified belief from opinion. This is the generaldefinition for pseudoscience, thought it covers all domains where conclusions are not based onevidence. For example, the National Science Foundation reported that at least 42% of Amer-icans believe astrology is Bsomewhat scientific^ or Bvery scientific^ and levels of these beliefsare increasing rather than decreasing (NSF 2014). The harm of these beliefs is real: unvacci-nated children die, savings are spent on psychics, and in the first month of 2014 two childrendied during an Bexorcism^ (Karimi and Sutton 2014).

Sci & Educ (2017) 26:93–105DOI 10.1007/s11191-017-9878-2

* Anne Collins McLaughlinAnne_McLaughlin@ncsu.edu

1 Department of Psychology, North Carolina State University, Box 7650, Raleigh, NC 27695, USA2 Department of History, North Carolina State University, Box 8108, Raleigh, NC 27695, USA

Less harmful though equally problematic aspects of these beliefs include the popularity ofprograms such as Ancient Aliens and misrepresentations of ancient cultures (e.g., the Maya inthemovie Apocalypto) or popular scholarship about civilizations which do not take into accountthe ingenuity of ancient cultures. One example is Jared Diamond’s book Collapse: HowSocieties Choose to Fail or Succeed (2005), where Diamond emphasized natural resourceexploitation and socio-political conflicts among the ancient Maya as societal Bfailures,^ ratherthan bringing attention to their resilience during periods of social, political, and environmentalstresses (e.g., centuries of successful agriculture in less than fertile soil (McAnany and Yoffee2010). Misrepresentations in popular media and incomplete scholarly pictures of ancientcivilizations have serious implications for the treatment of their descendants (Freidel 2007;McAnany and Negrón 2009; Pyburn 2006; Ren 2006). Further, unwarranted beliefs are related:belief in one predicts belief in others, suggesting that such beliefs represent a way of thinking(Lobato et al. 2014), even to the point where belief in one conspiracy theory positively predictsbelief in other logically contradictory conspiracies (Wood et al. 2012). Educators need provenmethods to teach the critical thinking skills needed to battle these unwarranted beliefs andreplace them with reasoned beliefs supported by evidence (Smith and Siegel 2004).

Reduction of unwarranted beliefs has been a challenge, with a common supposition beingthat increasing education, particularly science education, will result in a public better equippedfor critical thinking and analysis of unsupported claims. Critical thinking is a particular focus,defined as Bpurposeful, self-regulatory judgment which results in interpretation, analysis,evaluation, and inference, as well as explanation of the evidential, conceptual, methodological,criteriological or contextual considerations upon which that judgment is based^ (Facione 1990,p. 2). An important further component in critical thinking is the acceptance of the results ofcritical thinking, rather than reverting back to previous beliefs, or at least an acknowledgementthat the evidence supports some beliefs but not others (Smith and Siegel 2004). Unfortunately,knowledge of science is not a panacea for unwarranted beliefs with studies finding small or nodifferences for higher levels of science knowledge (Aarnio and Lindeman 2005; Bridgstock2003; DeRobertis and Delaney 1993; Goode 2002; Johnson and Pigliucci 2004; Ryan et al.2004) and some finding higher beliefs in the more highly educated (Rice 2003). Even teachersdid not differ greatly from the general public (Losh and Nzekwe 2011). Studies with 1000s ofarcheology students across multiple universities found that students across decades and withinthe last few years showed belief in the existence of Bancient astronauts,^ Atlantis, Big Foot, ora curse on King Tut’s tomb (Feder 1984, 1995, 2010; Harrold and Eve 1987). These studieshave shown little change in students’ preconceptions about the past over several decades,finding consistently over 10% of students responding that they either have Bno opinion^ or Bdonot know^ when asked about any pseudo-history or pseudo-archeological claim. This indicatesthat they feel ill-equipped to assess unwarranted claims. (Feder 2010). It may seem commonsense that critical thinking will develop merely from exposure to collegiate courses, butunfortunately such is not the case (Arum and Roksa 2011; Pascarella et al. 2011). Theconclusion was that it is not enough to teach science; how science is being taught and howto think (Paul 1995) needs to be revisited by explicitly defining critical thinking and expec-tations for students (Fitzgerald and Baird 2011).

Science courses created specifically to engender critical thinking have reduced pseudosci-entific beliefs (e.g., Dougherty 2004; Franz and Green 2013; Gray 1985; Kane et al. 2010;Wesp and Montgomery 1998). However, as mentioned by Kane et al. (2010), many of thesestudies did not contain a control group, did not use appropriate statistical tests, or did notprovide anonymity to respondents, meaning that the results could come from self-selection

94 A. C. McLaughlin, A. E. McGill

into the course or demand characteristics. Other evidence that demand characteristics did notaffect the data would be selective belief change, for example, when some beliefs may decreasewhile others would not. This would indicate that the participants were not universally loweringtheir reported beliefs due to demand characteristics. However, prior studies did not investigatebeliefs at this level (see Kane et al. 2010, for an exception). Studies of engendering criticalthinking skills have shown similar results, with the most effective courses providing directinstruction on critical thinking along with class exercises that require critical thinking onmaterials from the course (Abrami et al. 2008; Niu et al. 2013).

Although most courses meant to teach critical thinking and reduce pseudoscientific andparanormal beliefs have been science courses, critical thinking is not the sole property of thesciences. Ryan et al. (2004) put it well by stating:

The liberal arts – literature, philosophy, history, art—and the natural sciences belong to the sameintellectual tradition and as such, they have the same general goal … which is Bto develop the facultiesof the mind.^ … Certainly the humanities invite reflection and personal experience as bases for decision-making; however, like the sciences, the humanities also require a rigorous approach to reasoning andreaching sound conclusions.

Indeed, students with previous courses in the physical and social sciences and humanitieswere less likely to believe certain problematic claims about the past (Feder 1984, 2010), butpseudoscientific beliefs remain pervasive.

In the current study, we tested the effectiveness of a history course focused on exposingBpseudo-history,^ Bpseudo-science,^ and Bpseudo-archeology^ in reducing unwarranted be-liefs when compared to a research methods course in psychology. Pseudo- science, history, andarcheology can all be used pejoratively and acts of these often, though not always, involvesensational and outrageous claims about the past and/or the ways the world works. Thesepractices can utilize existing facts but in ways that are inconsistent with standard disciplinarypractices and may also simply exaggerate existing evidence to support false claims. Althoughit has been suggested that a pseudo-history course is a good way to address pseudoscientificbeliefs (Allchin 2004) and that pseudoscience and pseudo-history overlap in their features(Boudry 2013), no empirical studies have been undertaken. We measured beliefs pre- and post-semester, informed students their responses would be anonymous and could not adverselyaffect their grade in the courses, took general measures of scientific knowledge, and a measureof hindsight bias at post-test. Last, the history course included an honors section, allowing usto investigate individual differences in belief change as well as a comparison of each historycourse to the control.

2 Method

2.1 Participants

All materials and procedures were approved by the NCSU IRB. Participants came from threeundergraduate courses at North Carolina State University: HI 298 (Frauds and Mysteries ofHistory), an honors seminar of the same course (HON 290), and a control course forcomparison: PSY 320 (Research Methods). Forty-one males and 78 females completed thestudy. Participants also differed in the percentages of males and females in the courses, withthe psychology course containing a higher percentage of females to males. Demographics were

Critical Thinking Through History 95

recorded before the start of the first belief assessment and are broken down by class in Table 1.Differences between the three courses included their performance on a pre-test of generalscience knowledge, distribution of males and females, and variety of backgrounds (PSY 320tended to include psychology majors, HI 298 tended to include majors in the humanities andsocial sciences, and HI Honors was a mix of majors trending toward those in science andengineering fields.) Admission to the honors program at NCSU requires a 1300+ SAT score incritical reading and math or 30+ ACT and a 4.5 weighted or 3.75 unweighted GPA from highschool. Originally, it was assumed that the history classes would be combined, but due to thedifferences in the self-selected samples, they were analyzed separately.

2.2 Materials

Science knowledge was assessed using a 13-item quiz developed by the Pew Research center(Appendix 1; Pew 2013). Items covered science in daily life and Btextbook science^ such aschemical reactions. Norming data for the Pew quiz reported that those with a college educationaveraged 9.8 correct out of 13 (Pew 2013). The Pew measure was not correlated to pre-testbeliefs, and thus is used as a descriptor of the sample rather than a covariate in analyses.

The main measure of beliefs in pseudoscience and the paranormal (Appendix 2) wascomposed of statements across a variety of beliefs, adapted from belief scales used in similarexperiments (e.g., Bridgstock 2003; Feder 2010; Kane et al. 2010; Tobacyk 2004). Each beliefwas rated on a seven-point Likert scale with verbal anchors of B(1) I do not believe this at all,^B(2) I doubt this very much,^ B(3) I doubt this,^ B(4) I am unsure,^ B(5) I believe this,^ B(6) Ibelieve this very much,^ and B(7) I strongly believe this.^ Belief scores were calculated as themean rating of each item. The difference between ratings for an item pre-semester and post-semester was the dependent measure used in analyses. Reliability of the belief test questionswas assessed using the pre-test data with the resulting Cronbach’s alpha of 0.892, indicatinggood reliability.

Table 1 Participant demographics and pre-test scores

Demographics

Treatment ControlHonors History History PsychologyMean SD Mean SD Mean SD F

Age 19.9 0.90 20.24 2.20 20.14 2.92 0.162Year in college 2.90 0.82 2.62 1.02 2.24 0.95 5.150 *Pew science score 12.17 1.28 11.86 1.19 11.12 1.74 5.447 *Pre-overall beliefs 2.60 0.67 2.71 0.76 2.80 0.68 0.830Pre-test n n = 29 n = 33 n = 70Attrition 0 4 11Female/male 14/15 15/14 47/12Number of majorsColleges of science or engineering 22 4 5College of humanities and social science 1 17 49Other 6 8 5

*Participants differed by class in their year in college where those in the honors history course had more years incollege than the psychology course (p = 0.007) and Pew science score where those in the honors history courseoutscored those in the psychology course (p = 0.008). There was a higher percentage of females in thepsychology course compared to the history courses and difference in student majors

96 A. C. McLaughlin, A. E. McGill

2.3 Course Topics

The syllabus for the control course (Psychological Research Methods) is available atwww.lacelab.org/PSY230Fall2014.pdf. The course covered research design andmeasurement. The syllabus for the history course (Frauds and Mysteries in History)is available at http://history.ncsu.edu/images/uploads/HI%20298-001_McGill%20F14%281%29.pdf. The course used the textbooks BFrauds, myths, and mysteries:science and pseudoscience in archaeology^ (Feder 2010) and BFrom Stonehenge toLas Vegas: archaeology as popular culture^ (Holtorf 2005) and covered such topics asthe Piltdown Hoax and the building of the Egyptian pyramids. The Feder textpresented an empirical, positivist, and critical approach to analyzing and debunkingfrauds, mysteries, and misconceptions about the past. The Holtorf text provided amore constructivist, post-processual archeology-based, and post-modern approach tounderstanding interests in the past in popular culture and the ways people connectwith the past in the present.

Class time was devoted to analyzing how these different authors approachedknowledge and misinformation about the past and the values of both of theseapproaches to understanding frauds and myths. The course contained direct instructionon critical thinking, e.g., including Feder’s BQuick Start Guide^ (Feder 2010) and CarlSagan’s Bbaloney detection kit^ (Sagan 1996). These tools were then used throughoutthe course by the students, such as evaluating the likelihood of aliens being thesource of ingenuity in ancient cultures using Sagan’s rule that BIf there’s a chain ofargument, every link in the chain must work (including the premise) – not just mostof them.^ Students also learned common logical fallacies and fallacies of rhetoric,tropes in historical myths, and then applied them to course topics, such as the Bappealto ignorance — the claim that whatever has not been proved false must be true, andvice versa.^ Fallacies embedded in the frauds and mysteries of history were identifiedby the students throughout the semester to build their critical thinking skills. Studentsalso practiced analysis, evaluation, critical thinking, rational skepticism, and were tobe critical consumers of popular culture. Students honed their critical thinking skillsthrough website analyses, research on historical frauds and misconceptions, and classdebates about misuses and misconceptions about the past.

Not all of the examples that students were surveyed on were explicit examples of pseudo-history, pseudo-archeology, or pseudoscience. However, most of these are connected to theseconcepts in some way. For example, the concept of race as a biological construct is not anexplicit example of pseudo-history, pseudo-archeology, or pseudoscience, but it is taught as akind of social and historical Bmyth.^ The course covered the history of racial science, the waysthat scientific and historical information and concepts (e.g., connections between biology, race,and skin color; or connections between genes, skin color, and race) was and in some casescontinues to be misused to be make arguments about biological racial differences and racialsuperiority and inferiority.

Topics in the belief measure were classified as to whether they were covered in thehistory course (Appendix 2). The honors section used the same texts but the coursedemands differed in that students were also assigned a group project to exploreevidence behind a purported paranormal activity. For example, one group researchedclaims about the BBrown Mountain lights^ that reportedly appear over Brown Moun-tain, NC, and presented to the class.

Critical Thinking Through History 97

2.4 Design and Analytical Methods

Class Type (treatment: honors history, history, control: psychology) was a quasi-independentbetween-participant grouping variable. Belief Type (studied or unstudied in the historycourses) was a within-participant variable. No belief topics were covered in the control course,but the topics were still analyzed by Belief Type. This was to check that there were nodifferential effects for the topics studied in history compared to those not studied. For example,it might have been that the studied topics in the history courses were beliefs most likely tochange over time. If that were true, then we would see those beliefs change more for thecontrol course as well. By dividing the topics into Belief Type for the control course, we wouldknow whether such a confound affected the results: if a confound were present, we would seedifferences in belief change for the studied and unstudied topics, despite receiving noinstruction on either. The dependent measure was their belief change (as measured on theseven-point Likert scale for all questions) from the pre-semester assessment to the post-semester assessment.

Analyses were performed using SPSS 18. Correlations were performed on continuousvariables, such as pre-test scores on the various scales and demographic descriptors. The mainanalysis was a mixed-model ANOVA, with the between-participant factors of Class and thewithin-participant factor of Belief Type.

2.5 Procedure

Survey responses were collected online, outside of the classroom, and by a researcherwho was not the instructor for any of the courses. Participants were invited via emailssent to the course roster that contained a link to the pre-semester survey. Studentswho followed the link were presented with a consent form guaranteeing their answerswould be confidential and they would receive two extra credit points on an exam forcompleting both surveys. Consenting students continued to the pre-semester surveythat contained the demographic questionnaire, science knowledge test, beliefs inpseudoscience, and open-ended questions concerning their motivation for taking thecourse. Students who completed the pre-semester survey were emailed 1 week beforetheir last class and given the link to the post-semester survey, which contained beliefsin pseudoscience, and open-ended questions about their experience in the course. Theduration of each course was approximately 15 weeks.

3 Results

Participants of different classes differed at pre-test on their Pew Science scores, where thehonors history students scored significantly higher than the psychology course students(Table 1). All groups scored higher than the norm for college graduates (Pew 2013). Therewas no correlation between Pew Science score and pre-test overall belief scores (r2 = 0.02,p = 0.11, n = 117). Classes did not significantly differ on the belief measures at pre-test. AnANOVA revealed initial belief ratings differed according to biological sex, with femalestudents reporting higher beliefs, F(1115) = 8.33, p < 0.001. ω2 = 0.07. Mean belief scoresfor each class were low and similar to Dougherty’s findings for college students (Dougherty2004).

98 A. C. McLaughlin, A. E. McGill

A mixed-model ANOVA was performed with the factors of Class and Belief Type onchange in belief score from pre-semester to post-semester. Data met the assumptions for anANOVA by having a normally distributed dependent measure at the interval level assessed viahistogram, factors composed of independent, categorical groups, no significant outliers, andhomogeneity of variance as measured by Levene’s test (p > 0.05). Main effects were found forClass, where classes differed from each other in belief change (F(2116) = 17.50, p < 0.001,ω2 = 0.22) and for Belief Type, where there was more change for topics studied in the historyclasses, (F(1116) = 8.18, p = 0.005, ω2 = 0.05). Tests of simple effects showed significantlymore of a decrease in belief for studied beliefs vs. unstudied beliefs in the honors history class(p = 0.001) and no differences for the history class or psychology class (all ps > 0.05). Maineffects contrasts showed the honors history class changed their beliefs more than the historyclass (p = 0.017) and more than the psychology class (p < 0.001). The history class beliefchange was significantly higher than the psychology class (p = 0.038). As expected, thepsychology class beliefs were unaffected by whether beliefs were on the studied or unstudiedlist for the history classes. Bonferroni corrections were used in all contrasts. There was also asignificant interaction of Class by Belief Type on amount of belief change (F(2116) = 4.79,p = 0.010, ω2 = 0.12; Fig. 1), where beliefs decreased over the semester most for the honorsclass, some for the non-honors class, and did not change for the psychology class. Change wasgreater for the studied beliefs in the history classes, particularly the honors class.

Fig. 1 Beliefs measured pre-semester, post-semester, and recall of pre-semester beliefs in studied and unstudiedtopics from the history courses. Data represent change in belief scores from pre- to post-semester and werecalculated by subtracting mean post-semester belief ratings from mean pre-semester belief ratings. The greatestchange was from pre- to post-beliefs in the honors history course, followed by the history course, particularly forstudied beliefs

Critical Thinking Through History 99

Because there were differences in male and female levels of belief at pre-test, a secondmixed-model ANOVAwas performed as above, adding the between-subjects variable of sex.There were no changes in significance from the previous analysis, no main effect of sex, andno interactions with sex (all ps >/05).

4 Discussion

Unwarranted beliefs were low across all three classes at pre-test, similar to findings byDougherty (2004), and scientific knowledge was high. Even such already low unwarrantedbeliefs were further significantly reduced after exposure to a critical thinking-based historycourse, but not to a control science course in research methods. Important findings included thedifferences in belief change for students in the two history classes, both receiving similarcourse materials. The honors students scored more highly on a test of science knowledge andtrended toward less unwarranted beliefs at pre-test. These differences likely contributed to thehonors class lowering their beliefs the most of any class at the post-semester assessment, bothon topics covered by the course and topics not covered by the course. Further, the honors classmost reduced their beliefs for topics explicitly covered in the class. In essence, the effective-ness of a course on critical thinking via history was most effective for those who were bestprepared to absorb it and transfer it to other domains: honors students with some backgroundin science. Those in the non-honors history course also reduced their beliefs, replicatingprevious findings regarding the effectiveness of skepticism-oriented courses (Gray 1985).These findings support the importance (and effectiveness) of explicit instruction (Abramiet al. 2008).

There is a long literature on the difficulties in transfer of knowledge, where even closelyrelated information between subjects or courses often fails to transfer (Barnett and Ceci 2002).We find it promising that there was transfer of critical thinking to unstudied beliefs for studentsof both history courses, even if it was less for those in the non-honors course. Our findings viathe belief measures in the study were bolstered by student comments on the post-semestersurvey that indicated application of critical thinking to other parts of their lives. Representativecomments included BI think the most applicable things I learned from this class are guides onbeing a healthy skeptic. Being able to interpret current events and discoveries armed withFeder’s quick tips, Sagan’s baloney-detection kit, and with knowledge of credibility and thescientific method will allow me to make more informed decisions on what I believe to be true,as well as on what I need more evidence in order to make a more definite conclusion^ and B…I want to dive into the details and I always look for poorly constructed arguments. In my otherclasses, I find myself challenging what my professors say when they use false dichotomies,circular arguments, or reference questionable sources....^ Last, students referenced their chanceto apply their critical thinking skills in class with comments such as BThe course ... analyzed alot of misconceptions of history and did a good job of providing the students with the ability todo this as well. The in-class activities were very helpful and were a hands-on way to learn theclass material. I can easily say that I now have the skills and knowledge to properly assesshistorical information and form my own opinion on the reliability and credibility of theinformation.^

Although initial science knowledge did not protect against pseudoscientific beliefs, simi-larly to previous studies (Aarnio and Lindeman 2005; Bridgstock 2003; DeRobertis andDelaney 1993; Goode 2002; Johnson and Pigliucci 2004; Ryan et al. 2004), the honors history

100 A. C. McLaughlin, A. E. McGill

class did show the largest reduction in such beliefs due to the course, indicating that theirhigher science knowledge and cognitive abilities may have prepared them for the criticalthinking introduced by the course.

Limitations to consider include the potential effect of instructor on belief change and thepossibility that students responded to demand characteristics in the history class by reducingtheir belief ratings for the post-semester assessment. Because the control and treatment courseswere taught by different people, it may also be that the history instructor was particularlyengaging compared to the psychology instructor. However, because we found differences instudied beliefs compared to unstudied beliefs in the history courses, belief change was notentirely due to the instructor. We cannot rule out the possibility that the history students’ beliefchange was due to them thinking that their instructor expected that their beliefs should belower by the end of the semester (and thus reporting lower than actual beliefs post-semester).However, we believe the anonymity of study participation reduced the chances of suchdemand characteristics driving the results. Further, if students were responding to demandcharacteristics, we would expect them to lower their beliefs similarly on all topics, rather thanmore for the studied beliefs than the unstudied. One last limitation was that the control andtreatment groups differed in years of college experience, with the history classes averagingabout 1 year older. However, we do not believe that this age difference was driving theirbeliefs, as groups had no differences in their pre-semester belief scores.

In conclusion, the current study has shown that critical thinking skills can be taught via anon-science course, resulting in a reduction in pseudoscientific and paranormal beliefs. Thiseffect was strongest for beliefs directly addressed by the course, but transferred to other beliefs.Such generalization supports Davies’ (2013) supposition that critical thinking skills can begeneralized beyond the specific discipline in which they are instructed and practiced. Individ-ual differences in preparation and ability were related to belief reduction, where the most ablestudents benefitted most from taking the history course.

We recommend that instructors in the sciences and the humanities explicitly identify theways the class allows practice of critical thinking skills, define critical reflection, and integratecontent-based instruction with critical thinking tasks. Such tasks offer the opportunity to revealto students their own current beliefs while giving them a framework on which to alter thosebeliefs (Turgut 2011). Additionally, we recommend that other researchers consider the role ofthe humanities in the reduction of pseudoscientific thought, as suggested by Bunge (2010).Cross-disciplinary and collaborative scholarship on teaching and learning of critical thinkingskills could also be particularly valuable.

Compliance with Ethical Standards This research was approved by the North Carolina State University IRBand informed consent was obtained from all participants.

Conflict of Interest The authors have no conflict of interest regarding this project.

Appendix 1: Pew test of science knowledge

All radioactivity is man-made. Is this statement true or false?Correct Answer: FalseElectrons are smaller than atoms. Is this statement true or false?Correct Answer: True

Critical Thinking Through History 101

Lasers work by focusing sound waves. Is this statement true or false?Correct Answer: FalseThe continents on which we live have been moving their location for millions of years and

will continue to move in the future. Is this statement true or false?Correct Answer: TrueWhich one of the following types of solar radiation does sunscreen protect the skin from?Correct Answer: Ultravioletx-raysinfraredmicrowavesDoes nanotechnology deal with things that are extremely...Correct Answer: smalllargecoldhotWhich gas makes up most of the Earth’s atmosphere?Correct Answer: NitrogenHydrogenCarbon DioxideOxygenWhat is the main function of red blood cells?Correct Answer: Carry oxygen to all parts of the bodyHelp the blood to clotFight disease in the bodyWhich of these is a major concern about the overuse of antibiotics?Correct Answer: It can lead to antibiotic-resistant bacteriaPeople will become addicted to antibioticsAntibiotics are very expensiveWhich is an example of a chemical reaction?Correct Answer: Nails rustingWater boilingSugar dissolvingWhich is the better way to determine whether a new drug is effective in treating a disease?

If a scientist has a group of 1000 volunteers with the disease to study, should she...Correct Answer: Give the drug to half of them but not to the other half, and compare how

many in each group get betterGive the drug to all of them and see how many get betterWhat gas do most scientists believe causes temperatures in the atmosphere to rise?Correct Answer: Carbon dioxideHeliumRadonHydrogenWhich natural resource is extracted in a process known as Bfracking^?Correct Answer: Natural gasCoalDiamondsSilicon

102 A. C. McLaughlin, A. E. McGill

Appendix 2

Table 2 Belief measure with note of whether covered directly in the History courses

Covered in HIcourses

Not covered in HIcourses

Belief

x An ape-like mammal, sometimes called Bigfoot, roams the forests ofAmerica.

x The Ancient Egyptians visited Mesoamerica and taught groups like theMaya how to build pyramids.

x Alien visitors to earth taught ancient civilizations the technology to buildpyramids.

x Humans are more than 95% genetically similar.x A person’s race is simply determined by their skin color.x The concept of race is culturally constructed. That means that it only has

meaning within its cultural context.x The Ancient Maya people predicted that the world would end in 2012.x Planes and boats disappear in the Bermuda Triangle due to the area’s

mysterious properties.x Racial groups vary in their abilities because of biological differences

between them.x The Ancient Maya civilization collapsed, leading to the mysterious

disappearance of the Maya people.x An ancient curse placed on the tomb of Egyptian Pharaoh King Tut actually

killed people.x There is scientific evidence that there are different biologically determined

racial groups among humans.x There is good evidence for the existence of the Lost Continent of Atlantis.x The moon landing was faked.x There is a gene that determines someone’s skin color.x There is scientific evidence that giants once walked the earth.

x 9/11 was an Binside job.^x Men are genetically predisposed to be more violent than women.x Archeologists can locate ancient sites with dowsing techniques.x The Earth is approximately 6000 years old.x Scientists can prove the existence of other intelligent life forms in the

universe.x Creationism is a science similar to evolutionary biology.x In all societies, ideas about gender and differences between males and

females are the same.x The number 13 is truly unlucky.x The planets and stars affect peoples’ personalities.x Aliens from other worlds have visited the earth.x Breaking a mirror can bring bad luck.x Alternative medicine is more effective that Western biological medicine.x Crop circles were created by aliens as spacecraft landing strips.x Some people can communicate with the dead.x The Illuminati are a secret society with great influence and power over

politics and the economy.x The U.S. government hides evidence of alien beings.x Healing through faith is more effective than biological medicine.x Some dreams predict future events.x The planets and stars affect events in peoples’ lives.x Evolution is a theory.x People have been abducted by alien beings from other planets.x The Shroud of Turin is a centuries old linen cloth that bears the image of

Jesus of Nazareth.

Critical Thinking Through History 103

References

Aarnio, K., & Lindeman, M. (2005). Paranormal beliefs, education, and thinking styles. Personality andIndividual Differences, 39, 1227–1236.

Abrami, P. C., et al. (2008). Instructional interventions affecting critical thinking skills and dispositions: a stage 1meta-analysis. Review of Educational Research, 78(4), 1102–1134.

Allchin, D. (2004). Pseudohistory and pseudoscience. Science & Education, 13, 179–195.Arum, R., & Roksa, J. (2011). Academically adrift: limited learning on college campuses. Chicago: University of

Chicago Press.Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn?: a taxonomy for far transfer.

Psychological Bulletin, 128(4), 612–637. doi:10.1037/0033-2909.128.4.612.Boudry, M. (2013). Loki’s wager and Laudan’s error: on genuine and territorial demarcation. In M. Pigliucci &

M. Boudry (Eds.), Philosophy of pseudoscience: reconsidering the demarcation problem (pp. 79–100).Chicago: University of Chicago Press.

Bridgstock, M. (2003). Paranormal beliefs among science students. Australasian Science, 24(4), 33–35.Bunge, M. (2010). Knowledge: genuine and bogus. Science & Education, 20(5–6), 411–438.Davies, M. (2013). Critical thinking and the disciplines reconsidered. Higher Education Research &

Development, 32(4), 529–544.DeRobertis, M. M., & Delaney, P. A. (1993). A survey of the attitudes of university students to astrology and

astronomy. Journal of the Royal Astronomical Society of Canada, 87(1), 34–50.Diamond, J. (2005). Collapse: how societies choose to fail or succeed. New York: Penguin Group.Dougherty, M. J. (2004). Educating believers: research demonstrates that courses in skepticism can effectively

decrease belief in the paranormal. Skeptic, 10(4), 31–35.Facione, P. A. (1990). Critical thinking: a statement of expert consensus for purposes of educational assessment

and instruction. Millbrae: The California Academic Press.Feder, K. (1984). Irrationality and archaeology. American Antiquity, 49(3), 525–541.Feder, K. (1995). Ten years after: surveying misconceptions about the human past. Cultural Resource

Management, 18(3), 10–14.Feder, K. (2010). Frauds, myths, and mysteries: science and pseudoscience in archaeology (7th ed.). New York:

McGraw-Hill.Fitzgerald, J., & Baird, V. A. (2011). Taking a step back: teaching critical thinking by distinguishing appropriate

type of evidence. Political Science and Politics, 44(3), 619–624.Franz, T. M., & Green, K. H. (2013). The impact of an interdisciplinary learning community course on

pseudoscientific reasoning in first-year science students. Journal of the Scholarship of Teaching andLearning, 13(5), 90–105.

Freidel, D. (2007). Betraying the Maya. Archaeology Magazine, 60(2), 36–41.Goode, E. (2002). Education, scientific knowledge, and belief in the paranormal. Skeptical Inquirer, 26(1), 24–

27.Gray, T. (1985). Changing unsubstantiated belief: testing the ignorance hypothesis. Canadian Journal of

Behavioural Science, 17, 263–270.Harrold, F. B., & Eve, R. A. (Eds.). (1987). Cult archaeology and creationism: understanding pseudoscientific

beliefs about the past. Iowa City: University of Iowa Press.Holtorf, C. (2005). From Stonehenge to Las Vegas: archaeology as popular culture. Walnut Creek, CA: AltaMira

Press.Johnson, M., & Pigliucci, M. (2004). Is knowledge of science associated with higher skepticism of pseudosci-

entific claims? The American Biology Teacher, 66(8), 536–548.Kane,M. J., Core, T. J., &Hunt, R. R. (2010). Bias versus bias: harnessing hindsight to reveal paranormal belief change

beyond demand characteristics. Psychonomic Bulletin and Review, 17(2), 206–212. doi:10.3758/PBR.17.2.206.Karimi, F., & Sutton, J. (2014). Maryland mom kills two of her children during attempted exorcism. CNN.

http://www.cnn.com/2014/01/19/justice/maryland-exorcism-deaths/. Accessed 20 Feb.Lobato, E., Mendoza, J., Sims, V., & Chin, M. (2014). Examining the relationship between conspiracy theories,

paranormal beliefs, and pseudoscience acceptance among a university population. Applied CognitivePsychology, 28, 617–625.

Losh, S. C., & Nzekwe, B. (2011). Creatures in the classroom: preservice teacher beliefs about fantastic beasts,magic, extraterrestrials, evolution and creationism. Science & Education, 20, 473–489.

McAnany, P. A., & Negrón, T. G. (2009). Bellicose rulers and climatological peril? Retrofitting 21st centurywoes on 8th century Maya society. In In questioning collapse: human resilience, ecological vulnerability,and the aftermath of empire. Cambridge: Cambridge University Press.

McAnany, P. A., & Yoffee, N. (Eds.) (2010). Questioning collapse: human resilience, ecological vulnerability,and the aftermath of empire. New York: Cambridge University Press.

104 A. C. McLaughlin, A. E. McGill

Niu, L., Behar-Horenstein, L. S., & Garvan, C. W. (2013). Do instructional interventions influence collegestudents’ critical thinking skills? A meta-analysis. Educational Research Review, 9, 114–128.

NSF. (2014). Chapter 7: Science and technology: public attitudes and understanding. In Science and EngineeringIndicators 2014. National Science Foundation, 7–1–7-37.

Pascarella, E. T., Blaich, C., Martin, G. L., & Hanson, J. M. (2011). How robust are the findings of academicallyadrift? Change: The Magazine of Higher Learning, 43(3), 20–24.

Paul, R. (1995). Critical thinking: how to prepare students for a rapidly changing world. Rohnert Park:Foundation for Critical Thinking.

Pew (2013). Public’s knowledge of science and technology. Pew Research Center, Washington, D.C. (April).http://www.people-press.org/files/legacy-pdf/04-22-13%20Science%20knowledge%20Release.pdf.Accessed 23 Nov 2014.

Pyburn, A. (2006). The politics of collapse. Archaeologies, 2(1), 3–7.Ren, A. C. (2006). Maya archaeology and the political and cultural identity of contemporary Maya in Guatemala.

Archaeologies, 2(1), 8–19.Rice, T. W. (2003). Believe it or not: religious and other paranormal beliefs in the United States. Journal for the

Scientific Study of Religion, 42, 95–106.Ryan, T. J., Brown, J., Johnson, A., Sanburg, C., & Schildmeier, M. (2004). Science literacy and belief in the

paranormal—an empirical test. Skeptic, 10(4), 12–13.Sagan, C. (1996). The demon-haunted world: science as a candle in the dark. New York: Ballantine Books.Smith, M. U., & Siegel, H. (2004). Knowing, believing, and understanding: what goals for science education?

Science & Education, 13, 553–582.Tobacyk, J. (2004). A revised paranormal belief scale. International Journal of Transpersonal Studies, 23, 94–

98.Turgut, H. (2011). The context of demarcation in nature of science teaching: the case of astrology. Science &

Education, 20(5–6), 491–515.Wesp, R., & Montgomery, K. (1998). Developing critical thinking through the study of paranormal phenomena.

Teaching of Psychology, 25, 275–278.Wood, M. J., Douglas, K. M., & Sutton, R. M. (2012). Dead and alive: beliefs in contradictory conspiracy

theories. Social Psychological and Personality Science, 3(6), 767–773.

Critical Thinking Through History 105