HEURISTICS AS WARRANTS:

Leveraging Sourcing and Corroboration Heuristics as

Warrants in Historical Argumentative Writing

Ryan McCarty@RyanP_McCarty

rmccarty2875@gmail.comUniversity of Illinois at Chicago

LRA Annual Conference12/4/2015

Agenda

My Path to this Research Theoretical Framework Methods Findings and Discussion Implications

My Path to this Research

My time as a teacher

The importance of warrants, and warrants that may influence those in power

Using historical thinking heuristics to warrant claims

Historical Thinking Heuristics (& Close Reading) as Warrants

Sourcing identifying and evaluating the credibility of a source

based on its author or origin (Reisman, 2012)

Corroboration reading across multiple historical accounts to compare

content and identify differences, giving more weight to information common across accounts (Britt & Aglinskas, 2002)

Close Reading analyzing an author’s language use for three purposes:

to learn what a text says, to determine how a text works, and to evaluate its larger significance in relation to other texts and ideas (Adler, 1965; T. Shanahan, 2013)

Theoretical and Empirical Basis

Disciplinary Literacy (e.g. Moje, 2008; Shanahan & Shanahan, 2008)

Discourse Perspectives (e.g. Gee, 1989; Lankshear et al., 1997)

Multiple Text Comprehension (e.g. Perfetti et al., 1999; Wineburg, 1991)

Reading and Writing Connection (e.g. Fitzgerald & Shanahan, 2000; Tierney & Shanahan, 1996)

Argumentation & Historical Argumentation (Freeman, 2005; Toulmin, 1958; De La Paz & Felton, 2010; Monte-Sano, 2008)

Study Design

Quasi-Experimental and Mixed Methods 89 11th grade students, US History class,

diverse setting near major urban area Two instructors; 8 days total

Treatment Condition: Heuristics as warrants, warrant-generating questions (Freeman, 2011; Toulmin, 1958) and sentence frames (Graff & Birkenstein, 2007)

Comparison Treatment: Finding evidence to support different claims; Evaluating evidence

Research Questions and Hypotheses

Question 1: Does a treatment in using heuristics as warrants improve student performance on selecting and writing effective warrants, compared to a treatment in finding effective evidence?

Question 2: How do students perform on warrant selection and warrant writing items meant to reflect specific types of historical thinking?

Question 3: What sort of thinking does a subset of students in the treatment group engage in during the warrant selection and warrant writing tasks?

Hypothesis- Students in the treatment condition will select and write warrants more effectively than students in the comparison treatment

Central Questions of Unit

Did the explosion of the U.S.S. Maine cause the U.S. to invade Cuba?

Yes, the explosion of the U.S.S. Maine caused the U.S. to invade Cuba

No, the explosion of the U.S.S. Maine did not cause the U.S. to invade Cuba

Should the U.S. have invaded Cuba following the explosion of the U.S.S. Maine?

Yes, the U.S. should have invaded Cuba… No, the U.S. should not have invaded Cuba…

Text Set for Assessment

http://historicalthinkingmatters.org/spanishamericanwar/

Two newspaper articles, a song, two speeches, two telegrams, and a textbook excerpt

Methods

Figure 3.1. Example of a Warrant Selection Task

Example Warrant Selection Task

Figure 3.2. Example of Warrant Writing Task

Example Warrant Writing Task

Impact of Treatment on Warrant Selection & Writing

1a. Ability to Select Effective Warrants Warrant Selection Task

Ind. Samples t-test Warrant Selection Task “Correct” vs. “Not

Wrong” answers Ind. Samples t-test

1b. Ability to Write More Effective Warrants Coding for Aspects of Quality

Examine Means Coding for Historical Thinking

Ind. Samples t-test

Impact on Specific Types of Thinking

2a. Warrant Selection Performance by Type of Thinking Item Analysis by type of thinking

Ind. Samples t-test

2b. Warrant Writing Performance by Type of Thinking Coding for type of thinking

Ind. Samples t-test

Student Thinking During Tasks; Student Thoughts on Treatment 3a. Types of Thinking Displayed During

Task Subset of 4 students from Treatment Condition Think Aloud Protocol

Coding for type and percentage of historical thinking Coding for effectiveness

3b. Student Thoughts on Intervention Semi-structured Interview

Coding for type and percentages of historical thinking Coding for what learned, how different from typical

instruction Coding for what would make this easier (item 10)

Findings and Discussion

Research Question 1: Findings and Discussion

Ability to Select Effective Warrants Selection overall: No Statistically Significant Difference Heuristics M = 39.32 vs. Evidence M = 36.63, range = 26

Difficulty of foils

“Correct” vs. “Not Wrong”: No Statistically Significant Difference Heuristics M = 19.16 vs. Evidence M = 15.97, range = 22

p = 0.01; = .005 Ability to Write More Effective Warrants

Quality: No Statistically Significant Difference Accuracy slightly lower for Heuristics

Overall Hist. Thinking: No Statistically Significant Difference (M = 2.20 vs. M = 1.36), range = 4

p = 0.01; = .003

Claim: No, explosion of U.S.S. Maine did not cause the U.S. to invade Cuba

Evidence: The New York Times (Doc B) reported that Navy Leader Captain Schuley felt the explosion was an “accident”, likely caused by a coal bunker fire that ignited the ship’s ammunition. However, the New York Journal (Doc A) stated that “all” naval officers thought the Maine was “purposely” destroyed by a Spanish mine. It says the “brutal” Spaniards may have triggered the explosion while the men were sleeping.

Student Warrant: “While Doc A says it wasn't an accident, Doc B says it was an accident. The New York Times tells the truth about certain events that actually did happen. Unlike the New York Journal they always exaggerate and make up stories to make it more reliable. Doc B provides more evidence to show why because of a fire that could have occurred. Just because Doc A uses the word brutal doesn't mean - the explosion of the U.S.S Maine (caused) the U.S to invade Cuba.”

Writing Task Example: Heuristics Used as Warrant

Research Question 2: Findings and Discussion

2b. Warrant Selection Performance by Type of Thinking

Mean performance was slightly higher for treatment across types of thinking

Mean difference largest for corroboration 2c. Warrant Writing Performance by Type of

Thinking No difference in thinking types overall

Items cued different types of thinking Lowest performance on “Yes, should have caused”

Predicate/side

Table 4.6

Performance on Warrant Selection Task by Type of Thinking

Total Items

Group

M (SD)

Sourcing 16 Treatment 10.32 (2.27)

Comparison 9.73 (2.36)

Close Reading 16 Treatment

Comparison

10.60 (1.89)

9.80 (2.16)

Corroboration 16 Treatment

Comparison

10.68 (2.17)

9.67 (2.12)

Non-Historical 12 Treatment 7.84 (1.52)

Comparison 7.57 (2.27)

Note. n for Treatment =25. n for Comparison Treatment =30

Table 4.9

Performance on Warrant Writing Task by Type of Thinking- Items 1&2 Total

Highest Possible Total

Group

M (SD)

Sourcing 2 Treatment 1.00 (0.71)

Comparison 0.68 (0.61)

Close Reading 2 Treatment

Comparison

0.64 (0.70)

0.36 (0.56)

Corroboration 2 Treatment

Comparison

0.56 (0.58)

0.32 (0.48)

Sum of Historical Thinking 6 Treatment 2.20 (1.29)

Comparison 1.36 (1.03)

Note. For treatment, n=25. For Comparison Treatment, N=28

Student Thinking During Task; Student Thoughts on Treatment

3a. Heuristic Use During Think Aloud Warrant Selection:

All used at least 2 types of historical thinking Close reading effectiveness ranged from 67% to 100%

Warrant Writing: All used Sourcing; Consistently effective (14 of 15 idea units) Task may be easier for students

3b. Student Responses During Interview Heuristic Use

All show some hist. thinking; Primarily sourcing No close reading; Did not refer to text during responses

Suggestions for Making Work Easier: Time, practice, feedback Changes to task design

What They Learned: 2 emphasized heuristics as warrants, one said argument structure one found it confusing

Ivan on what he learned

I learned how to use sourcing, like how to connect the evidence and that could make a strong argument, just from sourcing. I didn’t know that before… now we used sourcing, as forwarding your claim [sic].

… So we used sourcing to see if it was reliable or not, but now we’re like, we’re seeing “So this is reliable… because this person was there, and when was this written, and how far or how close it was, would that change their viewpoint.

... I would like to say, “Hey this guy, he was there, he survived, he would know a little more about what happened there, than someone who was like running around ten years later, based on like information they got offline or something like that [sic].

Sonia on what she learned

I’m not really sure. Like the warrants thing confused me a little because we’ve been using reasoning from the beginning of the school year. So… having a new term (was) kind of, well confusing, how to use it, when to use it or what it is [sic].

It was somewhat easier, but it didn’t feel like I was doing much, like it was coming from me, so it was copy and paste… We usually write our own claims, well like the whole week we usually have our own claims, our own evidence, and then our own like reasoning and concluding sentence, everything was there for us (but here) we had to use what we were given…

Implications

Limitations

Design limitations Narrow definition of historical

thinking Refine Instrumentation

Warrant selection Warrant writing Think Aloud & Interview

Implications

Implications for Research Innovative approach to instrumentation; refine

further Refine treatment to positively impact student

learning Continued research of warrants, a neglected yet

essential area of study Implications for Instruction

Implications limited due to nonsignificant results Explicitly teaching of hist. argument writing Use of heuristics in combination; Close reading

in discipline

Next Steps

Qualitative analysis of essay writing data

Sampling of essays from each quartile Breaking into idea units, coding for aspect of

argumentation (i.e. claim, evidence, warrant) More refined analysis of type of historical

thinking present in warrant Looking across data sets Combining measures for MANOVA

analysis Formative Design-Based Research;

Transfer to issues students choose