resolving arguments accurately

Resolving Arguments Accurately

MIKE ALLEN AND NANCY A. BURRELL

Department of CommunicationUniversity of Wisconsin-MilwaukeeMilwaukee, Wisconsin 53201U.S.A.

ABSTRACT: This empirical investigation examined how ordinary language users resolveddisagreements over the solutions to categorical syllogisms. Forty-six participants completedpuzzles in logic. After completing the puzzles, participants were then randomly paired into23 to compare their answers and to resolve 159 disagreements. Results indicate that themost frequently used strategies for resolving disagreements centered on: (a) arguing overthe merits of the position (47% of the time) and (b) appealing to past solutions as a meansof addressing current disputes (28% of the time). In addition, the data revealed that themost frequently used strategy (arguing the merits of the positions) was no more effectivethan random choice (52% increase in correct solutions) while the strategy of appealing topast solutions significantly aided dyads in reaching correct solutions (70% increase incorrect solutions).

KEY WORDS: Resolving argument, logic, rationality, discourse, syllogisms.

Argumentation involves two general processes: an internal or intraper-sonal decision-making procedure, and an external process of social inter-action where reasons are provided to persuade others. Scholars studyingargumentation are interested in describing how ordinary and specializedactors use these two processes to draw conclusions and resolve disagree-ments. In this empirical investigation, we examined how an individual'sconclusions were tested during social interaction. We examined howordinary language users resolved disagreements about the solutions tocategorical syllogisms.

One test of any individually reasoned conclusion is the acceptability ofthe conclusion after interaction with others. The arguer will not alwaysconvince others or be convinced by others. Many arguments betweenindividuals remain unresolved. During the argumentation process, individ-uals consider the interests, concerns, and reasoning of others in decidingwhether or not to modify their positions.

Research by Jacobs, Jackson, Allen, and Patrell (1985) showed thatwhen individuals were solving abstract categorical syllogisms, they couldrecognize a valid conclusion argued for by others even when their initialconclusion was incorrect. Participants in this research had no priorexperience with formal logic or with categorical syllogisms.

Argumentation 4: 213-221,1990.© 1990 KluwerAcademic Publishers. Printed in the Netherlands.


What this indicates is that the process of exchanging information andreasons changes the decisions of individuals. Jacobs, Allen, Jackson, andPatrell (1985) suggest that individuals were capable of recognizing andcorrecting errors in reasoning during the course of an interaction.

Unfortunately, we know little about how individuals learn to recognizeand then correct their erroneous conclusions during the course of aninteraction. Usually investigations focus on the particular strategies used tojustify positions (Cf. Benoit, 1981; Miller, 1969; Putnam & Geist, 1985;Putnam & Wilson, 1987) or the function of the argument (Cf. Benoit,1983; Trapp & Hoff, 1985). Based on our review of the literature severalquestions come to mind. Which argumentative appeals are more effectivefor arguing about solutions to logical problems in interactive contexts?How does public argument alter individuals' conclusions which wereproduced by internal reasoning processes? Thus, the following researchquestions will guide our inquiry:

Q1: What are the most frequently used strategies for resolving disputesabout solutions to logical problems in interactive contexts?

Q2: To what degree are correct and incorrect conclusions associated withthe use of a particular strategy?

METHOD

The purpose of this study was to examine how argumentative interactionaffects an individual's solutions of logical puzzles. Participants were askedto solve categorical syllogisms individually and then were randomly pairedinto dyads to resolve their discrepant conclusions.

Participants

Forty-six participants from several introductory communication courses atMichigan State University were recruited to take part in this study.Participation was voluntary and students received extra credit in theircourses. None of the participants had any prior training in symbolic logicor set theory.

Procedure

Participants were given the following directions: "This is a study of logicalreasoning. For each pair of 16 premises in the booklet, write down whatconclusion, if any, can be drawn from the premises. If more than oneconclusion can be validly drawn, write down both in the space below thepremises. Write 'no conclusion' if none can be validly drawn. Conclusions

214

RESOLVING ARGUMENTS ACCURATELY

should be expressed in terms of the relationship between the A's and theC's." Participants were directed to work independently.

Next, participants were randomly paired into dyads and were directedto check their work with each other. Participants were told that the goal ofthe pairing was to try to agree on a correct set of answers. Dyads wereassigned separate rooms to compare their answers. Each room wasequipped with a tape recorder for the purpose of recording the inter-actions between dyads as they compared their answers to the categoricalsyllogisms. Recorders and microphones were in full view of participants.

Participants resolved or failed to resolve their disagreements in one ofthree ways: (a) they failed to reach a common answer; (b) one of theanswers originally made by Participant 1 was accepted by Participant 2 ascorrect (or just the opposite); or (c) the dyad agreed on an answer thathad not been selected previously by either participant.

Finally, the audiotapes of the 23 interactions were transcribed forcontent analysis. To insure accuracy in transcription, a second judgecompared the original audiotapes with typed transcripts.

Coding Scheme

We developed a coding scheme for categorizing various strategies dyadsused to resolve their disputes. These strategies consisted of the followingsix categories: (a) appealing to some past disagreement that had beensuccessfully resolved; (b) arguing over the merits of the positions; (c)discovering a "misunderstanding" of the instructions which led to anincorrect answer; (d) asking the opponent to justify his or her reasoningprocess; (e) pleading ignorance and allowing the other person to make thedecision, and (f) other; not covered by this coding scheme. The followingis a discussion of the various categories of the coding scheme.

Appealing to past precedent. This method of reaching a conclusioncentered on statements like, "This problem is just like the other one wherewe said...." The underlying feature of this strategy was not a presentationof reasons justifying an answer chosen by one of the participants, but anappeal that the answer to this problem was the same as an answer to aprevious problem because the problems were identical in structure.

Arguing over the merits of the positions. This strategy reconstructed thelogic employed in the decision-making process. Through this reconstruc-tion, partners justified their answers. For example, a participant might say,"Suppose that A is 1 and B is 2. That meets the condition that no A is B.Suppose you consider that no B is C. All that means is that C is not 2. Ccould be 1 or C could be 3 or C could be 3 and 1. You don't know.Therefore, the correct solution is that no conclusion can be drawn aboutthe relationship between A and C." When the other member of the dyad

215


conceded (often after an exchange of views and reasons), the eventualconcession demonstrated the superiority of the process.

Misunderstanding the instructions. This particular method of reaching aconclusion occurred when a member of the dyad recognized the "correct"answer by comparing responses with his or her partner and said, forexample, "You are right. I did not consider that some meant some -possibly all. Therefore, my answer is not correct." An important feature ofthis strategy was that the participant maintained that his or her process ofreasoning was correct. The error was simply a misapplication of theprocess based on a misunderstanding of the instructions.

Asking a question about the other person's reasoning process. Thismeans of reaching a conclusion occurred when Participant 1 "won" theissue by listening to Participant 2 and then asking a question thatParticipant 2 could not answer. For example, the first person said, "Bcould be all of A, therefore C is probably in A." The second person asked,"How do you know B is all of A?" This question triggered an inability ofthe first person to answer so he or she conceded to the "logic" of thesecond person without challenging that logic. This tactic differs from thestrategy of arguing over the merits because the former strategy involvesboth participants challenging and responding to each other.

Pleading ignorance. This strategy occurred when a person heard anexplanation by another and admitted the inability to make arguments. Forexample, after hearing the first person's reasoning, the second personmight say, "I don't know anything about this; I guess you must be right."This response concedes that the speaker did not simply misunderstand thedirections but did not have the knowledge available to solve the problem.

Finally, the instances that did not fit into any of the above categorieswere coded as Other. One hundred and fifty-nine decisions were codedoverall. In order to assess the reliability of the coding scheme, two judgesevaluated six interactions that entailed 42 judgments or about 25% of thedata. Intercoding reliability using the coding scheme was high (Cohen'sKappa = 0.88).

RESULTS

The 23 dyads answered a total of 368 problems, disagreeing on 159 of theanswers. The dyads discussed their disagreements about each of theanswers, seeking a solution to which both could agree. The dyadsattempted to move from a state of disagreement about the solution toagreement on a solution. The change in answers could result in one ofthree possibilities: (a) change from an incorrect to another incorrectanswer (N = 22 or 14%), (b) change from an correct to an incorrectanswer (N = 42 or 26%), or (c) change from an incorrect to correct

216


answer (N = 95 or 60%). Our analysis ignores the first answer andfocuses on the subsequent one. Thus, we simply divide the solutions intocorrect (N = 95) or incorrect (N = 64) final answers. See Table I for acomplete listing of the results.

Research Question One

What are the most frequently used methods for resolving disputes? Thedyads primarily used two of the six methods of resolving disputes: (a)arguing over the merits of the position (N = 75 or 47%), and (b)appealing to past precedent (N = 44 or 28%). The other methods ofresolving disagreements were used relatively infrequently (misunderstand-ing instructions N - 12 or 8%, asking a question N = 8 or 5%, pleadingignorance N = 9 or 6%, other N = 11 or 7%).

Disagreements usually were resolved by arguing on the basis of someposition applied to the particular issue or by appealing to some pastagreement and claiming the method of resolution was relevant to theresolution of this particular disagreement. Thus, the dyads usually weretrying to argue, that is, to give reasons to justify their choice of a solution.

Research Question Two

To what degree are correct and incorrect answers associated with the use ofa particular strategy? This question examined how the use of a particularstrategy was associated with whether the dyads reached a correct solutionto the categorical syllogisms. The first research question detailed howfrequently the strategies were used but not whether the strategies led tocorrect or incorrect solutions. Because only two categories (past precedentand arguing the merits of the position) received substantial use, only theyare considered in this analysis.

We used prediction analysis outlined by Hildebrand, Laing, and

Table I. Summary of Disagreement Resolution Methods.

Total Solutions Solutions

Method of Resolving Argument Number % Correct % Incorrect

Relying on Past Precedent 44 (28%) 31 (70%) 13 (30%)Arguing the Merits 75 (47%) 39 (52%) 48 (48%)Misunderstood the Instructions 12 (8%) 9 (75%) 3 (25%)Asking a Question 8 (5%) 6 (75%) 2 (25%)Admitting Ignorance 9 (6%) 5 (56%) 4 (44%)Other 11 (7%) 5 (45%) 6 (55%)

217


Rosenthal (1977). This is a method of association based on the frequencyof occurrences which disconfirm a hypothesized relationship. For exam-ple, if we hypothesize that the use of a particular argumentative strategysubsequently will lead to a correct answer, the incorrect answers aretreated as "errors." If these errors are less frequent than the hypothesizedrelationship would lead us to expect, A should be relatively high (it has amaximum value of 1.0). If the errors are about as frequent as they wouldbe if there were no relationship between the argument strategy and thecorrect answer (e.g., flipping a coin as opposed to reasoning), A shouldbe 0.

Arguing the merits of the position, the most frequently used category,only helped the dyads find a correct solution about half of the time (N =39 or 52%, A = 0.08). A 52% improvement is not much better thanflipping and as the A of 0.08 indicates, arguing the merits of the positiondid not significantly help the arguers produce a correct solution. This runscounter to most arguments that the exchanging of reasons over issueshelps to produce a correct solution. Argument does change conclusions,but that process does not necessarily guarantee a more accurate solution.

Appeals to the past helped the dyads correctly resolve a disagreementmore than arguing the merits of the position (N = 31 or 70%, A = 0.74).This is indicated by the fact that it increased the number of correctsolutions by 74% and by the A of 0.74.

One aspect of this reliance on precedent is whether the past precedentwas correct or not. Reliance on the previous precedent should replicatethe same result (correct or incorrect conclusion) with the present argu-ment. If a dyad obtained a correct solution to the categorical syllogism andlater argued that the current disagreement was the same as some pastargument, participants should agree on a correct solution. On the otherhand, if the past argument was resolved incorrectly, a dyad should reachan incorrect solution.

A chi-square test of this association demonstrates that past outcomesare related to future outcomes relying on that particular precedent (X2 =

38.7, df = 1, p < 0.05). When initial arguments were correct and reliedon for precedent, the subsequent arguments were also correct. Wheninitial arguments were incorrect, subsequent arguments relying on these asprecedent were also incorrect. The results point to the importance ofestablishing the rules for procedure very early in the argument becausethey affect the resolution of later arguments.

DISCUSSION

The results indicate that, when attempting to solve a logical puzzle,arguments about the merits of a position improve accuracy of the final

218


Table II. Frequencies of the Outcomes of Arguments Relying on Precedent.

Solution to Current Argument

Correct Incorrect

Incorrect 2 (10)1 11 (3)Solution toPast Argument

Correct 31(23) 0 (8)

X2=38.7, df= 1,p < 0.05

i Number in parentheses is the expected frequency.

decision only slightly. The other method, appealing to precedent, involvesthe establishment and application of valid decision rules about how toresolve disagreements early in a discussion.

One of the most interesting findings of this study involves the fact thatarguing about the merits of a position, presumably one of the mostcommon ways that people argue in social interaction, does not improvetheir chances of finding the correct answer to a logical puzzle. If theresults of this study are to be believed, arguers might as well resort toflipping coins.

But the same is not true when arguers rely on past precedent to solve alogical problem. Of course, the generality of these findings may be affectedby the nature of the task - a solution to a problem of formal logic. Butassuming the validity of the initial decision, the results are likely to bereplicated when future disagreements use the conclusions and standards toresolve disputes. However, if the initial decision is poor then the sub-sequent decision probably will be poor also.

The data point to the importance of establishing and maintaining goodprecedents in argument. In a real sense, the establishing of rules toconduct and manage disagreement is normal in ordinary conversation(Jacobs & Jackson, 1981; Jackson & Jacobs, 1980) and specializedcircumstances (Robert, 1970). If the goal is to interact in such a mannerthat disagreements can be resolved, the use of precedent is quite practical.Rather than disputants repeating arguments, recognizing that the currentproblem has the same structure as a past problem permits the arguers todetermine a solution quickly.

Some argument communities (science, medicine, law) act as though theoutcome for a community's arguments can be appealed to some consistent(if not objective) set of prior decisions. For example, a lawyer or judge canconsult the relevant precedents and/or procedural rules to determine howarguments should be resolved. Juries are given instructions about how to

219


resolve conflicts over the evidence. Scientists and medical doctors ofteninterpret the results of tests and procedures as objective. These communi-ties expect that as the evidence is acquired and the explanations are testedthrough argument, the social "truth" will emerge (Hollihan, Riley, &Freadhoff, 1986; Newell & Rieke, 1986).

This acquisition of truth in fields like science and medicine may takeyears or even decades as evidence is gathered, theories are tested, andarguments are made regarding the interpretation. By contrast, somecommunities expect arguments to continue with little opportunity orexpectation for a final resolution (religion, philosophy, politics). Membersof these communities may recognize acceptable and unacceptable forms ofargument but do not have methods or techniques for determining finalsolutions to specific issues (Benoit, 1981, 1985; Putnam, Wilson, Waltman,& Turner, 1986). For example, there is no expectation that the conclu-sions reached by a long dead philosopher binds current philosophicalargument in the same way that legal precedent is expected to restrict alawyer. In short, there are varying expectations about the use of priordecisions.

At an individual level, types of "truth" also emerge. Suppose a personattempts a strategy to win an argument. The attempt is rejected and he orshe is told by others participating in the interaction that the suggestion wasrude, inconsiderate, or wrong. This interaction should change the argu-ment strategy used by the person. The result is a dependency between thepublic and private spheres of argument that influence each other overtime.

REFERENCES

Benoit, P.: 1981, 'The Use of Argument by Preschool Children: The Emergent Productionof Rules of Winning Arguments', in G. Ziegelmueller and J. Rhodes (eds.), Dimensionsof Argument, Annandale, VA: SCA, pp. 624-642.

Benoit, P.: 1983, 'Extended Arguments in Children's Discourse', Journal of the AmericanForensic Association 20, 72-89.

Benoit, P.: 1985, Negotiating Shared Reality in the Interpersonal Arguments of MarriedCouples, paper presented at the Speech Communication Association Convention,Denver, Colorado.

Hildebrand, D. K., J. D. Laing, and H. Rosenthal: 1977, Prediction Analysis of CrossClassifications, New York: Wiley.

Hollihan, T., P. Riley, and K. Freadhoff: 1986, 'Arguing for Justice: An Analysis of Arguingin Small Claims Court', Journal of the American Forensic Association 22, 187-195.

Jackson, S. and S. Jacobs: 1980, 'Structure of Conversational Argument: Pragmatic Basesfor the Enthymeme', Quarterly Journal of Speech 66, 251-265.

Jacobs, S., M. Allen, S. Jackson, and D. Patrell: 1985, 'Can Ordinary Arguers Recognize aValid Conclusion If It Walks Up and Bites Them on the Butt?', in J. R. Cox, M. O.Sillars, and G. B. Walker (eds.), Argument and Social Practice, Annandale, VA: SCA,pp. 665-674.

220

RESOLVING ARGUMENTS ACCURATELY 221

Jacobs, S. and S. Jackson: 1981, 'Argument as a Natural Category', Western Journal ofSpeech Communication 45, 118-132.

Miller, G.: 1969, 'Some Factors Influencing Judgments of the Logical Validity of Argu-ments: A Research Review', Quarterly Journal of Speech 55,276-286.

Newell, S. and R. Rieke: 1986, 'A Practical Reasoning Approach to Legal Doctrine',Journal of the American Forensic Association 22, 212-222.

Putnam, L. and P. Geist: 1985, 'Argument in Bargaining: An Analysis of the ReasoningProcess', Southern Speech Communication Journal 50, 225-245.

Putman, L. and S. Wilson: 1987, Argumentation and Bargaining Strategies as Discrimina-tors of Integrative and Distributive Outcomes, paper presented at the First InternationalConference of the Conflict Management Group, Fairfax, Virginia.

Putnam, L., S. Wilson, M. Waltman, and D. Turner: 1986, 'The Evolution of CaseArguments in Teacher's Bargaining', Journal of the American Forensic Association 23,63-82.

Robert, H.: 1970, Robert's Rules of Order, Glenview, IL: Scott, Foresman Co.Trapp, R. and N. Hoff: 1985, 'A Model of Serial Argument in Interpersonal Relationships',

Journal of the American Forensic Association 22, 1-11.

resolving arguments accurately

Documents