qualitative & multi-method research · of data collection/creation and methods of data...

Fall 2017, Vol. 15, No. 2

Qualitative & Multi-Method ResearchContents

Letter from the Section PresidentMelani Cammett 1

Articles

Truth Seeking AND Sense Making: TowardsConfigurational Designs of Qualitative MethodsJoachim Blatter 2

Opportunities and Obstacles in Distributed orCrowdsourced CodingBrian R. Urlacher 15

Letting Easton Be Easton—An InterpretivistWilliam J. Kelleher 22

Symposium: Causal Explanation and the Study of RareEvents Through Large Sweeps of History

Introduction to the Symposium on Dale C. Copeland,Economic Interdependence and War (2015)Tim Büthe 29

The Central Methodological Claims and Contributionsof Economic Interdependence and WarDale C. Copeland 33

The Empirical Study of Great Power PoliticsTimothy McKeown 36

Evaluating “Competing” Explanations in EconomicInterdependence and WarSherry Zaks 40

Theoretical and Methodological Reflections onEconomic Interdependence and WarErik Gartzke 44

Rare Events and Mixed-Methods Research:Shaping the Agenda for the FutureDale C. Copeland 48

QMMR News and Announcements

2017 QMMR Section Awards 57 Letter from the Editors 59

Welcome Letter by QMMRSection President Melani Cammett

I am delighted to serve as the President of the QMMR section,which plays a key role in the discipline as a meeting point forscholars with diverse intellectual and methodological orienta-tions. The vibrancy of the section attests to the fact that quali-tative and multi-methods research continue to thrive in politi-cal science.

The section officers deserve special credit for carryingout the work of the section. I especially want to acknowledgePeter Hall, who recently completed his term as QMMR Presi-dent. Among his many contributions, Peter actively promotedthe deliberations around research transparency and integrity—an activity that has been a critical focus of the section overthe past couples of years. As ever, Colin Elman, the QMMRSecretary-Treasurer, works tirelessly on behalf of the section.It would not be an exaggeration to say that, without him, wewould not be where we are today.

Special thanks are also due to Tim Büthe and Alan Jacobsfor their important work on two fronts—as editors of Qualita-tive and Multi-Methods Research and for spearheading theQualitative Transparency Deliberations (QTD). First, duringtheir tenure as co-editors, Tim and Alan continued to elevatethe profile of our section’s flagship publication by publishinga variety of symposia on important topics. Thanks to theirefforts, building on the work of their predecessors as editorsof QMMR, it has become a scholarly publication rather thanjust a “newsletter,” a label which the section appropriatelydecided to drop from the title in 2016. The recent adoption ofDOIs for articles published in the newsletter symbolizes thevery real intellectual contributions of the symposia and ar-ticles featured in this publication. In addition, at the QMMRannual business meeting at the APSA meetings held in 2017,the membership voted to switch the newsletter to an all-digi-tal format under the incoming co-editors, Jennifer Cyr andKendra Koivu. Among other benefits, this will allow for morefunds to be devoted towards editorial work for this valuableresource.

continued on page 58

A Biannual Publication of the QMMR Section of the American Political Science Association

APSA-QMMR Section Officers President: Melani Cammett, Harvard University At-Large Members of the Executive Committee: President-Elect: Jason Seawright, Northwestern University Michael Coppedge, University of Notre Dame (2016-2018) Vice President: Henry Hale, George Washington University Pauline Jones Luong, University of Michigan (2016-2018) Secretary-Treasurer: Colin Elman, Syracuse University Jeb Barnes, University of Southern California (2017-2019)

Frederic Schaffer, University of Massachusetts (2017-2019)

QMMR Editors: Annual Meeting Program Chair (2018): Alan M. Jacobs, University of British Columbia Ryan Saylor, University of Tulsa Tim Büthe, Hochschule für Politik at the Technical

University of Munich, Germany & Duke University

https://www.doi.org/10.5281/zenodo.2563220

2

Qualitative & Multi-Method Research, Fall 2017

Truth Seeking AND Sense Making:Towards Configurational Designs of Qualitative Methods

Joachim BlatterUniversity of Lucerne, Switzerland

Introduction and Overview1

Within the qualitative social sciences, we can detect a widegulf between those who strive for revealing “the truth” aboutthe social world on the one hand and those whose goal is to“make sense” of it on the other. The former apply methodswhich are implicitly or explicitly rooted in positivist or realistepistemologies and ontologies whereas the latter apply meth-ods based on constructivist or interpretative epistemologiesand ontologies. Some of the characteristic expressions of thisgulf can be found in the work of Goertz and Mahoney whoexclude interpretative approaches in their characterization ofqualitative research.2 This is mirrored by Yanow who insistson the distinctiveness of interpretative research.3

Nevertheless, a closer view reveals that on both sides,among ‘truth-seekers’ and among ‘sense-makers,’ we find quitedistinct research goals, epistemological principles, and onto-logical presumptions, as well as a broad spectrum of methodsof data collection/creation and methods of data analysis/inter-pretation. The major claim of the following contribution isthat the internal diversity within both camps makes it possibleto develop a plurality of coherent qualitative methods whichallow to strive for truth-seeking and sense-making at the sametime. These methods are configurational in the sense that theycombine epistemological and ontological features of truth-seek-ing endeavors with those of sense-making projects. If appro-priately conceptualized and designed, they do this without

Joachim Blatter is Professor of Political Science at the Universityof Lucerne and a member of the Lucerne Cluster for ConfigurationalMethods (LUCCS). He can be found online at [email protected]. He would like to thank Alan Jacobs and Tim Büthe for theinvitation to present the following in QMMR, Alan Jacobs for excel-lent comments which lead to much clearer and more precise argumen-tations, and Alberto Alcaraz for his terrific job in language polishingand copy editing.

1 This contribution emerged out of endeavors to provide overviewsover qualitative methods in general (Blatter, Janning and Wagemann2007, Blatter, Haverland, van Hulst 2016, Blatter, Langer and Wage-mann 2017) and case study designs more specifically (Blatter andBlume 2008, Blatter and Haverland 2014). In line with major works(e.g. Brady and Collier 2004, Goertz and Mahoney 2012), in our firsttextbooks we described qualitative methods by comparing and con-trasting them to quantitative ones. In our more recent publications,though, we characterize distinct qualitative methodologies primarilyby comparing them to each other. The former approach facilitatesmutual understanding between qualitative and quantitative scholarsand stimulates multi-method research that combines qualitative andquantitative methods; the latter tries to do the same among truthseekers and sense makers within the community of qualitative socialscientists.

2 Goertz and Mahoney 2012.3 Yanow 2003.

losing their internal coherence and are therefore helpful forbuilding bridges across the aforementioned gulf.

In order to develop these configurative and coherent meth-ods, I start by reflecting on epistemology and argue that apragmatic epistemological stance consists of three components:a) a research goal specified by a research question; b) thecorresponding kind of aspired knowledge/explanation; and c)the adequate way to secure the validity (and reliability) of theresearch process and its results. For each component, I iden-tify principled differences between truth-seekers and sense-makers, but point to internal diversity on each side, as well.Next, I turn towards ontology and sketch the three compo-nents of a pragmatic ontological stance: a) presumptions aboutthe basic entities of the social world; b) presumptions aboutthe relationship among these entities; and c) understandingsof causes and causation. Once again, I identify principled dif-ferences between the two camps and internal diversity in eachcamp. Based on these premises, I present two tables in whichthe scrutinized alternatives are aligned in such a way that sixideal-typical methods emerge. Each of these methods is char-acterized by an internally coherent set of epistemological pur-poses/principles and ontological presuppositions. A third tablein which I lay out the corresponding methodological ap-proaches to data collection/creation and to data analysis/in-terpretation can be found elsewhere.4 Due to the space limita-tions of this paper, it could not be incorporated.

Figure 1 provides a first overview, indicating not only theconceptual poles in the epistemological and ontological di-mensions, but also the location of the six ideal-typical meth-ods. Furthermore, figure 1 specifies and substantiates the ma-jor claim of the contribution formulated above. I argue that anideal-typical method called Comparable Cases Strategy strivessingle-mindedly for revealing the truth (about the autonomouseffect of a single cause). In contrast, the Con-Textual Analysismethod aims entirely at making sense by creating enlighteningunderstandings of the empirical world. The four methods inbetween, though, can be conceptualized in such a way thatthey combine truth seeking and sense making without beingincoherent. Methodological approaches labelled Configura-tional Comparative Analysis and Co-Writing Cultures arestill predominantly committed to either truth seeking (theformer) or to sense making (the latter), but they incorporatealready some epistemological and ontological features of theother side. Causal-Process Tracing and Congruence Analy-sis (two specified forms of within-case analysis which areoften—alongside other forms of within-case analysis—lumpedtogether under the term ‘Process Tracing’), represent methodsthat come close to balancing the epistemological principles

4 Blatter 2016.


3


Source: Blatter, Haverland and von Hulst 2016: xxi

Figure 1: Locating Ideal-Typical Qualitative MethodsAccording to Their Coherent Positions in Respect to Epistemology and Ontology

and ontological presuppositions of truth-seekers and sense-makers.

Figure 1 reveals that my typology of methods containsonly ideal-types and not all logically possible combinations.Ideal-types are characterized by a conceptually coherent com-bination of specific epistemological purposes and principleson one side and specific ontological presumptions on the other.This means that I do not claim that these types exist in theirideal-typical form as methodological devices in textbooks oras distinct cultures among practitioners. At the end of thisessay, I point to potential uses of a typology of ideal-types.

Before I start, I would like to signal to the reader that theattempt to bridge a very broad spectrum of qualitative method-ologies and to develop ideal-types has two consequences inrespect to terminology:

1. I use core expressions like “explanation” and “cau-sation” as umbrella terms. This means that I treat “inter-pretation” and “understanding” as specific types of ex-planation and “constitution” or “construction” as a spe-cifictype of causation.5 Such a stance can be justified asfollows: A common terminology facilitates mutual understanding and helps to build bridges between truth-seekerand sense-makers. Furthermore, terms and terminologyframe the (scientific) discourse and set the limits of whatis accepted and what is not. Thus, if interpretation is notaccepted as a form of explanation, or if causality is limitedto the empiricist/analytic understandings that dominate

5 In contrast to Wendt 1999, but in line with Elster 2007 and Kurki2008.

the natural sciences, then these are fundamental forms ofexclusion.

2. The terms that are used to label the ideal-typicalmethods are based on a broad reading of the literature andare picked in order to correspond as much as possible toexisting terminology. Nevertheless, the ideal-typical ap-proach implies that the main criteria for choosing labelshas not been correspondence to existing terminology, butlinguistic coherence and conceptual correspondence tothe scrutinized features of the ideal-type.

Towards Pragmatic and Pluralist Epistemological Stances

In his widely cited textbook Approaches to Social Enquiry,Blaikie defines epistemology as “a view and justification forwhat can be regarded as knowledge—what can be known, andwhat criteria such knowledge must satisfy in order to be calledknowledge rather than belief.”6 Such a definition tends to leadto principled disputes about what we can know in the socialsciences given the facts that social scientists are parts of thesocial world they study, to the extent that some scholars claimthat the social word cannot be studied as an externally or ob-jective existing entity. A pragmatic approach, in contrast, startswith the assumption that we should accept all kinds of re-search goals which can be specified in distinct research ques-tions. For each research question, we can identify an under-standing of knowledge that is most appropriate for providing auseful answer to the question, and we can specify the criteriathat guide the processes of gaining this kind of knowledge. As

6 Blaikie 1993, 7.

4


we will see, the most important advantages of such a prag-matic approach are that it helps to overcome dichotomousthinking and that it paves the way towards epistemologicalpluralism.

If we accept that an epistemological stance should bebased on what we want to know (instead of what we can knowgiven some specific ontological presupposition, for example),then a specific epistemological stance encompasses three com-ponents:

a. the research goal expressed in a precise research ques-tion and translated into a corresponding research design,

b. the type of knowledge/explanation that we aspire inorder to answer the research question (e.g. if we want toknow, how -and not whether or how much- X influences Y,then we strive for a mechanism-based explanation), and

c. the principles and procedures that guide the processof acquiring this kind of knowledge and the correspond-ing criteria for evaluating the quality of concrete researchprojects.

In the following section, I do not only scrutinize the prin-cipled alternatives in respect to these three components, but Iindicate how we can overcome the dualism that is invoked byfocusing just on the principled alternatives.

Research Goals: Truth-Seeking,Sense-Making and their Combinations

Social scientists, especially those who pretend to do “qualita-tive” research, find themselves located between the hard/natu-ral sciences and the arts/humanities. Therefore, it is not sur-prising that some qualitative social scientists—in line withnatural scientists—adhere to “truth-seeking” as their prin-cipled goal of research, whereas others strive—in line withthose from the humanities—for “sense-making.” The proto-typical research goal for truth-seekers is to develop and testparsimonious hypotheses and models that correspond to themain features of an external world. In contrast, the prototypicalresearch goal of the sense makers is to develop and applycoherent paradigms and theories that provide orientationthrough meaningful interpretations of the world. In conse-quence, it seems that truth-seekers and sense-makers haveclearly distinct and seemingly incommensurable research goals.Nevertheless, each of these principled research goals allowsfor a range of possible specifications as to what a researchquestion is.7

Among the prototypical research questions that truth-seekers might try to answer are:

a) Which effects does a specific and concrete cause (X) have?b) Which configurations of conditions make a specific kind ofoutcome (Y) possible?c) Which underlying mechanisms (M) make causes producean effect?

7 Blatter, Langer and Wagemann 2017, 7-17.

Sense-makers ask questions that also take a wide range ofprototypical forms, such as:

A) Which fundamental structure (S) stabilizes and/or trans-forms a social/political system?B) Which interpretative signs and practices characterize a spe-cific culture (C)?C) Which paradigms-based, but specified theory (T) providesa better explanation?

These prototypical research questions signal the exist-ence of a plurality of specified research questions within eachcamp. In the following section, I want to show that the searchfor answers to some of these questions demands researchdesigns and methods that combine truth seeking and sensemaking. For example, I argue that the most productive answerto question c) is based on an understanding of a causal mecha-nism as a configuration of three kinds of social mechanisms.Furthermore, it demands a kind of Causal-Process Tracingthat aims to show not only that the identified mechanismscorrespond to an external reality, but also that they make senseinsofar as the social mechanisms are integrated in a coherentand comprehensive multi-level model of explanation. Likewise,to answer question C), we need a method that strongly com-bines sense-making and truth-seeking. As we will show lateron, the corresponding method, Congruence Analysis, drawson the abstract approach to theory-formation that sense-mak-ers adhere to, but the answer also depends on the systematiccomparison of the expectations that we can deduct from thosetheories with the kinds of empirical observations that truth-seekers demand in order to accept an explanation.

At this point, these statements are not much more thanclaims, but I try to clarify and justify them in the followingsection. The first step on this path is to show that we find asimilar plurality (instead of a dichotomy) when it comes to theprincipled kinds of knowledge that we are striving for andwhen we develop criteria for gauging the quality of the pro-cess through which we gain this kind of knowledge. Becausemost methods in the social sciences have been developed astools of explanation, we limit our discussion to explanatoryknowledge, although we generally share John Gerring’s viewthat “descriptions” and “comparisons” constitute forms ofknowledge that are at least as important.8

Explanations: Confirmed Thesis,Convincing Paradigm, and Options In-between

In the social sciences, there are multiple and quite differentunderstandings of what a ‘good’ explanation is. In order toovercome simple dichotomies without erasing fundamentaldifferences, we will develop a two-dimensional space for locat-ing distinct understandings of explanations. The first dimen-sion of this conceptual space refers to the level of abstract-ness and the second dimension to the level of generality. SinceCollier and Mahon’s path-breaking work on concept building,9

8 Gerring 2012.9 Collier and Mahon 1993.

5


we know that Sartori’s “ladder of abstraction”10 was a misno-mer: abstraction and generalization are not the same thing, andSartori was primarily concerned with the problems of generali-zation. The opposite of “abstraction” is “concretization”,whereas the opposite of “generalization” is “specification.”

The dichotomy between abstract and concrete conceptsshows up in the distinct procedures through which these twokinds of concepts are defined: Abstract concepts are definedthrough reflection on the relationships that one concept hasto other abstract concepts. The attributes that we select forcharacterizing our abstract concept have to be justified withreference to a theoretical discourse. Concrete concepts, in con-trast, are defined through the assignment of indicators thatrefer to observations. The categorical difference between ab-stract and concrete concepts shows up, once again, when welook at how the “negative pole” of the concept is getting de-fined. For an abstract concept, the negative pole must be de-fined through a substantial alternative concept (e.g. “monar-chy” or “autocracy” for the concept of democracy3). For aconcrete concept, though, it is enough to define the negativepole as simple negation or as the null point (non-democracies,zero degree of democracy).

When we reflect about the level of generality of a con-cept, we are not concerned with how the concept’s definingcharacteristics have been derived. Instead, we reflect on therelationship between the set of characteristics or attributesthat define a concept (“category” in Collier and Mahon’s ter-minology) and the set of entities in the world to which theconcept refers.12 The former is called the “intension” of a con-cept, the latter the “extension.” Collier and Mahon’s most im-portant insight is that only in classical systems of categoriza-tion does a higher intension (a concept that is more specifiedby a higher number of attributes) leads to a lower extension (alower number of entities that correspond to the concept). If weuse family resemblance or radial categories (systems of cat-egorization in which some attributes are possible but not nec-essary attributes of a category), there is no logical trade-offbetween intension and extension. This means that the exten-sion of a category that is located on a lower level of generalitymay exceed that of a category that is located on a higher levelof generality.13 This is because, in these systems of categoriza-tion, going down the ladder of generality means to select aspecific configuration out of a larger set of possible attributeswhich characterize a concept.

In line with such non-classical systems of categorization,we can define

a) a Paradigm (P) as the set of all theories that combine a spe-cific core concept (CC) with one or a plurality of different pe-ripheral concepts (PC): P = CC * [PC1 + PC2 + PC3 + …]14,

10 Sartori 1970.11 For the concept of “democracy” see Goertz 2006, 32.12 Collier and Mahon 1993, 846.13 Collier and Mahon 1993, 850.14 Schimmelfennig (2003) shows that a rationalist paradigm in In-

ternational Relations consists of rational choice theory as a core con-cept plus divergent conceptualizations of the primary goals that states

whereby in line with Boolean algebra ‘*’ means ‘and’ whereas‘+’ means ‘or’; andb) a Theory (T) as a specific combination of the core conceptwith one or a plurality of peripheral concepts: T = CC * PC1 *PC2.

The formulas reveal that a Paradigm has not only a largerintension (more attributes) but also almost certainly a largerextension (more empirical entities to which the concepts refersto) than a Theory. In consequence, we might redefine what“intension” means for non-classic systems of categorizationand for non-classic approaches to concept and theory forma-tion: In these contexts, “intension” would not refer to the num-ber of attributes that characterize a concept, but to the inten-sity by which these attributes are linked to each other. If weaccept this definition of intension, we end up also with whatintuitively makes sense for the non-classic systems of catego-rization: a higher level of intensity leads to a lower level ofextensity. The meaning of this insight, however, is markedlydifferent to the currently dominant understanding which isstill in line with the writings of Sartori.

So far, we have clarified the difference between abstrac-tion and generalization. Furthermore, we have pointed to thecategorical differences between abstract and concrete con-cepts, and introduced non-classic forms of categorization orconcept formation. In earlier contributions, we transferred theseinsights from concept-formation to the task of theory-formula-tion. Whereas “attributes” represent the elements that we usefor specifying “concepts,” “concepts” are the elements thatwe use to specify “theories.” Such a transfer paves the way toconceptualizing the relationship between paradigms and theo-ries in the language of concept formation.15

In the following, we build on these insights and turn to-wards a systematic mapping of the different types of explana-tion for which the different strands of research strive to (seefigure 2).

On the one hand, we might want to find out whether aconcrete cause (low level of abstraction) has a specific effect(low level of generality). The corresponding hypothesis thatprovides a preliminary answer can be deduced from abstracttheories, but it does not have to. Quite often, such a hypoth-esis is (seen as) nothing more than an unproven claim. In orderto test the causal claim of said hypothesis, the independentand the dependent variable must be clearly specified andoperationalized (concretized) by observable indicators. If wecan control for all other potential causal factors by comparingsimilar cases in these respects, the observed co-variationamong the independent and dependent variable providesenough empirical leverage for transferring the hypothesis intoa truthful thesis about a causal relationship within a clearlydelimited population of cases.

On the other hand, there are abstract paradigms, whichaspire to make sense and provide orientation for many instances

aspire (power, welfare or security) as peripheral concepts (Blatterand Haverland 2014: 180-182).

15 Blatter and Haverland 2014, 158; Blatter 2016.

6


Figure 2: Different Kinds of Explanation

paradigm and specifies the status of the selected peripheralconcepts as necessary conditions for the theory (Theory =Core Concept * Peripheral Concept a * Peripheral Concept b).This means that a theory is located on a lower level of general-ity, but it remains abstract in the sense that its conceptualelements are derived first and foremost by discussing theirrelationship to other abstract concepts and not by referring to(existing) indicators. Crucially important for an adequate un-derstanding of the kind of explanation that a theory providesis that the conceptual elements of a theory form a coherentwhole through their belonging to a common worldview/para-digm. Elements are only included into an explanatory frame-work if they conceptually fit with the other elements of thetheory and not if they enhance the fit to the empirical data.

Overall, the differentiation between levels of abstractionand levels of generality allows us to develop a pluralistic viewon distinct kinds of explanations. The former aspect reflectsthe differences-in-kind between truth-seekers who strive forexplanations on a low level of abstraction and sense-makerswho prefer explanations of a higher level of abstraction. Thelatter aspect makes us aware that there are differences-in-de-gree on both sides: Truth-seekers develop and not only testhypotheses which focus (ideal-typically) on causal relation-ships between a single independent variable of interest andthe dependent variable in a very limited population of cases,but also models which include causal relationships among aplurality of variables, conditions or mechanisms in a largerpopulation of cases. Sense-makers, in turn, do not only de-velop and apply paradigmatic lenses that provide insights andorientation in general, but also theories that are more tailoredfor specific contexts.

of social entities and for many facets of the social world (forparadigms, there are no boundaries of the population of casesto which they refer to). They are characterized by core con-cepts and a large set of peripheral concepts, whereby both thecore concepts and the peripheral concepts remain on an ab-stract level, so that it needs a lot of interpretative work in orderto connect empirical observations to these abstract concepts.

For an understanding of the difference between the othertwo types of explanations (theories, models), it is helpful toperceive them as less radical siblings of theses and paradigms.Like (hypo-)theses, models are located on a concrete level, butthey are not as narrowly specified—they take a broader set ofcausal factors into account for explaining the outcome. Thisstands in contrast to when we want to test a hypothesis. Insuch cases, we try to control for most factors and to focus onone single independent variable and one dependent variable.A model can be a statistical model (Dependent Variable Y =a*Independent Variable X1 + b*Indep. Variable X2 + c*Indep.Variable X3 + error), a set-theoretical model (Outcome = Condi-tion A * Con. B + Con. C * Con. D), a causal chain (Precondi-tion A -> Precond. B -> Precond. C -> Outcome) or a multi-levelmodel of a causal mechanism (Causal Mechanism = SituationalMechanism * Action-formation Mechanism * TransformationalMechanism). Crucially important—and the main difference toa theory—is the fact that models are integrated on an empiricallevel. A good model has a good “fit” to the empirical data. Thevarious elements of the model do not have to be conceptuallyconsistent in the sense of belonging to a single worldview.

Such a conceptual coherence is exactly what character-izes a theory in contrast to a model. As we have laid out before,a theory is a specified paradigm in the sense that it combinesone or a few peripheral concepts and the core concept of the

7


Quality Criteria: Different Ways to SecureValidity Between Neutrality and Positionality

Validity and reliability are the most basic quality criteria forresearch procedures. These two criteria point us to the mostimportant questions for judging our process of knowledgecreation: Do we study/explain what we claim to study/explain?Can we trust the results?

Usually, truth-seekers and sense-makers interpret andspecify these criteria quite differently: In respect to validity16,the former argue that we have to make sure that we describeand explain what we want to describe/explain by relying onformal logic. Principles of formal logic are “objective” and in-dependent from the standpoint of the applicant. In conse-quence, and in line with the goal to seek the truth, the firstapproach to validity aims at “objectivity” and prescribes “neu-trality” for the researcher. Sense-makers, in contrast, arguethat we have to secure adequate descriptions and explana-tions with the help of the associative and justificatory facul-ties that languages offer. These faculties depend on, and areshaped by, the specific language (theoretical lenses, concepts)that the researcher applies. In consequence, good researchhas to justify the selected theories and concepts by reflectingon their position in the scientific discourse and in the social/political practice and their relationship to other theories andconcepts.

Nevertheless, we get a more nuanced picture of the mean-ings of validity, and of how validity is sought, if we breakdown the analytic process into four components that are nec-essary for producing an explanation. For a comprehensive,albeit differentiated understanding of validity, it makes senseto distinguish:

a. whether we are concerned with the validity of descrip-tions OR with the validity of explanations, and

b. whether we are concerned with the validity of conceptsand conclusions for the cases we studied OR whether weare concerned with the validity of concepts and claims

16 Due to space restrictions, I cannot address the criterion of reli-ability in this contribution, but see Blatter 2016.

Table 1: Different Focal Points and Different Strategies for Securing Validity

beyond the studied cases. This distinction is often labelled“internal” versus “external” validity, whereby externalvalidity overlaps with reflection on the “generalization”of findings.

When we combine the two dimensions, we get four focalpoints that indicate where we try to secure validity (see Table1). Furthermore, Table 1 highlights that for each focal point wecan detect two distinct strategies for pursuing the correspond-ing task of validation. Within each cell, the first strategy is inline with truth-seeking, the second strategy with sense-mak-ing.

I briefly scrutinize the different strategies at the four focalpoints along the following sequence of a deductive researchprocess: a) concept specification; b) concept concretization(operationalization); c) conclusions from the created data tothe relationships among the concepts for the studied cases; d)reflections on the wider generalizations of these findings.

The validity of the specification of a concept depends onhow (much) we justify the assignment of specific attributes byrelating the selected concept to other concepts within the sci-entific discourse. Sometimes this procedure is presented asinvolving two steps: First, a “systematized concept” is de-rived from a “background concept:” the selection of a specificmeaning from the universe of possible meanings is justifiedwith the specific goals or purposes of the research project.17

Second, indicators are selected which “represent the universeof content entailed in the systematized concept.”18 Truth-seek-ers adhere to such a content-centered approach to conceptformation since it allows them to treat concepts as clearly ex-ternally-delineated and internally homogeneous elements.Sense-makers, in contrast, emphasize the context-dependentmeaning of concepts and the intersubjective construction ofmeaningful concepts. Accordingly, for them, the specificationof a concept involves a reflection on the position and the roleof a concept (its linguistic signifier) in the scientific discourse.The internal characteristics of the concept are not determinedby selecting the best observable representative of a homoge-

17 Adcock and Collier 2001, 531.18 Adcock and Collier 2001, 537.

Description Explanation

Securing validity for the cases under study

b) Valid Concretization:Selecting

convergent OR alternative indicators

c) Valid Conclusion:Linking abstract relationships to concrete observations through inference OR interpretation

Securing validity beyond the cases under study

a) Valid Specification:

Justifying attributesby referring to

content OR context

d) Valid Generalization:Presuming

causal OR constitutive scope conditions

8


neous concept but by reflecting on the (categorical and con-sequential) relationships of the concept to other abstract con-cepts.19

The validity of the concretization (often called operational-ization) of a concept depends primarily on the selection of thecorrect indicators. The most important test that confirms thisis whether the scores produced by an indicator are empiricallyassociated with scores of other (direct) measures of the con-cept. This kind of validity is often called “criterion validity”and the procedure is labeled “convergent validation.”20 Never-theless, Gary Goertz has made us aware of the fact that thescores of indicators should converge only if we believe theindicators to be consequences of our concept (which wouldbe termed a latent variable in quantitative research).21 If weperceive the relationship between indicator and concept notas causal (in a narrow sense, see below) but as functional orconstitutive (as sense makers often do), different valid indica-tors of a concept need not converge because they may beunderstood as alternative options for making the concept pos-sible.

The validity of the explanations that we derive for thecases we study depends primarily on whether the conclusionsthat we draw from observations/signs to unobservable rela-tionships between our concepts are consistent from the view-point of formal logic or whether they are coherent in the sensethat they are convincingly justified (explicitly, with means withthe help of language). The former is denoted by the term infer-ence (truth-seeking), the latter by the term interpretation (sense-making).

The validity of the generalizing conclusions that we drawfrom our results depends on the adequacy of some fundamen-tal presumptions. Once again, we can detect different proce-dures for strengthening what is also called external validity.Construct validation refers to procedures which start with thepresumption that specific causal relationships exist.22 For ex-ample, the Comparable Cases Strategy depends strongly onpresumptions about other factors (beyond the factor of inter-est) that might influence the dependent variable. Already thevalidity of the conclusions for the cases under study cruciallydepends on the correct identification of alternative factors ofinfluence. The only way to control for the influence of thesefactors is to take them as criteria for case selection (the se-lected cases must show no variation in respect to these fac-tors). The same is true when it comes to draw generalizingconclusions beyond the studied cases. We can generalize theresult of our cross-case analysis only for the population ofcases that show similar values in respect to the control vari-ables as our selected and analyzed cases, because only withinthis—often very small population of cases which share thesamescope conditions—we can be sure that our factor of interest isresponsible for a causal effect and not another factor.

In order to highlight the functional equivalency, and in19 Adcock 2005.20 Adcock and Collier 2001, 537-542.21 Goertz 2006, 14-15; 62-65.22 Adcock and Collier 2001, 542-543.

line with our valuation of linguistic coherence, we might callthe principled alternative to construct validity “constructionvalidity.” Like the former, the latter depends on a presumptionof relationships between the scope conditions and our out-come of interest. Whereas the former presumes causal rela-tionships in a narrow sense, the latter presupposes that thespecific material or ideational structures which are being fo-cused on have a constitutive effect on social actions and pro-cesses. For example, those who analyze discourses or narra-tives presume that these linguistic structures have a constitu-tive impact on the interests and interactions of social/politicalactors. In a Con-Textual Analysis, the analytic focus is not ontesting this presumption but on the formation and transforma-tion of these structures. In combination with the objectivistversus conventionalist understandings of knowledge, to whichtruth seekers and sense makers respectively adhere to, we canconclude: generalizations of truth-seekers depend on the truthof their causal presumptions in respect to control variables/scope conditions whereas generalizations of sense-makers, incontrast, depend on how well the constitutive presumptionsare accepted in the scientific discourse.

Towards Pragmatic and Pluralist Ontological Stances

According to Blaikie, ontology refers to “the nature of socialreality—claims about what exists, what it looks like, what unitsmake it up and how these units interact with each other.”23

Blaikie’s definition draws him immediately into the philosophi-cal debate on whether the social reality that we study existsindependent of the human mind. Quite similarly, Brady associ-ates ontology primarily with the question of deterministic ver-sus probabilistic causality.24 A pragmatic stance, in contrast,starts by emphasizing that it is legitimate to ask all kinds ofresearch questions, despite the fact that these questions im-ply very different assumptions about the nature of social real-ity. For each research question, we should use those assump-tions about the nature of social reality that allow us to developthe most useful answer to the question. Such a pragmatic viewof ontology not only helps us to build bridges between seem-ingly incommensurable ways of conceptualizing the nature ofsocial reality, but also allows us to differentiate distinct under-standings of causality among truth-seekers and among sense-makers.

As we did for epistemology, we distinguish three aspectsthat are necessary in order to specify a pragmatic ontologicalstance: a) presumptions about the basic entities of the socialworld; b) presumptions about the relationship among theseentities; and c) understandings of causes and causation. Foreach aspect, I briefly indicate the principled alternatives, butthe main goal is to point to the existing plurality within eachprincipled approach and to highlight the fact that some under-standings of causality allow for a combination of truth seekingand sense making.

23 Blaikie 1993, 6.24 Brady 2008, 225.

9


Basic Entities of Social Reality:Materialism, Idealism and Beyond Binary Concepts

There are two principled alternatives when it comes to concep-tualizing the basic entities of social reality: materialism andidealism. Materialists assume that the most fundamental factabout society is the nature and organization of material forces,therefore focusing on traits like the biological nature of hu-mans, natural resources, geography, and forces of productionand destruction. Idealists, in contrast, assign this role to thenature and structure of human and social consciousness, there-fore concentrating on aspects like dominant forms of knowl-edge, ideas and values on the individual or collective level.25

For sense-makers, it is important to locate an explanatoryendeavor within the general discourse about these basic enti-ties of social reality. This is because sense-makers’ main goalis to provide orientation. Therefore, it is important to locateindividual explanatory efforts within broader discourses. Do-ing so helps to avoid fully idiosyncratic explanations and tocommunicate research findings across different fields and(sub)disciplines. For truth-seekers, an explicit reflection on howthe concepts they apply relate to the basic entities of socialreality is not as important, since they concentrate on gaugingthe correspondence of explanatory models to a clearly delim-ited part of social reality. In consequence, truth-seekers areoften agnostic with respect to the question of whether theconcepts they apply in their explanatory endeavors arise froma materialist or an idealist nature of social reality. Arguably,this is the case with the concept of “actor preferences.” It is acrucial explanatory factor for rationalist explanations, but itcan reflect materialist and/or idealist motivations.

Relationship Among Entities:Elementarism, Holism and Beyond Binary Approaches

There are two principled alternatives when it comes to concep-tualizing the relationship among the basic entities of the socialworld: elementarism and holism. Elementaristic approaches tohypotheses- or model-building assume that the behavior/func-tioning of the parts of a system is determined by their internalproperties and the entirety of the system is the result of theinteractions among the autonomous individual parts. Holisticapproaches to theory-building, by contrast, claim that the be-havior of the particular elements is shaped primarily by theentire system (i.e. that entireties have an ontological status oftheir own and are more than the sum of their individual parts).26

Truth-seekers have a strong affinity to an elementarist world-view since they conceptualize the relationship among the enti-ties of a social/political system as causal (in a narrow sense),whereas sense makers assume it to be constitutive. It is impor-tant to note that this does not necessarily imply that the formeronly focuses on the consequences of individual agency andthe latter on the constitutive functions of social structures. Anelementarist approach can also strive for revealing the struc-tural causes of individual or collective action. Similarly, a holist

25 Wendt 1999, 23-24.26 Esfeld 2003.

approach can strive for the constitution of social structuresthrough individual (inter-)actions.27 In recent decades,Relationalism emerged as a less holistic alternative toelementarism, highlighting the ongoing and mutual re-consti-tution of parts and holes of a social system.28

Causation: Different Ways to Define Causes and theAdequate Methods to Prove their Empirical Relevance

Truth-seekers usually adhere to an elementaristic understand-ing of causes as individual “difference makers.” A difference-making understanding of causality stipulates that causes holda general property of making some sort of difference to theireffects. On the contrary, sense-makers are more inclined tofollow those who stipulate that causality is a relational con-cept (and not a property that a factor generally inhibits), andthat the dispositional influence of a cause on the effect mani-fests itself only under concrete circumstances.29 In the follow-ing, I will show that when we leave the definitional level andlook at the methods that are applied in order to prove therelevance of causes, we find a more diverse spectrum on bothsides than such a dichotomous categorization implies.

When truth-seekers try to conceptualize causation and toidentify the effects of causes, they embrace the “experimentaltemplate” as the gold standard. As Brady has shown,30 this isthe case because the experimental template combines a spe-cific understanding with an efficient way to trace this kind ofcausation: An experiment is based on the “counterfactual un-derstanding” of causation expressed by Hume as “if the firstobject had not been, the second had never existed,”31 andallows to control two important aspects: a) the “treatment/intervention” which is necessary in order clarify the directionof the causal relation, and b) alternative factors of influencewhich are necessary in order to isolate the causal effect of thefactor of interest.

Brady identifies two further ways to understand and totrace causation that are less focused on the identification ofthe effects of one specific cause. The “regularity approach” islinked to Hume’s other definition of “a cause to be an object,followed by another, and where all the objects similar to thefirst, are followed by the objects similar to the second.”32 Itfocuses on the identification of the multiple causes of a spe-cific effect. Finally, the “mechanism approach” to causation isconcerned with temporal processes and social mechanismsthat link cause and effect on a lower level of analysis.33

The argument that these understandings of causation andtheir corresponding qualitative methods imply elementarist andrelationalist ontologies can be formulated most clearly withthe help of the terminology of necessity and sufficiency. When

27 Wendt 1999, 22-29.28 Emirbayer 1997.29 Baumgartner 2015; Anjum and Mumford 2010; Mumford and

Anjum 2011.30 Brady 2008.31 Hume 1748 according to Brady 2008, 233.32 Hume 1748 according to Brady 2008, 233.33 Brady 2008, 242-245.

10


the counterfactual approach is applied within the ComparableCases Strategy (CCS), where we draw causal inferences froman observed co-variation of independent and dependent vari-ables in otherwise similar cases, we presume that the cause isa necessary AND sufficient condition for the effect. If we allowthe cause to be sufficient but not necessary—if we find aneffect without a cause—we cannot draw any logical conclu-sion. Similarly, if we allow the cause to be necessary but notsufficient—if we find the cause without the effect—we cannotdraw any logical conclusion either.

The regularity approach broadens this understanding byaccepting conditions that are necessary but not sufficient andconditions that are sufficient but not necessary. This impliesthat individual causes are most often INUS-conditions—in-sufficient but necessary parts of a compound condition that isitself unnecessary but sufficient for an effect.34 INUS condi-tions imply that explanations do not contain single, “indepen-dent” factors that have “autonomous” causal power. Instead,causation involves a configuration of causal factors and thatthe causal power of individual factors is contingent on theexistence or a specific expression of other causal factors. Inconsequence, the explanations that we strive for with a Con-figurational Comparative Analysis (CCA), the method that isbased on the ontological assumption of configurational cau-sality, imply a relationalist ontology in contrast to theelementarist ontology that we presume when applying meth-ods based on the experimental template. Nevertheless, CCArepresents only a very limited step towards a holistic orrelationalist ontology and its corresponding understanding ofcausality, since this method focuses on the empirical identifi-cation of the co-existence of a configuration of causal condi-tions and an outcome. It cannot provide empirical informationon the mechanisms which allow the divergent components ofa causal configuration to work together in such a way thatthey are able to produce the outcome. For that we need adifferent method that corresponds to an even strongerrelationalist ontology.

That is exactly what we get, if we follow those methodolo-gists who define and conceptualize the notion of “causalmechanisms” in line with major social theorists.35 According tosuch an understanding, a causal mechanism consists of threecomplementary social mechanisms that connect causal factorson different levels of analysis: “situational mechanisms” whichlink structures to actors; “action-formation mechanisms” asthe micro-foundations of mechanism-based explanations; and“transformational mechanisms” which link actions, includingcommunicative acts, to social structures.36 The correspondingideal-typical method, Causal-Process Tracing (CPT)37 is de-signed to help find the truth in the sense of developing anexplanatory model that corresponds to the empirical reality,but it is also committed to provide meaningful explanations.

34 Brady 2008: 227.35 e.g. Coleman 1990; Esser 1993; Elster 1998; Hedstroem and

Swedberg 1998.36 Blatter and Haverland 2014, 95-97.37 As laid out in Blatter and Haverland 2014, 79-143.

This is the case not only because of the strongly relationalistunderstandings of causation that comes with the scrutinizedconceptualization of causal mechanisms, but also because itguides the researcher towards applying one of the generic“action-formation mechanisms” that social theorists have de-veloped.38 This, in turn, affirms that explanations based onCausal-Process Tracing draw on basic social theory instead ofcreating a flurry of idiosyncratic explanations and mechanisms,as it is the case with other approaches to Process Tracing.39 Afinal reason for assigning Causal-Process Tracing and its cor-responding understanding of causation a centrist place in ourtemplate is the fact that the “bathtub” model of causation40

that is often invoked to figuratively represent the scrutinizedtheoretical understanding of causal mechanism has strong af-finities to figure 3, which we develop next and which helps usto scrutinize the (implicit) understandings of causation thatsense makers adhere to.

Kurki has reminded us that we can draw on Aristotle forreflecting on the meaning of causation.41 For a productive usein the current context, we have to transfer Aristotle’s famousfour causes into the context of the social sciences and trans-late them into the language of modern social science theory.Aristotle distinguishes four kinds of causes: material, formal,efficient and final causes. According to Kurki, material andformal causes are located on the structural level of analysis,whereas efficient and final causes are located on the lowerlevel of individual or corporate agency.42 Material causes canbe understood as the distribution of material resources andnatural conditions that enable and delimit the potential rangeand direction of action; formal causes can be conceived as thenormative-cognitive structures (discourses, frames) whichdefine the possible (imaginable) range and direction of action.Within a social science context, final causes can be under-stood as purposes which mobilize and motivate action. Finally,efficient causes refer to agents that produce action throughtheir pushing and pulling activities. Figure 3 shows how thesefour causes can be located within the conceptual space thatrefers to the ontological aspects that we scrutinized in the twoforegoing subsections—a conceptual space that Wendt usedfor locating theories in the field of International Relations.43

The location of these four kinds of causes is only the firststep for delineating different understandings of causation thatare in line with the goal of providing meaningful explanationsof the social world. The second step is to show how a specificway to refer to these generic causes can be coherently alignedto a specific methodological approach.

The strongly holistic approach presumes that material orideational structures (material and formal causes) stronglydetermine the formation and motivation of individual actors. In

38 Whereby rational choice is just one option, see Hedstrom andYlikoski 2010.

39 Blatter, Haverland and van Hulst 2016b, viii.40 Coleman 1990.41 Kurki 2008.42 Kurki 2008, 219-222.43 Wendt 1999, 29-32.

11


Figure 3: Different Kinds of Causes

consequence, approaches which can be labelled as Con-Tex-tual Analysis focus on revealing the formation and transfor-mation of these underlying structures. Most forms of Dis-course Analysis conform to this ideal-type.44

A first step in order to move from holism to relationalism isto broaden the range of causes that are taken into account onan equal footing. In this case, we strive for an explanation thatnot only includes all four causes, but also shows how the fourcauses mutually constitute, transform or stabilize each other.We reach such a complete picture when we combine a closeinteraction and communication between the subjects of thestudy and the social scientists’ theoretical knowledge whichleads to a “fusion of horizons.” Therefore, we call the corre-sponding ideal-typical method “Co-writing Culture.” Ethno-graphic studies are usually very close to this ideal-type.45

A more radical turn away from holism combines internalcoherence with external difference and plurality. Such an ap-proach strives for explanations that are internally coherentinasmuch as they combine and specify different causes thatcorrespond to a consistent materialist or idealist understand-ing of the nature of social reality (scientific paradigm). Insofar,the corresponding ideal-typical method, Congruence Analy-sis (CON), is fully in line with sense-makers’ striving for co-herence. But, in contrast to other sense making understand-ings of causality, it does not presume that providing orienta-tion demands a single integrated explanation. Instead, it as-sumes the productivity of a plurality of theories defined asinternally coherent and externally distinctive explanatoryschemas. A plurality of distinct theories makes it possible todevelop multiple and divergent explanations of a phenomenon.Within such a Congruence Analysis, it is important to reflect

44 Van Hulst, Blatter, Haverland 2016, xi-xii.45 See Van Hulst, Blatter, Haverland 2016, x-xi.

on the relationship between the different theories. In principle,they can complement or compete with each other. Further-more, within a CON the divergent theories must be put to asystematic empirical test in order reveal the (divergent) levelsof congruence between each causal schema and empirical ob-servations, a feature that brings CON close to truth-seekers’striving for correspondence (between the explanation and thereal world). Sense-makers can use the results of such a Con-gruence Analysis in order to reflect on the appropriate stand-ing of the theories in the (scientific/social) discourse. Truth-seekers, in contrast, can use the results of a congruence analy-sis in order to reflect on the truthfulness of the divergent ex-planations within the specific field of the study to which theyhave been applied.

Congruence Analysis, like Causal-Process Tracing, isbased on a relationalist understanding of causality inasmuchas both methods strive for explanations which include a multi-plicity of causal factors on different levels of analysis (struc-ture and agency). In a CON, though, each explanatory schemamust be theoretically coherent, whereas CPT is open to thepossibility of revealing the working of a combination of socialmechanisms in which each single social mechanism is alignedto a distinct theory or paradigm. In other words, CPT is appliedin order to create a comprehensive explanatory model that in-cludes a plurality of causal factors, whereas CON develops aplurality of coherent theories in order to produce a compre-hensive, in the sense of being multifaceted, understanding ofan empirical phenomenon.

12


Characterizing Different Ideal-Typical Methods

Tables 2a (above) and 2b (next page) summarize the delineatedcharacteristics of the six methods in their ideal-typical form.46

Concluding Remarks on the Potential Uses of the Typology

On a practical level, the typology of methods helped us towrite the introduction of our latest textbook on qualitativemethods in Political Science.47 There, we presented a set ofprototypical research goals and questions (as done in the sec-ond section of this essay). The core message for practitio-ners is that each prototypical research question implies a spe-cific configuration of epistemological principles and ontologi-cal presumptions with its corresponding methods of data cre-ation and data analysis.

On a methodological level, though, the presumption thatis implied in such an advice might be questioned. In researchpractice and in methodological contributions, we find applica-tions and descriptions of the methods (with the same or similarlabels) which do not correspond to our ideal-typical descrip-tions. For example, many scholars and methodologists pursue

46 Blatter 2016 contains not only a third table in which the corre-sponding methods of data collection/creation and data analysis/inter-pretation are summed up, but also a more comprehensive descriptionof four out of the six ideal-typical methods.

47 Blatter, Langer and Wagemann 2017 12-17.

Qualitative Comparative Analysis (QCA)—the most commonlabel for the family of methods that correspond to our ideal-type CCA—or Process Tracing (PT) as purely truth-seekingendeavours. Moreover, interpretive scholars rarely put a plu-rality of theories to empirical scrutiny in the systematic waythat is necessary for a good Congruence Analysis. Some schol-ars will challenge the presumption that the ideal-types pre-sented here represent the best ways to search for valid andreliable answers to the formulated questions. Nevertheless,these ideal-types might challenge, in turn, the presumptionthat one must be either seeking the truth or making sense.My claim is that if adequately designed, some methods allowfor pursuing both at the same time. Furthermore, the frame-work might be helpful to clarify which distinct strategies wehave to apply in the various steps of a research project whenwe use these methods either as pure truth-seekers, as puresense-makers, or as scholars who combine truth-seeking andsense-making.48

On the most general level, the framework and typologypresented here should stimulate a debate about the meaningof “multi-method research.” Currently, the dominant under-standing equates multi-method research with a combination of

48 For first attempts to do so in respect to PT and CCA, see Blatter2016; Blatter and Huber 2017. One important insight has been thatBayesian reasoning has to be applied differently.

Table 2a: Purposes and Epistemological Principles of the Six Qualitative Methods

13


Table 2b: Ontological Presumptions of Six Qualitative Methods

divergent methods (e.g. statistical analysis and Process Trac-ing, or within-case analysis and cross-case analysis) aiming atstrengthening the internal and/or external validity of results,whereby these results represent the answer to the same re-search question. The presented typology, instead, highlightsthe fact that divergent methods complement each other in amuch more fundamental way by allowing us to pursue differ-ent research goals and to answer quite distinct kinds of re-search questions.

References

Adcock, Robert. 2005. “What Is a Concept?” Political Concepts:Committee on Concepts and Methods Working Paper Series, no. 1.

Adcock, Robert & Collier, David. 2001. “Measuring Validity: A SharedStandard for Qualitative and Quantitative Research.” AmericanPolitical Science Review vol. 95, no.3: 529-546.

Anjum, Rani Lill, Stephen Mumford. 2010. “A Powerful Theory ofCausation.” In The Metaphysics of Powers, edited by Anna Marma-doro. London: Routledge, 143-159.

Baumgartner, M. 2015. “Parsimony and Causality.” Quality & Quan-tity vol. 49: 839-856.

Blaikie, Norman. 1993. Approaches to Social Enquiry: 1st Edition.Cambridge: Polity.

Blatter, Joachim. 2016. “Aligning Methodologies to Epistemologiesand Ontologies in Qualitative Research: An Ideal-Typical App-roach.” Paper presented at the APSA Annual Conference 2016.

https://www.unilu.ch/en/faculties/faculty-of-humanities-and-social-sciences/institutes-departements-and-research-centres/department-of-political-science/research/lucerne-cluster-for-configurational-methods-luccs/

Blatter, Joachim, Till Blume. 2008. “In Search of Co-variance, CausalMechanisms or Congruence? Towards a Plural Understanding ofCase Studies” Swiss Political Science Review vol. 14, no. 2: 315-356.

Blatter, Joachim, Markus Haverland. 2014. Designing Case Studies:Explanatory Approaches in Small-n Research. Houndmills, Basings-toke, Hampshire: Palgrave Macmillan.

Blatter, Joachim, Markus Haverland, Merlijn van Hulst. 2016a. “In-troduction to Vol. I: Backgrounds, Pathways and Directions ofQualitative Methodology.” In Qualitative Research in Political Sci-ence. Los Angeles et al.: SAGE Library of Political Science; avail-able at: http://ssrn.com/abstract=2937391.

—. 2016b. “Introduction to Vol. III: Mechanisms, Temporality andWithin-Case Analysis.” In Qualitative Research in Political Sci-ence. Los Angeles et al.: SAGE Library of Political Science; avail-able at: http://ssrn.com/abstract=2937391.

Blatter, Joachim, Samuel Huber. 2017. “Parsimony or Coherence:Alternative Principles for Designing Two Coherent Types of Con-figurational Comparative Analysis.” Paper presented at the ECPRJoint Sessions, Nottingham. https://www.unilu.ch/en/faculties/fac-ulty-of-humanities-and-social-sciences/institutes-departements-and-research-centres/department-of-political-science/research/lucerne-cluster-for-configurational-methods-luccs/

Blatter, Joachim, Frank Janning, Claudius Wagemann. 2007. Qualita-

14


Hedström, P., P. Ylikoski. 2010. “Causal Mechanisms in the SocialSciences.” Annual Review of Sociology, vol. 36: 49–67.

Hedström, P., R. Swedberg. 1998. “Social Mechanisms. An Introduc-tory Essay.” In Social Mechanisms. An Analytical Approach toSocial Theory edited by P. Hedstroem and R. Swedberg. Cam-bridge: Cambridge University Press: 1–31.

Kurki, Milja. 2008. Causation in international relations: reclaimingcausal analysis. Cambridge: Cambridge University Press.

Mumford, Stephen, Rani Lill Anjum. 2011. Getting Causes fromPowers. Oxford: Oxford University Press.

Sartori, Giovanni. 1970. “Concept Misformation in ComparativePolitics.” American Political Science Review, vol. 64, no. 4: 1033–1053.

Schimmelfennig, Frank. 2003. The EU, NATO and the Integration ofEurope. Rules and Rhetoric. Cambridge: Cambridge UniversityPress.

Van Hulst, Merlijn, Joachim Blatter, Markus Haverland. 2016. “In-troduction to Vol. IV: Interpretive and Constructivist Approaches”In Qualitative Research in Political Science. Los Angeles et al.:SAGE Library of Political Science; available at: http://ssrn.com/abstract=2937391

Wendt, Alexander. 1999. Social Theory of International Relations.Cambridge: Cambridge University Press.

Yanow, Dvora. 2003. “Interpretative Empirical Political Science: Whatmakes this not a subfield of qualitative methods.” Qualitative Meth-ods: Newsletter of the APSA Organized Section on Qualitative Meth-ods, vol. 1, no. 2: 9-13.

tive Politikanalyse: Eine Einführung in Methoden und Forschung-sansätze. Wiesbaden: VS Verlag.

Blatter, Joachim, Phil Langer, Claudius Wagemann. 2017. QualitativeMethoden in der Politikwissenschaft. Wiesbaden: Springer VS.

Brady, Henry E. 2008. “Causation and Explanation in Social Sci-ence.” In The Oxford Handbook of Political Methodology edited byJanet M Box-Steffensmeier., Henry E. Brady, and David Collier.Oxford: Oxford University Press: 217-249.

Collier, David, James E. Mahon. 1993. “Conceptual ‘Stretching’ Re-visited. Alternative Views of Categories in Comparative Analysis.”American Political Science Review, vol. LXXXVII, no. 4: 845–855.

Elster, Jon. 1998. “A Plea for Mechanisms.” In: Social Mechanisms.An Analytical Approach to Social Theory edited by P. Hedstroemand R. Swedberg. Cambridge: Cambridge University Press: 45–73.

Elster, Jon. 2007. Explaining Social Behavior: More Nuts and Boltsfor the Social Sciences. Cambridge: Cambridge University Press.

Emirbayer, Mustafa. 1997. “Manifesto for a Relational Sociology.”The American Journal of Sociology, vol. 103: 281-317.

Esser, H. 1993. Soziologie. Allgemeine Grundlagen. Frankfurt a.M.:Campus.

Gerring, John. 2012. “Mere Description.” British Journal of PoliticalScience, vol. 42, no. 4: 721–746.

Goertz, Gary. 2006. Social science concepts: A user’s guide. Princeton:Princeton University Press.

Goertz, Gary, James Mahoney. 2012. A tale of two cultures. Qualita-tive and quantitative research in the social sciences. Princeton:Princeton University Press.

15


Opportunities and Obstacles inDistributed or Crowdsourced Coding

Brian R. UrlacherUniversity of North Dakota

Scholars interested in translating large quantities of qualita-tive information into structured datasets have an expandingrange of options, each with different strengths and weaknessesto be balanced. Whether a researcher opts for an inductive1 ordeductive strategy2 for creating conceptual frameworks, in theend, the task of merging concepts with actual data is an “inter-pretive exercise.”3 For qualitative data—be it transcripts, fieldnotes, or texts—this exercise has traditionally been carried outby researchers, working alone or in small teams. Yet, relying onhuman coders can be slow and expensive and can raise ques-tions about inter-coder reliability and validity.4

The rise of computerized techniques for coding thereforehas tremendous appeal for researchers looking to leveragebreakthroughs in natural language processing and qualitativeanalysis software. As another way to improve speed and over-come some of the limitations of traditional coding methods,researchers are drawn to a middle ground between small teamsof human coders and automated systems: The emergence of“micro-task markets,” such as Amazon’s Mechanical Turk plat-form, promise to reduce the cost and the time associated withhuman coding while avoiding the technical challenges of natu-ral language processing.

While Mechanical Turk and other strategies for crowd-sourcing information collection and processing are beginningto revolutionize the social sciences, there are potential prob-lems associated with distributing research tasks. These prob-lems are just starting to come into focus and are not alwayswell understood. My own experience with “distributed” orcrowdsourced coding of data suggests that micro-task mar-kets like Mechanical Turk may pose reliability and validityissues that are more complex and less well understood thanthose associated with the work of more traditional researchteams. This essay begins with an introduction of micro-taskmarkets, specifically the Mechanical Turk system. I then de-scribe an inter-coder reliability and validity assessment I con-ducted using that platform. I conclude with recommendationsfor improving reliability and validity when using a distributedcoding approach and with a discussion of the strengths andlimitations of this approach for coding qualitative data.

Brian R. Urlacher is Associate Professor of Political Science andDirector of International Programs, CoBPA, at the University ofNorth Dakota. He is online at [email protected] andhttps://und.edu/directory/brian.urlacher.

1 Glaser and Straus 1967.2 Goertz 2006.3 Druckman 2005, 258.4 Schrodt and Van Brackle 2013.

Introducing Mechanical Turk

Amazon’s Mechanical Turk platform is a market place thattrades in Human Intelligence Tasks (HITs), which the com-pany defines as tasks “that human beings can do much moreeffectively than computers, such as identifying objects in aphoto or video, performing data de-duplication, transcribingaudio recordings, or researching data details.”5 On the sur-face, the platform is a way to execute complex computationaltasks, but behind the scenes human workers provide the intel-lectual power to make the system operate.6

Since its launch in 2005, Amazon’s Mechanical Turk plat-form has emerged as an invaluable research tool for scholars inmany academic disciplines, to the extent that—over the lastten years—it has become a standard tool of behavioral psy-chologists and is making inroads into other social science dis-ciplines. Mason and Suri provide a review of the uses of Me-chanical Turk by researchers in the behavioral sciences, not-ing the use of the system for surveys, experiments with ran-dom assignment, and experiments or simulations involving in-teractions between study participants.7 For researchers, theappeal of the Mechanical Turk system is the flexible employ-ment of workers to accomplish a range of tedious tasks, quicklyand cheaply.

Most of the scholarly applications of Mechanical Turkuse the platform to administer surveys or to carry out experi-ments. There have, however, been limited efforts to utilizeMechanical Turk in data collection and coding. Preliminarystudies suggest that this research application may be viable.For example, Alonso and Mizzaro found the document-codingby experts and workers hired through Mechanical Turk to beof comparable quality.8 In another study, Kittur et al. usedMechanical Turk to code Wikipedia entries.9 The MechanicalTurk worker coding was compared to coding produced byWikipedia administrators. Kittur et al. found that MechanicalTurk could, in some instances, be used to produce reliableresults, but they caution that the structure of the HIT was an

5 Mechanical Turk FAQ 2017.6 The Mechanical Turk system is a twenty-first century interpre-

tation of a famous eighteenth-century hoax perpetrated by Wolfgangvon Kempelen. Von Kempelen’s con involved disguising a chess mas-ter within a “mechanical device” named the Mechanical Turk. VonKempelen convinced the luminaries of Europe that he had invented amachine that had mastered the game of chess (Levitt 2006). Amazon’sMechanical Turk platform operates on a similar principle (albeitwithout von Kempelen’s deceit).

7 Mason and Suri 2012.8 Alonso and Mizzaro 2009.9 Kittur et al. 2008.


16


important factor driving the accuracy of the coding.These initial studies suggest that well designed coding

tasks can be effectively crowdsourced. However—in my ownexperience working with a traditional team of coders on re-search projects, and as a worker on Mechanical Turk—there isoften value in having coders with a base of knowledge on atopic. In a distributed work environment where the identity ofworkers is unknown, such a foundation of background knowl-edge cannot be assumed. Furthermore, coders working in moretraditional teams have the opportunity to learn and refine theircoding over multiple batches of material and through dialoguewith other coders. This kind of group and individual normingis almost certain to be absent in the context of distributedcoding where coders likely do not discuss their thought pro-cesses and may only code a few cases before moving on toother tasks.

Evaluating Distributed Coding

To better understand how to get the most accurate resultsfrom distributed coding, I carried out an inter-coder reliabilityanalysis and a validity test on three different groups of Me-chanical Turk workers. These groups included workers fromthe general Mechanical Turk workforce, from a subset of highlyproficient and experienced users identified by Amazon as “mas-ter” workers, and a third group of non-master workers whocompleted a brief online training module. The training moduleconsisted of approximately 600 words of background informa-tion on negotiations, conflict termination, comprehensive andpartial agreements, and third parties. This information was fol-lowed by six multiple choice or true false questions. Success-ful answers to the comprehension quiz were required beforecoders could access their first case. Coders were paid a $0.50bonus for completion of the training (twice the rate paid forindividual HITs).10

For this analysis, I created a seventy-case test datasetcomprised of qualitative descriptions of negotiations to endinterstate wars. The descriptions were culled from the UppsalaConflict Data Program’s (UCDP) conflict encyclopedia.11 Thequalitative descriptions ranged from 23 to 551 words with anaverage length of 140.6 words and provided a general over-view of negotiation activities occurring in a given conflict-year. After reviewing the descriptions, I identified three vari-ables relating to the onset and termination of negotiations thatI believed could be extracted from the texts. These three vari-ables were structured to provide greater precision than is in-

10 It is common for Mechanical Turk workers to be asked to com-plete training or certification processes before being allowed access toHITs. In my own experience as a Mechanical Turk worker, I foundthat some certifications were very labor intensive to complete, andthat compensation was never offered for work done to complete acertification process.

11 Earlier versions of the UCDP conflict encyclopedia allowed forusers to generate custom reports. I selected interstate conflicts withnegotiations occurring in a given year. I then requested qualitativedescriptions of these negotiations between 1975 and 2012. Therewere 70 descriptions of negotiations available for analysis.

cluded in the standard UCDP coding.The HIT involved coding three variables related to the

start of a new negotiation process, the way that negotiationsended, and if a partial agreement was reached in a given year.For the new negotiation variable three responses were pos-sible (New Negotiations, Negotiations Continued from Earlier,and No Negotiations Occurred). For the termination of nego-tiation variable four responses were possible (Negotiation Con-tinued, Negotiation Collapsed, Negotiation Ended with a Par-tial Agreement, Negotiation Ended with a Full Agreement). Fi-nally, a Yes or No question was posed to record if partial agree-ments were reached as part of a continuing processes. Foreach of these three variables, brief descriptions were providedto coders to explain each category. Selecting some responsesprompted coders to also extract dates from the text.

In assessing inter-coder reliability and validity, I used theproportion-of-agreement statistic, which is calculated by con-verting categorical variables into an array of dichotomous vari-ables (coded as 0 or 1) representing each category. The pro-portion of agreement statistic is the percent of cases in whichtwo coders (or in this case coding processes) produced iden-tical results (both 0 or both 1) for a given category. Fleiss et al.describe this procedure and note that the proportion of agree-ment measure is “the simplest and most frequently used indexof agreement.”12

The 70 descriptions of interstate negotiations drawn fromthe UCDP’s conflict encyclopedia were each coded four times(including my own coding) for a total of 280 observations.13 Atotal of 73 different Mechanical Turk workers participated incoding cases. This included 10 master workers, 17 non-masterworkers who did not receive training, and 46 non-master work-ers who completed the training module.14 While this is perhapsfewer cases and coders than might be desired in terms of sta-tistical power, the results in this limited study served the pur-pose of helping me to assess the viability of re-coding theUCDP’s conflict encyclopedia using Mechanical Turk.

Assessing Reliability

A comparison of the various sets of workers on MechanicalTurk showed disappointing levels of consistency in how thesame data was coded. Tables 1A-C provide the percentage-of-agreement scores for the three variables coded by MechanicalTurk workers. While the proportions are generally greater than0.5, simple randomness would meet that level of agreement for

12 Fleiss et al. 2003.13 I include my own coding here not as a “correct” measure but

rather as a way to evaluate how different pools of researchers per-formed compared to my own subjective coding of the UCDP descrip-tions.

14 Master workers completed an average of 7 HITs each. Non-master workers who were not asked to complete training, completed4.11 tasks on average. The average number completed for workerswho were asked to complete the training was 2 HITs, and more than70% completed only 1 HIT. I suspect that the one-time bonus forcompleting training may have motivated a number of workers tocomplete one hit even though they may have been only marginallyinterested in the actual task.

17


a dichotomous variable. For a three-category variable, randomchance would produce a proportion-of-agreement score ofabout 0.55. Similarly, a four-category variable would have abaseline expected proportion of agreement of 0.625. For com-parison, in projects, in which I have participated and whichhave used traditional teams of human coders, a threshold of0.9 (indicating 90% agreement) has often been used as a mini-mum acceptable level for intercoder reliability.

The first variable (see Table 1A), which related to the on-set of negotiations, had proportion-of-agreement scores dis-tributed fairly evenly between the minimum of 50%, which ac-tually falls below the baseline expected by chance, and themore respectable maximum of 91%. The second variable (seeTable 1B), which refines how negotiations ended, showed asimilar distribution. On the low-end, proportion-of-agreementscores approached what would be expected from randomness.On high end, agreement levels reached 95.7%—although mostcategories fell closer to the average score of 79%. Table 1Cpresents the results for the variable asking about negotiationsthat continued after parties reached a partial settlement. Thisdichotomous variable produced the lowest proportion of agree-ment scores with an average of only 62.4%.

Two issues seem to arise from these tables. First, the lowinter-coder reliability scores suggest either a high level of errorin the coding processes or real and meaningful differences inthe performance of the various pools of Mechanical Turk work-ers. A second issue relates to how my own coding tracked with

Table 1A: Proportion of Agreement by Coding Processes

that of other workers. While master workers generally pro-duced results closest to my own (80.9% on average), codersthat completed the training module seemed to be slightly moreout of sync with my own coding (75.6 percent on average).This difference is statistically significant and alarming in thatthe intent of the training was to produce results that would bemore comparable to what I would have produced on my own.15

Recommendations

It is difficult to know how much of the proportion-of-agree-ment results are attributable to the distributed coding approach.Intercoder reliability can be affected by the specific individu-als that participated in the coding and by the underlying ambi-guity in this project, including the coding protocols and data.Without a comparison to a broader set of human coders oper-ating in more traditional teams, it is difficult to know how muchthe limited learning and coder expertise are responsible for theresults.

For sources of error external to a specific project, a rea-sonable strategy for improving the quality of coded data wouldbe to code each case multiple times and use the modal re-sponse. While drawing multiple perspectives may help to evenout idiosyncratic error and reduce overall measurement error,there is a practical issue to be considered. Although coding

15 All statistical significance tests presented in this paper are set atthe .05 level. Standard errors are calculated using the formula forstandard error of a proportion.

General Workers Master Workers Researcher

General Workers (No Training) (With Training)

New Negotiations 57.1% 68.6% 61.4%

Continued 74.3% 74.3% 67.1%

No Negotiations 87.1% 88.6% 88.6%

General Workers (With Training)

New Negotiations -------- 54.3% 50.0%

Continued -------- 74.3% 84.3%

No Negotiations -------- 87.1% 90.0%

Master Workers

New Negotiations

-------- 75.7%

Continued

-------- 81.4%

No Negotiations

-------- 91.4%

18


Table 1B: Proportion of Agreement by Coding Processes

each case three or five or ten times may improve results (notethat I have not tested this in the context of Mechanical Turk),this solution increases both the costs associated with codingeach individual case and requires an additional layer of pro-cessing by a researcher to resolve disagreements betweencoders. It should also be noted that this approach only ad-dresses the problem of random error. The repetition solutionrests on the assumption that coders are on average able togenerate consistently valid results. It is to this assumptionthat I now turn.

Assessing Validity

While this project initially sought to produce more fine-grainedcoding of negotiations than what was available through theUCDP’s coding, two of the categories related to the termina-tion of negotiations run roughly parallel to the UCDP’s owncoding. Thus, as a validity check, I compared the various poolsof coders against information from the UCDP Peace Agree-ment Dataset.16 Table 2 presents the proportion-of-agreementstatistics for each of the different codings along with the num-ber of false positives and false negatives. In this case, a false

16 Harbom et al. 2006; Högbladh 2011.

positive is where a coder recorded an agreement that was notreflected in the UCDP’s Peace Agreement Dataset. On the otherhand, a false negative occurs when a coder fails to record anagreement that is documented in the Peace Agreement Dataset.

My own coding of the UCDP descriptions aligns with thePeace Agreement Dataset a little more than 90% of the time.Somewhat unexpectedly, given the findings above, trainedcoders are statistically comparable in their performance, withproportion-of-agreement scores of 88.5% and 91.4%. This wasalso statistically better than the scores for both master work-ers and untrained workers, which hovered around 80%. Thedifferences across groups seems to be driven by the reductionin false positives. That is, trained coders were much less likelyto incorrectly record a full or partial agreement based on thedescriptions than were master workers or untrained workers.

On the whole, the validity check is somewhat encourag-ing. In general, Mechanical Turk coders produced results thatwere not strikingly different from what the UCDP team pro-duced, although the results would not be strong enough tosatisfy most researchers. Still, two points are worth noting.First, the brief descriptions of negotiations provided to Me-chanical Turk workers are devoid of larger contextual informa-



Continued 62.8% 75.7% 71.4%

Collapse 80.0% 81.4% 77.1%

End with Partial Agreement 84.2% 80.0% 85.7%

Ended with Successful Negotiations 77.1% 81.4% 81.4%


Continued -------- 64.3% 62.3%

Collapse -------- 84.3% 77.1%

End with Partial Agreement -------- 87.1% 95.7%

Ended with Successful Negotiations -------- 75.7% 84.3%

Master Workers

Continued -------- 75.7%

Collapse -------- 84.3%

End with Partial Agreement -------- 88.6%

Ended with Successful Negotiations -------- 82.8%

19


Table 1C: Proportion of Agreement by Coding Processes

Table 2: Evaluation of Coding Processes Relative to UCDP Coding



Partial Agreement (with negotiations

continuing) 55.7% 61.4% 71.4%



continuing) -------- 57.1% 61.4%

Master Workers


continuing)

-------- 67.1%

General Workers (No Training) Proportion of Agreement False Negatives False Positives

UCDP Full Agreement 80.0% 1 13

UCDP Partial Agreement 81.4% 3 10


UCDP Full Agreement 88.5% 2 6


Master Workers

Full Agreement 81.4% 1 12


Researcher

Full Agreement 92.80% 0 4


20


At least part of what researchers will need to considerrelates to the nature of the task and the nature of the project.Some tasks are relatively intuitive for humans but may be moredifficult to automate. Where these tasks are simple and repeti-tive, there may be real value in pursuing crowdsourcing. Forsmaller projects, however, the technical barriers of setting up acrowdsourcing platform or machine learning algorithms maybe a prohibitive barrier. Yet, economies of scale make settingup a micro-task market negligible in large projects.

In spite of these limitations, crowdsourced or distributedcoding offers one clear advantage over the traditional approachto human coding. A distributed network of coders is more likelyto be representative across age, gender, race, and the North-South divide. It therefore allows scholars to avoid the culturalor gender bias that can arise from the lack of diversity in teamsof coders.19 Buhrmester et al. observe that Mechanical Turkworkers are not noticeably different than the general internetpopulation and are therefore more representative than what isoften found in studies that draw heavily from university com-munities.20 Furthermore, the Mechanical Turk pool of workersdraws globally, although as Ipeirotis documents, this globalworkforce is not globally representative as the vast majority ofworkers are based in either the United States or India.21

In short, there are potential advantages and opportunitiesfor researchers seeking to draw on distributed networks ofhuman coders. The speed, cost competitiveness, and repre-sentativeness of micro-task marketplaces are genuine advan-tages. Yet, my own investigation suggests that researchersshould proceed carefully, consider the appropriateness of atask, and scrutinize the results of distributed coding beforefully committing to crowdsourced data processing.

References

Alker, Hayward R. 1988. “Emancipatory Empiricism: Toward theRenewal of Empirical Peace Research.” In Peace Research: Achieve-ments and Challenges, edited by Peter Wallensteen. Boulder, CO:Westview Press: 219-241.

Alonso, Omar and Stefano Mizzaro. 2012. “Using Crowdsourcingfor TREC Relevance Assessment.” Journal Information Process-ing and Management vol. 48, no. 6: 1053-1066.

Buhrmester, Michael, Tracy Kwang, and Samuel D. Gosling. 2011.“Amazon’s Mechanical Turk: A New Source of Inexpensive, YetHigh-Quality, Data?” Perspectives on Psychological Science vol.6, no. 1: 3-5.

Druckman, Daniel. 2005. Doing Research: Methods of Inquiry forConflict Analysis. Thousand Oaks, CA: Sage.

Fleiss, Joseph L., Bruce Levin and Myunghee Cho Paik. 2003. Statis-tical Methods for Rates and Proportions. 3rd edition. Hoboken, NJ:John Wiley & Sons.

Glaser, Barney G. and Anselm L. Strauss. 1967. The Discovery ofGrounded Theory: Strategies for Qualitative Research. NewBrunswick, NJ: AldineTransaction.

Goertz, Gary. 2006. Social Science Concepts: A User’s Guide. Prince-ton, NJ: Princeton University Press.19 Alker 1998, 224.20 Buhrmester et al. 2011, 4.21 Ipeirotis 2010.

tion so it is difficult to know if a more accurate coding is rea-sonable to expect given the descriptions. Second, there seemsto be a strong bias on the part of Mechanical Turk workerstoward false positives. That is, they seemed more likely toreport that negotiations occurred, when there was little evi-dence to support this. This is most evident with untrainedMechanical Turk workers, who coded approximately 16.4% ofcases as false positives.

Recommendations

A striking portion of the error in the validity test resulted fromfalse positives. There is a case to be made that this may be astructural bias built into the Mechanical Turk platform. ManyHITs have tests embedded within to ensure that workers arecompleting tasks dutifully. There are considerable informationand power asymmetries between workers and those hiring fortasks.17 Thus, workers may be conditioned to look for “cor-rect” answers in every HIT and may have an aversion to cod-ing null results.

Fortunately, the training module seems to have effectivelycountered this tendency. Yet, providing training may have in-advertently limited some of the learning that occurs in codingmultiple cases. While master and non-master workers aver-aged around 3 minutes per HIT, it generally took workers 9minutes to complete the training and their first task. Conse-quently, the vast majority of workers who completed the train-ing finished only one HIT. In other words, this relatively brieftraining may have consumed the time workers were willing todedicate to the task, thus decreasing opportunities to accu-mulate experience through practice.

Given my experience with attempting to code qualitativedata through Mechanical Turk, I have come to the conclusionthat distributed coding may be effective when tasks arestraightforward and homogenous, but the Mechanical Turkplatform may also create incentives that bias results. This biascan potentially be countered through training but as the reli-ability analysis shows, researchers should proceed with cau-tion. Thus, McClelland’s cautionary advice in using humancoded event data seems to also hold for coding generatedthrough Mechanical Turk: “let the user beware.”18

Concluding Thoughts

The results of a single study of inter-coder reliability and va-lidity should not serve to legitimize or delegitimize an entireapproach to data collection and processing, but the findingspresented here do provide an opportunity to pause and re-flect. The distributed model of coding is not well understoodnor documented, and researchers should proceed with cau-tion. There are multiple practical and ethical considerationswhen thinking about using traditional approaches to humancoding, a crowdsourcing model, or a fully computational ap-proach.

17 In my own experience as a Mechanical Turk worker, tasks oftentake much longer than advertised and payment can be withheld with-out explanation or practical recourse.

18 McClelland 1983.

21


Harbom, Lotta, Stina Högbladh and Peter Wallensteen. 2006. “ArmedConflict and Peace Agreements.” Journal of Peace Research vol.43, no. 5: 617-631.

Högbladh, Stina, 2011. “Peace Agreements 1975-2011 - Updating theUCDP Peace Agreement Dataset.” In States in Armed Conflict2011, edited by Therése Pettersson & Lotta Themnér. UppsalaUniversity: Department of Peace and Conflict Research Report99.

Ipeirotis, Panos. 2010. “Demographics of Mechanical Turk.” NYUCenter for Digital Economy Research Working Paper CeDER-10-01.

Kittur, Aniket, Ed H. Chi, and Bongwon Suh. 2008. “CrowdsourcingUser Studies with Mechanical Turk.” In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Comput-

ing Systems, edited by M. Czerwinski and Al Lund. New York:ACM. 453-456.

Levitt, Gerald M. 2006. The Turk, Chess Automation. Jefferson NC:McFarland & Company Incorporated.

Mason, Winter and Siddarth Suri. 2012. “Conducting BehavioralResearch on Amazon’s Mechanical Turk.” Behavioral Researchvol. 44: 1-23.

Mechanical Turk FAQ 2017. https://www.mturk.com/mturk/help?helpPage=overview Accessed: November 9, 2017.

McClelland, C. A. 1983. “Let the User Beware.” International Stud-ies Quarterly vol. 27, no. 2: 169-177.

Schrodt, Philip A., and David Van Brackle. 2013. “Automated Cod-ing of Political Event Data.” Handbook of Computational App-oaches to Counterterrorism. New York, NY: Springer: 23-49.

22


Letting Easton Be Easton—

An Interpretivist

William J. KelleherIndependent Scholar

Surveying the field in mid-20th century, David Easton observedthat political science primarily lacks “a conceptual frameworkor systematic theory to give meaning, coherence, and direc-tion to ongoing research.”1 In his most prominent work, ThePolitical System, he writes, “In political science there has beenlittle deliberate effort to formulate a conceptual framework forthe whole field.”2 The problem he undertakes, then, is that of“defining political science,” and thus to “define the core of thefield.”3 This was Easton’s heroic mission.

After that publication, Easton enjoyed phenomenal suc-cess within the profession. Elected American Political ScienceAssociation president in 1969, studies conducted in the 1970sand 1980s showed that his peers ranked him fourth among themost prominent political scientists from 1945 to 1960, and sec-ond most prominent in the period 1960-1970. He ranked sev-enth among the twenty most cited political scientists in thedecade 1970-79.4

Since its publication in 1953, Easton’s “conceptual frame-work” for political science has conventionally been depictedas a seminal work of “positivistic behavioralism.” He has beencast as a leader in the study of political behavior with an un-derstanding of science as an effort to produce empirical hy-potheses that can be falsified or verified. For example, in a 2006survey of approaches to the study of politics, Mark Bevir clas-sifies Easton as one of the foremost “behavioralists,” and anadvocate of behavioralism as an “expression of the turn to-ward positivism.” Bevir then defines the “positivist concept ofscience” as seeking “universal, deductive, predictive, and veri-fiable theory.”5 Bevir reflects a well-established categorizationof Easton as a “positivist,” and the equation of positivism andbehavioralism.

Ironically, despite this wide spread characterization ofEaston’s point of view, a question remains as to how well thisclassic work, The Political System, has been understood. His-torian of the political science profession, John Gunnell, hasobserved that this book “has now become somewhat a pris-oner of the perspectives that have subsequently informed itsinterpretive history.”6 He questions whether the book has ever

William J. Kelleher is an independent scholar. He can be reached [email protected] and found online at https://independent.academia.edu/WilliamJKelleherPhD and at http://ssrn.com/author=1053589.

1 Easton 1953, 1971, 52.2 Easton 1953, 1971, 65.3 Easton 1953, 1971, 96.4 Lynn 1983; Robey 1982; Roettger 1978.5 Bevir 2006, quote at page 18 in online version.6 Gunnell 2013, 198. Besides Bevir, there are many other instances

supporting Gunnell’s observation. For instance, although Easton re-

been truly understood, and notes that, even “after two genera-tions,” the “actual text has never been fully and carefully ana-lyzed.”7 Therefore, “it is worthwhile reexamining in detail thestructure of the argument and determining its place in Easton’spath to the formulation of a systems analysis of political life.”8

He adds that given the passage of time since the book’s firstpublication in 1953, it “is in some respects possible today tounderstand the book better than it was understood in the 1950s,and it may be possible to understand the author better than heunderstood himself.”9

In this paper, I will undertake the same “reexamining”project as Gunnell, but from an entirely different perspective.This exegeses of the systematic political theory Easton intro-duces in The Political System focuses not on its history, buton its singularity; that is, its creative origins, intentionality,and interconnectedness.10 Taking this fresh look at the bookhas a surprising result. Freeing Easton from, as Gunnell hasobserved, the prison of “the perspectives that have subse-quently informed its interpretive history,” this essay will show,perhaps for the first time, that The Political System actuallymakes a case for a thoroughly interpretive, as opposed to apositivistic, political science.

The Need for a Systematic Theory of Politics

While Easton advocated developing the study of politics as ascience, his understanding of science is different from thatembodied in contemporary positivism. He understands the con-cept broadly as an organized frame of reference with an empiri-

jected “equilibrium theory” for, among other things, being too mecha-nistic to be a part of his systematic political theory, he is oftenassociated with it. For example, in his history of the Caucus for aNew Political Science, Clyde Barrow states that the “central focus”of Easton’s approach is “to understand how ‘decision-making’ (i.e.,authoritative allocations of values) facilitate the equilibrium of theoverall social system,” at page 217 (emp. ad.). Barrow 2008, pages216-217. See Kelleher 2017 for an extensive critique of Barrow’s mis-representation of Easton.

A more recent instance of conflating Easton with other writersunder the rubrics “behavioral political science” and “functionalism”is by Mary Hawkesworth who writes that for Easton, as for theseschools of thought, the political system is seen as functioning “so asto maintain homeostatic equilibrium.” See Hawkesworth, “Contend-ing Conceptions of Science and Politics,” page 46, in Yanow 2014.But we will show that, as to Easton, nothing could be further from thetruth.

7 Gunnell 2013, 198.8 Gunnell 2013, 198.9 Gunnell 2013, 205.10 I appreciate the encouragement Professor Gunnell has given me

in our private communications.


23


cal orientation, a defined subject matter (in this case, “poli-tics”), and a systematic approach to seeking causal knowl-edge about the subject. Referring to the literature of his time,Easton regretted what he saw as the “flight from scientificreason, especially in the area of political knowledge.”11 In-deed, Easton’s theory of the “political knowledge” which po-litical scientists have, and how it is to be verified, may be someof his least appreciated and most unexpected contributions topolitical science; therefore, they will be clarified in this paper.

Easton envisions science as the apex of human reason.12

He notes that he is far from the first to search for a science ofpolitics, but he is intent upon pursuing that goal. To have ascience of politics, Easton argues, political scientists must beginwith a clear idea of what “politics” is. “Where does the politi-cal begin and end, and how is it distinguishable if at all, fromother kinds of data that we call economic, sociological, psy-chological, and so on.”13 Such a theoretical demarcation isneeded “to identify the significant variables necessary to ex-plain political activities and to show their interrelations.”14

Hyperfactualism

Easton found that one of the consequences for political sci-ence of lacking systematic theory was the pervasive conditionhe called “hyperfactualism.” Rather dramatically he declaredthat in his day, the “American political scientist is born free butis everywhere in chains, tied to a hyperfactual past.”15

Hyperfactualism makes the assumption that “science”consists exclusively of objective and detached fact gather-ing.16 But, in Easton’s view, this assumption about the natureof science is way off the mark. He offers a different under-standing of science; that is, as an approach by researcherswho act not with a blank slate of a mind, but who are infusedwith and guided by a range of prior interests. In his view, the“aspect of the event selected for description as the facts aboutit, is determined by the prior interest of the observer; the selec-tion is made in the light of a frame of reference that fixes theorder and relevance of the facts.”17 This prior set of interests,

11 Easton 1953, 1971, 6.12 See The Political System, Chapter Three, pp 64-89. Throughout

the book he distinguished this understanding from the narrow posi-tivistic sense of 19th century mechanics, as we discuss in this paper.He also distinguishes between pure science (seeking knowledge initself), and applied science (knowledge used to solve problems in thetheory of the political system). See ibid, pages 87-89, and Kelleher2017.

13 Easton 1953, 1971, 92.14 Easton 1953, 1971, 93, 98.15 Easton 1953, 1971, 47. The phrase “born free but is everywhere

in chains” is borrowed from Rousseau’s Social Contract.16 Easton 1953, 1971, 66. Easton sees his views as part of the

“modern psychology of perception,” and mocks the “pure empiricisttradition” of “objective observation” as a theory of an “immaculateperception.” In “Easton Responds to Classical Critics,” online athttps://isistatic.org/journal-archive/pr/03_01/easton.pdf, ND (butearly 1970s), page 278f. This places him even further from positivis-tic notions of “scientific objectivity.”

17 Easton 1953, 1971, 53.

beliefs, and values serve as an interpretive framework thatguides research and gives particular facts, out of the whole,their special significance. Without such “theoretical assump-tions…it would be impossible for them to select meaningfulfacts.”18 Thus, no matter the reasons for the pursuit of re-search, it is “scientific” if it is conducted with a systematicapproach and from within an orderly “frame of reference.”19

His concept of science also includes an understanding of veri-fying or validating the fruits of research, which will be dis-cussed further in a moment. And, as we will see, these prin-ciples inform Easton’s theory of political knowledge.

Below the Level of Theory

But, without a systematic theory to guide their selections oftopics and related facts, how were political science research-ers in the past able to write anything about politics? Indeed,how could separate fields of social science, such as politicalscience and psychology, even develop without theoreticalguidance?

In Easton’s interpretation, it was a pioneer’s subjectiveunderstanding of “the intrinsic logic” of each field in the socialsciences that lead to their development as separate areas ofresearch.20 From his reading of history, Easton had learnedthat “sciences do not arise capriciously.”21 “Some process ofselection does take place.”22 Easton notes that specializationsin the social sciences have not developed as mere “historicalaccident,” but by “a rationale of their own.” Out of the com-plexity of “social life,” leading writers selected clusters of in-teractions for study.23 Original thinkers in various fields linkedtogether sets of recurring issues, problems, challenges, obser-vations, and questions, often led by an intuitive sense that theissues were related.24 Writers who shared overlapping intellec-tual passions helped develop specialized fields of social sci-ence.

For instance, psychology was pulled together, or sociallyconstructed, by writers who shared the intellectual passion tounderstand individual behavior.25 By the subject matter theyput together they distinguished their field from other socialsciences, such as economics, political science, or geography.Easton points out that those writers intuited a kind of “logic ofthe situation.” Those who made careful studies of particularaspects of society understood and shared certain “key ques-tions” which helped to define their professions. Unarticulated

18 Easton 1953, 1971, 53.19 Easton 1953, 1971, 53. Cf. Hawkesworth’s discussion of “pre-

supposition theory,” in Yanow et al 2014, 27-49.20 Easton 1953, 1971, 100.21 Easton 1953, 1971, 147.22 Easton 1953, 1971, 99.23 Easton 1953, 1971, 103.24 Easton 1953, 1971, 104.25 Easton 1953, 1971, 104. His references to “intuition” and “feel”

suggest that a “field’s intrinsic logic or rationale” is a metaphoricalpersonification of an abstraction, and that what he is actually refer-ring to is the intellectual interests and aptitudes of individual pioneer-ing researchers. In this sense, Easton was close to Polanyi’s concep-tion of “Intellectual passions.” See Polanyi 1958.

24


tacit factors, such as a personal sense of the subject and intu-ition, served as research guides in lieu of a defined conceptualframework. Thus, leading figures paved the way, followingtheir own inner sense of the subject matter.

For Easton, political science in particular, has risen largelyout of the personal interest of pioneering researchers in under-standing how policy for society is made and implemented. Inthe absence of a systematic theory defining the field of politi-cal science, “most students of political life do feel quite in-stinctively that research into the political aspects of life doesdiffer from inquiry into any other [aspect of social life].”26 “Quiteinstinctively” political science researchers select some kindsof facts rather than others.27 Among Easton’s most admiredpioneering political scientists is V. O. Key, in part because hehad a “feel for politics,” which distinguished his writings fromthe more crude hyperfactualists.28

Common Sense

Apart from instinct and feel, some political scientists haveused “common sense” notions of what is political as theirimplicit orientation to the subject. Common sense understandspolitics to include politicking—“maneuvering for position andpower.”29 It also understands politics as “an activity related insome vague way to problems of government or the making ofpolicy for the whole society,” and involving disputes “overthe policies accepted as authoritative for the society.”30 Eastonnotes that one of the characteristics distinguishing sciencefrom common sense is the “deliberate attempt to bring to thesurface what common sense leaves permanently concealed.”31

Thus, Easton wants “to raise these assumptions to the pointof consciousness for the purposes of careful examination.”32

Common sense can be misleading or mistaken, and can varyamong researchers, so an explicit systematic theory can centercollective attention, and critically sift the relevant knowledge,if any, from writings based on differing notions of commonsense, intuition, etc.33

The Axiom

Aspiring to make the field more scientific, Easton asks how theconduct of political science research can be made more ratio-nal than by following various forms of preconscious intuition.Many of those who doubt the efficacy of scientific rationalityhave been relying on the guidance of such nebulous factorsas feel, common sense, instinct, etc. But, as noted, this canlead to difficulties in a profession for several reasons, not theleast of which is that various writers might not share the samesub-conceptual feelings. Thus, Easton will attempt to liberatethe profession from its reliance on vague tacit notions by ex-

26 Easton 1953, 1971, 96.27 Easton 1953, 1971, 99.28 Easton 1953, 1971, 144, n12.29 Easton 1953, 1971, 127.30 Easton 1953, 1971, 127.31 Easton 1953, 1971, 54.32 Easton 1953, 1971, 59.33 Easton 1953, 1971, 60.

plicitly defining the core of the field in a way that both appealsto reason and can be criticized rationally. He begins this effortby presenting his basic axiom for political science.

Familiar to political scientists, it may fairly be stated that“politics,” the subject of political science, is that human be-havior undertaken in relation to the authoritative alloca-tion of values for a society.34 This axiom is meant to, amongother things, “sum up our common sense conception of poli-tics.”35 Once its key terms are defined, which Easton does, theaxiom raises the vagaries of undefined intellectual sensitivitiesto a clear conceptual level. Because it is in an articulated form,practitioners in the field will be able to rationally criticize theaxiom. As we will see, Easton understood his axiom as havingimplications for not only the scope, but also the methods ofthe field.36

Thus, Easton’s definition of “politics” is a theoretical po-sition, or interpretive framework, and, as such, a way of seeingsocial interactions, and a principle for organizing and makingsense of social observations. From the axiom-guided observa-tions of the complex behavior of individual humans on theground, which constitute political life, the vision of a persis-tent political system emerges. As Easton writes, with the axiomguiding observation it becomes clear that “political life consti-tutes a concrete political system which is an aspect of thewhole social system.”37 This insight calls attention to anotherfault of hyperfactualism, namely that it conceals “from stu-dents of political life the need to view the political system as awhole.”38

For Easton, such terms as “political life” and “politicalprocess” are general references to the same subject matter asthe “political system;” namely, all those human interactionsdone in relation to the authoritative allocation of values for asociety. The “political system” entails “all those kinds of ac-tivities involved in the formulation and execution of socialpolicy;” i.e., “the policy-making process.”39 In practice, thesespecific components are implicated by the axiom. That is, theaxiom guides the attention of the political scientist to thosecomponents. Because observers know what to look at, i.e.,“politics,” they will soon see far more than what the axiomstates, as a political system unfolds before them.40

34 Cf. Easton 1953, 1971, 128, 129 passim. My understanding ofthe part an axiom can play in a system of thought comes from thephilosopher of science, Robert S. Hartman. See Hartman 1967.

35 Easton 1953, 1971, 128.36 Easton advised that “at a high level of abstraction,” such as his

axiom, “theory needs to be free to develop unhampered by excessiveworries of verification.” Easton 1953, 1971, 315. This statementalone greatly distinguishes Easton from positivism, which, as we willdiscuss further below, insists that theories be verifiable.

37 Easton 1953, 1971, 97.38 Easton 1953, 1971, 78.39 Easton 1953, 1971, 129.40 Some readers of The Political System might find it ironic that

given its title, in the book, Easton does not elaborate specifically onthe five components of his theory (inputs, conversion, outputs, feed-back, and environment). Instead, he carries out an extended presenta-tion of that theory in two succeeding works, A Framework for Politi-cal Analysis, Easton 1965a. And A Systems Analysis of Political Life.

25


Political Life as Human Life

Throughout his discussion, Easton stated his axiom in slightlydifferent variations, and also mentioned some qualifying nu-ances he intended but which are not always specified in hisdefinition of politics. For example, he notes that out of all theforms of social activity some are “closely related to what wecall political life.”41 Indeed, one key element of Easton’s axiomis that it contemplates “life;” specifically, “the political aspectsof life,” or “the political side of life.”42 By “life,” of course, hemeans human life. He observes, for instance, that “What wehave in the concrete social world is a series of events in whichhuman beings are involved.”43 He also wrote, “As a social sys-tem, a society is a special kind of human grouping the mem-bers of which continually interact with one another and in theprocess develop a sense of belonging together [or,] commonconsciousness.”44 He notes that “we are human beings wholive in an organized society.”45 While this may seem too obvi-ous to merit mentioning, as we will see, his stress on human lifehas important methodological and epistemological implica-tions.46

The Axiom as a Rule of Relevance

Guided by his axiom, Easton writes that “Political life concernsall those varieties of activity that influence significantly thekind of authoritative policy adopted for a society and the wayit is put into practice. We are said to be participating in politicallife when our activity relates in some way to the making andexecution of policy for society.”47 For him, then, a phrase like“relates in some way to the making and execution of policy forsociety” becomes a normative criteria, or rule, by which politi-cal scientists can make crucial research decisions. He writes,“If the object of [systematic] theory is to identify all the impor-tant variables, some criteria are required to determine relevanceor importance…Without some guide to the investigator to in-dicate when a variable is politically relevant, social life would

Easton 1965b. While those works are rich in insights and guidance forthe conduct of political science, they will not be examined in thispaper. For Easton to elaborate on the elements of the theory of thepolitical system would have been a distraction from his central aim inThe Political System, which is to introduce his axiom, and to show theneed for a systematic political theory and the lack thereof in politicalscience. That is why he notes that the “merest hint of a theory [of thepolitical system] that does emerge is incidental here to the mainpurpose.” Easton 1953, 1971, page 5. But because he understoodthat observing political life through the lens of the axiom culminatesin a view of the political system, a book introducing the axiom andexplaining the need for it is aptly titled The Political System—theultimate implication of the axiom.

41 Easton 1953, 1971, 97.42 Easton 1953, 1971, 96, 126.43 Easton 1953, 1971, 53.44 Easton 1953, 1971, 135.45 Ibid, page Easton 1953, 1971, 103.46 As to why chimpanzee “politics” do not rise to the level of a

political system, see Kelleher, “Can Chimpanzee Politics Constitutea Political System?”

47 Easton 1953, 1971, 128.

simply be an incoherent wilderness of activities.”48 He hopedthat his axiom, and the framework implied by it, would helpresearchers to cultivate a “keen sense of where and how tolook for the locus of power and its influence.”49

Intimacy and Method

Easton is offering political science a more humanized form ofbehavioralism than the positivistic brand. His behavioralismenvisions a distinct conception of the relationship betweenthe researcher and the subject matter. He rejects the imper-sonal, if not alienated, relationship implicated in positivisticefforts at “objective” description. As we will see, Easton’smethod is more empathic. This is why the question of how toobtain “intimate knowledge” of political actors arises in hisapproach, whereas such an inquiry is immaterial, if not incon-ceivable, in the positivistic approach dominant in his time (andours).

Easton argued that his method would be able to “providethe basis for the kind of understanding of their data that stu-dents of political life seek.”50 He felt that this framework wouldfurther provide such matters as “a clear perspective on thefundamental problems of the logic behind scientific method,unambiguous terminology, the introduction of new techniquesand a deep awareness of the need to seek out intimacy withobserved phenomena.”51

Easton notes that political science is currently behind theother social sciences in its use of the “repertoire of techniquesfor controlled observation, such as the varieties of highly de-veloped forms of interview and objective participation, thecorrelation of data, experimentation, and the testing of theo-ries.”52 Clearly, he anticipated that researchers would draw uponthe wide variety of methods available in all the social sciences.For, it is not the method but the axiom that defines the field forEaston. He laments that the use of mere fact gathering “tech-niques has had the secondary result of keeping the researchstudent from intimate contact with his [or her] material.”53

Easton rejected those more positivistic orientations thatenvision a mechanistic theory of human action as a series oflearned, or programmed, reactions to stimuli. This robotic vieweschews empathy and blinds itself to the uniqueness of eachindividual, and to the role of meaning and volition in behavior.He found unacceptable “the damaging effects of this lack ofintimate knowledge about political activity on the products ofresearch.”54 That “damage” is, of course, the lack of humanunderstanding needed to solve some of the most pressingpolitical problems of the day.55

Also, in Easton’s methodology, the researcher can ap-proach the subject matter from different perspectives, such as

48 Easton 1953, 1971, 98.49 Easton 1953, 1971, 52.50 Easton 1953, 1971, 51.51 Easton 1953, 1971, 52, emp. ad.52 Easton 1953, 1971, 49.53 Easton 1953, 1971, 49.54 Easton 1953, 1971, 49.55 Easton 1953, 1971, 49.

26


putting the focus on the “political process,” or on “politicalbehavior.”56 The former, more systemic meaning, “refers to theconcrete system of intertwined activity which shapes authori-tative policy.”57 But the latter, more intersubjective “behav-ioral” approach, focuses on “a particular aspect of data, thepsychological.”58 When taking the behavioralist perspectiveas Easton understands it, political scientists “are studying thepolitical process by looking at the relation to it of the motiva-tions, personalities, or feelings of the participants as individualhuman beings.”59 This particular type of behavioral approach,Easton’s personal behavioralism, can obtain the intimateknowledge that students of political life seek precisely be-cause it attends to those “attitudes…motivations and feelingsof the human actor.” These are the moving elements of “actinghuman beings.”60

In Easton’s behavioralism, then, for the political scientistto obtain such intimate knowledge, empathic interpretationsof political behavior must be employed. Thus, Easton sees “abroad scope for direct field research…to make personal obser-vations in the field according to acceptable standards for thecollection of data.”61 His own efforts at making explicit thesubconscious, or tacit, theory assumptions of his fellow politi-cal scientists, discussed above, is, in part, a participantobserver’s act of empathic interpretation. What Easton addsto our understanding of political science knowledge, then, isthat it includes a significant element of empathically knowingthose feelings, meanings, and motivations experienced by ac-tors in relation to the authoritative allocation of values for asociety. Clearly, Easton’s idea of political science knowledgeincludes far more than what the statisticians and mathemati-cians would include. While important, their brand of abstractknowledge is unable to achieve the kind of intimacy Eastonenvisions.

The Axiom Compels Interpretive Methods

Easton’s axiom not just implies, but compels the inclusion ofqualitative/interpretive methods in political science. If politi-cal science is the study of behavior undertaken in relation topolitics, then the intentions of the actor are determinative ofthe behavior’s relevance. Whether an actor defines himself orherself as acting in a politically relevant way is a matter for thepolitical scientist to determine through empathic observation,or interview, or some related technique. The behavior of a crowdat a football game can be similar to that of a crowd at a politicalrally. The meanings and intentions of the actors, less than theirphysical movements, determine which crowd is a fit subject forpolitical science. Political groups, therefore, are defined firstby the political intentions of their members. Mere numbers ofpeople make no sense unless their shared meanings are known.Thus, even large-N studies require an element of empathy.

56 Easton 1953, 1971, 203.57 Easton 1953, 1971, 204-205.58 Easton 1953, 1971, 205.59 Easton 1953, 1971, 205.60 Easton 1953, 1971, 201.61 Easton 1953, 1971, 49.

How is a potential voting block to be known, except by theunderstanding of the shared intentions and meanings that re-late the individual persons to one another? What determinesthe relevance to political science as between a group of per-sons sitting in a living room for a Tupperware party, or a babyshower, and one gathered for a candidate fundraiser? It is theirrespective intentions and shared meanings. As Easton notes,observing behavior is not enough, the political scientist “mustalso be prepared to show what makes it political.”62 It’s theintentions.

Two Kinds of Interpretation—Theoretic and Empathic

Although he did not make the distinction explicitly, Easton’saxiom-centered systematic political theory appears to impli-cate two different kinds of interpretation. One can be under-stood as theoretic interpretation, and the other as empathicinterpretation. The theoretic interpretation entails the act ofintegrating data (or smaller sets of meanings) into a pre-exist-ing more general conceptual framework. An example of this isclassifying observed behavior according to the categories ofEaston’s theory of the political system—inputs, outputs, etc.The theory tells the political science researcher what to lookfor on the ground. A description of a particular pattern of be-havior, such as the actions of legislators, is a set of meanings,which can then be subsumed under the appropriate category;the conversion process, for instance. This interpretation givesthe behavior technical, or theoretic, meaning for the politicalscientist.

Positivistic political science also interprets events. It hasa theoretical framework into which its so-called descriptions ofbehavior are integrated. The result, as we have argued, is aportrayal of a mechanistic political behavior. In this view, be-havior is moved, but is not self-moved by people acting onmeanings. No empathy is needed.

The empathic interpretation suggested by Easton, adds asecond step of interpretation. That is, a researcher articulatesthe meanings that persons and groups of persons are actingupon in relation to politics. This set of meanings can includetheir definitions of the situations, their ideologies, their goals,strategies, etc. Such an interpretation is constructed by theresearcher in conjunction with the subjects. And that entails apersonal identification of the researcher with the subjects morebroadly as fellow humans. This is the way intimate knowledgeis obtained.63

The logical operation of theoretic interpretation is moretechnical than that of empathic interpretation, which is moreintuitive. But both types of interpretation, once given in writ-ten form, can be rationally criticized, and thus rejected, cor-rected, or confirmed by other members of the profession. To

62 Easton 1953, 1971, 192.63 Frederic Schaffer’s suggestions for interpreting what he calls

“experience-near” concepts offers some rules of art that would beuseful in the conduct of Easton’s behavioralism. However, his notionof “experience-distant” concepts is not intended to convey a fielddefining theory, as is Easton’s political system. For a discussion ofSchaffer’s rules of art see Büthe, 2016.

27


some extent, the two intellectual operations must be done to-gether. That is, the intentions of the actors must be empathicallyinterpreted as well as their overt behavior described as, forexample, of the legislative sort, in order to be classified withinthe appropriate category, e.g., the conversion process.

Conclusion

Easton’s axiom based systematic political theory, which in-cludes his theory of the political system, stands as a completealternative to the positivistic framework. That axiom may nowbe fairly stated to define “politics” as the volitional behaviorof persons and groups of persons who are intentionally en-gaged in matters related to the authoritative allocation ofvalues for society.64

The methodological principle, that the behavior understudy is that of “persons,” necessitates the rejection ofpositivism’s mechanistic and deterministic form of explana-tion. Easton’s assumption that humans create the meaningsby which they define the political situations they are in, andthen act upon those meanings, explodes the old positivisticnotion of personally detached “objective” observations ofbehavior, and compels the position of personally involvedempathic observation and interpretation.

Under Easton’s principles, then, the explanation of be-havior necessarily entails an exposition of the observer’s un-derstanding of the reasons the actors had for their behavior. Insome cases, groups and individuals may not fully understandtheir own motivations, or even misrepresent them. The politi-cal scientist can empathically observe and then clarify thesemotivations. The idea of causation remains, but the mechanis-tic tendency is factored out by the axiom’s implication of peopleas volitional, creative, intelligent, meaning making sentient be-ings. This soft theory of causation recognizes that no matterwhat was done, it could have been otherwise. In addition, thehope for a way to verify or refute hypotheses, in the manner oflaboratory physics, must give way to a requirement more likethe intersubjective confirmation or criticism of findings bypeers. No one can, for instance, replicate in a laboratory aparticipant observer’s experience and observations for lateranalysis. But peers can criticize the methods and logic used,and, based on their own experience and personal judgment,credit contributions to knowledge. In this sense, then, Easton’sinterpretive behavioralism would be conducted as a commu-nity project, working within a shared interpretive framework,and drawing from a plurality of methods.65

64 Understanding the behavior as volitional adds to the challenge ofexplaining how a political system is able to persist. Every day, largenumbers of individuals and groups act in ways that preserve theirpolitical system, although at any time they could choose to do other-wise, as they do in revolutions.

65 See Polanyi 1946, 1958. An activist in the paradigm politics ofhis day, and apparently in an effort to maintain his alliance with hiscontemporary “behavioralists,” Easton introduced what he called the“behavioralist credo” in Easton 1965a, page 7. He identified himselfwith the term “behavioralist,” but he understood he had vast differ-ences with his allies. As he commented following the Credo, “Nosingle way of characterizing [these postulates] is satisfactory to ev-

References

Adcock, Robert, Mark Bevir, and Shannon C. Stimson, Eds. 2007.Modern Political Science: Anglo- American Exchanges Since 1880.Princeton: Princeton University Press.

Adcock, Robert, and M. Bevir. 2010. “Political Science since WorldWar Two: Americanization and its Limits,” in R. Backhouse and P.Fontaine, eds., The History of Postwar Social Science. Cambridge:Cambridge University Press, 71-101.

Almond, Gabriel A. 1966. “Political Theory and Political Science.”American Political Science Review, vol. 60 (4), 869-879.

Barrow, Clyde. 2008. “The Intellectual Origins of the New PoliticalScience.” New Political Science 30 (2).

Berkenpas, Josh. 2016. “The ‘Behavioral Revolution’: A Genealogyof a Concept.” European Political Science, vol. 15 (2), 233-250.

Bevir, Mark. 2006. “Political Studies as Narrative and Science, 1880-2000.” Political Studies 54, 583-606.

Büthe, Tim. 2016. “Elucidating Concepts: Introduction to the Sym-posium.” Qualitative & Multi-Method Research vol. 14, no. 1/2: 40passim.

Clarke, Kevin, David Primo. 2012. A Model Discipline. Oxford Uni-versity Press.

Crick, Bernard. 1959. The American Science of Politics: Its Originsand Conditions. Berkeley and Los Angeles: University of Califor-nia Press.

David, Marian. 2016. “The Correspondence Theory of Truth.” TheStanford Encyclopedia of Philosophy. Edward N. Zalta (ed.). Onlineat, http://plato.stanford.edu/archives/sum2016/entries/truth-corre-spondence.

Easton, David. 1953, 1971. The Political System. Chicago: The Uni-versity of Chicago Press.

_________.1965a A Framework for Political Analysis. New Jersey:Prentice-Hall.

_________.1965b A Systems Analysis of Political Life. New York:John Wiley & Sons.

Easton, David, and Jack Dennis. 1969. Children in the Political Sys-tem: Origins of Political Legitimacy. NY: McGraw-Hill.

_________. 1967. “The Child’s Acquisition of Regime Norms: Po-litical Efcacy.” American Political Science Review, vol. 61(1), 25-38.

_________. 1969. “The New Revolution in Political Science.” Ameri-can Political Science Review vol. 63 (4), 1051-1061.

Farr, James & Raymond Seidelman, eds. 1993. Discipline and His-tory: Political Science in the United States. Ann Arbor: Universityof Michigan Press.

Finlayson, Alan, Mark Bevir, R. A. W. Rhodes, Keith Dowding,Colin Hay. 2004. “The Interpretive Approach in Political Science:a Symposium.” The British Journal of Politics & InternationalRelations, vol. 6 (2), 129-164.

Gunnell, John G. 1978. “The Myth of the Tradition.” AmericanPolitical Science Review, vol. 72 (1), 122-134.

_________. 1983. “Political Theory: The Evolution of a Sub-Field.”Political Science: The State of the Discipline. Ada W. Finifter, ed.Washington, D.C.: The American Political Science Association.

_________. 2013. “The Reconstruction of Political Theory: DavidEaston, Behavioralism, and the Long Road to System.” Journal ofthe History of the Behavioral Sciences, vol. 49 (2), 190–210.

Hartman, Robert S. 1967. The Structure of Value. Carbondale: South-ern Illinois University Press.

eryone.” Easton 1965a, page 6. His 1969 Presidential Address, en-couraged civic participation by political scientists, but made no changesin his political theory. See Easton 1969.

28


Parenti, Michael. 1973, 2010. Democracy for the Few. Belmont, CA.Wadsworth Publishing Company.

Polanyi, Michael. 1958. Personal Knowledge. Chicago: Universityof Chicago Press.

_________. 1966. The Tacit Dimension. Chicago: University of Chi-cago Press.

_________. 1959. The Study of Man. New York: Routledge & KeganPaul.

_________. 1962. “The Republic of Science.” Minerva, Vol. I (1)(1962), 54-73; republished in Minerva, Vol. 38, No. 1 (2000), pp.1-32.

_________. 1946. Science, Faith, and Society. London: Oxford Uni-versity Press.

Popper, Karl. 1963. Conjectures and Refutations: The Growth ofScientific Knowledge. NY: Basic Books.

Ricci, David. 1984. The Tragedy of Political Science. Yale UniversityPress.

Robey, John S. 1982. “Reputation vs. Citations: Who Are the TopScholars in Political Science.” PS, 15:200.

Roettger, Walter B. 1978. “Strata and Stability: Reputations of Ameri-can Political Scientists.” PS 11:9.

Schaffer, Frederic Charles. 2016. Elucidating Social Science Con-cepts: An Interpretivist Guide. New York: Routledge.

Somit, Albert and Joseph Tanenhaus. 1967. The Development ofAmerican Political Science: From Burgess to Behavioralism. Bos-ton: Allyn and Bacon, Inc.

Truman, David B. 1965. “Disillusion and Regeneration: The Questfor a Discipline.” American Political Science Review, Vol. 54 (4),865-873.

Winch, Peter. 1958. The Idea of a Social Science and Its Relation toPhilosophy. London: Routledge & Kegan Paul.

Wolin, Sheldon. 1969. “Political Theory as a Vocation.” AmericanPolitical Science Review, vol. 63 (4), 1062-1082.

Yanow, Dvora and Peregrine Schwartz-Shea, eds. 2014. Interpreta-tion and Method. NY: M.E. Sharpe, Inc.

Hawkesworth, Mary. 2014. “Contending Conceptions of Scienceand Politics.” In Yanow and Schwartz-Shea, eds. Interpretationand Method, pages 27- 49.

Kelleher, William J. 2015. “A Model Discipline, and How ‘GoodWork’ Hurts Political Science.” Perspectives on Politics, Volume 13(2), 446-448.

_________. 2017a. “Back to the Future: How Understanding DavidEaston Can Give Guidance to the Caucus for a New Political Sci-ence.” New Political Science vol. 39, no. 4: 473-486.

_________. “Can Chimpanzee Politics Constitute a Political Sys-tem?” Draft online at, https://www.academia.edu/20299604/Can_Chimpanzee_Politics_ Constitute_a_Political_System

_________. “The Clarke and Primo Liberating Conception of GoodWork in Political Science.” Draft online at, https:/www.academia.edu/10417226/The_Clarke_and_Primo_Liberating_Conception_of_Good_Work_in_Political_Science

_________. 2017b. “David Easton as Interpretivist.” Draft online as“David Easton’s Axiom Centered Systematic Political Theory.”For presentation at the Western Political Science Association Con-ference, British Columbia, Canada, April 2017. https://www.academia.edu/31337872/David_Easton_as_ Interpretivist

_________. “How They are Healing Physics Envy in Biology.”Draft online at,https://www.academia.edu/19807254/How_They_are_Healing_Physics_Envy_in_Biology

_________. “Normative Political Science - as Constituted by theFormal Axiology of Robert S. Hartman and David Easton’s Con-cept of the Political System.” Draft online at, https://www.academia. edu/4800305/Normative_Political_Science_-_as_Constituted_by_the_Formal_Axiology_of_Robert_S._Hartman_and_David_Easton_s_Concept_ of_the_Political_System

Lynn, Naomi B. 1983. “Self-Portrait: Profile of Political Scientists,”p. 106. In: Political Science: The State of the Discipline. Ada W.Finifter, ed. Washington: American Political Science Association.

Monroe, Kristen M. 1997. Contemporary Empirical Theory. Berke-ley and Los Angeles: The University of California Press.

29


Introduction to the Symposium

Tim BütheHochschule für Politik at the Technical University of

Munich, Germany & Duke University

Trade, Expectations, War and Peace

Dale Copeland’s Economic Interdependence and War—dis-cussed in this symposium primarily for its methodology—ad-dresses a classic yet very timely question: Do commercial tiesbetween countries increase or decrease the likelihood of mili-tarized conflict? To answer this question, the book developsnuanced arguments about the conditions under which eco-nomic interdependence is more likely to become a force forpeace and under what conditions it is more likely to exacerbatethe risk of a militarized escalation of a conflict of interest.1

As the magisterial conclusion of a twenty-year researchagenda,2 the book also presents a wealth of often meticulousempirical analyses of virtually “all the main cases [|] of greatpower conflict, starting in 1790.”3 Forty such conflicts are ana-lyzed in qualitative case studies in chapters 3-8 of the book—complemented in chapter 2 by a review of several large-N quan-titative studies of the relationship between interdependenceand war, which Copeland revisits to highlight what their find-ings suggest (and what they might have missed) about theimportance of economic considerations in great power con-flicts.

Economic Interdependence and War is a highly ambi-tious book, which sets out to “resolve,” once and for all, mul-tiple debates over the relationship between economic interde-pendence and war (some of which, as Erik Gartzke points out inhis essay for this symposium, go back hundreds of years) by

Tim Büthe is Professor and Chair for International Relations atthe Hochschule für Politik/School of Governance of the TechnicalUniversity of Munich (TUM), as well as a nonresident Senior Fel-low for the Rethinking Regulation project at the Kenan Institute forEthics at Duke University. He is online at [email protected] andhttp://www.buthe.info.

1 For rhetorical effect, Copeland occasionally slips into askingwhether-or-not questions or presenting starkly dichotomous out-comes such as war-or-peace (where peace is simply the absence ofwar, a sleight of hand for which Sherry Zaks takes him to task as partof her essay in this symposium). Importantly, however, the logic ofthe argument is probabilistic, and Copeland’s emphasis on conflictescalation or de-escalation as processes that play out over time im-plies the recognition of a range of intermediate outcomes betweenpeace and full-blown war.

2 See Copeland 1996.3 Copeland 2015, 2f.

Causal Explanation and the Study of Rare Events Through Large Sweeps of History:Symposium on Dale C. Copeland, Economic Interdependence and War

(Princeton: Princeton University Press, 2015: xiv + 489 pp.)ISBN: 978-0-691-16159-4 (pb)

“answer[ing] most of the outstanding questions surroundingthe issue.”4 That’s a tall order.

The theoretical core of the book’s answer to those out-standing questions is what Copeland calls “Trade Expecta-tions Theory.”5 It argues, in a nutshell, that as long as acountry’s leaders expect continued access to foreign marketsfor important inputs and for the country’s exports, economicinterdependence will reduce the likelihood of militarized con-flict by making such an escalation more costly. This part of theargument assumes that trade is economically beneficial forboth sides, that militarized conflict interrupts or at least re-duces and threatens such economic exchange, and that thedesire to continue to reap the benefits of flourishing tradecreates an incentive to maintain peaceful relations. Crucially,Copeland goes beyond classic liberalism by stipulating thatthis incentive to settle conflicts of interests peacefully oper-ates only as long as the political leadership expects that thecountry’s trade/economic partners also want to maintain thatrelationship. This condition is critical because it ensures that,if a country establishes or maintains economic openness inthe present, the net present value of the future consequencesis positive.6

By contrast, if this condition does not hold, greater eco-nomic interdependence may actually increase the risk of milita-rized escalation, Copeland argues. Why might expectationstake such a pessimistic turn? And why would expecting a fu-ture loss of market access push countries toward war? Startingout under autarky, the correct anticipation of a future loss ofmarket access7 should result in forgoing economic interdepen-dence in the first place. Economic openness therefore can co-incide with the expectation of declining access to foreign mar-kets for imports and exports only in the aftermath of an exog-enous shock to a previously held expectation of ongoing mar-ket access. And yet, Copeland cautions, such a turn of eventsis not uncommon. It may arise, for instance, from a change inany great power’s government “due to elections, coups, do-mestic instability, and the like,”8 whenever such a change isunanticipated and puts in power political leaders with a differ-ing set of preferences.

A change that removes or reduces the prospect of futuregains from trade also removes or reduces the above-mentioned

4 Copeland 2015, 1.5 Spelled out in detail in chapter 1 of the book (esp. pp.27-47).6 Copeland 2015, esp. 33ff.7 And expectations are in Copeland’s theoretical model assumed to

be always correct in the sense of making optimal use of all availableinformation in light of instrumentally rational pursuit of the politicalleaders’ national security maximization preferences.

8 Copeland 2015, 41; see also 43-46.


30


constraint. Consequently, if a conflict of interest were to arisefor unrelated reasons, and one side were tempted to use mili-tary force to gain an advantage in that conflict, contemporaryeconomic openness would no longer provide a counterweightagainst such temptations. In fact, by creating mutual but asym-metric dependency, economic openness might create just thepower imbalance that heightens the temptation to threaten oruse military force.

And it gets worse:9 Economic openness is economicallybeneficial because it allows for specialization, not just due tostatic comparative advantage of the kind David Ricardo andhis contemporaries already recognized, but also dynamically,for instance, to achieve economies of scale. Once dynamicspecialization takes hold, however, it is costly (and in the shortrun might be impossible) to undo. To sever a relationship ofeconomic interdependence therefore does not mean simplygoing back to the status quo; it means losses beyond the lostgains from trade. The sudden anticipation of a future in which,due to the expected loss of market access, a country expects tobe worse off than it would have been had it maintained autarky,can push the country that (due to unequal interdependence)expects substantially greater losses toward using military forceto prevent the other great power from closing or blocking thosemarkets. Copeland thus anticipates military escalation, not justas a way to overcome the intensified feeling of vulnerability,but also to prevent the realization of a (suddenly threatened)negative net present value of economic openness.

What’s At Stake? Theory & Policy

There is much at stake here for IR theory: Michael Hiscox“resolved” the long-standing debate between proponents ofthe Heckscher-Ohlin/Stolper-Samuelson model and proponentsof the Ricardo-Viner/specific factors model of international tradeby pointing out that the differing assumptions about mobilevs. specific factors may be more fruitfully treated as scopeconditions for the two models. When they are, the seemingincommensurability of the two models drops out, and Hiscoxfinds strong empirical support for both schools of thought—for each under the conditions when it should indeed apply.10

As Sherry Zaks points out in her contribution to this sympo-sium, Copeland’s analytical framework might be similarly readas an attempt to resolve long-standing disputes between “lib-eral” and IR-”Realist” scholars over the effect of interdepen-dence on interstate conflict. In such a reading, the book mightthen be said to offer an additional, fruitful way of overcomingwhat Joseph Grieco calls the “schools-of-thought problem inIR,”11 which would also overcome some of the concerns raisedby Timothy McKeown about the conceptualization of expec-

9 Copeland 2015, 2, 10f, 35f. Note that the underlying conflict ofinterest, which might escalate or be peacefully resolved, may ariseover completely unrelated matters or be itself prompted by economicinterdependence. By not taking a position on this issue, Copelandimplicitly suggests that TE Theory is meant to capture both possi-bilities.

10 Hiscox 2002.11 Grieco 2018 (forthcoming).

tations. Note, however, that Copeland rejects this reading ofthe book: As he explains in his rejoinder essay, he believesscholars should treat his Trade Expectations (TE) Theory as asui generis alternative to both liberal and Realist IR theory.

In addition, there is also much at stake for public andforeign policy at a time when the Trump administration is in-creasingly behaving like an overstretched declining hegemonin Gilpin’s classic War and Change. Copeland concurs withGilpin that such declining great powers are generally prone toescalating conflicts militarily—for instance in an attempt tohalt their relative decline through a “preventive” war.12 At thesame time, as long as the declining power remains the lessdependent country in the interdependent economic relation-ship (as President Trump tells us is the case for the UnitedStates in virtually all of its foreign economic relationships),Copeland expects that the leader’s “awareness of the deleteri-ous effects of the trade-security spiral” give such a leadersufficient “reason to avoid overly provocative policies.”13 Weshould of course keep in mind that the current U.S. admini-stration’s portrayal of the United States as being in long-termeconomic and military decline, might be a tactical move fordomestic political purposes, rather than something PresidentTrump and his policy advisors actually believe. But after read-ing Copeland’s book, the stakes appear even higher when try-ing to figure out what the most influential members of thecurrent U.S. administration truly believe.14

What’s At Stake Methodologically?

Important as the above considerations may be, we selectedthis book for a QMMR symposium not primarily for its contri-bution to IR theory nor its policy implications. Rather, wesought to encourage discussion of the big methodologicalclaims and contributions of the book. Most of the major meth-odological issues that Copeland addresses in Economic Inter-dependence and War also arise when studying various othersubjects across all the empirical subfields of political science.The book, moreover, makes a number of broad claims, such asthe claim that qualitative research involves a fundamentallydifferent kind of causal inference because qualitative research(or at least qualitative analyses of individual-level decisions)allow for observing causality “per se.”15 Such sweeping claims

12 Applying Copeland’s argument to particular cases and trying toderive policy implications from the book’s findings underscores theimportance of addressing the additional conceptual question of whatnotion of power underlies the claims about increasing/rising and de-clining powers—which, however, is beyond this symposium.

13 Copeland 2015, 48.14 Another, even more disconcerting reading of contemporary U.S.

foreign economic policy—which one might derive from Copeland’sanalysis even if he himself does not specifically suggests it—is thatPresident Trump is trying to sever U.S. economic ties to other coun-tries because he either intends to go to war against some of them or atleast wants to restore the ability to credibly contemplate and threatenwar to the President’s political tool kit, having already imposed thetrade-related costs on the U.S. economy, which might otherwise con-strain a President considering military escalation.

15 Copeland 2015, 51; emphasis added. Copeland reiterates this

31


—which the book delivers with great conviction—warrant care-ful reflection and scrutiny by QMMR scholars, as offered, e.g.,by Timothy McKeown in this symposium, before they are ac-cepted or rejected.16

One of the book’s important contributions is its persistentemphasis on causal mechanisms,17 followed by detailed em-pirical analyses of an impressive array of cases of great powerconflict, spanning 200+ years and ranging from the militaryescalation of the conflict(s) between post-revolutionary Franceand the major European powers (Britain, Austria, Prussia), 1790-1801, to the de-escalation (and ultimately the end) of the ColdWar between the United States and Soviet Union in the late1980s through 1991.18 Copeland stipulates that political scien-tists should focus more on causal mechanisms—in both theirtheoretical and their empirical work—and he strongly arguesfor the superiority of qualitative, causal mechanism-focusedresearch based on archival research (whenever practically pos-sible) over large-N, quantitative, statistical analyses.19 Consis-tent with these stipulations, Copeland presents, for each of hisforty cases, information not just about trade levels, politicalleaders’ expectations regarding their countries’ trading rela-tionships, and other initial conditions of the relationship be-tween the major powers in question. He also presents whatCollier and Brady have called “causal process observations”20

—in particular, information about the extent to which trade andeconomic openness were considered by political leaders intheir internal deliberations during major power conflicts (in-cluding stable or changing expectations regarding the futuredevelopment of trading relationships).

Why this focus on causal process or mechanisms? Cope-land considers it “self-evident”21 that rare events in interna-tional relations, such as great power conflicts that entail a real

belief in his rejoinder essay in his long footnote 33, despite employ-ing the language of covariance.

16 On this point, see also Brady’s (2013) and Elman’s (2013) es-says on Goertz and Mahoneys’ Tale of Two Cultures (2012), as wellas King, Keohane and Verba’s earlier discussion of the “fundamentalproblem of causality” (1994, 75ff; see also Pearl 2009).

17 Copeland 2015, 1f, 13f, 16-18, 43-50, 69f, 72f.18 For each case, Copeland provides something like an historical

narrative in the sense of Büthe (2002). His rationale for the book’soverarching historical scope (1790-1991) is discussed on pp.76f; anoverview of the cases is provided on pp.79-93.

19 McKeown’s and to a lesser extent Gartzke’s essay in this sym-posium addresses some of these criticisms of large-N/quantitative/statistical methods, with Gartzke raising the concern that some ofthose criticisms may to quantitative scholars appear like critiques ofa straw man version of such work. Some will also object to the book’stendency to equate large-N with quantitative and both with frequentiststatistics.

20 E.g., Brady 2010; Collier, Brady and Seawright 2010, esp. 184ff.Copeland also draws on the broader literature on process tracing,though some of the more recent key contributions (such as the impor-tant volume by Bennett and Checkel (2014) and approaches to caseselection for causal process tracing, informed by Bayesian ideas(Humphreys and Jacobs 2014, extending Bennett 2008) were pub-lished too late to be reflected in this book.

21 Copeland 2015, 77.

risk of military escalation, are “rare for a reason. Rare events ininternational relations are typically situations where a complexset of factors must come together for the event to occur,”22 i.e.,they are characterized by the kind of complex conjuncturalcausality underscored by J. L. Mackie’s classic notion of cau-sality.23 And due to this complexity, he submits, there cannotbe a single set or “bundle” of conditions that can explain alloccurrences of such a rare event. Instead, there must be mul-tiple possible sets of conditions, i.e., multiple “causal path-ways,” that result in the actual occurrence of the rare event.24

It is not clear why either of these claims should generally holdfor rare events, but Copeland is surely right that for phenom-ena that are characterized by complex conjunctural causalityand multiple causal pathways, an in-depth analysis of the causalmechanism is needed to establish a causal account. Extrapo-lating from the average effect observed for a subset of obser-vations is not possible when these conditions hold.25 Andstudying a phenomenon that is rare might indeed make it prac-tically feasible to study all empirically observed instances ofthe phenomenon within temporal scope conditions.26

While calling for more careful attention to causal mecha-nisms in IR scholarship, Copeland also notes that focusing onmechanisms entails potentially severe epistemological risks.Specifically, Copeland criticizes previous qualitative work oninterdependence and war (and causal process-tracing casestudy research in general) for its pervasive selection bias and,consequently, lack of external validity or “generalizability.”27

Focusing on causal mechanism, he cautions, tends to bias“qualitative researchers” toward presenting “cases that areparticularly useful in illustrating the way these causal mecha-nisms work in practice” while omitting “cases that do not fitthe model.” As a consequence, we may, upon reading suchstudies, “have confidence” that the hypothesized causalmechanism indeed was at work “for the cases selected, but wehave no way of knowing” whether this finding also holds “forthe broader population of cases.”28

In Economic Interdependence and War, Copeland strivesto overcome both problems (selection bias and generalizability)by conducting in-depth studies of what he considers “essen-tially the universe” of cases29—made possible by studying“rare events.” This raises the question: What exactly is, as amethodological prescription, supposed to be rare? The initialconditions, such as a conflict of interests among great powersthat trade with each other? Or the extreme outcome of greatpower war? Or “just” a significant change in the level of con-flict (e.g., a shift from escalation to de-escalation)? And what

22 Copeland 2015, 71; see also p.53.23 Mackie 1965.24 Copeland 2015, 77.25 See also Büthe 2014.26 See Linz and Stepan’s (1996) impressively comprehensive study

of the fate of third-wave democratizations during the early post-transition years for another example of such a study.

27 Copeland 2015, 13, 53, 70f, 75.28 Copeland 2015, 70.29 Copeland 2015, 70.

32


exactly is, as a matter of the empirical, historical record, rareabout the cases or episodes examined in this book? Thesequestions must be answered clearly to allow the reader to un-derstand: Forty cases of what? And as both Zaks and Gartzkediscuss in their respective essays, the answers have profoundimplications for the extent to which this book, as comprehen-sive as it is, indeed entails an analysis of something approach-ing the universe of cases. The answers also tell us to whatextent the book’s research design indeed is free from selectionbias (granting for the sake of argument the controversial claimthat studying the universe of empirically observed cases solvesthe problems of selection bias and generalizability).30 Indeed,Zaks and Gartzke are skeptical of Copeland’s assessment thatthis book has overcome the problem of selection bias, pre-cisely because they understand Copeland’s “true”explanandum to be how major powers relate to each other.And this outcome can range from highly conflictual (and rap-idly changing) to stably peaceful. Zaks hence suggests thatCopeland has selected out “non-conflict interactions” andtherefore likely suffers from omission-induced selection bias.

The Symposium

In order to make this symposium maximally accessible to thebroad readership of QMMR, including those who might notyet have read the 489-page book, we asked Dale Copeland toprovide, as the opening essay for the symposium, a brief sum-mary of what he today considers the key methodological claimsand contributions of the book. Copeland’s summary statementis followed by three reviews offering constructive critiques:

Timothy McKeown commends Copeland for advancingthe theoretical debate, then critically highlights three method-ological aspects of the book. He first discusses Copeland’scritiques of large-N statistical studies, especially the claim thatquantitative methods do not support causal inference. Sec-ond, he examines Copeland’s arguments about the distinctive-ness of case studies. Finally, McKeown explores Copeland’snotion of expectations (as the core concept of TE Theory) andexamines how we might establish the changing expectationsof political leaders.

Sherry Zaks approaches the book from a macro-method-ological perspective. Informed by the framework developed inher recent article in Political Analysis,31 she asks how exactlyCopeland’s theoretical approach is meant to be related to theneorealist and liberal schools of thought in IR. She finds thatthe book provides an ambiguous answer. This is critically im-portant methodologically because, she argues, the suitability

30 Arguably, it solves neither problem entirely, because the set ofobservable cases is generally already biased (as Frankfurt School/critical theory scholars have long pointed out) and because the infer-ential logic underpinning causal attribution in process tracing workusually does not provide a justification for out-of-sample extrapola-tion, unless the possibility of multiple causal pathways can be reli-ably ruled out (though cf. Büthe 2002 for an argument aboutprobabilistically increased confidence in generalizability under theassumption that multiple causal pathways can be ruled out).

31 Zaks 2017.

of different empirical strategies depends upon whether theo-retical claims are coincident, congruent, inclusive, or truly com-peting. Zaks also examines Copeland’s approach to rare eventsand case selection, as noted above.

Erik Gartzke joins the conversation as an IR scholar whohas himself contributed to the substantive literature on eco-nomic interdependence and war.32 He also comments as a meth-odologist who has written about analyzing rare events usingadvanced statistical methods. Viewing TE-Theory’s behavioralassumptions of political leaders, grounded as they are in aprobabilistic cost-benefit calculation, as a straightforward ex-tension of classic liberal IR theory,33 Gartzke commends Cope-land for the ambition of reconciling or “synthesizing” liberaland Realist approaches to International Relations. In the end,however, he is not convinced, due to lingering questions aboutendogeneity (which Gartzke, as many in econometricians, seemsto understand more broadly than Copeland) and what he con-siders incommensurate understandings of “conflict causation”(i.e., why conflicts arise and escalate) in the liberal and Realisttradition. Gartzke also takes issue with the book’s method-ological critiques and the alternative approach Copeland ad-vocates, questioning whether we can draw general causal in-ferences from case studies when the explanatory theory in-forming them employs a probabilistic notion of causation.

In his rejoinder, Dale Copeland addresses a number of thespecific critiques, but above all, he sketches an agenda forfuture work, viewed through the lens of a new “Approach toMixed Methods” (AMM). Note that in this AMM framework,the hypothesized causal explanation (e.g., the importance ofindividual decisions for the international-level outcome) is bakedinto the researcher’s methodological choices. This implies thatit may be impossible for one scholar to fully evaluate morethan one (type of) argument—which is useful reminder thatscience and scientific progress is ultimately a collective enter-prise.

References

Bennett, Andrew. 2008. “Process Tracing: A Bayesian Perspective.”In The Oxford Handbook of Political Methodology edited by JanetM. Box-Steffensmeier, Henry E. Brady and David Collier. NewYork: Oxford University Press, 2008: 702-721.

Bennett, Andrew, and Jeffrey T. Checkel, eds. 2014. Process Trac-ing: From Metaphor to Analytic Tool. New York: Cambridge Uni-versity Press.

Brady, Henry E. 2010. “Data-Set Observations versus Causal-Pro-cess Observations: The 2000 U.S. Presidential Election.” In Re-thinking Social Inquiry: Diverse Tools, Shared Standards (2nd

32 His own, substantive closely related research prompts Gartzketo caution that the total cost of war, even for the “winning” side, is sohigh that, if political leaders are in fact engaging in a rational cost-benefit calculations with good foresight, the potential gains fromcontinued trade (or losses from interrupted trade) should rarely besufficient to tilt the balance.

33 Even when the motivation for the cost-benefit calculation ismaximizing national security, as it is for Copeland (see 2015, 6, 29),since Copeland also assumes high fungibility of economic resourcesfor military ends.

33


The Central MethodologicalClaims and Contributions of

Economic Interdependence and War

Dale C. CopelandUniversity of Virginia

ed.), edited by Henry E. Brady and David Collier. Lanham, MD:Rowman and Littlefield, 2010: 237-242.

———. 2013. “Do Two Research Cultures Imply Two ScientificParadigms?” Comparative Political Studies vol.46 no.2 (February2013): 252-265.

Büthe, Tim. 2002. “Taking Temporality Seriously: Modeling His-tory and the Use of Narratives as Evidence.” American PoliticalScience Review vol.96 no.3 (September 2002): 481-494.

———. “Causal Process Tracing in Multi-Methods Research: ThreeProvocations.” Manuscript, Duke University, January 2014.

Collier, David, Henry E. Brady, and Jason Seawright. 2010. “Sourcesof Leverage in Causal Inference: Toward an Alternative View ofMethodology.” In Rethinking Social Inquiry: Diverse Tools, SharedStandards, edited by Henry E. Brady and David Collier. Lanham,MD: Rowman and Littlefield, 2010: 161-199.

Copeland, Dale C. 1996. “Economic Interdependence and War: ATheory of Trade Expectations.” International Security vol.20 no.4(Spring 1996): 5-41.

———. 2015. Economic Interdependence and War. Princeton:Princeton University Press.

Elman, Colin. 2013. “Duck-Rabbits in Social Analysis: A Tale of TwoCultures.” Comparative Political Studies vol.46 no.2 (February2013): 266-277.

Gilpin, Robert. 1981. War and Change in World Politics. New York:Cambridge University Press.

Goertz, Gary, and James Mahoney. 2012. A Tale of Two Cultures:Qualitative and Quantitative Research in the Social Sciences.Princeton: Princeton University Press.

Grieco, Joseph. 2018. “The Schools of Thought Problem in Interna-tional Relations.” International Studies Review, forthcoming. Athttps://doi.org/10.1093/isr/viy005 (last accessed 8/10/2018).

Hiscox, Michael J. 2002. International Trade and Political Conflict:Commerce, Coalitions, and Mobility. Princeton: Princeton Univer-sity Press.

Humphreys, Macartan N., and Alan M. Jacobs. 2014. “Mixing Meth-ods: A Bayesian Integration of Qualitative and Quantitative Ap-proaches to Causal Inference.” American Political Science Reviewvol.109 no.4 (November 2015): 653-673.

King, Gary, Robert O. Keohane, and Sidney Verba. 1994. DesigningSocial Inquiry: Scientific Inference in Qualitative Research.Princeton: Princeton University Press.

Linz, Juan J., and Alfred Stepan. 1996. Problems of DemocraticTransition and Consolidation. Baltimore: Johns Hopkins Univer-sity Press.

Mackie, J. L. 1965. “Causes and Conditions.” American Philosophi-cal Quarterly vol.2 no.4 (October 1965): 245-264.

Pearl, Judea. 2009. Causality: Models, Reasoning, and Inference.2nd ed. Cambridge: Cambridge University Press.

Zaks, Sherry. 2017. “Relationships Among Rivals (RAR): A Frame-work for Analyzing Contending Hypotheses in Process-Tracing.”Political Analysis vol.25 no.3 (Summer 2017): 344-362.

Economic Interdependence and War argues that dependentgreat powers may be inclined either toward peace or towardactions that can lead to war depending on whether their expec-tations of the future commercial environment are positive ornegative.1 States that are optimistic about their ability to haveaccess to raw materials, investment, and markets will be in-clined toward moderate policies that build their long-term eco-nomic power and that avoid pushing other states into restric-tive policies that set off destabilizing trade-security spirals(e.g., China 1985-2015). By contrast, great powers that believeothers are cutting them off from access to trade and invest-ment will fear a decline in their power and thus be more likely toinitiate military policies that prevent this decline through in-creased control over economic spheres of influence (e.g., Ja-pan 1930-41). Bridging the divide between liberalism and real-ism, the book thus seeks to show under what conditions eco-nomic interdependence can lead to changes in expectations offuture commerce, and thus either to stable international sys-tems or to ones that experience cold and hot wars.

The book offers two main ways to test its propositions:quantitative and qualitative. The first half of chapter two re-views the “greatest hits” of the large-N quantitative work oninterdependence and war to show that an expectational ap-proach can explain a number of the anomalies in the currentliterature. The main focus of my book, however, is on qualita-tive historical tests of trade expectations theory vis-à-vis com-mercial liberalism and economic realism. Large-N quantitativemethodologies are inherently limited when it comes to explor-ing the role of leader expectations in great power politics. Be-cause there are no surveys of leaders’ attitudes and percep-tions across historical time, quantitative researchers are forcedto use indirect measures of expectations—for example, the trendlines of the past three or four years of trade data that leadersare presumed to extrapolate out into the future. Such measuresare clearly second-best when there are available documentsthat reveal how leaders were actually thinking about the fu-ture, and how these expectations shaped their behavior. More-over, when dealing with rare events, quantitative research isalso constrained in its ability to establish exactly what causalrole particular variables might have played in the mix of factorsthat led to individual cases of war or the ending of rivalry

Dale Copeland is Professor of International Relations in theWoodrow Wilson Department of Politics, as well as a Senior Fellowat the Miller Center at the University of Virginia. He is online [email protected] or at http://politics.virginia.edu/dale-copeland/.

1 Copeland 2015.


34


across time.2

The book thus starts its empirical chapters with a shortoverview of the quantitative research to provide a useful “firstcut” test of the possible explanatory value of trade expecta-tions theory.3 The vast bulk of the study is then devoted to anextended analysis of the documents and the best secondarysources for 40 “case periods” from 1789 to 1991. These areperiods that essentially cover the universe of wars and majorcrises between great powers during that timeframe as well asdramatic shifts away from conflict such as the Russian-Ameri-can détentes of the early 1970s and the late 1980s. Becausethis is ultimately a study of rare events—great power wars orbehaviors that dramatically change the probability of suchwars—there is a necessary historical focus on the events them-selves and the reasoning behind leaders’ decisions to provokethem.4 Yet, to minimize selecting on the dependent variable,the years prior to key shifts in behavior are also examined. Thisallows us to see to what extent the planning for conflict, thelevels of tension, and the probabilities of war changed as thecore independent variables of the competing theorieschanged.5

The methodological approach of the book is underpinnedby three main claims: first, that rare events in great power poli-tics are rare for a reason; second, that we must go beyondmerely showing that a factor was present to show what causalrole it was playing in the “mix” of forces that led to the rareevent; and finally, that the real purpose of good qualitativetesting is to establish not simply whether a variable “matters,”but rather how often it matters, in what way, and under whatconditions. None of these claims, I believe, should be terriblycontroversial. But when put together in a coherent way, theyoffer a potentially distinctive approach to thinking about quali-tative testing in international relations. For the rest of thisshort overview, I will briefly summarize each of these claims.

Great power wars and the crises that raise the risk of warare rare events in international politics for one main reason:they are events of complex conjunctural causality, where a setof factors A, B, and C must come together simultaneously forthe event E to occur. Each of the factors is a necessary condi-tion in the mix, and yet when they combine, they become suf-ficient to produce event E. This “individually necessary, jointlysufficient” (INJS) logic is itself not enough, since for almost allphenomena in international relations, there will be multiplepathways to a specific type of event such as war or allianceformation. This means that other complex bundles of factors—perhaps A, J, and K, or D, J, K, and L—may also be sufficientfor E to arise.6

The reality of INJS and multiple causality in the onset ofrare event E over time and space means that we cannot simplyidentify a theory that poses factor A as important and then test

2 See Copeland 2015, 13, 69f, 73f and discussion below.3 Copeland 2015, 53-69.4 Copeland 2015, 75-78, esp. fn. 34.5 Copeland 2015, 76-77; Büthe 2002.6 See Bennett and Elman 2006; Copeland 2015, 71-72; Mackie

1980; Mahoney and Goertz 2006; Ragin 1987; 2008.

it against competing theories that specify factors B or J asimportant. It may be that A was present at time t through t + 10but event E only occurred at t + 10. If our investigation showsthat the leader of state Y was motivated to initiate E because ofA, but that she held off until necessary conditions B and Cwere in place, then the presence of A and not-E from t to t + 9does not hurt any theory arguing that A is a critical factor inthe cause of E.7 Rather, it is imperative to show by interpretingthe documents that the leader of Y was indeed driven by A toinitiate E and for the reasons the theory hypothesizes, even ifB and C are also important parts of the causal mix that led to E.8

The second claim is that to truly understand why particu-lar event E came about after certain necessary conditions werein place we must establish what causal role the factors A, B,and C played in the arising of E. We need to know more thansimply that factors A, B, and C were associated with E. That is,we need to know whether they were propelling leaders towardbehaviors that led to E, or whether the factors played more ofa facilitating or reinforcing role, or indeed were constrainingthe leader from taking actions that might otherwise have pro-duced E. These are terms that are often used in academic dis-course but rarely defined. A propelling factor is one that di-rectly involves a leader’s ultimate ends and desires or fears—her “reasons” for action. A facilitating factor is one that isincidental to a leader’s ends but needs to be in place before thedesired action can be carried out. A constraining factor issomething pulling an individual back from doing what shemight otherwise want to do. Finally, a reinforcing factor is onethat makes the potential effect of a key propelling factor thatmuch more likely to occur.9

When we are investigating historically the complexconjunctural causality underpinning rare events, understand-ing the functional role played by a variable is critical. Indeed,the very “support” or lack of support of a theory will dependon this understanding. If, for example, a leader’s domestic un-popularity is pushing her to initiate war, but a high level ofinterdependence is constraining her toward peace in the shortterm, then the commercial liberal argument would be upheld.Conversely, if the trade environment is actually propelling theleader to choose war to maximize the nation’s security, and thelevel of domestic popularity is simply a constraining or facili-tating factor, then depending on exactly how trade is pushingthe leader, the economic realist or trade expectations argu-ments have potential explanatory force.10 The case studies inEconomic Interdependence and War thus seek to establishthat combinations of great power interdependence and declin-ing expectations of the future trade environment were critical

7 Stated differently, because the absence of rare event E—in thisbook, for example, not-war or peace—is at any point in timeoverdetermined, the presence of A during a time of not-E does notdisprove a connection between A and E, since A is not specified in thetheory as a sufficient condition but only as an important necessarycondition within an INJS bundle.

8 Copeland 2015, 74f; see also Copeland 2000, 29ff.9 Copeland 2015, 72f.10 Copeland 2015, 73.

35


propelling factors for war, even as the book recognizes thatother factors in the INJS mix, including domestic and bureau-cratic variables, were often present as facilitating or as rein-forcing factors for conflict.

The final methodological claim follows naturally from thefirst two, namely, that in qualitative research, we should beexamining how often a factor was important, in what ways, andunder what conditions. The specification of the essential uni-verse of cases within a bounded timeframe helps us to reducethe selection bias that so often plagues qualitative work. Itforces us to include in our set of cases ones that might notwork well for our pet theory. And by doing so, we can start toidentify the conditions under which the theory is indeed mostlikely to work, or to not work. We might see, for example, that atheory’s causal logic is more likely to be born out in situationsof high levels of industrialization and mutual dependence onthird parties for key natural resources, and less likely to workwhen states are simply competing for markets in higher-endluxury goods. By covering a broad range of cases, it might alsobecome clear that a theory’s core factor A is propelling leadersto bring about event E only under conditions M and N, butwhen conditions B and C are present, factor A typically oper-ates more as a facilitating or constraining factor. Finally, wemay find that under certain conditions, factor A from a pettheory sometimes works in conjunction with factors D and Jfrom a competing theory for a specific event E, meaning thatboth theories answer “part of the puzzle” for why E occurred.Such fine-grained empirical findings should help spur futureextensions of a theory, including the specification of the con-ditions under which a theory’s independent variables will likelybe especially salient in a particular INJS mix. They may evenlead to the development of new theories that transcend thelimitations of the original deductive logic. At the very least,they can encourage an investigation of cases from outside astudy’s specified boundaries that will help to hone the under-standing of the conditions under which different theories willtruly work across space and time.11

All this means that in qualitative testing we should not beseeking definitive tests of whether theories stressing factor A“beat” theories focusing on factors D or J. Rather, knowingthat the events we are studying are rare, and thus are onlylikely to arise when particular bundles of factors are present,we can figure out exactly how often and under what condi-tions bundles with factor A in them are implicated in the arisingof events across time and space, and what specific causal rolefactor A played in the onset of the events. When there areevents E that arise as a result of a combination of A and D, wecan then assess both the relative causal salience of A and D inthe mix, and whether A and D were both propelling factors, orwere doing something else to bring about E. From the perspec-tive of wanting our theories to be practically valuable for policymakers, this approach to research can pay big dividends. Itcan tell a leader or official when and under what conditions toworry that factor A, or factor D, might lead a nation into a war

11 See Copeland 2015, 70f and 94ff.

or a destabilizing crisis. And even if, say, the bundle of D, J, K,and L explains only five percent of the wars over the last twocenturies, this fact is still important: if these factors are begin-ning to manifest, officials can take steps now to reverse theprocess, knowing that otherwise a war or crisis might breakout.

In sum, the methodological set-up of Economic Interde-pendence and War seeks to offer a balanced approach to thetesting of IR theories, one that reduces (but as the symposiumshows, does not fully eliminate) some of the problems thathave hung over qualitative methodology over the past fewdecades. By covering the essential universe of cases for aspecific time frame, and by examining what functional role vari-ables are playing within an INJS bundle, we can get a betterhandle on how often a theory works—or does not work—across time and space, and why. Moreover, by examining thecases that a theory cannot explain, we establish a basis forimproving the theory and for specifying clearly the conditionsunder which it is likely to be useful. The study of rare eventsshould always lead us to think in terms of multiple pathways toevent E—different complex bundles of factors that can explainthe various arisings of E at different times.12 We can still bebold in putting forward variables that we believe drive many,perhaps even the majority, of rare events for a particular timeperiod. But, instead of endlessly arguing over the “master ex-planations” of phenomenon E across world history, our dis-cussions can focus on the healthier debate about under whatset of conditions particular theories should explain outcomes,and, for real-world policy makers, whether current conditionsjustify the use of one particular theory or another.

References

Bennett, Andrew and Colin Elman. 2006. “Complex Causal Relationsand Case Study Methods: The Example of Path Dependence.”Political Analysis vol. 14, no. 3: 25-67.

Büthe, Tim. 2002. “Taking Temporality Seriously: Modeling His-tory and the Use of Narratives as Evidence.” American PoliticalScience Review vol. 96, no. 3: 481-493.

Copeland, Dale C. 2000. The Origins of Major War. Ithaca: CornellUniversity Press.

Copeland, Dale C. 2015. Economic Interdependence and War.Princeton: Princeton University Press.

Mackie, John. 1980. The Cement of the Universe: A Study in Causa-tion. Oxford: Oxford University Press.

Mahoney, James and Gary Goertz. 2006. “A Tale of Two Cultures:Contrasting Quantitative and Qualitative Research.” Political Analy-sis vol. 14, no.3: 227-249.

Ragin, Charles C. 1987. The Comparative Method: Moving BeyondQualitative and Quantitative Strategies. Berkeley: University ofCalifornia Press.

Ragin, Charles C. 2008. Redesigning Social Inquiry: Fuzzy Sets andBeyond. Chicago: University of Chicago Press.

12 Mackie 1980.

36


The Empirical Study ofGreat Power Politics

Timothy McKeownUniversity of North Carolina, Chapel Hill

sets forth an argument for studying the role of trade expecta-tions by relying completely on case studies, with the balanceof book largely devoted to presenting the cases. The justifica-tion for proceeding this way rests on several supporting pil-lars: a critique of statistical approaches, an argument for howcase studies can be used to deal with the weaknesses of thestatistical approach, a perspective on the nature of expecta-tions and the role of case studies in studying them, and adescription of how the case studies will be conducted andused. Each of these elements is discussed below.

The Critique of Statistical Approaches

Copeland is interested in studying events that happen quiteinfrequently—great power wars, or situations where great powerwar was a threat but did not actually occur. He identifies only40 such cases during 1790-1991. He realizes that a rare eventslogit or probit approach to studying the outbreak of thesewars could be taken, but suggests that doing so would notaddress a deeper problem—the presence of multiple causalpathways to the outcome of war, each of which operates in ahighly context-dependent fashion: “It is a self-evident pointthat rare events in international relations and comparative poli-tics, such as crises, revolutions, and wars, will be a functionnot of a single complex bundle of [individually necessary, jointlysufficient] conditions but rather different bundles of factorsoperating with different causal force at different places andtimes. That is, even when we think in terms of complex con-junctural causality, we must still think in terms of multiple path-ways to event E.”2 The combination of complex theory andlimited data thus creates an extraordinarily difficult challengefor statistical analysis. Although he does not use this analogyhimself, what he is describing seems to be akin to a situation inwhich a relatively small sample is being mapped into a rela-tively large, multi-dimensional contingency table, producingan over-abundance of sparsely populated or empty cells. Hisown discussion stresses the dilemma posed by enriching astatistical model to include interaction terms, suggesting thatthis complicates the interpretation of statistical results, andthat these complications are further deepened when theoryimplies the incorporation of threshold effects or other non-linearities in the statistical models.3 While these might be validconcerns at times, their importance depends on the specificsof the model being considered. In any event, it is an issue thatis not closely linked to sample size.

Copeland is also skeptical of the general idea that statisti-cal results support the drawing of causal inferences, notingthat: “The causal mechanisms that lead to peace or war will beinadequately understood if this is our sole or primary method-ology, given that quantitative methods are inherently aboutcorrelations and association between variables rather than cau-sality per se.”4 This is a venerable claim: The precise mannerby which statistical results inform judgments about causationhas been the subject of debate ever since Sewall Wright first

2 Copeland 2015, 77.3 Copeland 2015, 71-72.4 Copeland 2015, 51.

If international politics is about “power,” then what do gov-ernments do once they have it? In Economic Interdependenceand War, Dale Copeland suggests that they often use it to gaincontrol of resources on favorable terms, then use that controlto leverage additional gains in control and wealth.

Copeland revives a literature that appears to have becomemoribund—“systemic” theories of international relations. Hereturns to the venerable liberal versus realist debates and showsthat there is new life in old theoretical ideas. By doing so, hemoves the theoretical conversation towards a synthesis ofthese two viewpoints.

His primary tool for accomplishing this is his focus onwhat he terms “trade expectations,” which is used here to referto the outlook not merely for international trade, but rather abroader array of other economic goods obtained from the restof the world. When government officials expect that their na-tion will reap large gains from these international sources, theyare inclined to be more peaceful and cooperative. Conversely,when their expectations turn pessimistic they attach a lowervalue to a peaceful status quo and begin to look for ways touse their available resources and tools of influence to redresstheir declining fortunes.

While the theory that supports this argument is informal,the basic argument is simple and generally compelling. At itsmost basic, one could view international politics as countlessrepeated-play bilateral Prisoners’ Dilemma games along thelines of Axelrod.1 Thus, increasing the payoffs from mutualcooperation dampens the temptation to defect. The larger themutual gains from cooperation, the more the world resemblesthe liberal-idealist vision in which creating institutions andpractices to realize and lock in joint gains becomes a large partof diplomacy. In a world where those gains dwindle, the chancesof aggressive unilateral actions increase, the possibility of warrises, and the international system begins to resemble the tense,balance-of-power world typically depicted by realists.

Economic Interdependence and War engages in an ex-tended theoretical dialogue with idealist and realist theories inthe course of making the case for its own trade expectationapproach. However, here, the theoretical side of the book willbe set aside to focus more deeply on the book’s unusual andremarkable empirical strategy. After a chapter that reviews vari-ous statistical models of international conflict, the author then

Timothy McKeown is Professor of Political Science at the Univer-sity of North Carolina at Chapel Hill. He is online [email protected] and https://www.researchgate.net/profile/Timothy_Mckeown2.

1 Axelrod 1984.


37


presented the method of path analysis and fellow geneticistE.H. Niles criticized his use of it to support claims about causalprocesses.5 It is widely agreed that statistical results can in-form judgments about hypothesized causal relationships, butthat they do not automate the creation of causal theory.6 Thehistory of these discussions suggests that what is required inmaking valid causal inferences is less a matter of additional ordifferent evidence than the existence of a causal theory to betested. Statisticians do not always agree on their precise con-ception of how statistical work clarifies questions of causeand effect, but they do seem to agree that statistics has a greatdeal to offer those who are attempting to investigate causa-tion, and that concepts of covariation are close to the heart ofmost definitions of causality.7

Copeland also has a much more specific objection to theuse of quantitative analyses in a study that focuses on expec-tations: He claims that quantitative methods are burdened withan inherent difficulty in dealing with leaders’ expectations be-cause they are forward-looking—expectations are a forecastthat affects the current behavior of decision-makers.8 Just whythis poses a problem is not obvious from his discussion. Ex-pectations are indeed forward-looking, but they are formed onthe basis of information that has already been acquired. Whileas a practical matter it might be difficult to write an equationthat accurately captures how received information is combinedand evaluated to produce a forecast, this is an activity thatdoes not involve information that agents do not already pos-sess.

How Are Case Studies Different?

If we turn from this critique of statistical studies to what re-searchers do when they write case studies, we encounter adescription of case study practice that amounts to “statisticswithout numbers”—making non-quantitative judgments ofcovariation within and across cases and using the language ofstatistics to describe the process and its results. This method,sometimes termed “congruence analysis,”9 is common in com-parative case studies.

Copeland is very explicit about what he is doing: “[W]emust look at the competing theories in terms of the competingbundles of variables said to drive [the event] E [that we seek toexplain]. Then we can look to see how often [emphasis in origi-nal] it is the case that a bundle from one theory will do a betterjob explaining the arising of E than a bundle from a competingtheory.” When he explains that we can mitigate the risks of“selecting on the dependent variable” by looking to the periodleading up to a war, and examining whether planning for con-flict, levels of tensions, and probabilities of war “changed asthe core independent variables changed,” he is simply examin-ing contemporaneous covariation. Furthermore, “any usefulresearch agenda must revolve around the issue of how fre-

5 Wright 1921; Niles 1922, 1923; Wright 1923.6 Denis and Legersky 2006.7 Holland 1986; Glymour 1986.8 Copeland 2015, 74.9 George and McKeown 1985.

quently a factor plays a critical causal role within the complexcausality that is producing event E.” His “three key ques-tions” are (1) “how often” economic interdependence vari-ables “were important”? (2) assuming that they are important,how frequently do the trade expectations variables “do a bet-ter job of explaining” the case than realist or liberal theories(which do not incorporate explicitly the trade-related variables)?(3) Then, given the importance of other factors not part of thetheory, what is the “relative salience” of those variables com-pared to the variables highlighted by theory?10 Table 2.7 sum-marizes his answers to these questions for his forty cases. Ineach case the importance of economic interdependence vari-ables is evaluated as “negligible,” “moderate” or “strong,”and a “best theory” is often named.

This is a defensible way to proceed, given an interest instudying the effect of a factor that materializes in highly case-specific ways. Attempting to re-analyze the cases to create anonparametric statistical model of events would also face thepreviously noted difficulty posed by the relative rarity of casesand the relative richness of the generating causal processes.In this situation, case studies might be the best approach sim-ply because they are the only feasible approach. However,that still leaves open the question of whether the presentationof the cases is best treated as an exercise in the qualitativeassessment of covariation. George and McKeown suggestedthat “process-tracing” methods are an alternative to congru-ence analysis.11 That alternative might be especially relevantin a study that focuses on the formation of expectations.

The Nature of Expectations and How to Study Them

If expectations of future economic gain are such a central partof the historical process leading to war or peace, then it mat-ters a great deal how expectations are formed. Copeland iscritical of the current literature on this point, claiming that itsmodels are “static and backward-looking.”12 It is necessarythat expectations be backward-looking because they are basedon information already received, but they need not be static ifinformation is being updated. Taking note of rational expecta-tions arguments in microeconomics, he posits for himself aneven more demanding theoretical assumption about the ratio-nality of decision-makers: not merely that they are rational onaverage, but that every decision-maker is rational.13 Disagree-ments among national decision-makers (it is unclear how heviews the possibility of divergent expectations across nationalboundaries) can occur, but we are not told how they are re-solved—only that their individual estimates are assumed to bedevoid of systematic bias. Inasmuch as there is no formal modelhere, it is not clear what is gained by making such a strongassumption, except perhaps providing a justification for notdelving into possibilities of divergent perceptions in the casestudies. It is easier to see what is lost by such an approach: theopportunity to ground a theory of how expectations affect the

10 Copeland 2015, 77-78.11 George and McKeown 1985.12 Copeland 2015, 16.13 Copeland 2015, 17 fn.

38


chances of war in a treatment of them that takes seriously thecognitive capacities and limits of individuals, the ways in whichtheir advisory systems operate to provide them with informa-tion, and the role of larger organizational processes of foreignministry bureaucracies, intelligence agencies, and military of-ficers and civilian national security officials, and of leaders’informal personal networks and contacts. Case studies of de-cision processes would seem to be the empirical approachbest suited to uncovering evidence in support of a behavioraltheory of expectations, but that is not how cases are usedhere. His empirical chapters have the more modest objective ofestablishing just what the decision-makers’ expectations were,and coming to some judgment about how salient the expecta-tions were in the decision-makers’ definition of the situationthat they confronted. The cases suggest that, while such ex-pectations are common, and they frequently seem to be sa-lient to decision-makers, the role that they play in each case isheavily context-dependent, and that it is currently not fruitfulto try to develop a single overarching measure of trade expec-tations. Without having one, we cannot simply re-analyze theregression models while including a measure of trade expecta-tions. That might not be such a loss if we can succeed inestablishing that both realist and idealist theories have ne-glected trade expectations at their peril—a more limited goal,which seems to have been attained.

Copeland suggests that statistical analyses of causationmerely provide “indirect” evidence of the factors leading towar,14 and that “in-depth qualitative research has the advan-tage over quantitative methodologies in that it can unpack theexact mix of causes that go into particular cases.”15 These claimsseem to suggest that, while quantitative analyses are restrictedto gathering information about variables already postulated asrelevant, case study research can uncover evidence of hereto-fore unsuspected considerations impinging on governments’decision processes when they examine government documentsor other sources of evidence about the decision process. If weagree with Copeland that this is a deficiency, it is one that isnot inherent in statistical analysis, as one can always expandthe data-set and re-analyze the data if one has happened uponnew information deemed relevant. But the extant quantitativeliterature is only “indirect” because of the kinds of variablesthat quantitative research has generally considered, and whichare featured in his discussion of quantitative research in chap-ter two: they are publicly observable characteristics of gov-ernments or nations, not variables chosen to capture the na-ture of the information flow reaching high-level decision mak-ers. As such, the quantitative models implicitly assume thatthese publicly observable national features affect all decision-makers in a uniform way—differences from time to time or gov-ernment to government in how information is processed areassumed to be theoretically unimportant, though from an em-pirical standpoint their possible existence might motivate afixed-effects approach in a pooled model. Copeland’s assess-ment suggests that he realizes that something is lost when the

14 Copeland 2015, 74.15 Copeland 2015, 53.

decision process is not systematically examined, but the fail-ure to do so is not due to the choice of research method. Forexample, a text-as-data researcher could upload a large corpusof foreign policy documents and analyze them statistically,and even evaluate how changes in the content of the corpusare related to changes in policy decisions. This evidence wouldbe a good deal more “direct” in the sense that it is taken frominternal decision processes, but what has changed is less thereliance on statistical method than the model used and thechoice of variables to be included.

Using government documents and other sources to tracethe evolution of the decision process in a case could provideinsight into the inner thinking and planning of leaders. Thisseems to be Copeland’s approach, and he claims in his theorychapter that “the best way to investigate [expectations] isthrough careful documentary analysis.”16 This is what he doesfor the cases where the supply of primary documents is rela-tively abundant. When it is not, he relies on historian’s ac-counts. Because these are typically based on a reading ofprimary documents, they provide evidence about the decisionprocess, but with an intervening layer of analysis. Since themethods used by individual historians to support their judg-ments about the decision process are typically not recover-able—they seldom have “methods” chapters in their books—we are left, as Copeland is, with the choice of either ignoringtheir conclusions, or else treating them as a panel of experts,each of whom is highly knowledgeable about the details of acase, but who is not generally in a position to help us recon-struct the research process that led to their conclusions. In ascholarly age that is highly concerned with the transparencyand reproducibility of the research process, this seems like asignificant limitation, but another feature of the current era—the exploitation of “crowdsourced” judgments and actions insupport of research—provides a way to think about how alarge body of historical writings could be pressed into serviceto support research on international relations even when thebasis for any single judgment is unclear. However, if any re-search strategy deserves to be described as “indirect”, thisone certainly does.

How should we assess the threats to the validity of causalinferences drawn from archival material (or, in some cases, fromothers’ readings of the archival material)? Copeland does dis-cuss the possibility that one’s own theoretical framework fil-ters the reading of the case material so as to exclude discor-dant information. “The only thing one can do, therefore, is toseek to be as self-conscious as possible about such potentialbiases.”17 Although he adds that we need “objective handlingof documentary evidence” and a “careful methodologicalsetup,”18 some more specifics about these points would bewelcome. Trying to be honest is never a bad idea, but it is notenough: document creation, distribution, redaction or destruc-tion, and public release can all be viewed as the product of

16 Copeland 2015, 74.17 Copeland 2015, 95.18 Copeland 2015, 96.

39


strategic choices,19 and hence it is wise for scholars to adopttheir own strategies for coping with the challenges posed bystrategic release and denial. The preserved and released writ-ten record of foreign policy decisions over-emphasizes “na-tional interest” justifications for policy choices,20 while de-emphasizing the influence of informal communications, espe-cially with private actors (in the US case, Foreign Relations ofthe United States, the official compendium of declassified ma-terial, rarely includes any documentation of private communi-cation with US government officials over foreign policy). Italso tends to slight the importance of covert operations, which,along with signals intelligence and code-breaking, are typi-cally among the most closely held material and are often re-leased very late or not at all.

Conclusion

Copeland’s book opens the door to three significant accom-plishments in the development of a deeper understanding ofinternational relations. First, it clears a path forward towards arealist-idealist theoretical synthesis by identifying a set of con-ditions that could account for why each of these two generalorientations successfully describes international politics onlypart of the time. Second, it re-opens discussion of a topic thatwas at the forefront of theorizing about international relationsin the period before the Cold War and the intellectual domi-nance of balance of power thinking—the relation of “power”to “plenty.” Paul Reinsch, often regarded as the founder ofmodern American scholarship on international relations, openedhis discussion of world politics at the dawn of the twentiethcentury not with a discussion of naval competition or Euro-pean alliance systems, but with a map of the Chinese railwaysystem.21 Jacob Viner inaugurated the journal World Politicswith an extended discussion of how scholars of the previouscentury had struggled to make sense of the relation betweenthe pursuit of wealth and the pursuit of politico-military ad-vantage.22 Albert Hirschman developed a highly influentialtheory of how Nazi Germany used its trade policy not simply toacquire wealth, but as an instrument of national power.23 Theselines of inquiry receded with the onset of the Cold War, but notbecause they had been resolved. Third, his account directsour attention to the possibility and desirability of writing abehavioral theory of foreign policy-making that is richer andmore accurate than those based on a treatment that positsunitary rational actors. This is an exciting set of research ob-jectives to pursue, and we can thank Dale Copeland for light-ing the way forward.

19 Colaresi 2014.20 Anderson 1981.21 Reinsch 1925 [1902].22 Viner 1948.23 Hirschman 1945.

References

Anderson, Paul A. 1981. “Justifications and Precedents as Constraintsin Foreign Policy Decision- making.” American Journal of PoliticalScience vol. 25, no. 4: 738-761.

Axelrod, Robert. 1984. The Evolution of Cooperation. New York:Basic Books.

Colaresi, Michael P. 2014. Democracy Declassified: The SecurityDilemma in National Security. New York: Oxford University Press.

Copeland, Dale C. 2015. Economic Interdependence and War.Princeton: Princeton University Press.

Denis, Daniel J. and Joanna Legerski. 2006. “Causal Modeling andthe Origins of Path Analysis.” Theory and Science vol. 7, no. 1. Athttp://theoryandscience.icaap.org/content/vol7.1/denis.html (lastaccessed 6/23/2018).

George, Alexander L. and Timothy J. McKeown. 1985. “Case Stud-ies and Theories of Organizational decision making,” In Advancesin Information Processing in Organizations, Volume II, Researchon Public Organizations edited by Robert F. Coulam and RichardA. Smith. Greenwich, CT: JAI Press, 21-58.

Glymour, Clark. 1986. “Comment.” Journal of the American Statisti-cal Association vol. 81, no. 396: 964-966.

Hirschman, Albert O. 1945. National Power and the Structure ofForeign Trade. Berkeley: University of California Press.

Holland, Paul W. 1986. “Statistics and Causal Inference.” Journal ofthe American Statistical Association vol. 81, no. 396: 945-960.

Niles, H. E. 1922. “Correlation, Causation and Wright’s Theory of‘Path Coefficients’.” Genetics no. 7: 258-273.

________. 1923. “The Method of Path Coefficients: An Answer toWright.” Genetics no. 8: 256-260.

Reinsch, Paul S. 1925 [1902]. World Politics at the End of the Nine-teenth Century as Influenced by the Oriental Situation. New York:Macmillan.

Viner, Jacob. 1948. “Power versus Plenty as Objectives of ForeignPolicy in the Seventeenth and Eighteenth centuries.” World Politicsvol. 1, no. 1: 1-29.

Wright, S. 1921. “Correlation and Causation.” Journal of Agricul-tural Research vol. 20: 557-585.

_______. 1923. “The Theory of Path Coefficients: A Reply to Niles’sCriticism.” Genetics no. 8: 239-255.

40


Evaluating “Competing” Explanationsin Economic Interdependence and War

Sherry ZaksDartmouth College

Dale C. Copeland’s Economic Interdependence and War tack-les head on one of the central debates in grand IR theory:whether and how economic interdependence between greatpowers affects the likelihood of war. With realist scholars ar-guing that interdependence is a source of vulnerability and acause of conflict, and liberal scholars arguing that interdepen-dence is a source of stability and a mitigator of conflict, IR’smain paradigms remain unable to fully account for both themassive amount of interdependence among great powers andthe conflicts between them. Initially framing his book as a sortof middle ground, Copeland argues that the “impact [of inter-dependence] can cut both ways” and so we must shift thequestion to “when and under what conditions will the trade[...] between nations lead to either peace or military conflict?”1

Copeland argues that the bifurcating factor is whether leadershave positive or negative expectations about their country’sfuture trade relationships: anticipation of continuity or pros-perity should foster restraint, whereas anticipation of declineor cut-offs should prompt aggression.

The theoretical importance of this variable is nontrivialand Copeland successfully demonstrates through rich archi-val analysis that leaders’ trade expectations often play a cen-tral role in conflict initiation and restraint. However, the theo-retical framing and corresponding analysis suffer from tworelated shortcomings that leave many questions unansweredand ultimately detract from the potential contribution of thiswork. First, Copeland’s attempt to situate trade expectationstheory vis-à-vis realism and liberalism is at best ambiguous.Copeland is unclear about whether his framework is workingagainst, alongside, or as a part of the alternative explanationshe examines. This form of theoretical ambiguity has direct meth-odological implications. As such, the second shortcoming is aresearch design that is incapable of fully testing the range ofpredictions that follow from the respective frameworks and therelationships among them. I draw on the Relationships AmongRivals framework—a set of analytic guidelines for testingnested and competing hypotheses—to examine the theoreti-cal and methodological weaknesses and to offer suggestionsfor future work examining the role of trade expectations.

Trade Expectations Theory vis-à-vis Realism and Liberalism

Beginning with our training and evident up through our publi-cation standards, political science exhibits a strong preference—if not bias—in favor of novel theoretical frameworks that quash

Sherry Zaks is a postdoctoral fellow at the John Sloan DickeyCenter for International Understanding at Dartmouth College. She isonline at [email protected] and http://www.sherryzaks.com

1 Copeland 2015, 1.

out competing explanations. While this preference is under-standable in terms of seeking greater contributions, it oftenmanifests as an ambiguity in theoretical framing and an incom-pleteness in empirical analyses as authors erroneously frametheir theories as occupying a mutually exclusive position withrespect to alternative explanations.2 In reality—and much morefrequently—two theories may also be coincident (i.e., bothcould simultaneously account for the phenomenon, with evi-dence in favor of one not affecting the other), congruent (i.e.,not only do both account for the outcome, but evidence infavor of one theory also supports the other theory), or inclu-sive (i.e., one theory represents a novel extension of the other).3Without specifying how alternative explanations relate to oneanother, scholars risk dismissing alternative theories withoutsufficient justification and leaving readers with dubious guid-ance on how and when to use the different frameworks movingforward.

Unfortunately, Copeland is not immune from this criticism.At various times throughout the book, Copeland presents thethree main theories he addresses as mutually exclusive expla-nations, such as when he writes:

“[t]he main goal of the case studies is to test the logic oftrade expectations theory directly against its main alter-natives”4 (emphasis added), and

“[t]he empirical chapters have shown...the superior ex-planatory power of trade expectations theory over its ri-vals.”5

At other times, he frames them as coincident explanations, inwhich multiple logics contribute to the outcome:

“rare events in international relations [...] will be a func-tion of [...] different bundles of factors operating with dif-ferent causal force at different places and times,”6

or as congruent, in which evidence for one theory also sup-ports the alternatives:

“[B]oth liberals and realists have a point. They have justnot specified the conditions under which they might beright. [Trade expectations theory] will help resolve thisproblem,”7

“[t]rade expectations theory [...] offers a new variable [...]as a way to link liberal theory’s emphasis on the benefitsfrom trade with realism’s concern for the potentially sig-nificant costs of adjustment that a state would face were itto be cut off...,”8

At yet other times, Copeland portrays the relationship betweenthe three theories as inclusive, in that one theory constitutes a

2 Zaks 2017.3 Zaks 2017, 348.4 Copeland 2015, 50.5 Copeland 2015, 432.6 Copeland 2015, 77.7 Copeland 2015, 24.8 Copeland 2015, 429.


41


fully subsumed extension of another:

“At its foundation, [trade expectations theory] is funda-mentally realist.”9

In this section, I discuss the source and consequences of thisambiguity, identify the actual relationships between trade ex-pectations theory and its “competitors,” and lay out the impli-cations for research design and future work in internationalrelations theory.

Having a clear sense of where trade expectations theorysits relative to alternative explanations of conflict under inter-dependence is especially important in this context. This is thecase because Copeland does not merely test his variable againstsome other variables. Rather, he situates trade expectationstheory to take on the two main paradigms in international rela-tions in an attempt to explain one of the most high-stakesoutcomes. Any support for Copeland’s framework will neces-sarily have critical implications for the future of internationalrelations theory. As a result, this analysis demands explicitguidance on how (and whether) to apply realism, liberalism,and/or trade expectations theory moving forward. On this front,however, the reader is left wanting.

This ambiguity is primarily attributable to Copeland’s un-derlying ideological preference for realist theory. He arguesthat, by maintaining the foundational assumption that statesare primarily concerned with security (rather than materialgains), trade expectations theory is “fundamentally realist” innature.10 Upon closer consideration, however, this frameworkneed not be confined to either paradigm—arguably, one couldmake an equally compelling case that trade expectations theoryis fundamentally a liberal one.11 To be sure, where a new hy-pothesis can help bolster any theoretical framework, research-ers should be explicit about how it fits into an existing theory,what needs to change, and how to go about testing it. Main-taining paradigmatic preferences can—and in this case does—undersell the contribution of new theories.

Drawing on the framework for specifying the relationshipsamong rival hypotheses outlined in Zaks,12 I demonstrate be-low that the trade expectations variable represents a usefulextension to both economic realism and liberalism that shouldbe applied in contexts of great power dependence in order tomake more robust predictions about the likelihood and timingof conflict. Copeland’s framework is not intended as a replace-ment for either paradigm, nor does support for trade expecta-tions theory directly undermine its competitors. Moreover, thelogic of trade expectations is largely consistent with both para-digms and the predictions on the outcome differ only whererealism and liberalism fall short. As such, ruling out mutual

9 Copeland 2015, 27.10 Copeland 2015, 27.11 Specifically, Copeland acknowledges that dependence is in part

defined by the benefits of trade and that there are circumstances inwhich the benefits of trade outweigh vulnerability (35). He also de-parts from structural realist assumptions by acknowledging the roleof unit-level characteristics in one’s trade partner (39, 41).

12 Zaks 2017.

exclusivity is straightforward. The question now becomeswhether trade expectations is a variable that exists indepen-dently of realist and liberal logic, or whether this frameworkcan be incorporated into both of the paradigms. I address eachone of these in turn.

Turning first to realism, I argue that, in its current state,realism does not offer a unique hypothesis that accounts forthe variability in the likelihood of conflict under interdepen-dence. Therefore, it does not make much sense to think aboutrealism and trade expectations theory as simultaneously andindependently offering an explanation for the outcome. Rather,realism posits that the vulnerability resulting from dependenceshould drive states toward more aggressive tactics.13 Tradeexpectations theory incorporates this prediction, but addsnuance to it by considering both the benefits of trade and anassessment of the future. As such, the trade expectations hy-pothesis is both fully compatible with the realist paradigm andpicks up the slack where realist predictions fail. Trade expecta-tions theory therefore constitutes an extension that enhancesthe predictive power of the realist paradigm under conditionsof great power interdependence.

Liberalism, in contrast, offers a set of specific hypothesesabout the conditions under which conflict is more or less likelyin contexts of great power interdependence. Domestic-levelvariables (such as regime type or economy type), commitmentproblems, and the “shadow of the future” have all been ad-vanced by liberal scholars to account for interstate conflictand peace. It is worth noting that Copeland does not disaggre-gate liberal theory into those constituent hypotheses. I argue,however, that doing so is crucial to achieving precision in boththeoretical and methodological discussions. Trade expecta-tions theory is certainly not exclusive to the first two hypoth-eses—i.e., finding support for a leader’s fear of future cut-offswould not make it impossible for regime type or bargainingproblems to also play a comparable role. Both factors couldeasily coexist, but independently of trade expectations. Thus,Copeland’s framework is coincident with these two liberal ex-planations: any or all of the three can be contributing factorsto a given instance of conflict or peace and each must betested separately.

As for the “shadow of the future” hypothesis—the liberalnotion that two states’ anticipation of multiple future interac-tions helps promote cooperation and mitigate conflict betweenthem—14 trade expectations theory represents a congruent andinclusive extension to liberalism on this dimension. In essence(though he does not frame it as such) Copeland’s frameworkposits that the shadow of the future is a variable rather than aconstant: leaders do look to the future of their trade relation-ships to inform present behavior, but that future is not alwaysassessed favorably. Moreover, trade expectations theory drawson other liberal notions to account for variation, such aschanges in leadership or regime. Thus, trade expectations theorycan be framed as a modification and enhancement of theshadow of the future hypothesis, which means it can also be

13 Copeland 2015, 22.14 Fearon 1998.

42


fully integrated into the liberal paradigm by incorporating ad-ditional considerations (such as threat and security) into pre-dictions based on future trade assessments.

In sum, by constituting a compatible extension of bothrealism and liberalism, the trade expectations hypotheses rep-resent an unprecedented contribution to international relationstheory. The trade expectations framework borrows key insightsfrom competing paradigms in order to better parse the effect ofinterdependence on the likelihood of conflict. From a theoreti-cal standpoint, Copeland implicitly demonstrates that both para-digms are excessively blunt, but in different directions: realismis naively pessimistic about the effects of interdependenceand liberalism is naively optimistic on the same score. Then, hemanages to develop a single theoretical framework capable ofadding explanatory and predictive accuracy to both at once.Unfortunately, Copeland’s predisposition toward realism andambiguous framing of where trade expectations theory sitswith respect to alternative accounts of conflict overshadowsthe true scale of this contribution. In reality, scholars fromboth traditions can proceed by incorporating the insights fromtrade expectations theory into either lens.

Methodological Implications

This more nuanced identification of how trade expectationstheory relates to alternative explanations of conflict under in-terdependence has critical implications for research design andinference. I now seek to show that the nature of the relation-ships among competing hypotheses shapes the type of testsone can conduct as well as the inferences one can draw.

Unfortunately, Copeland’s empirical analysis falls shorton two fronts. First, his analysis and the implications are ob-fuscated by the tendency to force tests of multiple theories toconform to the “testing Theory X against Theory Y” construc-tion. In reality, this purported goal of Copeland’s empiricaltests is only possible under a limited number of circumstancesand is constrained directly by the relationships among com-peting theories. Second, drawing on the central tenets of eachtheoretical framework, I argue that his identification and selec-tion of cases is too narrow to address the full range of hypoth-eses and relationships he wishes to test.

Constructing Tests

Copeland notes that the goal of his empirical analysis is notonly to establish the overall importance of trade expectations,but to identify how strong of a role they played relative to the“bundles” of other factors present in each case.15 Yet, one ofthe primary insights that follows from the characterization ofrelationships among the “competing” hypotheses above isthat testing the relative “causal force” is only possible whentwo explanations can simultaneously but independently bringabout the outcome (i.e., under relationships of coincidence).Under mutual exclusivity, we would not consider asking aboutthe relative force, since only one explanation is possible at atime. Instead, researchers who want to get some leverage onthis question are limited to testing 1) which explanation is

15 Copeland 2015, 77.

present in a given case, and 2) which theory explains moreoutcomes across the universe of cases (i.e., the frequencywith which one or the other has explanatory power). Underinclusive relationships (i.e., when Theory Y is integrated intoTheory X as an extension), the question of relative salience isnon-sensical: if one is extending an existing theory, we wouldnot consider the two theories to be separate. Researchers arelimited to testing whether the expanded theory 1) performsbetter (either in terms of prediction or explanation) in a givencase, and 2) whether it generates more accurate predictionsthan it did prior to the extension.

Thus, to get traction on the analytic purchase of tradeexpectations theory vis-à-vis existing explanations, Copelandmust construct tests and select cases that are tailored to theconstraints described above. Testing trade expectations theory“against” realism is a matter of constructing tests to identifywhether incorporating this variable enhances realism’s predic-tive (and explanatory) capacity. Given that trade expectationstheory is best situated as a modification and extension of real-ism, the tests should be constructed to assess whether thisexpanded version of realism predicts and explains more caseswith better precision than it did in its original state.

The depth and breadth of evidence Copeland analyzes toassess leaders’ evaluations of future trade relations is impres-sive by any measure. However, the analysis becomes clunkydue to the unclear implications of supporting one theory overanother. By forcing himself into the “testing theory X againsttheory Y” box, Copeland overlooks and/or undersells his owncontribution to both realist and liberal theories. In some cases,for example, he concludes that “trade expectations theory andeconomic realism [are] best,” while in other instances, he ar-gues that his framework outperforms realism.16 These conclu-sions, however, leave two questions unanswered that harkendirectly back to the relationship between his framework andrealism. First, how are researchers supposed to proceed—should they be employing both theories or different ones un-der different conditions? Second, how can we reconcile thattrade expectations theory can simultaneously outperform real-ism yet also be a “fundamentally realist” theory?17 Thus, whilehis tests demonstrate in some sense that the current state ofeconomic realism is underpowered and that trade expectationstheory performs better, his research design is incapable of test-ing the extent to which the trade expectations hypothesis en-hances the predictive power of realism in the context of inter-dependence. As a result, his readers may likely come awayfeeling as though they learned something, but uncertain ofhow to apply it in a systematic way as they move forward.

Testing trade expectations theory with respect to liberal-ism is necessarily a more complex task. Since liberalism posesa variety of explanations intended to parse conflict and peaceunder interdependence, he must proceed carefully with caseselection, evidence gathering, and analysis to test the full rangeof variables and predictions. Copeland’s assessment of liber-alism highlights additional problems with his analysis. First, as

16 Copeland 2015, 88 (in Table 2.7)17 Copeland 2015, 27.

43


I alluded to previously, Copeland does not fully disaggregateliberalism into its three constituent hypotheses regarding howconflict (and peace) can result from interdependence. Sincethe first two liberal explanations—the role of unit-level vari-ables and bargaining dynamics—are coincident with histheory,18 each should have been addressed by searching forconfirming and disconfirming evidence for each hypothesisindependently.

The second issue with testing trade expectations againstliberalism is that Copeland at no point acknowledges theshadow of the future hypothesis, which is both surprising andproblematic given his steadfast calls for a more dynamic andforward-looking theory of state interactions. Because liberal-ism includes a dynamic theory, it is all the more important forCopeland to explicitly address how trade expectations theorydiffers—whether it should be incorporated as a modificationor replacement, and what the implications are for the shadowof the future concept moving forward. By not disaggregatingliberalism into its constituent theories, the reader is often leftwith a surface-level rejection of the liberal paradigm, but withno specific identification of where liberalism goes wrong.

An Incomplete Universe: Case Identification and Selection

The central tenets of each theory as well as the relationshipsamong them should be synthesized to inform how cases aredefined and selected for a given analysis. Datasets ought tobe constructed according to theoretically-informed inclusioncriteria to guard against selection effects and confirmation bias.I argue that the universe of cases Copeland identifies is incom-plete in a way that systematically favors trade expectation andrealist explanations (yet still prevents even the fullest testingof these hypotheses). I do feel it is important to begin thisdiscussion by giving Copeland full marks on the ambition ofthis analysis. Many qualitative analyses include one tenth thenumber of case studies—or fewer—found in Copeland’s bookand are still considered thorough. As such, the following cri-tique is not to argue in favor of increasing the N for the N’ssake, but to argue that the universe is systematically incom-plete: the cases it omits are likely to reveal something differentabout the way the world works.

The problem manifests first by defining “case periods” asnecessarily comprising “great power cases of crisis and war”and requiring that they be “marked by a particularly salientissue or important event.”19 By restricting the sampling frameto interactions that eventually culminated in a crisis, Copeland,despite his argument to the contrary, is selecting out cases oflong-term stability. The justification he puts forth—that byconsidering the non-conflictual periods surrounding the cri-sis, he does address negative outcomes on the dependentvariable—20 only goes so far to mitigate this problem. Whilethis tactic demonstrates that trade expectations theory outper-forms previous explanations in terms of predicting the specific

18 I.e., finding support for trade expectations theory has no effecton the validity of these other hypotheses.

19 Copeland 2015, 79.20 Copeland 2015, 77.

timing of a given conflict, it still only examines instances inwhich crisis was imminent. Yet, for every great power conflictthat arose in the context of economic interdependence, onecan imagine that there are many other interdependent dyadswhere no conflict ever occurred.

The omission of non-conflict interactions is problematicon several dimensions. First, it narrows the scope of cases inwhich Copeland could and should be testing the predictivevalue of trade expectations theory. According to the frame-work, when states have positive assessments of future stabil-ity in their trade relationships they should be less likely toengage in conflict. Consequently, non-conflict cases are justas important to evaluating trade expectations theory as thegreat power crises are. Furthermore, these cases are especiallycritical to evaluating how trade expectations theory performsrelative to liberalism. Since the foundational assumptions inliberalism lead scholars to predict that on the whole, economicinterdependence is a mitigator, rather than a facilitator, of con-flict, rejecting the liberal paradigm in favor of trade expecta-tions theory is nearly impossible without engaging liberal theoryon its own turf. This issue is the qualitative incarnation of thenonignorability problem addressed in King: the missing datais in part a function of the explanatory and outcome variables.21

Specifically, the outcome No Conflict—which is a crucial partof the theoretical frameworks under examination—is highlycorrelated with missingness in Copeland’s dataset.

Conclusion

Economic Interdependence and War is a behemoth of empiri-cal work on two centuries of great power conflicts. Moreover,Copeland successfully demonstrates that leaders’ assessmentsof the future stability of their trade relationships play a centralrole in many of these crises. Unfortunately, a critical theoreti-cal oversight and the corresponding methodological implica-tions undermine the extent of the book’s contribution. As aresult, readers are likely to come away certain they have learnedsomething, but uncertain of precisely how to synthesize theirnew insight with their existing knowledge.

In this review, I have highlighted the importance of identi-fying the relationships among competing hypotheses andbreaking out of the ingrained frame of the mutual exclusivity.Indeed, Copeland himself is clearly aware that in many cases,multiple explanations may work at once to bring about an out-come. But, by lacking a comprehensive framework for identify-ing the range of ways hypotheses may relate to one anotherand the implications for research design and analysis, eventhe most careful scholars can fall back onto the crutch of “test-ing Theory X against Theory Y.” Identifying the relationshipsamong competing explanations informs with greater specific-ity the types of tests that one can conduct and, in turn, shouldinform how the universe of cases is defined and how specificcases are selected for analysis.

When Copeland presents trade expectations theory in atheoretical context, his characterization of the relationship be-

21 King 2001, 51.

44


tween his framework and its predecessors leans more towardcoincidence or congruence (i.e., that the explanations may worktogether or that one may be an extension of the other). Yet,when he moves on to discuss his research design, the tradeexpectations hypothesis is suddenly framed as occupying anexclusive position. This level of ambiguity about where tradeexpectations theory sits relative to realism and liberalism cas-cades through the analysis and has both theoretical and meth-odological implications. On the theoretical side, this framingleaves the reader wondering what it is we are going to get fromthe book: is it an extension to realism or liberalism? Is it afusion of both? Is it a replacement? On the methodologicalside, readers are left wondering how, for example, to test atheory that is both “competing with” realism and “fundamen-tally realist.”

Copeland concludes by positing that “trade expectationstheory resolves the problems for established liberal and realisttheories.”22 In reality, however, his analysis falls short of thisgoal. Despite providing an impressive wealth of evidence infavor of the trade expectations hypothesis, the theoretical andmethodological ambiguities muddy the inferences we can drawand the implications for future research. The crucial addendumis that the trade expectations theory has the latent potential toachieve this resolution between the realist and liberal para-digms and to guide future applications of both. By recastingtrade expectations as an extension of both theories, Copelandcould demonstrate precisely the need to modify both main IRparadigms to more accurately deal with conflict under interde-pendence.

References

Copeland, Dale C. 2015. Economic Interdependence and War. Prince-ton: Princeton University Press.

Fearon, James. 1998. “Bargaining, Enforcement, and InternationalCooperation.” International Organization vol. 52, no. 2: 269-305.

King, Gary, James Honaker, Anne Joseph & Kenneth Scheve. 2001.“Analyzing Incomplete Political Science Data: An Alternative Algo-rithm for Multiple Imputation.” American Political Science Re-view vol. 95 no. 1 :49-69.

Zaks, Sherry. 2017. “Relationships Among Rivals (RAR): A Frame-work for Analyzing Contending Hypotheses in Process-Tracing.”Political Analysis vol. 25 no. 3: 344-362.

22 Copeland 2015, 428.

Theoretical and MethodologicalReflections on EconomicInterdependence and War

Erik GartzkeUniversity of California, San Diego

What causes peace? Dale Copeland’s detailed and ambitiousbook, Economic Interdependence and War, has an answer. Atleast under certain conditions, the ties that bind nations to-gether in webs of commerce can lead them to prefer to avoid, orat least delay, active conflict. The basic claim of a commercialpeace is hardly new. Scholars like Montesquieu and Smithdetailed this connection at the dawn of the commercial age.Two features make the book distinctive and might be said toadvance this timely and important research agenda. First,Copeland sets out to integrate ideas of a negative commercialpeace that originated in liberal internationalist theory into aRealist neoclassical framework. Two lines of thinking aboutworld affairs that were dialectical may perhaps have reached asynthesis. To achieve this synthesis, Copeland emphasizesprospective thinking about commerce by leaders in evaluatingtheir nations’ foreign security policy: It is the anticipation oftrade, rather than its mere concurrence, that Copeland deemscritical in achieving major power peace. Second, the book isbroad and ambitious in its empirical scope as well as its theo-retical domain, offering panoramic analyses of eras more akinto period narratives than to the narrowly focused vignettesthat are common in the social sciences.

The world is complicated, made more complex seeminglyevery year by the march of trade and technology. This com-plexity suffuses both Copeland’s subject matter and his at-tempt to explain the ties between commercial relations andconflict in international affairs. Copeland can only be praisedfor taking on such a challenging subject. At the same time, it isthe responsibility of an author to clarify and simplify for hisreaders, and to prevail in its position through logical and em-pirical precision. I will try to keep things simple here, thoughregrettably this means I must gloss over or omit the rich nu-ance that is abundant in this 504-page text.

The Liberal-Realist Synthesis

Wealth from trade, to say nothing of the arms trade itself, canfacilitate war, but as liberals emphasize, it also creates commonties that constrain actors, making war costlier. It is this interde-pendence, and its pocketbook effects, that are said in liberaltheory to prevent war. There are numerous versions of thebasic argument, so many in fact that one could fill several

Erik Gartzke is Professor of Political Science and Director of theCenter for Peace and Security Studies (cPASS) at the University ofCalifornia, San Diego. He is online at [email protected] and http://erikgartzke.com.


45


books with the varieties of liberal peace theory.1 Whatever thedetails, however, the basic form is one of making war relativelyexpensive (or peace relatively cheap). Commercial loses asso-ciated with conflict pose costs for leaders and countries (econo-mists call these opportunity costs) that, if anticipated, canlead them to prefer peace more often. It is important to notethat in no form does this argument fail to put itself in the future.If these costs, or the opportunities to avoid them, are in thepast then subsequent action can do no good (or cause nomore harm). It is only when one’s actions today shape out-comes tomorrow that opportunity costs matter for decisionmaking. Similarly, a failure to anticipate opportunity costs makesthem irrelevant.

In Copeland’s conception, as in other opportunity costarguments, nations still want to fight. However, the text arguesfor a key theoretical innovation in focusing on future eco-nomic gains or losses. But the distinction between future andpast trade that Copeland emphasizes is not salient. Specifi-cally, whether states are currently not trading, but will if theyremain at peace, or whether states are currently trading andwill continue to do so if they do not fight, is said to provepivotal in Economic Interdependence and War. But there isnothing in the opportunity cost framework, or in Copeland’sparticular version of the theory, that would make distinguish-ing among different pasts germane. Both are functionallyequivalent. In each case, it is anticipation of the future thatdrives choices.

In basic analytical terms, the effect of trade in the oppor-tunity cost framework is like that of altering the payoffs in thefamous Prisoners’ dilemma game to transform it into the almostequally famous chicken game. If the cost of war goes upenough, both sides prefer conceding to mutual destruction. Ofcourse, each side still prefers an outcome where it is the adver-sary that swerves or backs away. 2 A similar challenge existsbetween trade partners. One need to look no further than thecurrent headlines to see two states, the United States andChina, trying to cow one another over the imposition of riskyactions made possible not in spite of, but because of, trade.

This does not preclude Copeland’s conception of themechanism underlying trade’s effect on peace from operatingeffectively. However, at least three problems with this logicremain. First, how much is enough? Big wars are very expen-sive. It is a rare case where two nations’ bilateral trade is on ascale that would make eliminating all trade (let alone some of it)sufficiently costly to fundamentally alter the accounting priceof a war, or even of a significant militarized dispute, once stateshave reached the point where they consider military escalationof a dispute a rational choice. In my own research, I havefound that in the post-World War II era, less than four percentof all dyads look this way.3

Second, there is the vexing question of endogeneity, herein the sense of reverse causation. Endogeneity implies that the

1 For examples of such books, see Doyle’s Ways of War and Peaceand Russett’s Grasping the Democratic Peace.

2 Snyder 1961.3 Gartzke Typescript.

states that are most likely to fight are least likely to trade to-gether. For example, the United States worked hard to preventU.S. and other western merchants from trading with the SovietUnion. Copeland dismisses these and other endogeneity prob-lems rather blithely. It is not clear why, since they are of con-siderable concern in the literature.4

Third and most vexing, I do not think Copeland’s theorysucceeds at reconciling contrasting predictions of conflict cau-sation used by different paradigms. The admittedly implicittheory of war underlying liberalism emphasizes a cost-benefitcalculus. Realist theories rely on some form of power relations.Realism does not intensively consider war costs (though theseappear in the nuclear domain). So, how does one reconcile thepredicted effects borne of opportunity costs with relativepower, in an environment where prosperity itself may prove tobe a problem? Copeland’s solution is ingenious but confused,and confusing. The simplest description is that, in prospect,opportunity costs somehow manage to intersect with and rem-edy concerns about relative power. If a state expects futuretrade to make it more powerful, it will not fight today. If insteadstates perceive that they will be prevented from prospering,they can prefer war. “Such a state will tend to believe thatwithout access to […] markets needed for its economic health,its economy will start to fall relative to other less vulnerableactors.”5

This sounds very plausible, especially if one’s mind is stillfocused on the liberal logic of war, as well as peace. But thereare at least two problems in attempting to fuse liberal (costs)and Realist (power) conceptions in this way. First, let us as-sume for a moment that everyone can see the future equally, sothat the future is subject to common conjecture (everyone cansee and evaluate everyone else’s problem). If I know that bycontinuing to trade instead of fight I will be stronger in thefuture relative to you, then of course this is terrific for me. ButRealist you is not going to like this future. If instead you willgrow more from trading than I, then Realist me hates interde-pendence and you are the happy one. Looking into the future,if we agree on what we see, we cannot escape the Realistrelative gains problem.

The second problem with using the future to segue from afocus on costs to power has to do with perceptions. My initialsupposition about what leaders know about the future mustbe relaxed. Whether it is world affairs or labor relations, theorigins of disputes seem most often to lie in what actors do notknow, rather than what they do. In fact, the text is clear thatleaders do not know the future. One of the key features ofbeing forward-looking is that one can get things wrong—lead-ers may not have the same beliefs about their future economicprospects. If there is no common conjecture, then war inCopeland’s world depends on whether Realist leaders antici-pate compatible or incompatible effects of interdependence. Ifboth you and I think that we will be relatively advantaged bytrade, then we can remain at peace, at least for a while. If in-stead either of us is pessimistic about the magnitude of future

4 See, for example, Morrow’s “How Could Trade Affect Conflict?”5 Copeland 2015, 2.

46


economic gains, then we may prefer war. This of course meansthat the theory really hinges on the question of beliefs andtheir origins, since what leaders happen to anticipate deter-mines how they will behave, determining whether trade begetspeace or war.

There is an enormous literature on decision making andhuman cognition. It is unreasonable to expect an author tocapture all or even most of what other authors are writing. Butgiven the centrality of perception to assessing the accuracy ofpredictions from Copeland’s framework, some additional detailon how leaders form beliefs would be especially helpful. Here,the book says too little.

Material variables condition who gets what in this story,while the information actors have about material variables con-dition how they go about getting there. Copeland ends upwith this conclusion without consciously embracing it, dis-cussing the process of discovery about relative power andbenefits in a manner not unlike Bayes Rule, but without explic-itly addressing key insights that follow from tying uncertaintyto decision-making and conflict in this manner.

The result is no longer a story about opportunity costs. Itis instead a story about beliefs about opportunity costs, orindeed beliefs in general. If states have incompatible concep-tions of the effects of trade, whether real or imagined, this canlead them to fight even when the actual mutual benefits oftrade should have prevented conflict. Copeland notes thiseffect without tying it to the appropriate causal mechanism ofconflict. If what leaders believe about trade is at the core ofwhether they fight, then it is beliefs that drive conflict, notmaterial conditions.

In sum, Copeland’s book seeks to meld liberal commercialpeace theory with a Realist account of the origins of greatpower war. These two theories have fundamentally differentexplanations for the origins of conflict and thus are difficult toreconcile. Copeland’s solution is to look to the future, focus-ing attention on what leaders anticipate about events and cir-cumstances. However, this does not actually resolve logicaltensions between the two paradigms. Nor does it respond ef-fectively to fundamental criticisms of each perspective’s theoryof war. It also leads to important questions about the origins ofperceptions that are not resolved in the text. These structuralproblems are obscured under an enormous amount of descrip-tion and argument.

The Methodological Approach ofEconomic Interdependence and War

I thus turn, briefly, to addressing the extremely detailed empiri-cal research in the text. There is much in Copeland’s extensivecase studies that informs and provokes, making it impossibleto do it justice in a short essay. So I will just address a few keyissues.

A basic problem, which Copeland disputes, involves endo-geneity. The text concentrates on great powers. Separatingstates into big and small appears to be a hold-over from thebilliard ball days, and from an age when anything more than adichotomy overloaded the human imagination. But focusing

on great powers is likely to greatly worsen the endogeneityproblem discussed above: One of the factors determining ma-jor power status is economic size, and economic size is in-creasingly driven by integration into the global economy. Thus,Copeland’s sampling procedure incorporates his key indepen-dent variable.

A related concern about Copeland’s case selection, ascomprehensive as the empirical analysis might appear to be, isthat it inhibits our ability to assess alternative explanations.One way in which war can be averted, possibly helped bytrade, is if actors lack the basis for competition or conflict inthe first place. This is positive (rather than negative) peace.This is how the West was tamed, not by guns but by civiliza-tion. In settings as diverse as marriage, the workplace, and theEuropean Union, constituent actors have found sufficient com-mon cause to interact cooperatively. Institutions and normsbridge gaps not to compel, but to facilitate cooperation. Work-ers in knowledge industries cannot be effectively forced to becreative. Instead, they are given incentives to further the inter-est of their employers by sharing a stake in the firm’s future.Conquest in such an environment is counterproductive, sincethere is no practical way to ensure continued productivity andprofit except by allowing workers to remain unfettered by coer-cion, which undermines conquest.6 In the second half of the20th Century, the United States as hegemon has not conqueredto rule, possess and extract. Instead, it has led a coalition ofnations, made more powerful and willing based on prosperityand collective security. Common interests have made war inthe Western world all but unthinkable for over seven decades.Copeland recognizes this important peace-producing dynamic,but he instead focuses on explaining the tensions betweenWest and East.

Another issue has to do with inference in an inherentlyprobabilistic setting. The assertion in modern analysis is notthat states that do not trade always fight or that those thattrade never do. Copeland is mindful of this and too sophisti-cated to suggest that interdependence is more than one amongmany factors influencing international affairs. Still, this posestwo challenges for his chosen mode of analysis. First, the mar-ginal impact of a variable or process depends on how often itmakes a difference in terms of given outcomes in a populationof actors or circumstances. How do we know whether the casesCopeland has chosen to study are representative of tenden-cies in world affairs? How can we tell how much trade mattersif the examples he shows us are not representative? One indi-cation of a clear problem in case selection is his focus onrivals; one cannot expect to learn a great deal about peace bylooking disproportionately at those that fail to achieve it. Trademay operate at an earlier stage, preventing states from becom-ing rivals. Second, case studies, even a considerable numberof them, cannot clearly delineate tendencies.

There are also a number of admittedly technical and spe-cialized errors that will mar the text in the eyes of experts, thought

6 Examples of this type of argument abound. See, Brooks’s Pro-ducing Security; Rosecrance’s The Rise of the Trading State; andAngell’s The Great Illusion.

47


they are unlikely to be noted by general readers. Copelandclaims to have replicated statistical studies by other authors.But in doing so, he makes glaring omissions. For example, heincorrectly interprets interaction terms from tabular results anduses models to make out-of-sample predictions.

Another concern about the analytical methods is that thetext notes the importance of both exogenous and endogenousfactors in a leader’s choice of interdependence. This implies atwo-stage process and creates challenges for inference (evenunder the best of conditions). For example, it is not clear howthe text is able to resolve whether causes are endogenous orexogenous since this depends in part on a prior stage of cau-sation. If domestic politics is treated as an exogenous factor,for example, but the choice of regime type is influenced bytrade and other variables, then regime type is endogenous.Copeland identifies at least six exogenous factors as confound-ing variables in the analysis. The variables should be factoredsystematically into case selection, as well as explication withincases. Yet, this is difficult, given limited degrees of freedom (inother words, there are more variables than cases).

Finally, Copeland’s defense of his methodological ap-proach is that he is analyzing rare events. The discussion ofthe strengths and weaknesses of quantitative versus qualita-tive methods for assessing rare events in a multi-causal envi-ronment highlights basic challenges to inference facing allmethods involving observational data. Over-sampling on fail-ure (in this case war) does not necessarily tell us more aboutthe causes of peace, especially if one believes that causationis multiple and jointly contingent. And there is lots of peace inthe world, for a multitude of reasons, so to the extent thatCopeland in fact has an explanation for peace, he is, fortu-nately, not studying a rare event at all.

Ignoring what one wants to explain because the otherthing is rare gets the whole enterprise backward. Tests of theo-ries of commercial peace involve contrasting interdependencewith many factors that interfere with conflict, such as distance,interests, capabilities, alliances, etc. By looking mostly just atfailures, one cannot distinguish the causes of the overwhelm-ing proportion of successes. Again, because the discussion isso detailed, the words sound right, and many readers have nobasis on which to doubt the veracity of the author’s claims,highly uncertain inferences may be believed as assertions offact.

Conclusion

Interdependence as a setting for research is challenging (somemight say daunting), precisely because the processes involvedare multifaceted and complex. Copeland’s book is ambitious,and in the best sense of that word it moves research forwardby encapsulating both insights and areas deserving additionalattention. I have focused disproportionately on areas where Ithink it falls short, but this is only because the goal of promot-ing peace and international stability through a better under-standing of commerce and conflict is clearly one of the mostimportant in the modern world. I applaud the aspiration of theauthor to make the world better, more peaceful. At the same

time, I fear Copeland’s particular approach, seeking to synthe-size two traditions that each have problems addressing thefundamental process of interest, and conflict itself, is not likelyto prove productive, at least partly due to the methodologicalchoices he makes. I hope that future iterations of this scholarlyobjective are even more careful in their causal logic and in themethods they use for the empirical assessment of that logic inour fractious world.

References

Angell, Norman. 1933[1912]. The Great Illusion. New York: Putnam.Brooks, Stephen. 2005. Producing Security: Multinational Corpora-

tions, Globalization, and the Changing Calculus of Conflict. Prince-ton: Princeton University Press.


Doyle, Michael. 1997. Ways of War and Peace: Realism, Liberalism,and Socialism. New York: Norton.

Gartzke, Erik. “Interdependence Really is Complex.” Typescript.University of California.

Morrow, James D. 1999. “How Could Trade Affect Conflict?” Jour-nal of Peace Research vol. 36, no. 4: 481-489.

Rosecrance, Richard. 1985. The Rise of the Trading State: Commerceand Conquest in the Modern World. New York: Basic Books.

Russett, Bruce. 1993. Grasping the Democratic Peace: Principles fora Post-Cold War World. Princeton: Princeton University Press.

Snyder, Glenn H. 1961. Deterrence and Defense. Princeton: PrincetonUniversity Press.

48


Rare Events and Mixed-MethodsResearch: Shaping theAgenda for the Future

Dale C. CopelandUniversity of Virginia

I very much appreciate the opportunity to respond to threethoughtful critiques of the methodology of my book, Eco-nomic Interdependence and War.1 Timothy McKeown, SherryZaks, and Erik Gartzke offer important and constructive com-ments on the mixed-methods approach I adopted in the book.Adopting the positive spirit found in the critiques, I will notattempt a blow-by-blow rebuttal of their arguments, but ratherseek to show how some of their insights can be used to ad-vance the book’s original intention—that of building a distinc-tive approach to mixed-methods research for the study of rareevents in international relations (IR) and comparative politics(CP). The big take-away point from this essay is that there isno one best way to do social science research, even in a per-fect situation where we have a great deal of evidence availablefor both quantitative and qualitative analyses. The right over-all approach, and indeed the right balance or “mix” of quantita-tive and qualitative methods, depends fundamentally on thetype of phenomenon one is trying to explain and the type ofrelationship between independent and dependent variablesthat one is considering. Once we get a handle on these ele-ments, I will argue, many of the issues and concerns broachedby McKeown, Zaks, Gartzke and other scholars about the roleof qualitative research in the broader project of social sciencecan be addressed and at least partially resolved.

Two questions animate the following discussion. First, towhat extent does the type of event that a researcher wishes toexplain affect the selection and balance of different methodsused to explain this event? There are of course many ways tocut up the concept of “type of event,” but I will focus on twodimensions in particular: whether the event is rare or common;and whether the event is brought about by the decision ofindividual leaders of groups or through macro social processesthat no specific individuals control (I will later bring in a thirddimension, namely, whether the individuals that may be re-sponsible for an event E are aware ahead of time that differenttypes of endogeneity may be at play). The second questioncomes out of the first, namely: Is the researcher primarily inter-ested in explaining specific historical events, or in explainingthe general effects that a particular causal variable will have asit varies across space and time? This question relates to the

Dale Copeland is Professor of International Relations in theWoodrow Wilson Department of Politics, as well as a Senior Fellowat the Miller Center at the University of Virginia. He is online [email protected] or at http://politics.virginia.edu/dale-copeland/.

1 Copeland 2015. To save space, I will often refer to the book asEIAW and its trade expectations theory as TET.

familiar distinction between causes-of-effects research andeffects-of-causes research.2 Yet when we combine this distinc-tion with the answers to the first question on the type of event,we can better understand why there is no one best way toapproach methodology or mixed-methods research in the so-cial sciences.

As I have argued in EIAW and in the introductory essayfor this symposium, when events are rare in IR and CP, they aretypically rare for a reason. That is, it requires a number ofconditions to come together simultaneously for the event E tooccur. Wars between great powers, as shown in the book, andsocial revolutions in CP, are two clear examples of such com-plex conjunctural causality. Consequently, although an “indi-vidually necessary, jointly sufficient” (INJS) bundle of factorsA, B, and C is needed before event E comes about, differentbundles of factors may also lead to E (multiple causality orequifinality). Yet once we’ve established that complex causal-ity is at work, we also need to know whether the event E is theproduct of individual decision-makers such as leaders, as ingreat power war, or a function of changes in larger social vari-ables, as with unanticipated social revolutions such as theEnglish, French and Russian revolutions.3 In situations whereindividual leaders choose to bring about E, especially when Ehas potentially dire consequences for their societies and theirown families, we will be naturally interested to know why theydid so, that is, the reasons that propelled them into such mo-mentous decisions.4 When, on the other hand, the event Ecomes about because of macro-historical changes in society,we will be more interested in establishing exactly how the se-quencing of these changes led to the crossing of a thresholdthat brought about the rare event E.5

In the study of rare events, because these events involvecomplex conjunctural causality, we will typically be more con-cerned with explaining why specific events came about whenthey did (a causes-of-effect approach) than in seeing to whatextent the variable of our pet theory explains changes in E ingeneral, all else being constant (the effects-of-causes ap-proach). Yet if these rare events are brought about by indi-vidual leaders, as is the case in this book, the answers to “why?”and “when?” can best be ascertained by a study of the internalthoughts, attitudes, and motivations of the decision-makersthemselves. In many situations, as with studies of voting orpurchasing behavior in the United States or other countries,surveys and interviews can give researchers access to theseinternal processes of decision-making.6 Given the nature ofgreat power politics, however, where there are no surveys of

2 Mahoney and Goertz 2006; Goertz and Mahoney 2012.3 Skocpol 1979; Goldstone 1991.4 None of the three critiques engages my point that analysts doing

research on complex causality need to determine what causal role A,B, and C are doing within the INJS bundle that leads to E. Documen-tary work is particular valuable here, since it can reveal whether afactor is propelling the leader to choose E (the “reasons why” he orshe acts) as opposed to simply constraining or facilitating the leader’sbehavior (72-73).

5 Pierson and Skocpol 20026 For an example of this, see Welzel and Inglehart 2007.


49


leaders and interviews are either impossible or unreliable (74),one must focus on an intensive investigation of the docu-ments that show what leaders were thinking as they changedtheir behavior (either toward war or risky hard-line behavior ortoward greater cooperation). In the study of macro phenom-ena such as revolutions, documents are less important sincethe events are driven by the larger changes in society, not bythe conscious choices of individual leaders.7

The above discussion suggests a 2 by 2 framework thatlays out four main ways to approach the study of differenttypes of events (what I will call for convenience the Approachesto Mixed Methods framework, or AMM). On the vertical di-mension of AMM, we have either a causes-of-effects or aneffect-of-causes approach (CE versus EC). On the horizontaldimension, we can distinguish between methods that seek toinvestigate “internal phenomena” (IP) such as thoughts andmotivations versus those that are interested in observing “ex-ternal phenomena” (EP) such as changing social and classstructures, population densities, resource levels, and thelike.The first approach, investigating causes-of-effects throughthe study of internal phenomena (CEIP), is the primary methodemployed in the book. CEIP is recommended when the type ofevent being explained is both rare and the product of the con-scious decisions of individual leaders of states or key socialgroups. Since rare events are likely a function of complex causalconjunctures, researchers will be most interested in establish-ing what particular bundle of INJS factors were at work forspecific events in history. What separates CEIP from a causes-of-effects logic that relies on observations of external phe-nomena (CEEP) is the way the events come about—by indi-vidual decisions of leaders rather than by a mix of social aggre-gates. In the book, crises/wars or the ending of cold wars aredriven by choices of leaders, and thus we need to look “inside

7 Indeed, in instances of this kind of phenomenon, the leaders wantto avoid event E (the revolution). This fact means that documentswill at most reveal the mistakes leaders made in not anticipating orcountering the larger social changes that, in retrospect, researchersknow led to revolution.

the heads” of leaders and their support staff to figure out whythey made a dramatic shift in behavior. But for things such asunexpected social revolutions, our explanations rest on thegeneral social conditions, not on the decisions of leaders perse. Here, the CEEP approach of such scholars as Skocpol8 andGoldstone9 thus makes sense.10

Both effects-of-causes approaches are, by definition, look-ing to explain the general effect of a particular variable overmany cases. For a typical large-N study in IR and CP, onetends to use the fourth approach, namely, the observing ofexternal phenomena such as trade levels, regime-type, and lev-els of development to establish the relative contribution of aparticular causal variable A to the overall explanation of varia-tion in event E. This is the ECEP approach to research. Yet, it isworth remembering that when reliable mass surveys of socialphenomena are available, one can also do effects-of-causesresearch with internal phenomena (ECIP).11 Documents canalso be useful in ECIP research, and indeed in my book I amalso interested, as McKeown notes, in the general effect ofchanging trade expectations across many different kinds ofscenarios. If I can show that rising or falling expectations ofthe commercial environment had a dramatic effect on the likeli-hood of conflict across many different scenarios, it suggeststhat TET’s main variable “A,” trade expectations, may be play-ing an important causal role within different INJS bundles thatlead to war or to the ending of cold war, even if A is not impli-cated in all bundles that do so. Nothing in the discussion so

8 Skocpol 1979.9 Goldstone 1991.10 Note that for civil conflicts that are not spontaneous but rather

the result of deliberate decisions by individuals leading distinct andwell-organized social groups, the CEIP method may be useful, espe-cially when combined with CEEP (thus the value of documentaryanalysis for the U.S. Civil War and the 1931-49 Chinese Civil War).

11 This is obviously a point most relevant to American Politics(AP) research but, as Inglehart’s work in CP shows, cross-countrysurveys can be quite revealing of the intervening social attitudes thatcan shape political outcomes (Inglehart 1990; Welzel and Inglehart2007).

Table 1: Approaches to Mixed Methods

Internal External_______________________________________________________________________________What one is interestedin finding out:

Causes-of-Effects 1. Causes-of-Effects 2. Causes-of-Effectsvia Internal Phenomena via External Phenomena.

(CEIP) (CEEP)

Effects-of-Causes 3. Effects-of-Causes 4. Effects-of-Causesvia Internal Phenomena via External Phenomena.

(ECIP) (ECEP)

Nature of the Phenomena Investigated:

50


far indicates one must choose only one of the four approaches.The task is therefore to determine what relative balance oneshould maintain between the four different modes of investi-gation, depending on the type of event being studied and howone anticipates explaining it.

The McKeown and Gartzke Critiques

Let me now turn to the three critiques before us, so that we cansee how they might contribute to a deeper understanding ofmixed methods research in IR and CP—in particular, to know-ing when and how to use emphasize certain methods, and withwhat relative “mix.” All three authors seem to accept that itmight be worthwhile to investigate “how often” a variable suchas trade expectations explains dramatic shifts within a casetoward crisis and war or toward the ending of a cold war ri-valry.12 In this sense, the effort to define a “universe” of cases,bounded by a time frame and the type of actor (great powers,for example), can be a valuable goal in qualitative work, since itallows us to see just how often one theory’s causal logic ex-plains specific cases better than alternative arguments.13 Yetdespite this agreed starting point, McKeown and Gartzke areconcerned that I overemphasize the differences between quali-tative and quantitative techniques, and the advantages of theformer. Moreover, in accordance with Zaks, they also see mycases as being overly shaped by my consideration of whathappens on the dependent variable. Zaks’s main concern, how-ever, has to do with my initial set-up. Aside from the fact thatshe believes I undersell the true potential explanatory powerof trade expectations theory, she suggests that I am uncertainas to what I am really doing with the theory. Sometimes I seemto be trying to use evidence to defeat rival theories (a “mutu-ally exclusive” approach). At other times, she argues, I appearto be showing that different theories can both be “right” forspecific cases, either because their variables work together (a“coincident” approach) or because the evidence could sup-port competing arguments (a “congruent” result). In what fol-lows, I will first deal with the McKeown and Gartzke critiques,and then turn my attention to Zaks’s more all-encompassingtheoretical and methodological points.

McKeown argues that I use a combination of the congru-ence method and regular process tracing to establish the em-pirical value of my deductive argument. The congruencemethod focuses on the co-variation between changes on anindependent variable and changes on a dependent variablewithin a case over time, or across cases. Process tracing differsfrom the congruence method insofar as it is more focused onidentifying the intervening processes and variables that link atheory’s main independent variable or bundle of INJS condi-tions to the behavior or outcome of interest.14 McKeown isright that I employ both methods. But his analysis misses acrucial point: namely, that when the type of event being stud-ied is a rare event that comes about through the consciousdecisions of a few powerful men or women, the way one uses

12 Copeland 2015, 77-78; 92-93.13 Copeland 2015, 75-76.14 See George and McKeown 1985; George and Bennett 2005.

congruence or process-tracing methods is quite different fromthe way one studies non-rare events or macro-historical events.I am indeed interested in the overall co-variation between tradeexpectations and the probability of conflict between great pow-ers and, in this sense, I want the theory to be relevant to thoseinterested in the effects-of-causes across many cases. It is forthis reason that I advocate analyzing “how often” a particularvariable from a theory can offer some explanation for specificevents over time and space. But because of the complex con-junctural nature of causality in great power conflicts, my mainfocus is on establishing explanations for particular phenom-ena, namely, dramatic shifts in behavior within designated caseperiods. Thus, the bulk of my work is devoted to CEIP analy-sis, the first quadrant of the AMM framework. In fact, we can-not establish “how often” a theory succeeds within an effects-of-cause approach until we’ve done the spade work on ex-plaining the individual cases—which is determined by the typeof event in question and the fact that it arises from complexdecision-making of individual leaders and officials. In short,for rare events with individual decision-makers, CEIP must comebefore ECIP and is a necessary step for the latter.

Yet the charge that I have a distorted set of cases becauseof ill-advised selecting on the dependent variable—a claim allthree authors make—misses what was said in the book aboutthe unique nature of events that are both rare and driven bythe conscious decisions of powerful elites. Complex con-junctural causality of this type means that elites, when ratio-nal, are fully aware of two fundamental forms of endogeneity:the risks of spirals and feedback effects when they act; and theability they have to effect changes in other key conditions ofstate action. Gartzke thinks that I ignore the issue ofendogeneity. The truth is the exact opposite: I deliberatelyincorporate endogeneity into the deductive logic of the theory,for the simple reason that individuals, who shape the majorevents of their nations, are only fully rational if they under-stand the likely effects of their actions on the future costs,benefits, and risks for their groups. Thus, the leaders of de-pendent states in my deductive set-up understand that if theyare too aggressive in their efforts to control access to rawmaterials and markets, they can cause other states to cut themoff, which might lead to a spiral of mistrust and hostility. Theyalso know that if they can increase trade, this may make theother more inclined to peace, which in turn could lead to afurther increase in trade (a virtuous cycle, rather than a nega-tive spiral). Finally, they know that if they do need to go to warto protect their trade spheres, they must first “get their ducksin a row,” that is, they must adjust the conditions necessaryfor initiating war, e.g., building domestic and allied supportand increasing arms spending.15

The realities of complex causality and endogeneity in thestudy of great power conflict means that in testing a theorythat captures these realities, one necessarily must focus on

15 Copeland 2015, 75 fn. 28. The importance of endogeneity andfeedback loops is covered throughout the book (9-12, 39-43, 48-49,74-75, 428-29), and is incorporated into a causal diagram summariz-ing the theory (49).

51


careful documentary process tracing of the major shifts in greatpower behavior, either toward crisis and war, or toward theending of a long-term rivalry.16 What we need to know, aboveall, is whether the actors acted for the reasons hypothesized—did they take their nations into crisis/war or decide to end andangerous rivalry because of changes in dependence and tradeexpectations, or for some other set of reasons? If our goal is toexplain dramatic shifts in behavior, we can only know the rea-sons for the shifts if we look at internal phenomena, that is, atthe beliefs, attitudes, and ends of the individuals making thedecisions. In short, we must emphasize CEIP over the otherthree methods, even if we supplement CEIP with them.

But why emphasize dramatic shifts in behavior as a basisfor defining case periods? None of the three authors analyzesmy stated criteria for defining a case period, or the reasoningbehind them.17 Nor do they discuss the fact that I deliberatelychose a “hard era”—the period from 1790 to 1991—for thetesting of trade expectations theory.18 That being said, it is an

16 I should note for clarity that Zaks wrongly states that I onlylook at actual conflicts (crises or wars) as rare events E’s, and that Iomit “non-conflict interactions.” Aside from the study of efforts toend the Cold War during the bipolar era of 1945 to 1991, I also discussperiods when states stayed peaceful in multipolar settings despitepotential new reasons for crisis or war (Russia in the upheavals in theEastern Mediterranean 1839-40 and Japan in the late 1920s, for ex-ample). As I state upfront, I am interested in changes in the probabil-ity of war due to dramatic shifts in behavior either toward crisis andwar initiation, or due to unexpected efforts to end or calm a dangerousrivalry (vii, 3 fn. 2; see also 92, fn.37 on why most détentes inmultipolarity are not examined).

17 In addition to using major shifts in behavior within an existinggreat power relationship or across regions as a basis for designating acase period, I also use the relative degree of independence of events asa criterion to ensure that years of struggle between great powers werenot divided up too finely. As I noted, it would make little sense totreat the series of great power wars from 1803 to 1815 as “separate”case periods, since they were all shaped by one overarching fact: thehegemonic aspirations of Napoleon. Treating them as independentevents would create a bias in favor of whichever theory did a good jobfor the 1803-15 period as a whole (giving it more “hits” than itdeserved). A quick scan of some of my key case periods shows thatmy criteria for case periods almost always works against my theory,rather than for it. Since TET works well for periods such as theNapoleonic Wars, the 1900-14 period in Europe, and the 1930-41period leading up to the Pacific War, counting these three periods as,say 10 or 15 separate cases would have greatly inflated the relativesuccess rate of my theory versus competing theories. It is worthnoting as well that I deliberately added in what might be seen aspotentially marginal cases that do not work for TET such as theBelgium Crisis of 1830-31 and the French and Austrian interventionsof 1815-22 in order to preempt charges of historical bias (91, 96).

18 As I pointed out, had I chosen the universe of cases for the 1550to 1750 period, critics would have argued that TET and economicrealism would have been more likely to work, since this was the hey-day of mercantilism. The period after 1790, on the other hand, wasincreasingly informed by the liberal economic thought of Smith,Ricardo, and modern trade theory. The 1790-1991 era thus consti-tutes more than a fair test of realist-based theories (TET and eco-nomic realism) versus established liberal arguments (76). The factthat the latter does poorly even during the era of liberal economics is

understandable concern that a book that focuses on explain-ing rare events such as crises/wars and ending of cold warrivalries might be downplaying, to use Zaks’s words, the regu-lar periods of “stability” prior to dramatic changes in greatpower behavior. So aren’t there obvious distortion effects fromemphasizing evidence based on the dependent variable? Thereare two ways to deal with such a question. The first is to admitthat some distortion inevitably comes into play whenever re-search spends the bulk of its time explaining more “extreme”events,19 but then to argue that one has to try to minimize anydistortions through certain techniques. In my case, I use twoimportant techniques to avoid overstating the value of mytheory or its causal significance. First, by examining the “uni-verse” of cases for a bounded time frame (1790-1991) and typeof actor (great powers), we automatically bring into the empiri-cal analysis cases that have nothing to do with economic in-terdependence—hurting the overall explanatory power of anytheory claiming a link between commerce and conflict. In thisbook, this was one-quarter or ten of the 40 case periods underconsideration (92-93).

Second, for each case period, I deliberately spent timeexamining the periods of “stable” relations prior to the dra-matic shifts in behavior that constitute the rare event. Thisallows us to see what shifts in independent variables led to achange in decision-making, and whether changing trade ex-pectations were critical to the new decisions, and if so, in whatway. Yet, as mentioned above, by using documentary processtracing and the CEIP approach rather than simply observingthe arising of intervening processes through CEEP, we canunderstand why the actors changed behavior, as compared tothe reasons they had for maintaining a stable relationship priorto that “decision point.”

In short, far from ignoring periods of “stability,” the studyspends a great deal of time—especially for controversial caseswhere my evidence cuts against traditional accounts—in try-ing to understand the stability and what led to shifts in behav-ior within case periods.20 Notwithstanding this clarification, itis true that the book spends most of its time on the qualitativestudy of why leaders choose to initiate crises and war. Yet, allthree authors miss the reasons laid out for such a focus. In anysituation of complex conjunctural causality and rare eventschosen by individual leaders, we would not expect a changefrom a “stable” pattern of behavior unless all the necessaryconditions in an INJS bundle were met.21 Furthermore, the

certainly a count against it.19 Collier, Mahoney, and Seawright 2004.20 For example, the case study of Japan’s shift from cooperation in

1920s to conflict and ultimately war in 1941 spans one and a halfchapters (chs. 4-5), given the case’s complexities and the difficulty inshowing the errors of traditional historiography. The book also looksat periods of tension that either did or did not shift to détente (1956-62, 1963-73, and 1984-91).

21 Note that stability is not defined here as “peace” per se, exceptinsofar as peace means the absence of war. In most periods of no-war(not-E), great powers are in a low-level struggle for territorial andmilitary position. They simply are not seeking costly wars or thecrises that could lead to them.

52


absence of any one of these necessary conditions would leadto the prediction that there would be no dramatic shift in be-havior.22 Overall stability in a great power relationship, in short,is highly overdetermined. Moreover, since a leader who hasdecided to engage in war may take years to endogenouslycreate the conditions necessary to execute it, what we reallyneed to know is when the decision for war or crisis is made,and why. This requires an investigative focus on the “con-flict” side of great power relations. For this to be done ad-equately, we must look at documents on the internal thinkingand planning of the key officials of a country. For example, wecannot understand why World War II broke out in Europe inSeptember 1939 without examining the period after January1933—when Hitler and his military laid down and implementeda plan to get ready for major war within eight years.23

Gartzke, and to some degree McKeown, seem to eye sus-piciously the effort to support trade expectations theorythrough the use of documents, as opposed to either large-Nanalysis24 or the process tracing of “publicly observable na-tional features.”25 Gartzke suggests that this opens up the flood-gates for the study of perceptions and thus misperceptions,and for subjective and cherry-picking analysis of what thedocuments say and mean. McKeown does not go quite thisfar, but he charges that much of my documentary process trac-ing relies on secondary sources and thus on the interpreta-tions of historians who do not reveal their methods. This means,for McKeown, that my evidentiary base for conclusions is“indirect” and thus similar to my claim that large-N quantita-tive work can only “indirectly” get at expectations. The impli-cation is clear: perhaps my documentary work is not so differ-ent from most qualitative process tracing and large-N quanti-tative work after all.

Seen within the full scope of what I was trying to do in thebook, and my upfront efforts to specify how to deal with suchpotential problems, these critiques are off the mark. First of all,it is the very fact that expectations are a form of perceptionthat forces us to see that we must study “internal phenomena”when the documents are available and reliable (CEIP over CEEP,and ECIP over ECEP as primary approaches). But I assume inthe deductive structure of the theory that leaders’ expecta-tions are rational, that they are reasonable given the informa-

22 All three critics overlooked an important footnote in the book,drawing parallels between the study of rare events in the medicalprofession, such as the “Sudden Unexpected Deaths” (SUDs) ofyoung people, and the method I advocate for considering rare eventsin IR and CP. SUDs are studied by taking a universe of actual deaths,to see “how often” the deaths were the result of asthma, heart failure,or stroke and the conditions that contributed to the event (such as hotdays and excess exercise). While researchers may examine the daysand weeks of the individual’s life leading up to the SUD, the focus onthe INJS conditions for the SUD itself is critical, since a stable situa-tion of “life” is so overdetermined for the average young person (see78, fn.34).

23 Copeland 2015, 135-140.24 See Gartzke’s essay for this symposium.25 See McKeown’s essay for this symposium.

tion available.26 Thus, it is up to a Gartzke to show that thisassumption of rational expectations is not upheld in the evi-dence, and that domestic and psychological distortions andfilters played a big role in the determinations of events. He hasprovided no such evidence (and of course he could only do sothrough opposing case studies).27

McKeown’s charge that I rely on the distorted opinionsof historians overlooks a key aspect of my methodological set-up. In pretty well every case where commercial variables wereimportant to the shift in behavior, I focused on original docu-ments either from archives (e.g., the Cold War cases) or frompublished document collections. Secondary sources were ex-amined to make sure the reader understood the scope of thehistorical debates, and to employ quotations from archivaldocuments discovered by historians when these documentswere not available to me. But by comparing my own assess-ment of original documents to the historians’ arguments andby looking for interpretations consistent with all, not just some,of the evidence, I was able to show that the documentaryevidence was often at odds with the views of the historianswhose work I was considering.

One might argue that this is perhaps easy to do, since thenotion that trade expectations drive behavior is almost com-pletely ignored by historians, who tend to focus on domesticand ideological factors to explain big events. But that is ex-actly the point. Political science provides us with new theoreti-cal lens to reexamine old events and to come up with newways to explain them. Thus, it is up to Gartzke, McKeown, andother scholars to show, with evidence, exactly why my inter-pretations of evidence are indeed distorted, and why their in-terpretations are better. The fact that Gartzke and McKeowncan neither point to specific examples of distortions in my casestudies nor show how a different set of cases or different evi-dence would lead to contrary interpretations suggests thattheir charges of distortion are at this stage merely specula-tive.28 I ended my methodological chapter with a call for schol-ars to jump into the historical debates about specific cases andto correct my interpretations when errors are clear, thus help-

26 Copeland 2015, 17 fn. 1; 27-28.27 He also does not discuss the fact that I accept that one must

often relax the assumption of rational expectations when one goesfrom a deductive theory to the real world (17 fn.1, 27-28, 27 fn.22).

28 Here, I would recommend a careful re-examination of my studiesof six case periods of Japanese history from 1880 to 1941, periodsthat occupy the bulk of the first three empirical chapters of the book.After an examination of both documents and the best secondary workout there for the six periods, I conclude that none of the commercialtheories can explain the Sino-Japanese conflict 1931-37. For the otherfive case periods, economic realism provides the best explanation fortwo (1880-95 and 1905-22), trade expectations theory for anothertwo (1895-1904 and 1938-41), and both liberalism and TET workwell for the period of relative stability in U.S.-Japanese relationsfrom 1922 to 1931. The fact that TET does well for the lead-ups tothe Pacific War and the Russo-Japanese War is perhaps significant.But it is hard to argue that the distortions of historians and lack ofattention to documents biased the results in favor of TET for the1880-1941 period as a whole.

53


ing the field to build an increasingly accurate sense of howoften different theories work and under what conditions. I sug-gested that only when this is done can we say that there is agrowth of knowledge in the IR field.29 I still hold to this recom-mendation, and I encourage Gartzke, McKeown, and otherscholars to participate in this long-term process of true knowl-edge accumulation.

My second point of rebuttal to McKeown’s implied chargethat there is not much that is new here is more fundamental,since it concerns the larger question of how to do mixed-meth-ods research well. Once we establish that we are dealing with atype of event that requires a deep investigation of the thinkingand planning of leaders via a CEIP approach, we must delegateto a secondary status the CEEP process-tracing that focuseson changing “national features”. But as we’ve seen, there isnothing in my set-up that says one cannot employ all fourapproaches simultaneously as ways to reinforce the confi-dence in the findings that arise out of the primary method ofCEIP. It is all a question of the relative mix or balance betweenthem. For most of my cases, I employed documentary processtracing while also inserting observations of aggregate nationalfeatures that would reveal the general social environment lead-ers were grappling with. For example, I brought in changinglevels of democracy versus authoritarianism across many ofthe cases to show how these macro variables might constrainleaders or in fact facilitate or drive their plans for conflict orpeace. This sounds like CEEP analysis, and it is. But it is verymuch secondary to CEIP precisely because of the type of phe-nomenon the book is trying to explain. Likewise, as mentionedabove, I wanted to determine whether changing trade expecta-tions shape conflicts across many different types of countries,regions, and historical periods. But my interest in establishing“how often” TET works well across the 40 case periods, whileakin to a medium-N congruence test within the ECIP approach,is only done after the CEIP operations are performed for ex-plaining specific cases.

Finally, I examined the “greatest hits” of the large-N quan-titative work on trade and conflict, to see to what extent tradeexpectations logic can explain the results.30 This employs thefourth approach of ECEP in a fruitful and supportive way. Bybringing in countries that are not great powers (and indeedmake up most of the observations in such large-N studies), wecan determine whether TET’s explanatory power is confinedonly to great powers. Moreover, using large-N can help toreduce any lingering anxiety that the results of the case stud-ies are a function of the cherry-picking of evidence or of selec-tion effects in general.31, 32

We can see in this way that a mixed-method approach thatexamines many or all of the cells within the AMM framework

29 Copeland 2015, 94-96.30 Copeland 2015 53-69.31 Copeland 2015, 3.32 Gartzke disputes the way I interpret the large-N results of

quantitative scholars studying trade and war (including his own re-sults). Yet his criticisms amount to vague one-sentence dismissalsrather than clear and sustained arguments.

can serve to reinforce confidence in findings that perhaps havecome out of an emphasis on one of the cells. But for scholarsof IR and CP, the question is always: Where should I devotemy effort, given that time is limited and the skills required fordoing each cell well are quite different and themselves taketime to develop? The typical way to answer this question, atleast in the IR field over the last two decades, has been to startwith large-N quantitative tests and devote the bulk of one’stime there, if the data is reliable and abundant. King, Keohane,and Verba’s 1994 book (“KKV”), because of its bias towardcausal inference based on quantitative methods, has for al-most a quarter of a century reinforced this inclination of gradstudents and young scholars to see quantitative methods assuperior, at least if good large-N evidence is available. Yet, aswe have seen, when events are rare and driven by a few pow-erful leaders, documentary process tracing via CEIP must bethe primary method to test competing causal logics (again,presuming reliable and ample documentation). Other methods,including large-N ECEP, may be useful supplements, but muchis lost if they are made to be foundational.33

The Zaks Critique

The Zaks essay offers quite a different critique of the book,which combines an effort to show that my theoretical argu-ment is actually more all-encompassing than I recognize withan argument that the theory is inconsistently tested. Zaks of-fers useful advice to help correct the latter, namely, that a re-searcher should be clear whether his or her theory, relative toalternative arguments, is positioning itself as mutually exclu-sive, coincident, congruent, or inclusive. As I see it, Zaks’sessay seeks to show that once the imprecise elements of thecurrent version of trade expectations theory, both deductively

33 This is not the place to jump into the debate on causal inference.But it is worth noting that KKV and most studies challenging itsimply assume that there is no real difference between documentaryprocess tracing and process tracing involving observable social aggre-gates. Yet, in the latter, causal inference is indeed mere “inference”(informed conjecture), since one is only observing co-variation andinferring a causal relationship, say, by trying to hold as much aspossible constant as one’s independent variable varies. As in thenatural sciences, more cases are better—and experiments the appar-ent gold standard—since no single case or observation really saysanything on its own (inference comes only from comparison). Yet indocumentary process tracing, as in murder trials, one can use evi-dence on the internal thoughts and intentions of individuals in spe-cific cases to, at the very least, eliminate possible causal explanations,even if one only rarely finds “smoking gun” evidence for a particulartheory about event E. This gives us huge leverage when debatingalternative causal accounts linking variables A, B, and C to a particu-lar event E. The documents may show that C did co-vary with E, butthat it played the role of constraining the individual to avoid E, ratherthan pushing them into choosing E (as pre-1914 domestic upheavalsmade German leaders wary about general war; see Copeland 2000:ch.3). Any theory arguing that C was the propelling reason for Ecould then be eliminated from contention. If one can use this ap-proach to eliminate all but theory J, and J fits the evidence, thelikelihood that J provides a good causal account of the arising of E isgreatly enhanced.

54


and empirically, are cleared up, the theory would have strongvalue as an extension of both liberal and realist thinking aboutthe mechanisms linking economic interdependence to conflictor cooperation.

I greatly appreciate Zaks’s effort to make TET a strongerand perhaps “grander” theory of international relations. Over-all, however, I believe Zaks pushes this agenda too far. If I wereto adjust TET along the lines Zaks advocates, there would bemuch that would be lost, both for scholars and policy-makers.In particular, instead of having three or four big approaches (ifwe include neo-Marxism) explaining some of the cases some ofthe time, we would have only a single reformulated theory(TET) trying to explain almost all of the cases all of the time. AsI will show, each of the competing arguments is indeed sepa-rate from TET, and by being so, brings something valuable tothe theoretical and empirical table.

Using her Relationship Among Rivals framework,34 Zakscorrectly points out that the book has some looseness of lan-guage that leads to confusion as to what trade expectationstheory is seeking to do for the field of IR. She notes that attimes I pose TET in a mutually exclusive way, as though it is adirect competitor to the alternative arguments of commercialliberalism and economic realism (and neo-Marxism). Yet, at othertimes, I suggest that variables from multiple theories mightwork together to explain particular cases (coincident explana-tions) or that they might be congruent in that the specificpieces of evidence might support multiple theories. Finally,Zaks indicates that I underestimate the value of TET, and thatit in fact has “inclusive” power—namely, that it can and doesincorporate the strengths of both liberalism and realism as itleaves behind their weaknesses. In recommending that I ac-cept its inclusive strengths, she suggests that I break out ofthe political science habit of setting up mutually exclusive “com-peting” theories that seek to “beat” one another. Indeed, ifTET is as powerful as she implies, it would probably be able toavoid empirical conclusions suggesting coincidence or con-gruence, since evidence that seemed to support liberalism orrealism would really be evidence for what physicists wouldcall “special cases” of the broader, more inclusive theory.35

Let me start by saying that I quite like Zaks’s Relation-ships Among Rivals framework and believe it can help hone aresearcher’s sense of exactly what he or she is doing. Indeed,I wish the framework had been available prior to the writing ofthe book—it would have helped me avoid falling subcon-sciously into language that implied that I was setting up amutually exclusive empirical competition between TET and thealternative theories. Zaks’s observations on this language areaccurate. But as the book’s methodology chapter and empiri-cal cases make clear, I am above all what Zaks might call a“coincidence scholar.” That is, I believe strongly that—giventhe type of event I am studying, namely, complex rare eventswhere individual leaders are conscious of the magnitude of

34 Zaks 2017.35 The most famous example is Newtonian physics, which now

operates as a special case within general relativity and quantum phys-ics.

their decisions both in terms of endogeneity and sheer socialcosts—different theories may indeed explain different aspectsof a particular case, and thus all be simultaneously “right” attimes. Across the 40 case periods, I regularly give two or moretheories credit for capturing parts of the causal reasoning ofleaders that pushed them into war/crisis or to end a cold warrivalry. Hence, for the 30 of 40 cases where commerce is in-volved in the shifts in behavior of great powers, the percent-age of “successes” for the main three theories—TET, commer-cial liberalism, and economic realism—adds up to more than100%. This of course would be impossible if establishing amutually exclusive approach to testing had been my true in-tention.

Nevertheless, it is important to distinguish between amutually exclusive approach to deductive theory buildingversus to empirical analysis.36 Because human beings are ableto hold multiple motives in their heads simultaneously, it ismore than likely in practice that they will sometimes act be-cause of “mixed motives,” believing that a particular action will“kill two or more birds with one stone.” In the study of Nazismand the start of World War II, for example, I was more thanwilling to admit that Hitler and the German military leaders mayhave had more than one “reason” for initiating war in 1939.Who, after all, would want to deny that ideological objectives,national pride, and personal desires for glory—and not simplysecurity concerns tied to Germany’s commercial dependenceon other great power spheres—were important in such a case?When such “coincidence” occurs, the goal of researchers is touse documents and the sequencing of aggregate social changesto evaluate two things: what causal roles did the variables ofdifferent theories have and with what relative causal saliencefor the final result? Here, I believe, a careful use of documentsand a study of the sequencing of social processes can revealanswers to these questions.37

Yet coincidence in the real world does not require, or evensuggest, that one should develop coincident deductive theo-ries. In the book, I was careful to state exactly how TET differsin its causal logic from liberalism, economic realism, and neo-Marxism. TET follows economic realism by assuming that lead-ers in anarchic realms care only for the security of the state andsociety, and thus are worried about their long-term power po-sitions in the system. In this sense, as I made clear, both TETand economic realism differ fundamentally from commercialliberalism, which assumes that leaders and social groups aredriven by the overall utility/welfare they derive from various

36 The four-fold RAR framework of Zaks, by this way of thinking,is really a four by two framework, since for each of her original fourcategories, one can distinguish between the deductive stage of theory-building and the more inductive stage of quantitative or qualitativeanalysis and testing.

37 Moreover, if we seek to stay objective, our answers can fruit-fully adjust over time through debate and reflection. For example, inthe case of World War II, my initial assessment that TET was thestrongest theory (Copeland 1996) changed as I thought more aboutthe documentary evidence and sequencing. At least for Hitler’s rea-soning, I argue in the 2015 book, economic realism offers the superiorexplanation (133-42).

55


outcomes.38 This fundamental difference in assumptions aboutactor ends leads to very different causal reasons for why thetrade environment might have positive or negative effects onthe likelihood of great power conflict. TET and liberalism mightboth predict, in terms of co-variation, that higher trade mightlead to a lowering of the likelihood of conflict. But while liber-alism argues that this is due to trade’s ability to act as a con-straint on preexisting domestic pathologies that might other-wise lead to conflict—the gains from trade increase the oppor-tunity cost of going to war—TET would argue that a stategrowing in power through trade is likely to believe that contin-ued peace enhances its long-term security. Liberals thus see afall in trade as unleashing the preexisting domestic patholo-gies—meaning that the state is propelled into war by unit-level variables, not by security fears—while TET’s logic indi-cates that falling trade tied to negative expectations for futuretrade make leaders inclined to war for fear of a decline in powerand a long-term loss in security.39

By keeping the deductive logics of TET and liberalismquite distinct—“mutually exclusive” in theory, if not in prac-tice—we can go to the documentary evidence and ask: Did theleaders make dramatic shifts in behavior for liberal reasons, orfor the reasons expected by TET or some other theory? Insome of the cases, such as World War II, we do see someevidence of “mixed motives” and thus coincidence. But formost of the cases analyzed, there was little or no evidence thatfalling trade unleashed preexisting domestic pathologiespresent in the initiating state. Rather, conflict was propelled bysecurity fears for the future, either for economic realist or TETcausal reasoning.

What about Zaks’s argument that TET is either congruentwith economic realism or that TET incorporates it because, inZaks view, economic realism does not have true variables, whileTET does? Unfortunately, the first notion—that TET is con-gruent with economic realism, or with neo-Marxism, becauseall three theories might predict an increase in conflict whenthey observe an increase in trade levels—overlooks some-thing crucial. As I explained in the book, even when theoriesmake similar predictions that factor A may lead to event E, if thetheories offer different reasons for why A leads to E, then thetheories are distinct and can be tested as such.40 The problemof course for large-N work, or for process-tracing involvingthe observation of external phenomena (CEEP and ECEP), isthat simply observing co-variation between A and E, or be-tween A and intervening variable B and then E, will not tell uswhich theory best captures the causal reasoning of leaders fora particular case.41

By unpacking the causal reasoning behind economic real-ism and neo-Marxism, we see that despite some similaritieswith TET’s causal logic, they are indeed separate and poten-tially powerful theories with real variables and should be treated

38 Copeland 2015, 6-7, 27.39 Copeland 2015, 7, 18-20, 33-38, 50.40 Copeland 2015, 23, 73.41 See footnote 33 (above in this essay) on causal inference.

as such.42 Economic realism starts with an assumption that inanarchy, great powers must assume the worst about adversar-ies. Given this, as a particular great power becomes more de-pendent on trade flows, its sense of vulnerability to cut-offsrises, and it will seize new opportunities to expand militarily toincrease control over access, thereby reducing this vulnerabil-ity. Thus, increasing dependence levels and heightened op-portunities for expansion are its core independent variables.Both economic realism and TET are “realist” in their assump-tions that anarchy and uncertainty, plus the search for secu-rity, will lead great powers to worry about long-term power. ButTET assumes that states do not assume “worst case,” andthus are sensitive to changes in the likelihood that other greatpowers will trade with them into the future (the “expectationsof future trade” variable is thus key). TET also incorporatesthe defensive realist notion that endogeneity matters—thataggressive policies by one state may lead others to cut thatstate off from trade, which in turn may only increase the state’swillingness to expand militarily. So, while economic realismpredicts more conflict when opportunities to reduce depen-dence by expansion arise, TET predicts that great powers willonly expand when new threats to commercial access arise andwhen these threats cannot be reduced by diplomatic mea-sures.43, 44

What we see from all this is that the way the differenttheories are specified will drive the way they are tested. De-spite some admittedly loose language, the methodologicalagenda set forth in EIAW is one of laying out the distinctcausal logics for the various deductive theories—to showunder what conditions they would expect factor A to lead toevent E or to not-E. Because there is often overlap in the pre-dictions of the theories—TET and liberalism both predict fall-ing trade will lead to conflict, while TET and economic realism/neo-Marxism predict increasing trade can also lead to conflict

42 The same holds for liberalism versus TET. Zaks contends thatTET can absorb the insights of liberalism once it properly capturessuch things as liberalism’s use of the shadow of the future. Yet, as Ijust reiterated, the two theories’ starting assumptions regarding actorends are fundamentally different. Note also that liberalism’s shadowof the future variable is held constant in TET by the assumption ofnon-myopic rational actors with no “discounting” of the future (36fn.38). TET, in short, holds unit-level variables affecting estimates ofthe expected value of future trade constant, while varying the likeli-hood of the other being willing to trade, while liberalism, consistentwith its unit-level focus, allows for myopic actors with high discountfactors, even as it stipulates that current cooperation will continue ifnon-myopic rational actors have clear means (including institutions)of punishing defections (17, 37).

43 Copeland 2015, 7-11, 21-23, 42-43, 50, 428-430.44 Neo-Marxism predicts that great powers will be more expansive

when trade rises, because key economic elites within a country fear afuture cut-off from access to cheap raw materials, markets, and placesto invest surplus capital (22-23). Neo-Marxism’s assumption thatgreat power decisions for conflict are driven by pressure from eco-nomic elites out for profit, not national security, makes it a distincttheory whose causal reasoning can be tested against the documents (Ifound only a couple of cases where this causal mechanism had anyreal empirical support).

56


—the only way to sort out which theory or theories does wellfor particular cases is to look at the documents. Observing co-variation between A and E/not-E through CEEP and ECEP willnot suffice, because the causal reasons for action remain hid-den.45

In the end, TET, commercial liberalism, economic realism,and neo-Marxism remain distinct theories that resist efforts tocreate overarching hegemonic (“inclusive”) approaches, andthis is a good thing. Having distinct theories allows us tocapture more fully the reality that in situations of complex cau-sality, there are almost always multiple possible paths to rareevents such as war. It also allows our documentary analysesto reveal the conditions under which different bundles of fac-tors will indeed create a path to event E. Finally, it makes ourtheories and evidence relevant to real-world policy-makers.Such policy-makers need to know more than simply that thepresence of a specific factor increases the overall risk of E bysome percentage—the kind of thing effects-of-causes research,especially ECEP, is useful for. Rather, they need to know, forthe specific case they are dealing with, which of the differenttheories best applies, given the conditions present. TET maydo well across a majority of cases, but this does not mean oneshould apply it to every current real-world case. Theories thatbring in the pathologies that may be present within dependentstates, such as liberalism and neo-Marxism, may be more ap-propriate, even if they don’t have broad success across thelarger universe of historical cases. The reasons why contem-porary Iran or North Korea might launch a war should trade fallmay be very different than why Japan attacked Pearl Harbor,and U.S. and European leaders need to be “armed” with differ-ent theories that facilitate a proactive comparison of the differ-ences.

Conclusion: The Way Forward

This essay has sought to show that there is no one best wayto test rival theories in IR or CP, nor is there one best way tobalance or “mix” methods to maximize leverage and the causalunderstanding of political phenomena. It all depends on thetype or nature of the event one is studying, and the type ofcausal relationship one seeks to investigate. Rare events ofthe kind that EIAW examines involve individual leaders ofstrong states that can not only choose to initiate crises andwar (or to end cold wars), but also to spend months or years“preparing the ground” for a momentous state-level decision.In such situations, careful documentary process-tracing touncover the thinking and planning of leaders and officials isthe best means to establish what complex bundles of factorsled to the decision to dramatically change behavior from thestable pattern of behavior that preceded it. In the language of

45 Process-tracing using the identification of intervening variablesB and C between A and E/not-E can help, of course, but if done onlythrough CEEP and ECEP, this method only exposes more levels ofco-variation, begging the question of why A leads to B and C beforeleading to E/not-E. To stop this ad infinitum process, at some pointdocuments and CEIP/ECIP will have to be used. See also fn. 33 oncausal inference.

the Approaches to Mixed Methods (AMM) framework laidout above, CEIP is the foundational approach and the othershelp to support or reinforce the findings. For other types ofdependent variables, such as rare social revolutions or morecommon shifts in exchange rates and levels of development,simple comparative-historical process-tracing of macro socialforces (CEEP) or large-N correlational analysis (ECEP) wouldbe the primary techniques, with other AMM methods beingused to build confidence in the purported causal mechanismsor to show how leaders can influence the broader trends inaggregate social forces.

The main questions that a researcher must ask him orherself prior to spending years learning methodological tech-niques and gathering and analyzing data are these: What am Istudying, and what am I interested in finding out? The threecritiques in this symposium help us hone our ability to answersuch questions. They allow us to see that scholars must beclear in separating methods that look at co-variation in observ-able external phenomena, through process-tracing or quanti-tative analysis, from deep investigation of the thoughts, atti-tudes, and intentions of powerful individuals—and when bothmethods are used, to identify the strengths and weakness ofeach, relative to the type of event being examined. They alsohelp us remember that the way we set up our theories deduc-tively, and our clarity in defining assumptions, will likely notonly shape whether we are engaging in a cause-of-effects oran effect-of-causes investigation, but whether we are buildingtheories to “beat” the competition (mutual exclusivity) or arewilling to accept that different theories might “work” for differ-ent aspects of a single case or event (coincidence). My bookputs forward one way to establish a balance between differentmethods in a single research program. But the right “mix” forany mixed-methods research, even when the evidence for allfour individual AMM methods is plentiful and reliable, willalways, and inevitably, be unique to the study being under-taken.

References

Collier, David, James Mahoney, and Jason Seawright. 2004. “Claim-ing Too Much: Warnings about Selection Bias.” In Rethinking So-cial Inquiry: Diverse Tools, Shared Standards, edited by Henry E.Brady and David Collier. Lanham: Rowman and Littlefield.

Copeland, Dale C. 1996. “Economic Interdependence and War: ATheory of Trade Expectations.” International Security vol. 20,no.4: 5-41.

Copeland, Dale C. 2000. The Origins of Major War. Ithaca: CornellUniversity Press.


George, Alexander L. and Timothy J. McKeown. 1985. “Case Stud-ies and Theories of Organizational Decision-making.” In Advancesin Information Processing in Organizations, Volume II, Researchon Public Organizations, edited by Robert F. Coulam and RichardA. Smith. Greenwich, CT: JAI Press.

George, Alexander L. and Andrew Bennett. 2005. Case Studies andTheory Development in the Social Sciences. Cambridge: MIT Press.

Goertz, Gary and James Mahoney. 2012. A Tale of Two Cultures:

57


Qualitative and Quantitative Research in the Social Sciences.Princeton: Princeton University Press.

Goldstone, Jack A. 1991. Revolution and Rebellion in the Early Mod-ern World. Berkeley: University of California Press.

Inglehart, Ronald. 1990. Culture Shift in Advanced Industrial Society.Princeton: Princeton University Press.

King, Gary, Robert O. Keohane, and Sidney Verba. 1994. DesigningSocial Inquiry: Scientific Inference in Qualitative Research. Prince-ton: Princeton University Press.

Mahoney, James and Gary Goertz. 2006. “A Tale of Two Cultures:Contrasting Quantitative and Qualitative Research.” Political Analy-sis vol. 14: 227-49.

Pierson, Paul and Theda Skocpol. 2002. “Historical Institutionalismin Contemporary Political Science.” In Political Science: State of

2017 QMMR Section Awards

the Discipline, edited by Ira Katznelson and Helen V. Milner. NewYork: W. W. Norton.

Skocpol, Theda. 1979. States and Social Revolutions: A ComparativeAnalysis of France, Russia, and China. Cambridge: Cambridge Uni-versity Press.

Welzel, Christian and Ronald Inglehart. 2007. “Mass Beliefs andDemocratic Institutions.” In The Oxford Handbook of Compara-tive Politics, edited by Carles Boix and Susan C. Stokes. Oxford:Oxford University Press.

Zaks, Sherry. 2017. “Relationships Among Rivals (RAR): A Frame-work for Analyzing Contending Hypotheses in Process-Tracing.”Political Analysis vol. 25, no. 3: 344-62.

Giovanni Sartori QMMR Book AwardThis award recognizes the best book, published in the calen-dar prior to the year in which the award is presented, whichmakes an original contribution to qualitative or multi-methodmethodology per se, synthesizes or integrates methodologicalideas in a way that is itself a methodological contribution, orprovides an exemplary application of qualitative methods to asubstantive issue. The selection committee consisted of HillelD. Soifer (Temple University), chair; Harris Mylonas (GeorgeWashington University); and Ron Krebs (University of Min-nesota)

Winner of the 2017 Award: Katherine J. Cramer, The Politics ofResentment: Rural Consciousness in Wisconsin and the Riseof Scott Walker. Chicago: University of Chicago Press, 2016.

Prize citation, written jointly: The Politics of Resentment is acompelling account of the urban/rural political divide in con-temporary Wisconsin. Among many excellent books underconsideration, the committee found the book especially laud-able for several reasons. One important innovation found inthe book is the thoughtfully developed and precisely delin-eated concept of rural consciousness. The committee was alsoimpressed by the depth and incisiveness of Cramer’s research,including multiple visits with the same subjects over a periodof years. Her book provides a window into the current politicalmoment, a deep understanding of a fundamental political di-vide and illuminates crucial aspects of American political iden-tity. The book is especially noteworthy, however, for its lucidpresentation of its ethnographic work, or what Cramer calls the“method of listening.” In clear language, Cramer explains herapproach and discusses how she addressed methodologicaland practical dilemmas in the execution of her research. Theresult, as a model of research design and presentation, is apiece of scholarship that will become a touchstone of researchmethods classes.

Alexander George Article/Chapter AwardThis award recognizes the journal article or book chapter, pub-lished in the calendar prior to the year in which the award ispresented, which—on its own—makes the greatest method-ological contribution to qualitative research and/or providesthe most exemplary application of qualitative research meth-ods. The selection committee consisted of Bear Braumoeller(Ohio State University), chair; Neta Crawford (Boston Univer-sity); and Laia Balcells (Georgetown University).

Winner of the 2017 Award: Kurt Weyland, “Crafting Counter-revolution: How Reactionaries Learned to Combat Changein 1848.” American Political Science Review vol.110 no.2(May 2016): 215-231.

Prize citation, written by the committee by Bear Braumoeller:This article explores a neglected topic, counterrevolution, inthe context of the uprisings of 1848 and the subsequent resto-ration of order. In so doing, it provides insights relevant to theArab Spring while highlighting the importance of cognitiveheuristics and biases in revolution and selective learning incounterrevolution. The case study itself is outstanding bothin detail and in theoretical relevance and illustrates the utilityof well-crafted single-case studies. In scope, depth, and theo-retical relevance it stands as an exemplary work of social sci-ence.


58


Sage Paper AwardThis award recognizes the best paper on qualitative and multi-methods research presented at the previous year’s meeting ofthe American Political Science Association. The selection com-mittee consisted of Eva Bellin (Brandeis University), chair;Janet Lewis (U.S. Naval Academy); and Erica Simmons (Uni-versity of Wisconsin-Madison)

Winner of the 2017 Award: Tasha Fairfield and Andrew Char-man, “Explicit Bayesian Analysis for Process Tracing: Guide-lines, Opportunities, and Caveats.”

Prize citation: We are delighted to present Tasha Fairfield andAndrew Charman the Sage Award for best paper presented atAPSA 2016. Their paper, “Explicit Bayesian Analysis for Pro-cess Tracing: Guidelines, Opportunities, and Caveats” con-tributes to a new, growing area of work, applying Bayesianlogic to qualitative analysis. Specifically, Fairfield and Charmanargue that a Bayesian logical framework can contribute impor-tantly to process tracing—especially when scholars disagreeon inferences—and in doing so, can serve as an importantbridge between qualitative and quantitative research. WhileBayesian techniques are not appropriate for all qualitative workor all projects that employ process tracing, the paper does anexcellent job building its case clearly and persuasively andshows how the technique can be useful for some researchquestions. The paper deftly attends to both technical and con-ceptual issues; it provides helpful, step-by-step guidelines onimplementation; and it considers how use of Bayesian tech-niques may improve analytic transparency. Please join us incongratulating Fairfield and Charman on this important contri-bution.

David Collier Mid-Career Achievement AwardWinners of the 2017 Award: Tim Büthe, Hochschule für Politik/

School of Governance at the Technical University of Munichand Duke University and Alan M. Jacobs, University of Brit-ish Columbia

Prize citation, written for the committee by Peter Hall: The DavidCollier Mid-Career Award was established in 2010 to honor aremarkable scholar who was also a founder of the qualitativeand multi-method movement in political science. It is presentedannually to recognize distinction in methodological publica-tions, innovative application of qualitative and multi-methodapproaches in substantive research, and/or institutional con-tributions to this area of methodology. The award is made thisyear by a selection committee composed of (University of Texasat Austin), Elisabeth Jean Wood (Yale University), JamesMahoney (Northwestern University), and Peter A. Hall (HarvardUniversity, chair) and it goes jointly to Tim Büthe of the Tech-nical University of Munich and Duke University and to AlanJacobs of the University of British Columbia. They were nomi-nated by a group of scholars, led by Kathleen Thelen andincluding Orfeo Fioretos, Walter Mattli, Lisa Wedeen, andDeborah Yashar.

Alan and Tim are so well-known to this Section that I needhardly list their achievements. For several years, they havebeen the editors of QMMR, the flagship publication of theSection—to which they have brought insight, verve and strik-ing editorial skill. The quality of this publication has been out-standing. They have also made a remarkable contribution tothe discipline as Co-Chairs of the Qualitative TransparencyDeliberations, a mammoth enterprise involving dozens of schol-ars designed to identify the issues that efforts to render re-search reports more transparent raise for scholars using mul-tiple kinds of qualitative methods. Last but very far from least,Tim and Alan have both made important contributions to ourunderstanding of qualitative and multi-methods in their ownscholarship. These include Tim Büthe’s 2002 APSR article on“Taking Temporality Seriously” as well as the methodologicalsophistication of his prize-winning 2011 book with Walter Mattli,New Global Rulers, as well as Alan Jacobs’ important 2015APSR article on Bayesian approaches to mixed methods andhis lovely chapter on process tracing and ideas in the recentbook on process tracing edited by Andrew Bennett and JeffCheckel as well as the methodological sophistication of hisprize-winning book, Governing for the Long Term. In short,there could be no more deserving recipients of this award.

section president’s letter continued from page 1

Second, Tim and Alan went beyond the call of duty aseditors of the newsletter to lead a major effort to consider therecent drive towards greater research transparency in the dis-cipline. Along with the QTD steering committee, they launchedand oversaw a nearly 18-month process to deliberate over theimplications of this move for a wide array of approaches toqualitative empirical data collection and analysis in the disci-pline. Housed on a web platform, the QTD entailed a multi-stage process aimed at providing an inclusive outlet for schol-arly discussion and debate. In the first stage, the QTD con-sisted of open discussions aimed at setting the agenda formore in-depth and focused deliberations. This initial step ledto the formation of 13 different working groups focused onparticular themes or methodological approaches that emergedout of the open discussion format. In the second stage, theseworking groups, each with its own online discussion board onthe QTD platform, led virtual consultations and deliberationsmoderated by 42 scholars of diverse backgrounds from largeand small colleges and universities both within and beyondthe United States. Based on these exchanges, each workinggroup drafted a report, which was posted on the QTD discus-sion board and open to comments until mid-November of lastyear. At present, the members of the various working groupsare revising their reports to incorporate the feedback they re-ceived through the QTD portal and through a series of panelsat the APSA 2017 meetings. The work of the QTD is a broad,collective effort and all contributors—whether committee mem-bers or commentators—should be applauded for their input.We owe a special debt of gratitude to Tim and Alan, who de-voted over a year of their time to oversee this important pro-

59


cess. The exemplary process they have spearheaded is a testa-ment to the encompassing spirit of the QMMR section. As anopen venue for deliberation about the potentially momentousshifts of the drive towards research transparency for somecorners of political science, the QTD has provided a vital ser-vice for the discipline as a whole.

The QMMR section continues to attract a broad member-ship. With about 740 members, the QMMR is the second larg-est section of the more than 40 sections in the APSA. At theAPSA meetings last August, QMMR sponsored five differentshort courses on both well-established and novel method-ological approaches. (Full information on these courses is avail-able on the APSA QMMR website.) After the elimination ofgraduate student membership fees as of January 2018, a change

Melani [email protected]

Letter from the Editors

Tim Büthe Alan M. [email protected] [email protected]

With this issue, we complete our term as co-editors of Qualita-tive and Multi-Method Research. It has been an honor to editQMMR, which we had the good fortune of taking over as awell-established venue for cutting-edge work and high-qual-ity debate about qualitative and mixed-method research in po-litical science, thanks to the splendid stewardship of the edi-tors who preceded us: John Gerring (2003-2006), Gary Goertz(2006-2011), and Robert Adcock (2011-2014).

We have sought to maintain not just the high standardsestablished by our predecessors, but also the commitment tomake and keep QMMR a publication that reflects the method-ological pluralism of the QMMR community and appeals to abroad range of QMMR section members. The conversationsthat have unfolded across the articles and symposia publishedduring our tenure leave us optimistic about both the possibil-ity and the fruitfulness of civil and respectful intellectual en-gagement across sometimes profound ontological, epistemo-logical, methodological, and subfield divides—even when theunderlying disagreements themselves cannot be reconciled.

These conversations have been made possible by themany colleagues who have contributed to QMMR over thelast three years, with whom we have greatly enjoyed interact-ing and from whom we have learned a great deal. Thank you!

We also thank the section and the Consortium for Qualita-tive Research Methods (CQRM) for their support; AllisonForbes (vols. 13 & 14), Andrew McCormack (vol.15:1) andAlberto Alcaraz Escárcega (vol.15:2) for their copyediting as-sistance; and Joshua Yesnowitz, QMMR’s longstanding pro-duction editor, for his many years of service. And we are grate-ful to Sebastian Karcher and Sarah-Anna Hogan of CQRM fordoing the bulk of the work to bring QMMR more fully into the

digital age: Each essay now has a unique “digital object iden-tifier” (DOI), with links from each issue’s online table of con-tents to an open-access post of the essay. This change makesQMMR content more findable, accessible, and citable, and wehope that it gives QMMR authors and publications additional,much-deserved attention.

We have, during our time as editors, sought to openQMMR to a broader range of authors by scanning conferenceprograms for innovations and debates, of which we wouldotherwise have been unaware, and by inviting, reviewing andpublishing unsolicited submissions. That said, looking backon volumes 13 through 15, we realize that our efforts to diver-sify QMMR authorship clearly paid insufficient attention togender balance. The underrepresentation of women, who havecomprised only 15% of the authors in QMMR during the lastthree years, is a serious problem that we regret we did not do abetter job addressing.

All the more, we are absolutely delighted that Jennifer Cyrand Kendra Koivu have agreed to take on the QMMReditorship, starting with the next issue. Jennifer and Kendrahave already been shaping qualitative and multi-method re-search in our discipline for many years, through (among otherthings) their publications and their roles as co-founders andco-organizers of the annual Southwest Workshop on MixedMethods Research. We could not imagine this publication beingin better hands, and look forward to seeing the directions inwhich Kendra and Jennifer will be taking the QMMR conver-sation.

adopted in response to a proposal from the APSA GraduateStudents Association, we hope to welcome even more youngscholars working in the qualitative and multi-methods tradi-tions.

The QMMR section is distinguished by the rigorousscholarship of its members, who collectively represent a widerange of methodological and theoretical approaches. TheQMMR commitment to intellectual diversity and open dialogueand debate helps to foster an inclusive ethos, which can onlystrengthen the discipline overall.


60

Qualitative and Multi-Method Research Nonprofit Org. Department of Political Science U.S. Postage 140 Science Drive (Gross Hall), 2nd Floor PAID Duke University, Box 90204 TUCSON, AZ Durham, NC 27708 Permit No. 271 United States of America

Qualitative and Multi-Method Research (ISSN 2153-6767) is published twice yearly on behalf of the American Political Science Association’sOrganized Section for Qualitative and Multi-Method Research. It is edited by Tim Büthe (tel: 919-660-4365, fax: 919-660-4330, email:[email protected] or [email protected]) and Alan M. Jacobs (tel: 604-822-6830, fax: 604-822-5540, email: [email protected]). Theproduction editor is Joshua C. Yesnowitz (email: [email protected]). The manuscript editor is Andrew McCormack. Published withfinancial assistance from the Consortium for Qualitative Research Methods (CQRM); http://www.maxwell.syr.edu/moynihan/cqrm/About_CQRM/. Opinions do not represent the official position of CQRM. After a one-year lag, past issues will be available to thegeneral public online, free of charge, at http://www.maxwell.syr.edu/moynihan/cqrm/Qualitative_Methods_Newsletters/Qualitative_Methods_Newsletters/. Annual section dues are $9.00. You may join the section online (http://www.apsanet.org) or byphone (202-483-2512). Changes of address take place automatically when members change their addresses with APSA. Please do notsend change-of-address information to the newsletter.