the ideological cohesion of political parties. an evaluation of the method of deriving mps’ policy...
TRANSCRIPT
The Ideological Cohesion of Political Parties
An Evaluation of the Method of Deriving MPs’
Policy Positions from Parliamentary Speeches1
Hanna Bäck
Department of Political Science
Lund University
Marc Debus
Mannheim Centre for European Social Research (MZES)
University of Mannheim
Wolfgang C. Müller
Department of Government
University of Vienna
Paper prepared for presentation at the seminar “Current Research in European
Union Politics”, at Oxford University, Oxford February 15th
2011.
1 The authors thank Julian Bernauer, Marcelo Jenny, Thomas Meyer and Jochen Müller for their helpful
contribution to our research project and the MZES and Riksbankens Jubileumsfond for providing research
funding. We are also grateful for the excellent research assistance provided by Daniela Beyer, Maiko
Heller, Julia Schnur and Christian Roth. A previous version of this paper was presented at the ECPR Joint
Sessions in St Gallen, 12-16 April 2011.
Abstract
Analyses of West European political parties show that a high degree of cohesion inside party
groups exists. There are, however, internal groups inside a political party with programmatic
viewpoints that diverge from the one of the party core. The aim of this paper is to explore ways
of estimating the ideological cohesion of political parties. Studies on party unity normally
analyze roll-call votes, but MPs may not behave sincerely when their voting decision is known,
suggesting that such analyses do not reflect political actors’ real policy positions. Thus, we
estimate MPs’ policy positions, and thereby intra-party ideological cohesion, by studying the
spread of positions inside parties derived from an analysis of parliamentary speeches of MPs.
This analysis is performed by using the fully computerized Wordscores technique. In the case
study performed here of Austrian MPs, we have the unique opportunity to validate the policy
positions derived from an analysis of parliamentary speeches by comparing them with those
derived from MP survey responses. In addition, we perform a multivariate analysis, studying the
relationships between a number of important individual-level characteristics and MPs’ policy
positions. Our analyses provide support for the idea that we can derive measures of MPs’ policy
positions and thereby party cohesion, by analyzing parliamentary speeches.
1
Introduction
Analyses of parliamentary voting behavior in West European countries show that a high degree
of cohesion inside the party groups exists (e.g., Bowler et al. 1999; Depauw 2003; Sieberer
2006). There are, however, internal groups inside a parliamentary party with programmatic
viewpoints that diverge from the one of the party core, and previous research shows that these
groups may play an important role in government formation and portfolio allocation (e.g.,
Giannetti and Benoit 2009). Our central aim with this paper is therefore to provide estimates of
the programmatic cohesion of political parties, defined as the “general agreement within a party
organization on specific issue positions” (Kitschelt and Smyth 2002: 1129).
Studies on party cohesion or unity in parliaments normally use roll-call votes to analyze the
degree of cohesion. For this reason, studies on this topic normally concentrate on parliaments
that provide data on the representatives’ voting behavior. Most prominently, these legislatures
are the US Congress (e.g. Poole and Rosenthal 1997; Krehbiel 2000), the European Parliament
(e.g. Hix et al. 2006), and the German Bundestag (Saalfeld 1995a). These studies show that
there is considerable variation in the ideological positions of the MPs in one parliamentary
group. However, MPs may not behave sincerely when their voting decision is known, which
implies that an analysis of roll-call or recorded votes is not likely to reflect the “real” policy
positions of political actors (see Carrubba at al. 2006). MPs should have more leeway in
expressing their own policy positions by giving speeches, for example since speeches typically
are less consequential for parliamentary government than voting.
Therefore, we estimate the policy positions of MPs, and thereby intra-party cohesion by
studying the spread of positions inside parties derived from an analysis of the parliamentary
speeches of MPs. This analysis is performed by using the fully computerized Wordscores
technique (Laver et al. 2003). Recent work shows that this technique provides plausible results
in the analysis of parliamentary speeches (e.g., Giannetti and Laver 2009; Laver and Benoit
2002). Promising as these results are, they may still constitute too narrow a basis for the external
validation of the approach of measuring party cohesion on the basis of a computerized analysis
of parliamentary speeches. The question we ask in this paper therefore is, how valid are the
measures of MPs’ policy positions, and thereby ideological cohesion, that we draw from a
computerized analysis of parliamentary speeches? As a first stage of a comparative project, we
here focus on analyzing the speeches made in the Austrian parliament at the end of the 1990s.
In the present case study, we have the unique opportunity to compare policy positions
derived from an analysis of parliamentary speeches with those derived from MP survey
responses at the individual level. This enables us to gauge the value of analyzing parliamentary
speeches with the help of various techniques of content analysis to derive MPs’ policy positions.
In addition, we perform a multivariate analysis, studying the relationships between a number of
important individual-level characteristics and MPs’ deviations from the party core. In the
multivariate analysis of Austrian MPs, we find that MPs who hold regional district seats and
who have a shorter parliamentary experience are more likely to adopt positions in speeches
which are not congruent with the position of the party core. In addition, we find that there is a
relatively high correlation between the MP policy positions derived from our analysis of
speeches and the positions drawn from an elite survey performed among Austrian MPs. All in
all, we suggest that these results support our approach to measuring ideological cohesion.
2
Defining and measuring ideological cohesion
What do we mean by “ideological cohesion”?
The concepts “unity”, “cohesion” and “discipline” have sometimes been used in a confusing
manner in the literature on political parties (see Giannetti and Benoit 2009). We here make a
clear distinction between voting “unity” and “programmatic cohesion”, where unity refers to the
behavioral phenomenon of MPs in a party or other group voting together, or as a bloc in
parliament. Or as Sieberer (2006: 151) puts it, “the degree to which members of a group act in
unison”. Programmatic cohesion instead refers to the “homogeneity of policy preferences”, or
the general agreement within a group in terms of policy positions (Giannetti and Benoit 2009:
5). In this paper, we focus on the latter concept, that is, programmatic or ideological cohesion,
However, since much of the literature focuses on voting unity, we also draw on such research.
For example, Carey (2007: 93) distinguishes between three distinct sources of voting unity
within legislative parties: cohesiveness, discipline, and agenda control. Thus, one reason why
MPs belonging to the same party vote as a bloc is that the members have similar preferences.
Another reason why MPs of the same party vote together is that party leaders use a
“combination of carrots and sticks […] to reward voting loyalty and deter or punish breaches in
discipline”. Lastly, party leaders may use their ability to steer the agenda, in order to avoid that
proposals that would divide the party come to a vote in the legislature. Hence, a highly unified
voting record may or may not signal a high level of ideological cohesion within the party.
Carey (2007) has presented a number of hypotheses explaining voting unity across parties
and systems, drawing on the so called “competing principals theory”, based on the more general
principal-agent framework. The main idea is that members of parliament can be seen as agents
facing several different principals, and since these principals are likely to control resources to
influence the voting behavior of MPs, divergence in the demands of “competing” principals is
likely to reduce voting unity within parties. Carey (2007: 93) argues that “virtually all
legislators are subject to the influence by at least one principal: their legislative party
leadership”, and whether they are influenced by other, competing principals depends on the
institutional context in which they act. All in all, this suggests that a deriving measures of
ideological cohesion within parties by studying MPs’ voting behavior may be problematic.
To be sure, making plenary speeches is political behavior and as such akin to parliamentary
voting. The major difference, however, is that voting is extremely structured behavior. It
provides the actors with three options only: voting “yes”, “no”, or abstaining. Speeches very
often will arrive at the same baseline with the speaker declaring his or her support for a proposal
or rejecting it. Yet, speaking allows for much more behavioral variation and allows the “sender”
to express different degrees of acceptance/non-acceptance of a proposal and different reasons
for it. We here assume that speeches reveal information about the individual speakers and
collectively about their parties and the party system. Extracting such information is by no means
a simple task. In this paper we discuss the challenges and look in depth at one promising
research technique that is designed to extracting actors’ positional data from political text.
3
Approaches to measuring cohesion
A survey of the attempts to systematically measure party cohesion and the closely related
literature on measuring party positions suggests six different approaches: party coding from the
literature, expert surveys, elite surveys, mass surveys, roll-call analysis, and the extraction of
individual positional data from speeches (or other texts). We briefly review these approaches.
Party coding. While many authors have arrived at ad hoc judgments of party cohesion and
national rank-orders of parties, the International Comparative Political Parties (ICPP) Project by
Kenneth Janda (1980) is the main systematic attempt to do so. Student coders have coded
relevant publications to generate a wealth of variables on 158 political parties. With respect to
party unity, Janda (1980) aimed at measuring legislative cohesion, ideological factionalism,
issue factionalism, leadership factionalism, and strategic factionalism with the help of a number
of scales. These were found to be highly correlated (1980: 154). Impressive as this work is,
obviously any coding from documents has two inbuilt limitations: (i) the scope and quality of
previous research, and (ii) the grip the researcher can establish on these sources in terms of
collecting them and extracting equivalent information. We can expect that coding will lead to
more accurate results the better researched the parties are and the more accessible the sources.
Expert surveys. The second holistic approach is surveying experts. This approach has seen a
fair amount of methodological discussion recently and we will refrain from summarizing all the
arguments here. Rather we confine ourselves to highlighting a few of the methodological
limitations that seem to remain even in carefully designed expert surveys. Steenbergen and
Marks (2007) and Marks and Steenbergen (2010) have provided evaluations of their own expert
surveys. With regard to party positions on the issue of European integration they show that their
results are less reliable the closer to each other the party positions are, the less salient the issue
is to the individual party, and the more divided the party is over that issue (2007: 354). For our
research question this would imply that expert judgments on party cohesion are likely to vary
between policy areas. Another potential problem is that the experts’ own opinions can distort
the results, e.g. experts may place parties closer to them more to the political center and other
parties more towards the extremes (Benoit and Laver 2006; Curini 2007). With regard to intra-
party disagreement, experts may have “biased” perceptions of the level of cohesion depending
on how “close” they are to the evaluated parties.2
Mass surveys. Party positions have been inferred from the positions of party voters. The
increasing availability of voter surveys makes it an economic shortcut in getting at party
positions, and previous research has shown relatively high correlations between party positions
drawn from voter surveys and from other sources (Ray 2007; Marks et al. 2007). An advantage
is that the range of intra-party opinions can easily be calculated from such data. Yet,
notwithstanding the relatively high correlations with other data, voters remain conceptually
different from parties, and researchers interested in representation or electoral competition, need
independent measures for both the demand side (voters) and the supply side (the party).
2 As concerns the practical limitations of expert surveys for answering our research question, the literature agrees that
expert surveys become problematic when they ask about events in the more distant past. As we are interested in
generating longitudinal measures of party cohesion for several salient policy dimensions, expert surveys are
somewhat limited since the relevant question about party unity has been asked only in the Chapel Hill surveys, and
thus is available only for one issue – European integration – and only for the period since 1999.
4
Elite surveys. Several studies have therefore instead relied on surveys among representatives
(e.g. Müller and Jenny 2000), for example arguing that information about individual politicians’
preferences is appropriate to use when we study events like coalition formation since the
politicians are the actors involved in coalition bargaining, and they are thus the actors who make
the decisions that lead to the outcomes that we wish to explain. A drawback is however that
surveys among political elites are costly and face the problem that politicians are often less
willing to answer than academics who dutifully fill-in expert surveys. For such reasons elite
surveys are rare and mostly single-country studies which may not compare well with each other,
and are therefore also rarely discussed (see however Mair 2001 and Bäck 2003).3
Roll-call analysis. The classic behavioral measures of party cohesion are parliamentary
voting and voting participation. Abstention often is considered as a mild measure of
disagreement even though there are many other plausible reasons for not participating in a
parliamentary vote. Clearly, voting is the most consequential activity of MPs and therefore it
should and will always constitute a central focus of studies of party cohesion. Yet, limiting the
study of party cohesion to parliamentary voting in general and roll-call votes in particular, limit
our ability to understand cohesion due to a number of reasons. First, in many parliaments and
periods the number of roll-call votes is very small. Second, even when roll-call votes amount to
figures that can be meaningfully analyzed statistically, it is not clear that they are representative.
Rather they may be endogenous to precisely the characteristics of voting behavior – party
cohesion – that we want to study (Carruba et al. 2006, 2008). On the one hand party leaders are
likely to employ roll-calls as a means to enforce party discipline upon MPs when the agreement
among them is low (as roll-call votes can be better observed by party leaders – Saalfeld 1995b).
On the other hand, parties that are united on an issue may demand roll-call votes to expose the
fact that other parties are split. Both would lead to a biased sample, as votes on those issues will
be overrepresented where at least one of the parties has a potential for cohesion problems.
Leaving aside the problems of small n and selection bias, information on voting behavior,
valuable as it is, has limited and context-specific information content only. Agreement or
disagreement of MPs with their party is dichotomous information, and employing the more fine-
grained categories of “voting with the party”, “abstaining”, and “voting against the party”
provides just three categories. Among the crucial missing information are the policy status quo
and the direction of the proposed change. The reversion point – what is going to happen if the
majority fails to deliver its vote – is particularly relevant and can account for high amounts of
party cohesion in terms of behavior that camouflage substantial differences in terms of attitudes.
If, for instance, a left party attempts to move the (rightist) status quo just an inch to the left, the
new policy may be liked much more by its right competitors than by its own left wing. Yet, as
even an inch is a real improvement, the party is likely to be united in terms of behavior although
some of its MPs may fundamentally dislike the new policy position and are frustrated about the
magnitude of the move. While the vote will not reveal this, the speeches may do so.
3 Another potential problem with such surveys is that elites may answer strategically as elites may want
to appear more united than they actually are.
5
Measuring cohesion by analyzing parliamentary speeches
The speeches thus are likely to provide some of the motives for behavior and real preferences.
Analyzing speeches should thus fill important gaps that remain after the analysis of voting
behavior. And the analysis of speeches should reveal information that is qualitatively different
from what can be extracted even under the best circumstances from the analysis of roll-call
votes. Analyzing speeches thus brings us closer to understanding party cohesion in terms of
preferences rather than behavior. In short, the competitive advantages of drawing on speeches
are the following: First, for many parliaments and periods we have simply more cases available
as recorded votes are rare in many parliaments while speeches are not. Second, with regard to
the issues covered, we reduce selection bias. Although particularly controversial issues may
cause more speeches, even the most uncontroversial ones will attract some. Third, voting is a
blunt instrument, whereas speeches allow for much more fine-grained evaluations of the issue at
stake. Support may range from enthusiastic to choosing “the least evil” and rejection may cover
the entire range from full-hearted “demonizing” to the balance “just tipped” to the negative.
The analysis of speeches also has some disadvantages. First, there may be individual MPs
who never or rarely speak. Consequently, we may have more speeches than roll-call votes
collectively but most likely not for each individual MP. Second, while the collective coverage of
issues by speeches is likely to be much broader than that of roll-call votes for most parliaments
and periods, most issues will attract speeches from a minority of MPs only. Third, while
extracting information from parliamentary votes is often a relatively straightforward task,
analyzing speeches is a relatively demanding enterprise. In addition, when combining debates
we assume that the speeches of MPs are structured by their party affiliation regardless of the
subject matter. Yet, if policy areas tend to have their own specific terminologies and jargons
such inter-party differences may be belittled by the differences between different policy areas in
a words-based quantitative analysis. Consequently, in multivariate analyses we should control
for potential influence of policy areas and issue (as measured by individual debates).
If not all MPs speak in a debate does this introduce a selection bias that can be compared to
the selection bias of roll-call votes? Here we must distinguish between specialization and
preferences, where we are interested in the latter. Clearly, we expect MPs specializing in a given
subject area to be more likely to speak than their colleagues. In that respect each debate will
produce a biased sample. The important question for us is, does specialization go together with
specific preferences? The literature on legislative committees suggests two potential biases:
Weingast and Marshall (1988: 345) suggested, when studying the US Congress, that self-
recruiting committee members are high-demanders and as such preference outliers (see also
Fenno 1973). As members of the committee in charge of a proposal tend to be systematically
overrepresented among the plenary speakers on a topic, the same logic could apply and lead to
serious selection bias. Yet, the distributional theory of Weingast and Marshall has a rival in the
informational theory of Gilligan and Krehbiel (e.g. 1994). They share the idea that committees
are likely to attract preference outliers but theorize that their distribution will be bi-polar,
consisting of, for instance, high- and low-demanders, falcons and doves, conservatives and
liberals, etc. This would imply that analyses of parliamentary speeches will overestimate the
policy differences between parties. A study by Müller et al. (2001: 274–9) on the Austrian
parliament demonstrates that both potential biases are unlikely to occur. Based on MP survey
6
data they show that Austrian parliamentary committees do not systematically consist of high-
demanders or bi-polar preference outliers.
Another potential problem of selection bias is due to the fact that the party leadership may
have strategic reasons for showing that the party is more unified than it is, be it for vote-seeking
reasons, or for office-seeking reasons, where unified parties may be more attractive in the polls,
and as coalition partners (see e.g. Bäck 2003). Proksch and Slapin (forthcoming) argue that in
systems where party unity is important for the leadership, backbenchers will receive less time
on the floor when they take policy positions that are further away from the party leadership.
They also find support for this hypothesis in Germany, with less floor time being allocated to
more deviant MPs, and they conclude that this may lead to that an analysis of speeches
underestimates the ideological differences within parties. Whether this is indeed the case
however depends on whether party leaders have such a strong control over the debate agenda
that they are able to fully exclude deviant members from speaking – otherwise the potential bias
is only relevant for studies of floor time and not for the analysis of the policy positions derived
from speeches, which would suggest that ideological cohesion is not overestimated
Another measurement problem may result from the need of speakers and parties to have in
mind a debate’s dramaturgy. Simply put, a good speech is a speech not already given. MPs thus
have to pay tribute to the normative expectation that their speeches should be different from the
speeches of their party comrades, even if they do not disagree a jota with them. Seen from the
perspective of the party leadership, all speeches should pay tribute to the party’s punch line and
core arguments. But to make them worth listening they should also contribute something new:
illustrations and applications. Thus, a party strategist may plan a debate’s dramaturgy as
follows: the lead speaker develops the general party argument; subsequent speakers may deepen
particular aspects of the argument; further speakers may discuss the proposal from the
perspective of different social groups or from their geographical constituency perspective;
finally, a wrapping-up speech will repeat the party’s punch line. The parties paying tribute to
such dramaturgical considerations may constitute a challenge for textual analysis as it may
imply the usage of different vocabularies and styles of rhetoric.
Applying computerized content analysis to speeches
In this paper we derive MPs’ policy positions by applying computerized content analysis to
parliamentary speeches. This approach has been used in some previous studies. For example,
Giannetti and Laver (2005, 2009) make use of a dataset that covers the policy positions of each
minister estimated on the basis of their speeches in parliament. They apply the Wordscores
technique developed by Laver, Benoit and Garry (2003; cf. Lowe 2008) to get information on
the policy positions of the members of the Italian 1996 cabinet. Laver et al. (2006) apply the
Wordscores technique to speeches of French presidential candidates in 2002 and their programs.
Likewise to the research design applied in this paper, Laver and Benoit (2002) use computerized
content analysis to estimate positions of Irish MPs on the basis of their speeches held during one
single debate. The results show that MPs belonging to one party group share similar
preferences, but also that differences inside parties and thus policy conflict inside parties exist.
Bernauer and Bräuninger (2009) apply the Wordscores technique to the German case and
estimate the policy positions of Bundestag MPs 2002–2005. They show that Wordscores
delivers plausible results when estimating the preferences of political actors on the basis of their
7
speeches instead of party programs or election manifestos. Also, studies on the policy
preferences of members of the European Parliament reveal similar positions on the basis of
parliamentary speeches, regardless of whether estimated by Wordscores or the more recent fully
computerized technique for content analysis, “Wordfish” (see e.g. Slapin and Proksch 2008).
Fully computerized techniques of content analysis have advantages and disadvantages. The
main advantage of such approaches is that position estimation is left completely to computer
algorithms. This maximizes reliability and clearly is less time-consuming than some form of
hand coding once one has downloaded the speeches. In addition, potential reliability problems
associated with MRG/CMP-style hand coding (see Volkens 2001) or the “dictionary procedure”
(Laver and Garry 2000) do not arise (see, e.g., Mikhaylov et al. 2008; Benoit et al. 2009).
Some caveats of this approach have also been discussed, for example regarding the
standardization method for Wordscores estimates (Martin and Vanberg 2008; Benoit and Laver
2008), and on the method itself (Budge and Pennings 2007; Benoit and Laver 2007). Despite
this, the evidence at the level of the positioning of collective actors is impressive. For example,
in a comparative analysis that covers party positions for 13 West European countries for a time
period of around 30 years, Bräuninger et al. (2010) find a clear relationship between the scores
of the MRG/CMP left-right index (“rile”) with left-right positions estimated by Wordscores.
What is the evidence from the analysis of speeches using such computerized techniques?
Clearly, here validation suffers from the scarcity of reference points external to the computer-
based analysis as we simply do not have much reliable data on individual actors and even less
can be accessed. So, how can the validity of deriving policy positions from speeches be
evaluated? We can identify three different approaches: (i) relying on face validity and aggregate
comparison, (ii) testing for construct validity, and (iii) testing on the basis of individual data.
Previous work on speeches have mainly employed face validity and aggregate comparison. If
the aggregates of the individual positions correspond to the party positions estimated by other
methods, then the individual measures are considered prima facie credible. Testing for construct
validity means checking whether the individual measures generated can explain other observed
variables “in a way that is consistent with theoretically derived predictions” (Bollen 1989: 188).
One such theory is that factional membership generates clear behavioral expectations. Applying
this research strategy Giannetti and Laver (2005, 2009) have looked at the spending patterns of
ministries and the left–right position of Italian factions and find that ministers’ policy positions
derived from speeches using Wordscores can be used as predictors of budget spending.
Laver et al. (2006) use individual-level data to evaluate the positions derived from a
Wordscores speech analysis in the setting of the French Semi-presidential system. Typically,
presidential candidates not only make speeches but also release policy programs. Thus we have
individual-level information from two different types of text. Yet, as both are subject to the
same method of analysis (i.e. Wordscores) this does not take us very far. Fortunately, the
authors also have an expert survey at hand for external validation. This convincing research
design however does not travel to parliamentary systems. While it may be possible to extract
individual positions for “the ones” in a Semi-presidential system or legitimate to equate them
with the corresponding party positions, it is more demanding to extract such positions in
parliamentary systems. In conclusion, more individual-level validity testing of computer-based
analyses of political actors’ speeches is what we need. And this is what we set out to do in this
paper. In addition, we also apply the approaches of checking for face and construct validity.
8
Research design and data
An Austrian case study
For our empirical analysis we have chosen to focus on Austria, mainly because we have access
to identified survey data drawn from an elite survey among MPs in the Austrian parliament
(Nationalrat) during the same period as we draw our speeches (Müller et al. 2001). MP Surveys
have been performed in a number of countries, which allows us to compare measures of
ideological cohesion drawn from an analysis of speeches and from elite surveys at the aggregate
level. However, most such survey data is archived in non-identified form, which implies that we
cannot make the comparison at the individual-level, i.e. comparing MPs’ policy positions from
different sources. Our data allows us to perform such an analysis, which is especially fruitful at
this stage, when speeches have only been collected in a small number of parliaments.
The Austrian case is also advantageous for other reasons. Most importantly, its electoral
system offers the opportunity to study the effects of (some degree of) institutional variation
while holding other features constant. Specifically, its three-tier system creates more than one
type of MP: those elected from party lists in 43 multi-member constituencies (regional level), at
the Land or national levels. At the regional and Land levels candidates are accountable to the
voters via a system of preference votes. Such votes constitute a potentially powerful instrument
at the regional level as winning a sixth of the party vote or half of the votes required for a seat is
sufficient for changing the party list. Moreover, the system is easy to handle, as the names of the
local candidates are printed on the ballot paper. In contrast, at the Land level a candidate needs
to win as many preference votes as votes are required for a seat and the candidates’ names need
to be written-in. This makes it much harder for voters to coordinate for sanctioning individual
candidates or promoting others (Müller and Scheucher 1994). Although actual changes of the
party lists through preference votes happen only rarely, MPs are aware of the potential power of
this feature of the electoral system. It also has indirect effects, as preference votes can be
relevant for future list rankings. Sometimes challengers hope to get the party’s attention while
incumbents do not want to leave the field and therefore compete for preference votes as well. A
strong result in preference votes may even increase an MP’s standing in the party.4
In our observation period altogether five parties were represented in the Austrian parliament
(Social Democrats (SPÖ), Christian Democrats (ÖVP), Freedom Party (FPÖ), Liberal Forum
(LIF), and the Greens (G). The effective number of parliamentary parties (based on seats) was
3,47. Throughout the period the party system was tilted more to the right (a center-right
majority) however government office was shared by the grand coalition of the SPÖ and ÖVP.
Predictors of ideological cohesion
As mentioned above, the aim of this paper is to evaluate the approach of measuring ideological
cohesion by analyzing parliamentary speeches. We here suggest that one way of performing
such an evaluation is to study the relationships between a number of important explanatory
4 Most MPs combine candidacies at two or even three levels (the neat three-types distinction thus does
not hold, as we may face “contamination” effects). Although local strongholds make many seats save
ones, small changes in party fortunes can shift around quite a few seats (Müller 2005).
9
features and the MP policy positions that we derive from an analysis of parliamentary speeches
(“construct validity”). We therefore briefly review the literature cohesion and unity with the aim
of finding the most important individual-level features that can be used to predict the MPs’
ideological deviations from the party core, a sort of individual-level measure of cohesion.
According to Giannetti and Benoit (2009), scholars who have tried to explain cohesion and
unity have mainly focused on the variation across different political systems by introducing
three main sets of explanatory variables: institutional features, party system features and parties’
internal structure. Since we here focus on understanding individual-level behavior, it is less
straightforward to apply the explanations given in the previous literature. However, we suggest
that we should still be able to draw on this literature when predicting individual-level cohesion.
Several authors have stressed the role of electoral rules when explaining the cross-national
variation in voting unity. Carey (2007) argues that in systems where candidates compete with
other MPs within their party for electoral support, voting unity should be lower since such
electoral systems encourage personal vote-seeking. Some authors have also recognized that
there may be differences within countries in the incentives to cultivate a personal vote (see e.g.
Norris 2004; Becher and Sieberer 2008). This is especially the case when we are dealing with a
mixed electoral system such as the German one, where some legislators are elected by plurality
in single-member districts and some MPs gain their seat through closed party lists. In such
systems, MPs who are directly elected through a constituency vote (or in the Austrian case, MPs
holding a seat in a regional electoral district) are more likely to deviate from the party line,
when holding a speech or voting, than the MPs getting a seat through the PR list (national) vote.
Carey (2007: 95) also makes a distinction between government and opposition parties,
suggesting that the leadership of governing parties have more motivation and more resources, in
terms of office and policy payoffs, to impose discipline on and provide rewards to MPs, in order
to induce them to follow the party line (see also Becher and Sieberer 2008). Since the national
party leadership also controls executive office, the MP should be even more dependent on the
national party leadership for making a political career, thereby rendering unity and cohesion
higher in governing parties. In our multivariate analysis we therefore include a variable
describing if the MP belongs to a governing or opposition party.5
Several authors have stressed the role of the party-internal hierarchy, as reflected in MPs
holding party and parliamentary leadership positions, influences many aspects of their behavior
(e.g. Searing 1994; Müller et al. 2001), including the observation of the party line in roll-call
voting (Saalfeld 1995a). It is easy to see the rationale of such behavior. For one thing, MPs
holding leadership positions tend to have greater influence on the party line. All else equal, MPs
holding leading positions should stick to the party line for the reason that it is likely to represent
their own preferences. Also, the rule of reciprocity (Fenno 1973: 95; Weingast 1987) should
work better at higher levels of hierarchy. In contrast to a backbencher, a committee chair, for
instance, has his/her own turf to defend. MPs in leadership positions thus may gain more
5 Carey (2007) discusses the role of federalism and argue that MPs operating in federal systems are more
likely to be subject to demands from competing principals, rendering party unity lower in federal than in
parliamentary systems. Drawing on this institutional explanation, we expect that there may also be a
variation within federal systems in terms of the importance of sub-national principals – in some regions,
career-seeking MPs may be more dependent on the support of state-level party leadership than in others.
In our analysis, we therefore include dummies of the Austrian states to account for such variation.
10
practical use from reciprocity. Finally, MPs in leading positions tend to have more to lose from
potential punishment as they may be withdrawn from such positions or their term not renewed
(Damgaard 1995; Saalfeld 1995a). We therefore include a variable measuring an MP’s
leadership function inside the parliamentary party group.
In each parliament some MPs serve their first term (“freshmen”) while others can look back
to a career of considerable length. Long careers indicate proven ability to survive politically.
This may be due to strong independent bases of power, but it may also be due to
accommodation with the party leadership. The literature on political socialization suggests that
the more acquainted MPs get with their parliamentary roles, the more they will tend to support
established patterns of power (Dawson et al. 1977: 16). Hence, we would expect that MPs with
longer parliamentary careers are more in line with their party. Yet, the same outcome may also
result from a better understanding of the rules of the game and the MPs learning how to shape
the party line. To account for this, we include a measure of the length of the parliamentary
career of each MP (number of years in the Nationalrat) in our multivariate analysis.
To sum up, we identify a number of features that should correlate with ideological cohesion
within parties and thereby with MPs’ ideological deviations from the party core. We expect that
MPs who are directly elected (or hold a regional seat in the Austrian case) are more likely to
deviate from the party line, and that MPs who hold leadership positions, who have a long
parliamentary career, and who belong to the governing parties are less likely to do so.
Specifying the computerized content analysis
Our analysis of MP policy positions is based on 2,284 speeches of 164 members of the Austrian
Nationalrat in policy areas that belong to the economic policy dimension, and 1,343 speeches
from 129 Austrian MPs in debates on societal issues or – as Laver and Hunt (1992) and Benoit
and Laver (2006) put it, “social issues”, which covers the positions of political actors on
conflictual issues on the order of society like abortion policy or the rights of homosexuals.
Before turning to the analysis, we discuss one caveat of the Wordscores approach by
focusing on the diverging results that appear when selecting different reference texts. As pointed
out by Laver et al. (2003), the reliability and validity of estimations of the positions of political
texts by using the Wordscores method depends highly on the characteristics of the selection of
reference texts. “Reference texts” should have the following characteristics: First, to obtain
valid results, the selected texts should be of the same character as the one whose position is
unknown. The risk of obtaining invalid results will increase if, for instance, one tries to estimate
the position of an election manifesto when using a speech of a politician as a reference, because
the word structure in both texts will be less homogeneous. Second, the selected documents
should cover aspects of all policy dimensions in which one is interested. Because election
manifestos include a large number of aspects that were salient in the actual election campaign as
well as from a long-term perspective, this sort of documents are adequate “reference texts”.
However, sometimes manifestos are no lengthier than a booklet or a flyer and therefore include
only a small number of words. As there is no exact limit for covering and scoring words in
“virgin texts”, a third criteria is that “reference texts” should be as long as possible.
Thus, if one is interested in the policy-area specific position of an election manifesto, it
would be most suitable to use election manifestos as “reference texts”. When measuring policy
positions of speeches, however, one would generally argue to use speeches as “reference texts”,
11
because the risk of obtaining invalid results by using Wordscores should increase if, for
instance, one tries to estimate the position of an election manifesto when using a speech of a
politician as a reference, because the word structure in both texts will be less homogeneous. It is
common knowledge that people speak in another way than they write texts.
Yet, when selecting speeches of parliamentary party groups leader only, which is the most
straightforward reference text that can be used when analyzing MPs’ speeches, we will run into
a another problem, because speeches are generally shorter than written programmatic
documents like election manifestos. In the Wordscores estimation process, the latter will result
in a smaller number of scored words in a speech given at the parliamentary floor and, thus, in a
position of the speech with a high standard error. We thus selected both election manifestos and
speeches of the parliamentary party group leaders as reference texts. We score the speeches by
the parliamentary party group leaders and the election manifestos of the national parties for the
2002 national elections in Austria on the identified policy dimensions with the economic and
societal or social policy position provided by the Benoit and Laver (2006) expert survey. We
focus on these two policy dimensions because they structure Austrian party competition
according to the Lipset and Rokkan’s (1967) cleavage theory.
Because not every MP gives a speech in parliament that belongs to the macro-policy
dimensions in the time period selected here, we cannot estimate a position of every MP on the
two policy dimensions under study. In a number of cases, by contrast, MPs give at least two
speeches on an issue that belongs to the economic or social policy dimension during the
legislative period. In those cases, we combine all speeches given of an MP that belong to the
economic or social policy dimension into one document, include this text in the Wordscores
analysis and use this score as a proxy for the MP’s position on the respective dimension. This
procedure has the advantage that we do not estimate the positions of very short speeches, which
may give biased results (Bernauer and Bräuninger 2009). We therefore use all speeches given in
the Austrian parliament to estimate the degree of intra-party cohesion in the Austrian parties.
Table 1 provides an overview on the specific policy domains that have been allocated to the
economic and societal policy dimension. The speeches of Austrian MPs are allocated according
to the committee affiliation of each member of the Nationalrat. Every speech held during one
specific debate has been assigned to the same policy area.
***Table 1***
12
Empirical analyses of cohesion
Descriptive analyses of Austrian MPs’ policy positions
In this section, we analyze the positions of MPs in a descriptive manner, and compare the
positions derived in the analysis of parliamentary speeches of Austrian MPs with the results of a
survey of members of the Austrian Parliament. Table 2 below shows the degree of correlation
between the Wordscores estimates and the self-placement of MPs on both policy dimensions
under study. The correlation between the scores of the MP survey and the estimated policy
positions on the basis of MP speeches is not extremely strong, in particular in case of the
economic policy dimension. Thus, the self-placements of MPs differ quite substantially from
our measurement on the basis of their speeches. The correlation is, however, significantly
positive, indicating that both techniques measure similar policy preferences for each MP.
***Table 2***
Table 3 and Figure 1 present the results of our computerized content analysis with the help of
the Wordscores algorithm of parliamentary speeches for the members of the Austrian parliament
in more detail. The lower the estimated scores of single MPs and parliamentary parties are, the
more the respective actor adopts economically left-wing or socially progressive policy positions.
The higher the estimated positions are, the more market-liberal and the more conservative the
MP. The table reports the mean value of the policy position of MPs by parliamentary party
group that gave at least one speech belonging to one of the two macro-policy dimensions. In
addition, we report the self-placements of the members of Austrian parliament on the basis of
the MP survey by Müller et al. (2001).
While the mean position of parties on the respective dimension reflects the common ordering
of the Austrian parties on an economic left-right dimension and a societal policy dimension in
case of the MP survey (Jenny 2006), the party positions extracted on the basis of parliamentary
speeches deviate from the common perspective on Austrian party competition to some degree.
This is in particular the case for the economic left-right policy dimension. There, the Liberal
Forum is located to the left of the economic left-right spectrum and placed between Greens and
Social Democrats. Furthermore, the FPÖ is the right-wing party in economic issues when
referring to the Wordscores analysis of parliamentary speeches, but is located at the centre when
using data from the MP survey. There are, however, no remarkable differences between the
results of both approaches when it comes to the policy dimension that deals with the order of
society. The Greens and the Liberals adopt progressive policy positions on average, while the
SPÖ MPs have a moderate position on the order of society. The only difference in party
placements is the position of the FPÖ. While the ÖVP and FPÖ adopt very similar conservative
positions on societal issues when looking at the mean placement of their MPs in the elite survey,
according to the analysis of speeches the FPÖ is clearly the most conservative party.
One possible explanation for the surprising moderate economic policy position of the SPÖ
expressed in the MPs’ speeches, could be the sitting coalition government of the two parties. On
the one hand, maintaining a stable government, particularly after the short-lived 1994–1995 one
between the same two parties, might require parties to contain conflict. On the other hand, a
13
governing party is expected to deliver on its promises and this was difficult for the SPÖ in a
parliament with a right-of-center majority. Hence, not making too leftist claims might have been
aimed at avoiding dissonance between government policy and parliamentary speeches.
***Table 3***
To describe the degree of intra-party programmatic heterogeneity in more detail, Figure 1 shows
the positions of each MP on the respective policy dimensions under study by differentiating
between parliamentary parties. To start with the SPÖ as the largest party in the studied
legislative period, this party’s MPs show a large diversity in their policy positions in societal
affairs. While most of the SPÖ MPs are located around the centre of that dimension, there is a
significant number of Social Democrats that adopt either very progressive or very conservative
positions on the order of society. There is also a quite large intra-party heterogeneity in the
economic policy domain of the SPÖ, which is, however, not as large as in the case of social
issues. The latter is in particular the case when referring to the data from the MP survey, where
the SPÖ MPs show a clearly stronger cohesion in economic issues compared to societal policy.
***Figure 1***
The scatter plot of the policy positions of the People’s Party MPs delivers a different picture.
While there is also a group of MPs in the case of the ÖVP that shows progressive and very
conservative societal policy positions when referring to the Wordscores data, the MP survey
shows a different picture. The latter data source shows, by contrast, that the economic policy
position of the People’s Party MPs clearly varies between market liberalism on the one hand
and a preference for a strong welfare state on the other hand.
When referring to the MP positions extracted by the analysis of parliamentary speeches, the
economic and societal policy preferences of the Freedom Party MPs are clearly located more to
the right when compared to SPÖ and ÖVP MPs. When looking at the MP survey data, however,
this pattern becomes less clear. According to their self-placements, FPÖ MPs are mostly located
at the centre of the economic left-right dimension. Likewise to the Wordscores data, there are a
few FPÖ MPs with moderate, but not clearly progressive positions on societal issues, while
most FPÖ MPs hold very conservative views on the order of society.
In case of the two smaller parliamentary groups – the Greens and the Liberals – the degree of
programmatic diversity is much smaller compared to the larger parties SPÖ, ÖVP and FPÖ.
MPs of the Liberals and Greens have, however, quite similar economic positions when applying
the dataset that is based on the analysis of parliamentary speeches. The results of the survey
among MPs are very different in this regard: here, the MPs of the Austrian Greens place
themselves clearly on the left of the economic policy dimension, while the members of the
liberal parliamentary party group adopt clearly market liberal positions. However, MPs from
both parties have similar progressive positions on the order of society, which is in line with the
findings from the analysis of parliamentary speeches.
14
To sum up, the descriptive analysis of our estimates on MP policy positions that are based on
a content analysis of parliamentary speeches shows some, but not perfect, face validity when
comparing it with data from MP self-placements on similar policy dimensions.6
A multivariate analysis of Austrian MPs’ policy positions
To further evaluate the measures of MPs’ policy positions we have derived from an analysis of
parliamentary speeches, we calculate various OLS regression models where our dependent
variable measures ideological cohesion at the individual level, that is, the absolute distance
between the policy position of an MP and the position of the party core. Instead of analyzing
programmatic cohesion for each policy dimension separately, we refer to the Euclidean
distances in our two-dimensional policy space (e.g., Hinich and Munger 1997; Kam et al. 2010).
The position of the party core is measured as the arithmetic mean of the policy positions of
all its MPs.7 Thus, a positive coefficient suggests that a feature increases the deviation between
an MP’s estimated policy position and the position of the party core, whereas a negative
coefficient suggests that a feature decreases the MP’s deviation from the party line. The main
explanatory variables are the length of the parliamentary career of each MP, his or her potential
leadership function inside the parliamentary party group, and whether an MP was a member of
the governing party. In addition, we include a party strength variable in terms of seat share into
the analysis as a control, since ideological cohesion can be expected to be lower in larger
parties. We also incorporate variables that reflect contextual and institutional factors such as the
aspect whether the party of an MP is currently part of the coalition government and whether the
MP holds a regional district seat or not. In addition, we include, in some of our models, a
number of dummy variables describing the different Austrian states, in order to control for the
fact that there may be regional differences in the level of intra-party cohesion among MPs.
Table 4 presents the results of an OLS regression with the Euclidean distance of an MP from
the party core as the dependent variable. In every model, we include the Euclidean distance
between an MP and the party core measured on the basis of the elite survey by Müller et al.
(2001) in order to evaluate whether the two measures are correlated when we control for a
number of other important individual-level features. Beside this variable, the first model
includes institutional factors and dummy variables for each Austrian state only.
The variable that identifies whether the respective MP belongs to the government camp or
not shows a significant effect. As the positive sign of the coefficient indicates, MPs from
governing parties show a larger distance from the party core than opposition MPs. This is not in
line with our hypothesis, since we expected the leadership of governing parties to have more
incentives and resources to impose discipline on and provide rewards to MPs. The variable that
6 The weak face validity of our MP positions could indicate that there is an incorrect specification in our
analysis of parliamentary speeches that belong to the economic policy dimension. One potential reason
for this could be the allocation of speeches to the respective policy dimension is problematic. We have
here made the assumption that members of specific committees should mostly give speeches related to the
committee’s policy profile. It could be that not every speech given by an MP that is a member of a
committee listed in Table 1 has an economic policy or social policy background. Thus, one incentive for
next steps in this project is to find further possibilities to identify the policy area a speech belongs to. 7 The results of the regression analyses do not significantly change when we use an exogenous measure
for the party core position like the party position drawn from the Benoit and Laver (2006) expert survey.
15
identifies how the MP is elected has a statistically significant impact. If he or she is holds a
regional district seat then the MP has a significantly higher distance from the mean position of
the party. This is completely in line with our hypothesis, thus supporting the idea that MPs who
represent regional districts have stronger incentives to cultivate a personal “profile”. While the
regression analysis provides a positive effect of the distance between MP and party core
measured via the survey, the estimated coefficient is not significant in the first model.
Model 2 includes party organizational features only. Here, we expected that the length of the
parliamentary career of an MP should affect the degree of his or her deviation from the party
core. As the regression results reveal, there is no significant effect of this variable on the
distance between an MP and the parliamentary party core. There is also no significant effect of
the variable that identifies if the MP holds a leadership position. However, we now find a
significant effect of the distance between an MP and the party core measured by the MP survey.
Model 3 replicates model 2, but includes the strength of parliamentary parties as a control
variable. This variable has the expected positive effect, implying that MPs are more likely to
deviate from the party line the larger the party group.
***Table 4***
In model 4 we combine all explanatory variables from the different theoretical approaches. The
results of the regression analysis reveal that MPs elected directly in the sense that they hold a
regional district seat and parliamentarians from the governing parties deviate to a stronger
degree from the party core. While the finding mentioned first is in line with our expectations,
the second is not. Additionally, the effect of parliamentary career becomes statistically
significant in model 4: the longer a politician served as an MP, the less likely it is that he
deviates from the party line. This finding is clearly in line with our expectation: MPs with
longer parliamentary careers and, thus, a better understanding of the rules of the game, give
speeches in the parliament which are more in line with the policy positions of their party.
Finally, we get a positive and significant effect of the distance between an MP and the party
core measured on the basis of the elite survey by Müller et al. (2001), indicating that there is a
clear relationship between the measurement of deviation from the party core on the basis of
parliamentary speeches and MP self-placements, even when controlling for other features.
Additionally, we present one model (5) in table 4 where we exclude the MP survey variable,
in order to evaluate whether our findings are altered when we do not control for an MP’s policy
position as measured in the survey. The results of our analyses hold when excluding this
variable, and actually some results become slightly stronger, for example the effect of holding a
regional district seat is now significant at the 0.05 level as opposed to the previous 0.10 level.
To sum up, our findings (see also Table 5) indicate that several variables drawn from the
previous literature can be used as predictors of the distance between the MPs and the party core.
Most importantly, in the analysis of Austrian MPs, we find that MPs who hold regional district
seats and who have a shorter parliamentary experience are more likely to adopt positions in
speeches which are not congruent with the position of the party core.
***Table 5***
16
Concluding remarks
The aim of this paper was to evaluate whether a computerized content analysis of speeches of
MPs can be used to estimate the programmatic cohesion of parties. In this paper we have
discussed the pros and cons with applying this methodological approach in comparison to the
main alternative approaches, such as relying on expert surveys and parliamentary voting
behavior. The conclusion in this discussion is that there are a number of advantages with
analyzing parliamentary speeches when we aim to measure the positions of various actors
within parties, and that the main alternative source available, the analysis of recorded or roll-call
votes, is severely limited. In addition, we have also set out to evaluate the validity of the
measures we derive from an analysis of speeches made by political actors in parliament. The
question is, can speeches justifiably be used to evaluate the policy positions of MPs, which can
be used to create measures of ideological or programmatic cohesion within parties?
To answer this question, we have performed a case study of Austrian MPs and the speeches
that they held during one legislative period during the late 1990s. In this case, we have access to
both MP survey data, where respondents were asked to place themselves along a number of
ideological dimensions, and we have access to the speeches that the respective MPs have made.
When comparing the policy positions derived from an analysis of parliamentary speeches with
those derived from MP survey responses at the individual level, we find that there is a relatively
high (but far from perfect) correlation between the two measures. In addition, we have tried to
validate the policy positions drawn in the computerized speech analysis by performing a
multivariate analysis where we analyze the MPs’ deviations from the party core as a dependent
variable, and where a number of important explanatory features are included as predictors. This
analysis gives rather encouraging results, showing that MPs who hold regional district seats and
who have a shorter parliamentary experience are more likely to adopt positions in speeches
which deviate from the position of the party core.
These results set incentives for further studies measuring intra-party heterogeneity of parties
on the basis of speeches, and we suggest that future research makes use of the rich and easily
accessible data source that parliamentary speeches constitute. One such avenue of research is to
use the policy positions of politicians derived from speeches to predict who becomes a cabinet
member, for example, Bäck et al. (2011) show that politicians are more likely to become
ministers if they take a policy position that is closer to the PM or the coalition agreement.
Of course, some of our results are less encouraging, for example the correlation between MP
surveys and our estimates derived from the speeches is far from perfect, and not all explanatory
features included to predict whether an MP deviates from the party line exerted significant
effects in the expected direction. We therefore need to take a closer look at the specifications
made in the Wordscores analysis, for example to evaluate whether the selection of reference
texts and reference scores could have a decisive impact on the final results, and whether an
alternative allocation of speeches to specific policy areas provide better results. In addition, we
aim to apply alternative content analysis techniques to our data, to evaluate whether various
other approaches give similar results, for example using “Wordfish” (Slapin and Proksch 2008),
a fully computerized technique likewise to Wordscores, but also using semi-manual techniques
that are based on dictionaries (e.g., Laver and Garry 2000; Schonhardt-Bailey 2005).
17
References
Aylott, Nicholas. 2003. “After the Divorce: Social Democrats and Trade Unions in Sweden.” Party
Politics 9: 369–390.
Bäck, Hanna. 2003. Explaining Coalitions. Evidence and Lessons From Studying Coalition Formation in
Swedish Local Government. Uppsala: Acta Universitatis Upsaliensis.
Bäck, Hanna, Marc Debus, and Wolfgang C. Müller. 2011. “Intra-party Conflict and Ministerial Selection
in Coalition Governments”. Paper presented at the ECPR Standing Group on Parliaments Conference
in Leiden, 24–25 November.
Becher, Michael and Sieberer, Ulrich. 2008. “Discipline, Electoral Rules and Defection in the Bundestag,
1983-94.” German Politics 17: 293–304.
Benoit, Kenneth and Laver, Michael. 2006. Party Policy in Modern Democracies. London, New York:
Routledge.
Benoit, Kenneth and Laver, Michael. 2007. “Benchmarks for Text Analysis: A Reply to Budge and
Pennings.” Electoral Studies 26: 130–135.
Benoit, Kenneth and Laver, Michael. 2008. “Compared to What? A Comment on ‘A
Robust Transformation Procedure for Interpreting Political Text’ by Martin and Vanberg.” Political
Analysis 16: 101–111.
Benoit, Kenneth, Mikhaylov, Slava and Laver, Michael. 2009. “Treating Words as Data with Error:
Uncertainty in Text Statements of Policy Positions.” American Journal of Political Science 53: 495–
513.
Bernauer, Julian and Bräuninger, Thomas. 2009. “Intra-Party Preference Heterogeneity and Faction
Membership in the 15th German Bundestag. A Computational Text Analysis of Parliamentary
Speeches.” German Politics 18 (3): 385–402.
Bollen, Kenneth A. 1998. Structural Equations with Latent Variables. New York: John Wiley & Sons.
Bräuninger, Thomas; Debus, Marc and Müller, Jochen. 2010. “Estimating Hand- and Computer-Coded
Policy Positions of Political Actors Across Countries and Time.” University of Mannheim:
Unpublished Manuscript.
Budge, Ian, Klingemann, Hans-Dieter, Volkens, Andrea, Bara, Judith and Tanenbaum, Eric (eds). 2001.
Mapping Policy Preferences. Estimates for Parties, Electors and Governments 1945–1998. Oxford:
Oxford University Press.
Budge, Ian and Pennings, Paul. 2007. “Do they work? Validating computerised word frequency estimates
against policy series.” Electoral Studies 26: 121–129.
Carey, John M. 2007. “Competing Principals, Political Institutions, and Party Unity in Legislative
Voting.” American Journal of Political Science 51 (1): 92–107.
Carrubba, Clifford J., Matthew Gabel, Lacey Murrah, Ryan Clough, Elizabeth Montgomery, and Rebecca
Schambach. 2006. “Off the Record: Unrecorded Legislative Votes, Selection Bias and Roll-call Vote
Analysis.” British Journal of Political Science 36 (4): 691–704.
Curini, Luigi. 2007. “The Ideology of Italian Political Experts in Comparative Perspective. An Unfolding
Analysis based on Benoit-Laver Expert Survey.” Italian Political Science 1.
Damgaard, Eric. 1995. “How Parties Control Committee Members”, in Herbert Döring (ed.), Parliaments
and Majority Rule in Western Europe. New York, NY: St. Martin’s Press, 308–325.
Debus, Marc and Bräuninger, Thomas. 2008. “Intraparty Factions and Coalition Bargaining in Germany”,
in Giannetti, Daniela; Benoit, Kenneth (eds.), Intra-Party Politics and Coalition Governments in
Parliamentary Democracies. London, New York: Routledge, 121–145.
Depauw, Sam. 2003. “Government Party Discipline in Parliamentary Democracies: The Cases of
Belgium, France and the United Kingdom in the 1990s.” Journal of Legislative Studies 9 (4): 130–146.
Fenno, Richard F. 1973. Congressmen in Committees. Boston: Little, Brown.
Giannetti, Daniela; Benoit, Kenneth (eds). 2009. Intra-Party Politics and Coalition Governments in
Parliamentary Democracies. London, New York: Routledge.
Giannetti, Daniela; Laver, Michael. 2005. “Policy Positions and Jobs in the Government”. European
Journal of Political Research 44: 91–120.
18
Giannetti, Daniela and Laver, Michael. 2009. “Party Cohesion, Party Discipline, and Party Factions in
Italy”, in Daniela Giannetti, and Kenneth Benoit (eds), Intra-Party Politics and Coalition Governments
in Parliamentary Democracies. London, New York: Routledge.
Gilligan, Thomas W. and Keith Krehbiel. 1994. “The Gains from Exchange Hypothesis of Legislative
Organization.” Legislative Studies Quarterly 19: 181–214.
Hinich, Melvin and Munger, Michael C. 1997. Analytical Politics. Cambridge: Cambridge University
Press.
Hix, Simon, Noury; Abdul and Roland, Gérard. 2006. “Dimensions of Politics in the European
Parliament”. American Journal of Political Science 50: 494–511.
Janda, Kenneth. 1980. Political Parties. A Cross-National Survey. New York: Free Press.
Jenny, Marcelo. 2006. “Programme: Parteien im politischen Wettbewerbsraum”, in Herbert Dachs et al.
(eds.), Politik in Österreich. Das Handbuch. Vienna: Manz, 305–321.
Kam, Christopher, Bianco, William T., Sened, Itai and Smyth, Regina. 2010. Ministerial Selection and
Intraparty Organization in the Contemporary British Parliament. American Political Science Review
104: 289-306.
Kitschelt, Herbert and Smyth, Regina. 2002. “Programmatic Party Cohesion in Emerging Post-
Communist Democracies.” Comparative Political Studies 35: 1228–1256.
Klingemann, Hans-Dieter, Volkens, Andrea, Bara, Judith, Budge, Ian and McDonald, Michael. 2006.
Mapping Policy Preferences II: Estimates for Parties, Electors and Governments in Central and
Eastern Europe, European Union and OECD 1990–2003. London: Routledge.
Krehbiel, Keith. 2000. “Party Discipline and Measures of Partisanship.” American Journal of Political
Science 44: 212–227.
Laver, Michael and Benoit, Kenneth. 2002. “Locating TDs in Policy Spaces: Wordscoring Dáil
Speeches.” Irish Political Studies 17: 59–73.
Laver, Michael, Benoit, Kenneth and Garry, John. 2003. “Extracting Policy Positions from Political Texts
Using Words as Data.” American Political Science Review 97: 311–331.
Laver, Michael, Benoit, Kenneth, and Sauger, Nicolas. 2006. “Policy Competition in the 2002 French
Legislative and Presidential Elections.” European Journal of Political Research 45: 667–697.
Laver, Michael and Garry, John. 2000. “Estimating Policy Positions from Political Texts.” American
Journal of Political Science 44: 619–634.
Laver, Michael and Hunt, W. Ben. 1992. Policy and Party Competition. New York: Routledge.
Lipset, Seymour M. and Rokkan, Stein. 1967. “Cleavage Structures, Party Systems and Voter
Alignments: An Introduction”, in Seymour M. Lipset and Stein Rokkan (eds), Party Systems and Voter
Alignments: Cross-National Perspectives. New York, London: The Free Press, 1–64.
Lowe, Will. 2008. “Understanding Wordscores.” Political Analysis 16: 356–371.
Mair, Peter. 2001. “Searching for the Positions of Political Actors”, in Michael Laver (ed.), Estimating
the Policy Positions of Political Actors. London: Routledge, 10–30.
Marks, Gary, Hooghe, Liesbet, Steenbergen, Marco R., and Bakker, Ryan. 2007. “Crossvalidating Data
on Party Positioning on European Integration.” Electoral Studies 26 (1): 23–38.
Marks, Gary and Steenbergen, Marco R. 2010. “Reliability and Validity of Measuring Party Positions:
The Chapel Hill Expert Surveys of 2002 and 2006.” European Journal of Political Research 49: 687–703
Martin, Lanny W. and Vanberg, Georg. 2008. “A Robust Transformation Procedure for Interpreting
Political Texts.” Political Analysis 16: 93–100.
Mikhaylov, Slava, Laver, Michael and Benoit, Kenneth. 2008. “Coder Reliability and Misclassification in
Comparative Manifesto Project Codings.” Chicago: MPSA Annual National Conference, April 3–6.
Müller, Wolfgang C. 2005. “Austria: A Complex Electoral System with Subtle Effects”, in Michael
Gallagher and Paul Mitchell (eds.), The Politics of Electoral Systems. Oxford: Oxford University
Press, 397–416.
Müller, Wolfgang C. and Jenny, Marcelo. 2000. “Abgeordnete, Parteien und Koalitionspolitik:
Individualle Präferenzen und Politisches Handeln im Nationalrat.” Österreichische Zeitschrift für
Politikwissenschaft 29 (2): 133–152.
19
Müller, Wolfgang C., Marcelo Jenny, Barbara Steininger, Martin Dolezal, Wilfried Philipp and Sabine
Preisl-Westphal. 2001. Die österreichischen Abgeordneten. Individuelle Präferenzen und politisches
Handeln. Vienna: Fakultas WUV.
Müller, Wolfgang C. and Christian Scheucher. 1994. “Persönlichkeitswahl bei der Nationalratswahl
1994.” Österreichisches Jahrbuch für Politik 1994: 171–197.
Poole, Keith T. and Rosenthal, Howard. 1997. Congress. A Political-Economic History of Roll Call
Voting. Oxford: Oxford University Press.
Proksch, Sven-Oliver, and Jonathan B. Slapin. Forthcoming. “Institutional Foundations of Legislative
Speech”. Forthcoming American Journal of Political Science.
Ray, Leonard. 2007. “Validity of Measured Party Positions on European Integration: Assumptions,
Approaches, and a Comparison of Alternative Measures.” Electoral Studies 26 (1): 11–22.
Saalfeld, Thomas. 1995a. Parteisoldaten und Rebellen. Fraktionen im Deutschen Bundestag 1949–1990.
Opladen: Leske und Budrich.
Saalfeld, Thomas. 1995b. “On Dogs and Whips: Recorded Votes”, in Herbert Döring (ed.), Parliaments
and Majority Rule in Western Europe. New York, NY: St. Martin’s Press, 528–565.
Schonhardt-Bailey, C (2005). “Measuring Ideas More Effectively: An Analysis of Bush and Kerry’s
National Security Speeches”, Political Science & Politics 38, 701–11.
Sieberer, Ulrich. 2006. “Party Unity in Parliamentary Democracies.” Journal of Legislative Studies 12
(2): 150–178.
Slapin, Jonathan B. and Proksch, Sven-Oliver. 2008. “A Scaling Model for Estimating Time-Series Party
Positions from Texts.” American Journal of Political Science 52: 705–722.
Steenbergen, Marco R. and Marks, Gary. 2007. “Evaluating Expert Judgments.” European Journal of
Political Research 46 (3): 347–366.
Volkens, Andrea. 2001. “Quantifying the Election Programmes: Coding procedures and controls”, in:
Budge, Ian, Klingemann, Hans-Dieter, Volkens, Andrea, Bara, Judith and Tanenbaum, Eric (eds.),
Mapping Policy Preferences. Estimates for Parties, Electors and Governments 1945–1998. Oxford:
Oxford University Press, 93–109.
Weingast, Barry R. 1987. “A Rational Choice Perspective on Congressional Norms”, in Mathew D.
McCubbins and Terry Sullivan (eds.), Congress: Structure and Policy. Cambridge: Cambridge
University Press, 131–146.
Weingast, Barry R. and William J. Marshall. 1988. “The Industrial Organization of Congress; or, Why
Legislatures, Like Firms, Are Not Organized as Markets.” Journal of Political Economy 96: 132–163.
20
Tables and figures
Table 1. Allocation of speeches from specific policy areas to macro-policy dimensions
Macro-policy dimension Policy area
Economic policy dimension Labour and welfare issues, Budget, Finance, Industry, Economy
Societal policy dimension Family, Equal opportunity, Interior, Justice, Culture, Health
Table 2. Correlations between the self-placements of Austrian MPs on macro-policy dimensions
and the positions drawn from the Wordscores estimation
Policy position according to Wordscores
analysis
Economic policy position
according to elite survey
Societal policy position
according to elite survey
Speeches and manifestos as reference
texts
0.241* 0.368*
Note: * significant at 1%. Elite survey data drawn from Müller et al. (2001).
Table 3. Policy-area specific positions of Austrian MPs by party and data source
Party
Policy position
Economic dimension (SD, N) Social dimension (SD, N)
Wordscores data MP survey data Wordscores data MP survey data
SPÖ 9.92 (3.29, 66) 1.93 (0.49, 70) 9.85 (5.68, 58) 3.69 (1.10, 70)
ÖVP 11.62 (3.41, 55) 4.19 (1.02, 50) 11.67 (6.11, 46) 5.45 (0.89, 50)
FPÖ 11.84 (2.78, 40) 4.01 (0.92, 42) 13.46 (3.77, 35) 5.44 (0.79, 42)
Greens 6.06 (2.11, 8) 1.75 (0.38, 9) 7.56 (3.81, 8) 1.67 (0.17, 9)
LIF 7.92 (2.86, 9) 5.10 (0.55, 9) 9.24 (2.65, 8) 1.80 (0.49, 9)
Note: The lower the policy position score of a political actor, the more leftist or progressive is his/her
economical/societal policy position. The higher the score, the more market-liberal and socially
conservative position he/she adopts. While the MP survey data can vary between 1 (very left/progressive)
and 7 (very right/conservative), the Wordscores estimates range between -5 (very left/progressive) and 25
(very right/conservative; standardization method according to Laver et al. 2003).
21
Table 4. Regression analyses with deviation from party core (by Wordscores) as dependent variable
(1) (2) (3) (4) (5)
MP survey variable
Distance MP–core 0.27 0.50* 0.33* 0.28*
(0.13) (0.14) (0.09) (0.09)
Institutional variables
Regional district seat 1.27* 1.42* 1.26**
(0.29) (0.32) (0.27)
Governing party member 1.47** 1.70** 1.70**
(0.06) (0.11) (0.13)
Party organizational vars.
Parliamentary career -0.00 -0.03 -0.06* -0.07*
(0.03) (0.02) (0.02) (0.02)
Party leadership function 0.85 1.01 0.87 0.37
(1.17) (1.03) (0.81) (1.10)
Party size 7.24** -1.00 -0.86
(1.15) (0.99) (1.17)
Constant 3.79** 4.67** 2.95** 4.19** 4.01**
(0.42) (0.74) (0.61) (0.54) (0.27)
State dummies included Yes No No Yes No
Observations 148 148 148 148 148
R-squared 0.15 0.02 0.07 0.16 0.13
Note: By parliamentary party group clustered standard errors in parentheses; * significant at 10%; ** significant
at 5%; *** significant at 1%
22
Table 5. Summary of results
Confirmed (+) / Not confirmed (-)
Distance MP – party core (according to the MP survey) +
MP holds regional district seat +
MP is a member of the governing party -
Length of MPs parliamentary career +
Party leadership function of MP (-)
Note: signs in parentheses imply that the estimated effect was not statistically significant.
.
23
Figure 1. Policy positions of Austrian MPs by party and estimation method
ÖVP
Wordscores estimates MP self-placements -5
05
10
15
20
25
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
-5 0 5 10 15 20 25
Economic policy left-right
MPs Party core
12
34
56
7
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
1 2 3 4 5 6 7
Economic policy left-right
MPs Party core
SPÖ
Wordscores estimates MP self-placements
-50
51
01
52
02
5
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
-5 0 5 10 15 20 25
Economic policy left-right
MPs Party core
12
34
56
7
So
cia
l po
licy p
rogre
ssiv
e -
co
nse
rvative
1 2 3 4 5 6 7
Economic policy left-right
MPs Party core
FPÖ
Wordscores estimates MP self-placements
-50
51
01
52
02
5
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
-5 0 5 10 15 20 25
Economic policy left-right
MPs Party core
12
34
56
7
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
1 2 3 4 5 6 7
Economic policy left-right
MPs Party core
24
Greens
Wordscores estimates MP self-placements
-50
51
01
52
02
5
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
-5 0 5 10 15 20 25
Economic policy left-right
MPs Party core
12
34
56
7
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
1 2 3 4 5 6 7
Economic policy left-right
MPs Party core
LIF
Wordscores estimates MP self-placements
-50
51
01
52
02
5
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
-5 0 5 10 15 20 25
Economic policy left-right
MPs Party core
12
34
56
7
So
cie
tal po
licy p
rogre
ssiv
e -
co
nse
rvative
1 2 3 4 5 6 7
Economic policy left-right
gespol mean_soc_mps