revisiting assessing the quality of democracy: measuring democratic competition and participation in...

8/7/2019 Revisiting Assessing the Quality of Democracy: Measuring Democratic Competition and Participation in Latin Americ

1/20

REVISITING ASSESSING THE QUALITY OF DEMOCRACY:

Measuring Democratic Competition and Participation in Latin Americasince the Left Turn

Miguel CentellasUniversity of Mississippi

[email protected]

Abstract

This paper revisits the challenge of empirically measuring the quality ofdemocracy in Latin America in the wake of the regions left turn since the1990s and the rise of classic populist leaders in several countries. Buildingon an earlier effort by David Altman and Anbal Prez-Lian to measurethe quality of democracy, this paper develops a revised measure of

effective competition. The paper develops this new measure through anexploratory assessment of nine key cases. The paper then tests the validityof this measure against the Freedom House index, a widely acceptedassessment of the quality of democracy. Finally, the paper then measuresthe nine cases, at various points in time, along two dimensions consistentwith Robert Dahls understanding of polyarchy: effective competition andeffective participation (based on the authors previous work).

Prepared for presentation at the 2011 Annual Meeting of the Midwest Political ScienceAssociation, Chicago, March 31-April 3.

I thank Carlos Gervasoni, Mark P. Jones, Felipe Botero, David Altman, and Anbal Prez-Lin for their assistance and advice.


2/20

Centellas 1

Revisiting Assessing the Quality of Democracy

Introduction

Three decades since the third wave of democratization hit Latin America, concernremains over the quality of democracy in many countries. Although there have not beenany outright democratic reversals, questions remain over the consolidation of individualdemocratic regimes and long-term prospects of democracy throughout the region. Thereturn of populism in the 1990swhether neopopulists such as Perus Alberto Fujimorior classic populists, such as Venezuelas Hugo Chvezraised doubts about the quality ofdemocracy and suggested the emergence of new forms of illiberal regimes (Smith andZeigler 2008). Similarly, the ability of social movements to drive out sitting presidents inplaces like Bolivia raised concerns about the ability of existing democratic regimes toeffectively channel social demands. Throughout it all, however, no Latin Americandemocracy has completely broken down and reverted to outright authoritarianism. Still,we recognize that there are significant differences between the quality of democracy in acountry like Venezuela and Costa Rica. The dilemma, of course, is that such differencesare difficult to empirically measure, which limits our ability to pursue research questionsin which quality of regime is an important variable.

This paper revisits the challenge of measuring the quality of democracy in LatinAmerica. The first part provides a brief overview of the concept of democracy as it hasbeen used in much of the comparative democratization literature, focusing on the threekey dimensions of polyarchy: competition, participation, and civil and political liberties(Srensen 2008). It then makes the case for anchoring the assessment of the quality ofdemocracy on a defining feature of modern democracy: elections. From here, the paperdevelops indicators for measuring the effective competition and effective participation ofdemocratic regimes, building on previous work by David Altman and Anbal Prez-Lin(2002). A close look at several recent cases shows that the measures developed byAltman and Prez-Lin do not perform well under certain conditions. Thus, a key

contribution of this paper is to suggest some corrective measures that better reflectdifferences in competitiveness between electoral democracies of different quality. Next,the indicators are applied to a selection of nine Latin American cases across their mostrecent elections. The measures are then compared to existing measures two commonlyaccepted measures for democracy: the Freedom House and Polity IV indexes. Finally, thepaper turns to a discussion of the advantages and limitations of the proposed measures.

Assessing the Quality of Democracy

Discussions about the quality of democracy can take many different forms. Typically, theterm is used to describe the extent to which a democratic regime achieves the normativestandard for democracy. Such discussions often focus on various outputs of democracy,such as the rule of law (Maravall and Przeworski 2003; ODonnell 2004), policies thatreduce inequality (Rueschemeyer 2004; Bermeo 2009; Carbone 2009), or efforts toprotect or expand civil liberties (Beetham 2004). While these issues are importantandperhaps difficult to separate from our basic understanding of what democracy is andwhat it mattersthey move us further away from conceptual discussions of democracy asa type of political regime.

In their seminal article, Philippe Schmitter and Terry Lynn Karl (1991) warnedabout the confusion generated by imputing additional values into democracy. We may


3/20

Centellas 2

very well wish for democracies to produce a number of other normative values; but thereis no inherent reason that every democracymustproduce the same values. Schmitterand Karl thus push for a procedural (or minimal) definition of democracy:

Modern political democracy is a system of governance in which rulers are heldaccountable for their actions in the public realm by citizens, acting indirectly through

the competition and cooperation of their elected representatives (1991, 76).

Their definition is self-consciously derived from Joseph Schumpeters formulation ofdemocracy as an institutional arrangement for arriving at political decisions by meansof competitive struggle for the peoples vote (1975, 269). While acknowledging, likeRobert Dahl (1971), that a number of procedural minimal conditions must exist fordemocracy to function, these are essentially reduced to background (or prerequisite)conditions, not specific indicators of democracy itself. To the seven institutionalguarantees proposed by Dahl (1982),1 Schmitter and Karl include two others: electedofficials must have autonomy from other actors (e.g. the military) and the polity must beself-governing (i.e. it must enjoy full sovereignty). But none of these guarantees bythemselves define democracy.

The Freedom House index (FH) is one of the most commonly used indicators ofthe quality of democracy. While Freedom House self-consciously points out that its indexmeasures the level of political rights and civil liberties, and not democracy per se, FHscores are often used to define democracies, as well as their differences in quality. A lookat two outliersMonaco and Liechtensteindemonstrates that Dahls institutionalguarantees do not automatically match up with democratic regimes. Freedom Houseconsistently rates both Monaco and Liechtenstein as free. And though neither countryqualifies as a democratic regime, their populations do enjoy a wide range of civil libertiesand political freedoms.

Other measures of democracy suffer from similar problems, as highlighted in areview of existing measures of democracy by Gerardo Munck and Jay Verkuilen (2002).The three strongest indices identified by Muchk and Verkuilen were those produced byMichael Alvarez et al (1996), Michael Coppedge and Wolfgang Reinicke (1991), and Axel

Hadenius (1992). The three weakest indices were those produced by Freedom House,Mark Gasiorowski (1996), and Tatu Vanhanen (2000). Munck and Verkuilen praise thefirst group primarily for their strong conceptualization, sophisticated operationalization,and transparent coding procedures. However, these are not without important shortfalls.The Alvarez et al measure is very good at distinguishing between authoritarian anddemocratic regimes, but it lacks the ability to distinguish variations within each regimetype. The Coppedge and Reinicke polyarchy index is better able to distinguish variationswithin democracies, but it relies primarily on measures of Dahls procedural guarantees;as such, it is a more robust or empirically objective version of the Freedom House index.Similarly, the Hadenius democracy index includes two dimensions: elections andpolitical liberties. But the electoral dimension is primarily devoted to assessing whethervoters have optionsit does not then assess how or whether competitive elections

1 These are: 1) Control over government decisions is constitutionally vested in elected officials. 2)Elected officials are chosen in frequent and fairly conducted elections in which coercion iscomparatively uncommon. 3) Practically all adults have the right to vote in the election ofofficials. 4) Practically all adults have the right to run for elective office 5) Citizens have a rightto express themselves without the danger of severe punishment on political matters broadlydefined 6) Citizens have a right to seek out alternative sources of information [which] areprotected by law. 7) Citizens also have the right to form relatively independent associations ororganizations, including independent political parties and interest groups (Dahl 1982, 11).


4/20

Centellas 3

translate into competitive regimes. Thus, this measure is primarily an index of how freeand fair elections are. One of the popular indexes, the Polity IV index, did not rate nearthe bottom, but was also heavily criticized by Munck and Verkuilen for a number ofproblems in conceptualization and operationalization. Still, it should be noted that PolityIV data relies primarily on coder assessments of indicators (Marshal, Gurr, and Jaggers2009). This is very similar to the Freedom House procedure, with the difference that it

looks specifically at regime characteristics, rather than civil liberties and political rights.Recently, The Economisthas developed yet another democracy index (see EconomistIntelligence Unit 2010). An improvement over FH measures, this index combines coderassessments with other sources of data, most notably public opinion (used to assess thepolitical culture dimension). Nevertheless, we are lacking robust empirical measures ofdemocracy that can distinguish between democratic regimes (across cases and acrosstime) that are transparent, reliable, and focus on the core dimensions of democracy:competition and participation.

This paper adheres closely to Dahls two procedural dimensions of democracy:contestation (competition) and inclusiveness (participation). This follows the minimalapproach developed by Vanhanen (2000), whose index relied exclusively on measures ofparticipation and competition derived from existing electoral data, and the Altman and

Prez-Lin (2002) measure for political competition. The chief advantage of such anapproach is that it can be easily calculated and, like the Polity IV index, be constructedfor historical cases that predate other indices (such as Freedom House, whose index onlydates back to 1972).

Vanhanens index is conceptually strongand therefore appealing for a numberof reasons. Where it falls short was in its operationalization. Vanhanens measure forcompetition merely subtracts the percentage of votes (or seats, where vote data is notavailable) for the largest party from 100; the measure for participation is merely thepercentage of voters as a share of the countrys total population (2000, 253). Althoughhe adjusts the scores to distinguish between systems with parliamentary dominance(parliamentarism) and executive dominance (presidentialism), the procedure is crudeand relatively arbitrary: weighting each election 25/75, depending on the case

(Vanhanen 2000, 255).Most recently, Daniel Pemstein, Stephen Meserve, and James Melton (2010)have suggested a compromise approach that uses a Bayesian latent variable approachto combine various existing indices into one scale. While methodologically sophisticated,their approach suffers from an inherent weakness: it combines indices developed withdifferent underlying assumptions and assumes they tap into the same latent dimensions.For example, scholars who use the Freedom House index as a measure of democracyshould remember that FH scores are not designed to measure democracy per se. FHscores measure the degree of civil liberties and political rights that are effectivelyexercised in a society. But it remains unclear whether these are prerequisites or effects ofdemocracy. Another problem is that their method requires scholars to continue to relyon existing indices that may or may not be updated and may or may not be updated inthe same way (their own internal methodologies may change). Such a procedureundoubtedly allows for a great degree of confidence that one is correctly identifyingdemocracies because of general scholarly consensus inherent in the use of multipleindices. But it does not actually allow one to measure democracy independently as newdata becomes available for any country or countries of interest.

This paper seeks to build upon the basic conceptual understanding shared byDahl, Schmitter and Karl, and Vanhanen that democracy is best understood as a regimetype defined by the combination of high levels of participation andcompetition relative


5/20

Centellas 4

to other types of regimes.2 This paper goes further, however, to propose measures foreffective competition and participation that are conceptually grounded, transparent, easyto replicate, and consistent with other established indicators for democracy. To do so,this paper begins with the three core dimensions of democracy:

Effective Civil Rights and Political Liberties: Civil rights and political liberties are the

background condition for democracy. Thus, we should not accept as democratic anyregime that does not provide minimal protections for civil rights and political liberties. Itis unlikely that a regime can exhibit high levels of competition and participation in theabsence of effective civil rights and political liberties. Further, levels of competition andparticipation should be strongly correlated with levels of civil rights and politicalliberties, since these are understood as institutional guarantees that make democracypossible. FH scores serve as a useful indicator of this dimension, and all the casesincluded in this paper are rated as either free or partly free by Freedom House duringthe relevant years; the measures developed below for participation and competition willbe cross-referenced with FH scores.

Effective Participation: Participation is a key dimension of democracy. Although robust

views of democracy recognize participation beyond the electoral arena (whether in civilsociety or other areas) as essential to the vitality of democracy, any democracy minimallyincludes broad levels of participation in elections for public offices. Higher levels ofelectoral participation reflect public confidence in and affinity for the democratic regime.Although electoral participation can also take many forms (e.g. donating time or moneyto campaigns), voting is the minimal way in which citizens can participate in democraticdecision-making. Thus, voter turnout is a key indicator. However, measures that rely onturnout run into problems, such as differences in voter registration. This paper thereforedevelops an alternate measure for effective participation (described below) that bettercaptures the degree to which citizens actively (or effectively) participated in elections.

Effective Competition: Competition is perhaps the most crucial dimension of democracy.

While many authoritarian regimes can reach high levels of participationand can do sofor extended period of timeit is unlikely that a competitive regime can long avoid thetransition to a democracy. Axel Hadenius and Jan Teorell (2007) and Srensen (2008)outline the conventional wisdom, first developed by Dahl (1971), that democracyanddemocratic consolidationis more likely in cases where states established competitiveregimes first, and then increased participation. Today, there are many competitiveauthoritarian regimes identified in the literature (Levitsky and Way 2002; Schedler2006; Bunce and Wolchik 2010). However, we would expect that higher levels ofcompetitionwhen government and opposition forces are more evenly matchedreflecta higher democraticness of electoral competition. Competition is more meaningfulinthe context of stable party systems. This paper develops a measure for competition thattakes into account not only the balance between government and opposition forces inany election, but also their relative overall weight and the stability of the party system.

Measuring Effective Competition

2 Dahl had earlier identified at least four basic types of regimes, measured along the dimensionsof competition and participation: closed hegemonies, inclusive hegemonies (high competition,low participation), competitive oligarchies (high participation, low competition), and polyarchies(high competition and participation). See Dahl 1971, 7.


6/20

Centellas 5

This paper measures political competition building on an indicator for effectivecompetitiveness developed by Altman and Prez-Lin (2002). Their measure relies onthe distinction between government and opposition political parties. This paper definesgovernment parties as those parties represented in the legislatures lower chamber thatare the parties of the chief executive and/or are members of a formal governmentcoalition. Consequently, opposition parties include all other political parties represented

in the legislatures lower chamber but that do not formally participate in the governmentcoalition (even if they support the government some of the time).

Altman and Prez-Lin developed measures for the size of the typicalopposition and government parties. These measures are designed to account forfragmentation but are weighted in favor of the largest parties. The size of the typicalopposition party (O) is calculated as:

! =!!!

!!

where oiis the share of seats of the i-th opposition party. The size of the typicalgovernment party (G) is calculated as:

! =!!!

!!

where giis the share of seats of the i-th government party. These measures make itpossible to calculate the relative size of government and opposition blocs, rather thanusing aggregate shares of seats for government and oppositionwhich disregards thedegree of fragmentation within rival political blocs.

Using these two measures, Altman and Prez-Lin then develop an index ofcompetitiveness (C):

! = 1

! !

100

The value for C ranges along a scale from zero (when either the government or theopposition controls all seats in the legislature) to 1 (when the government and oppositionare perfectly balanced).

When Altman and Prez-Lin first developed their index of competitiveness, itperformed very well when tested across 18 Latin American electoral democracies.However, their dataset did go beyond the mid-1990s. Thus, they were unable to test theirindex in the context of the same number of illiberal democratic regimes that emergedsince then. This past decade, in particular, has seen a number of elections that producedgovernments that are not easy to place on a democratic scale. For example, figures suchas Hugo Chvez in Venezuela, Evo Morales in Bolivia, and Daniel Ortega in Nicaragua

part of the populist wing of the regions recent left turncan be described as semi-authoritarian or delegative democrats. Venezuela is particular prickly, because in twoelections (2000 and 2010), competition remains relatively high using the Altman andPrez-Lin measures (0.70 and .80, respectively). The 2005 election is even worse:Although the C index score drops to 0.45, this is still relatively high considering that pro-Chvez parties held more than 91 percent of the legislative seats. This is becausealthough Chvezs MVR party by itself won nearly three quarters of all seats, it was alliedwith a number of micro-parties, reducing the measure for the typical government partyto only 0.56.


7/20

Centellas 6

Although their model still performs relatively well, I wanted to modify the Altmanand Prez-Lin index to reflect to important dimensions missing from their index ofcompetitiveness. The first is the difference between high levels of competition in whichthe presidential party or coalition held a majority of seats in the legislature from those inwhich it did not. It seems obvious, perhaps, that (all things being equal) competition ishigher in regimes where the president faces an opposition-dominated legislative branch.

Because the Altman and Prez-Lin index only takes the absolute distance between thegovernment and opposition blocs. For example, Mexico in 2009 receives a similar scoreto Colombia in 2010 (0.94 and 0.95, respectively), even though in 2009 Mexicos FelipeCalderons PAN held less than a third of the seats in the lower house and ColombiasJuan Manuel Santoss coalition holds more than two thirds of the lower house seats. Thefirst adjustment involves reducing the competitiveness measure for cases in which thepresidents party holds a majority of the seats in the lower chamber.

A second dimension missing from their index is the amount of electoral volatilityin the regime. If we assume that parties matteror are indispensible for democracy(Lipset 2000)then we should be concerned with the stability of the party system as acritical component of a democratic regime. In systems in which party systems collapse,competition may become meaninglessor, at best, contingent. For example, Bolivia in

2002 and Chile in 2001 both score a near-perfect 0.96 on the competitiveness index. Yetin Chile, the high competition was between two well-established party blocs (the center-left Concertacin and the center-right Unin), whereas in Bolivia it was between ahodgepodge coalition of all the traditional parties against three new anti-establishmentparties (MAS, MIP, and NFR). Only a year later, Bolivias Gonzalo Snchez de Lozadawas driven out of office. The second adjustment involves reducing the competitivenessmeasure to account for the degree of electoral volatility.

Based on the above adjustments, I calculate a new index ofeffective democraticcompetiveness (EDC):

!"!!"# > !!"", !!"!"# = ! 1 !!"# !!""100

(1 !)

!"!!"# !!"", !!"!"# = ! !where SGOVis the total seat share of the government coalition, SOPP is the total seat shareof all other parties, and Vis the measure of electoral volatility.3 EDC values range along ascale from zero (no competition) to 1 (government and opposition are perfectly balancedand there is no volatility).

Measuring Effective Participation

The second dimension of democracy is political participation. Democratic competition isnot very meaningful if only a small minority of a countrys citizens actively participate.

Although participation can take on many forms, this paper limits its scope to electoralparticipation. One simple way to do this would be to look at voter turnout. However,while high voter turnout may be ideal and suggest something about broad support for orconfidence in the electoral regime and low voter turnout may suggest something aboutcitizens perceptions about its legitimacy, voter turnout alone hides some differences key

3 The measure of electoral volatility is measured using the formula developed by MogensPedersen (1979): ! =

!

! !!,! !!,!!! where p is the vote share for the i-th party at election t.


8/20

Centellas 7

differences. The legal voting age and voter registration laws vary across countries.Additionally, many countries have legal requirements for votingthough sanctions andenforcement vary widely.

This paper measures effective participation on the basis of voter turnout as theshare of the voting age population (Vote/VAP). Unlike standard voter turnout data thatmeasures turnout based on laws regulating voter registration rolls (which vary across

countries and may exclude significant numbers of citizens), this measure uses a standardbenchmark. An alternative would be to calculate turnout on the basis of voter turnout asthe share of the total population (Vote/Pop), but this could potentially boost scores forcountries that had lower minimum voting ages.

To account for mandatory voting laws, the measure for effective participation isthen adjusted to account for the share of invalid votes (votes cast blank and/ornullified by electoral authorities). This would account for cases in which voting wasmandatory, but a large number of voters either deliberately cast blank or null votes, aswell as cases where the electoral fraud was used to nullify large numbers of votes.Additionally, using invalid vote data is useful even if such instances are rare. After all,to the extent that citizens make honest mistakes in casting their ballots, which result innull ballots, they are not effectively participating in the electoral process.

The measure for effective democratic participation (EDC) is simply:

!"# =!"

!"#

!"#

!"=

!"#

!"#

where Vtis the total number of voters, Valis the total number of valid votes case in anyelection, and VAPis the total voting age population. Like the index of competitiveness,EDP values range from zero (no citizens voted) to 1 (all voting age citizens voted and castvalid votes).

Case Selection and Data

Because this is an exploratory paper testing the viability of proposed measures, it relieson a limited selection of cases. Although a larger set of cases would offer more robustassessment, a limited number of carefully selected cases allows for an analysis that teststheoretically-derived empirical measures or indicators against accepted understandingsof individual cases. This follows the logic of Dahls (1971) original development ofpolyarchy, which defined the dimensions of democracy by looking at cases commonlyviewed to be democratic. Thus, the validity of the measuresparticularly the measure foreffective democratic competitiveness (EDC)should correspond (at least roughly) withcommon views of the quality of democracy across the selected cases.

[Table 1 here]

This paper uses nine Latin American cases that represent a diversity of range inboth quality of democracy and other factors (see Table 1). With two exceptions (Peru andNicaragua), the data looks specifically at elections not included in the Altman and Prez-Lin dataset. In contrast to their data (which looked at the crest of the s0-called thirdwave of democracy in the region), the late 1990s was a turbulent time in Latin America.The selection includes two recent transition to democracy (Mexico and Nicaragua), aswell as three cases of democratic decline (Venezuela, Bolivia, and Nicaragua). Theselection also includes cases that are part of the shift to the left, whether understood asthe populist left (Venezuela, Bolivia, and Nicaragua) or the pragmatic left (Brazil,


9/20

Centellas 8

Peru, and Costa Rica), as well as three cases that recently shifted to the right (Mexico,Chile, and Colombia). The selection includes three established democracies (Colombia,Venezuela, and Costa Rica), which nevertheless have gone in different trajectories, andan assortment of new democracies ranging in age from Peru to Mexico. Finally, thesealso include a mix of federal and unitary states, different voter registration laws, anddifferent electoral systems.

The unit of measurement was the legislative election. Thus, each country in thesample was disaggregated by election year, producing a total of 37 observations. In caseswhere presidential and legislative elections coincided, the effective participation figurewas calculated on the basis of reported legislative turnout and valid votes. Measures forcompetition were calculated on the basis of aggregate legislative votes of the party listtier for all cases (ignoring SMD tiers in MMP systems), as well as the reported nationaltotals for Chile.

Election data was gathered from a wide variety of available sources, includingDieter Nohlens two-volume Elections in the Americas (2005), and two online sources:Adam Carrs Election Archives and Manuel lvarez-Riveras Election Resources onthe Internet. This data was used to calculate electoral volatility,4 government andopposition seat shares,5 and the share of valid votes. Vote/VAP data was obtained from

International IDEAs Voter Turnout database. Unfortunately, elections in 2009 and 2010(as well as Nicaraguas 2001 election) are not included in the database, which drops ninecase-years from analysis.

Measuring the Quality of Democracy in Latin America since the Left Turn

Using the measures described above suggest three different groupings among the nineselected Latin American electoral democracies. Four cases consistently scored high onboth the effective democratic competition and effective democracy participation: Brazil,Chile, Costa Rica, and Mexico. Three cases consistently score high on the participationscale, but low on competitiveness: Nicaragua, Peru, and Bolivia. Two cases consistentlyscore low on both indices: Venezuela and Colombia. The three zones coincide with three

of the spaces in Dahls framework, suggesting three democratic regime types: highquality democracies, uncompetitive participatory democracies, and limited democracies.

Figure 1 shows the dispersal of country-election scores along the two axes, withmost countries scores clustered in the same zone. The three outliers are Peru 1995,Brazil 1998, and Nicaragua 2006. The Peru outlier is explained by the low quality of the1995 election, which took place in the context of a collapsed party system (volatility was0.719) and with high number of invalid votes (46.1%). The Brazil outlier reflects higherthan typical electoral volatility and the formation of a supermajority governing coalitionthat involved every major party except the PT. It is noteworthy that in the subsequentelection years (2002 and 2006), Brazil has moved into the high quality democracy group.Nicaragua 2006 is more ambiguous: it reflects improvement along the competitivenessdimension (due to a sharp decline in electoral volatility and the formation of a minority

government), but worsening on the participation dimension (due to nearly 40 percentinvalid votes case). These outliers are consistent with mainstream interpretation of thethree cases: democracy has improved in both Brazil and Peru in recent years, while it hasrecently declined in Nicaragua.

4 Calculating voter turnout for the first election year for each country in the sample required usinglegislative vote shares for the previous election.5 A number of the cases in the sample did involve multi-party coalition governments. This wascommon in Bolivia (pre-2005), Brazil, Chile, and Colombia (post-1998).


10/20

Centellas 9

[Figure 1 here]

Overall, the scores correspond quite well with qualitative explanations for thedifferences in democratic quality in the region and across time. But it is also important tocompare the scores with established measures of democracy. This paper uses two readily

available indices as benchmarks: FH and Polity IV scores. Despite the caveats about FHscores, they remain the most commonly used indicator for quality of democracy acrossthe world. Moreover, their scores use not only event or formal institutional data (as doesPolity IV), but rely on a broad evaluation along a number of dimensions related todemocracy on the ground. In contrast, Polity IV is recognized as a stronger measurewith the benefits of clear criteria that can be applied retroactively (whereas FreedomHouse only goes back to its 1972 survey). But Polity IV measures rely primarily on eventdata and descriptions of formal institutional structures, without including assessment ofqualitative effects on real politics.

This paper performs a number of different simple bivariate correlation tests. Thefirst series of tests look at the bivariate correlation between the proposed EDC index andboth indices (FH and Polity IV). This is based on a normative assumption that effective

electoral competitiondefined as a relative balance between government and oppositionforces in the legislatureis an essential component of democracy. The second series oftest look at the relationship between the proposed EDP index and both benchmarkindices. This is based on the normative assumption that the effective participation ofvotersdefined as high voter turnout and valid vote preferencesis also an essentialcomponent of democracy.

Simple bivariate analyses confirm a strong relationship between the proposedmeasures and both benchmark indices (see Figures 2 and 3). In bivariate tests, theproposed effective participation (EDP) and effective competition (EDC) measures werecorrelated with FH scores at statistically significant levels. Of the two measures, the EDCindex was most strongly correlated, with a Pearson correlation of 0.648. These resultswere different from those reported by Altman and Prez-Lin in their sample. They

reported a Pearson correlation of 0.58 for voter turnout (Vote/VAP) and FH scores, anda correlation of 0.49 for their competitiveness index (both at statistically significantlevels). With the important caveat that the samples come from different time periods, themeasures proposed in this paper perform as well or significantly better than thoseproposed by Altman and Prez-Lin. The proposed EDP and EDC measures were alsocorrelated with Polity IV scores at statistically significant levels, even though the Pearsoncorrelations cores were lower (0.465 and 0.0539). Altman and Prez-Lin did not testtheir measures against Polity IV scores.

[Figures 2 & 3 here]

Because the proposed measure for effective democratic competitiveness (EDC)was significant modified from the original Altman and Prez-Lin index (C), it wasimportant to test how well the original measure did in bivariate analysis with FreedomHouse and Polity IV measures for the sample of countries analyzed in this paper. Theoriginal C index scores were significantly correlated with both Freedom House and PolityIV indices, but with interesting differences. The Pearson correlation between the C scoresand FH scores was 0.393, substantially lower than between proposed EDC and FHscores. But the Pearson correlation between C scores and Polity IV was 0.590, slightlyhigher than the Pearson correlation between proposed EDC and Polity IV scores. Thus,


11/20

Centellas 10

while Altman and Prez-Lins measures fare worse against FH scores than themeasures proposed in this paper, they fare better when compared with Polity IV scores.

A closer inspection at Polity IV scores for the sample countries, however, suggeststhat Polity IV may be a less useful measure than Munck and Verkuilen (2002) suggest.One way to do this is to look at two subsets of cases: Brazil, Colombia, and Mexico (seeFigure 4) and Bolivia, Peru, and Venezuela (see Figure 5). According to the Polity IV

index, Brazil and Colombia have been static (with scores of 8 and 7, respectively) from1997 through 2009, while Mexico shifted only once, in 2000: Brazil scores an 8 from1998 through 2009, while Colombia scores a 7 throughout the same time period.Because the Polity measures use event data, Mexico jumps up from a 6 (the minimumscore to quality as democratic) in 1999 to an 8 in 2000 because of the transfer of powerto the opposition. Yet there has been speculation in the past that Vicente Foxs 2000 winsuggests the possibility that Mexicos PRI was willing to relinquish power earlierasmarked by 1997 midterm legislative elections, when for the first time in the regimeshistory opposition parties held a majority of the seats. In contrast, Polity IV measuresshow no changes in either Brazil or Colombia, despite significant changes over time.Brazils party system appears to have consolidated, with volatility now at 10-15 percentever year, down from 20.2 percent in 1998. Brazils 1998 election also produced a

supermajority coalition aimed exclusively at sidelining the PT and other leftist parties;since 2002, Brazilian presidents have enjoyed narrower coalitions. Likewise, Polity IVmeasure suggest no change in Colombias democratic politics, despite significantchanges over time: Electoral volatility remained above 40 percent from 1998 through2006, with a drop to 29.2 percent in the most recent 2010 election. Meanwhile, thebalance between government and opposition forces in the legislature was differentduring this time period, with the two forces most closely balanced in 2002 and lowest in1998 and 2010.

[Figure 4 here]

A comparison between Bolivia, Peru, and Venezuela also suggests limitations of

Polity IV scores. Peru begins with a low score of 1 on the Polity index in 1997, whichcorresponds to the tail end of the Fujimori regime. From there, however, it jumps up to a5 (just below the democratic threshold) in 2000, before leaping up to a 9 in 2001 (theyear Alejandro Toledo was elected president), where it has remained since. In contrast,Venezuela begins at a high score of 8 in 1998, drops gradually throughout the Chvezyears to 5 by 2008, then sharply to -3 in 2009. Bolivia, meanwhile, begins 1997 with ahigh score of 9, drops slightly to 8 in 2003 (the year Snchez de Lozada was forced out ofoffice), then drops slightly again to 7 in 2009. Thus, although Bolivia under Morales hasbeen as polarized as in Venezuela, Bolivia remains in the democratic range, whileVenezuela under Chvez scores considerably worse than Peru under Fujimori. Likewise,it is odd that Venezuelas score should collapse suddenly in 2009, about the time whenthe opposition reorganized and mounted a remarkably successful challenge to Chvezsparty in legislative electionswhile Venezuelas Polity score was only slightly affected bythe 2005 election, in which Chvez supporters won more than 90 percent of legislativeseats (due in large part to an opposition boycott). In contrast, the proposed effectivedemocratic competitiveness scores suggest slightly different trajectories for the threecountries (see Figure 5).

[Figure 5 here]


12/20

Centellas 11

The proposed EDC measures better correspond to the three cases: Venezuela in2005 scores about on par with Peru in 1997, but then recovers substantially by 2010.Bolivias score falls significantly between 2002 and 2005, reflecting the collapse of theparty system and a highly polarized political climatethough the scores never fall as lowas Peru 1995 or Venezuela 2005. Interestingly, Bolivia scores about the same in 2009 asin 2005, despite the fact that Morales won reelection by a wide margin. But, as several

Bolivia observers have noted, Bolivias democracy has not eroded to the extent as it hasin Venezuela; moreover, radical decentralization reforms that have virtually federalizedthe country suggest an increase in competitiveness. It may seem counter-intuitive thatEDC measures suggest significant improvement in Venezuela, bringing its score almostback to the 1998 level. Time will tell whether the significant gains made by a unified,coordinate opposition in the most recent legislative elections are a substantive return todemocratic competition.

Although it is possible to collapse the EDC, EDP, and FH scores into one singlefactor score for quality of democracy (similar to the approach pursued by Altman andPrez-Lin), this paper argues against such a procedure. Democracy is best understoodas a multidimensional conceptone that involves competition, participation, and civiland political rightsbut without any a priori decision about how there relative weights

interact. Any approach that combines these dimensions into a single factor score erasesimportant differences across regimes that may do well on some dimensions than onothers. If we look only at competition and participation, two regimes may have similarfactor scores despite one having higher levels of participation and the other scoringhigher on competition. Because such differences matter, they should not be ignored.

For the sake of comparison, however, Figure 6 shows the relationship between acombined effective democracy scores calculated as the average between EDC and EDPscores and FH scores. This produces mixed results: On the one hand, the correlationbetween the combined competition and participation index to FH scores is stronger thanbetween FH scores and either index alone (r=0.71104; p>0.0001). The country-yearunits are clearly more tightly contained. But this poses a problem of loss of information:The sharp differences between clusters of country-year units fadesparticularly at the

low end of the scale.[Figure 6]

Conclusion

Assessing the quality of democracy involves two conceptually distinct questions: The firstis how to distinguish between better and worse democracies. This is something mostdemocracy indices attempt to do. But it is equally important to distinguish democraticregimes along the different dimensions of democracy. Democracy measures that collapsedifferent dimensions into a single factor score lose significant information. This papersuggests a new way to measure democracy that is consistent with and grounded in Dahls

conceptualization of democracy along three dimensions: competitiveness, participation,and political rights and civil liberties.There are, of course, limitations to the proposed measures. First, the measures

rely on readily available, accurate, and useful electoral data. Although the measures aresimple enough to calculate, the validity of those calculations depends on the reliability ofthe available data. Some cases present significant problems in this regard. For example,Argentina was originally included in the selection of cases, but was abandoned becauseof significant problems with finding clear, accurate vote and seat data at the individualparty level for the recent elections.


13/20

Centellas 12

Second, the measures rely on a very minimal conceptual framework of democracythat puts the focus on electoral politics. This heavily ignores other important indicatorsof democracy, such as a vibrant civil society, civil-military relations, or socio-economicinequality. Freedom House indicators do take such concerns into account, but notsufficiently for critics of procedural democracy. Additionally, Freedom House does notmeasure regime characteristics; it provides qualitative expert assessment of how well

freedom is exercised in practice in a society.Third, an important limitation is that the effective competitiveness index relies

on a calculated balance of opposition and government blocs in the legislature that doesnot include the relative powers that legislatures and executives haveor the wayindividual executives choose to expand their powersis clearly limiting. This could beparty addressed by inserting such information into the equationperhaps somethinglike the parliamentary powers index developed by M. Steven Fish and MatthewKroenig (2009). However, this still retains a dilemma: Should competition be measureddifferently in parliamentary, presidential, or semi-presidential systems? The adjustmentapplied to the above cases (all presidential) punishes governments that have majoritysupport in the legislature. This potentially deflates the scores for parliamentary systemsand elevates the scores of minority presidents.

Finally, as currently operationalized, the measures focus exclusively electoralcompetition in the lower chamber of the legislature. The balance between governmentand opposition may vary significant between chambersas may the types of powers thechambers themselves possess. Recalibrating the measures to include this information is,of course, possible. But it remains a significant undertaking.

However, the advantages of the proposed measures outweigh their costs. First,the measures are easy to calculate and require little or no qualitative assessment by theresearcher. This is appealing to those interests in large-N statistical analysis, since theyare not required to understand individual cases in the way specialist country coderswould have to be. This is also appealing because measures are derived completelyempirically and should be replicable across any set of researchers with access to the sameelectoral data set. Of course, country-specialists are able to adjust a countrys scores if

they merit them. For example, identifying and disaggregating parties that are in factcoalitions of parties (or even factions) would alter the measures for G (size of typicalgovernment party) and/or O (size of typical opposition party).

A second major advantage is that the two proposed measures could be calculatedas soon as data is availableas early as the day election results are released. Most indicesare calculated in house by either a proprietary set of researchers or ad hoc collection ofcommissioned country specialists. This means existing indices are only available oncethe editors or researchers compile them. In contrast, the proposed effective democraticcompetitiveness (EDC) measures can be calculated as soon as final election results arepublished. A caveat is that that the proposed effective democraticparticipation (EDP)measure does rely on the availability of Vote/VAP, which often has its own lag (e.g.Vote/VAP measures for elections taking place in 2009 and 2010 were not available forthe cases included in this paper). However, Vote/VAP measures could ostensibly becalculated independently. Or, alternatively, EDP scores could be calculated using othermeasures: calculating voter turnout as share of total population (as proposed byVanhanen) or simply using official voter turnout figures. Both solutions have theirlimitations, and researchers should be aware of them. However, effective democraticcompetitiveness measures worked best when compared either to FH or Polity IV scores,suggesting that theycouldbe used independently.

The final advantage of the proposed measures is that they are more dynamic thanmany existing measures. The proposed EDP index responds to subtle changes in both


14/20

Centellas 13

voter turnout and voters willingness to engage in the electoral process by casting validvotes. The proposed EDC index also responds to subtle changes in party alignments,electoral volatility, and shifts in government coalitions. In fact, the EDC index can berecalculated at any given moment to reflect changes in coalition dynamics as partiesleave or enter the government coalition (though the volatility component would remainstatic). A dynamic index is particularly useful to compare across democratic regimes or

to understand the effects of incremental changes within individual regimes overrelatively short periods of time.

Although the proposed measures work well in the nine selected cases included inthis paper, more research is clearly needed. The next step is to expand the data set tocases from other regions and including parliamentary and semi-presidential systems. Afuture research agenda will also include testing the measure in different contexts, as wellas beginning to apply the measuresboth as dependent and independent variablestotest for relationships to some of the concepts often thought to affect or be affected by thequality of democracy, such as economic development, socio-economic inequality,corruption, human development, and others.

References

Altman, David and Anbal Prez-Lin. 2002. Assessing the Quality of Democracy: Freedom,Competitiveness and Participation in Eighteen Latin American Countries.Democratization 9 (2): 85-100.

Alvarez, Michael, Jos Antonio Cheibub, Fernando Limongi, and Adam Przeworski. 1996.Classifying Political Regimes. Studies in Comparative Political Development31 (2): 1-37.

lvarez-Rivera, Manuel. Nd. Election Resources on the Internet. http://electionresources.org/Beetham, David. 2004. Freedom as the Foundation. Journal of Democracy 15 (4): 61-75.Bermeo, Nancy. 2009. Does Electoral Democracy Boost Economic Equality? Journal of

Democracy 20 (4): 21-35.Bhlmann, Mark, Wolfgang Merkel, and Bernhard Wessels. 2007. The Quality of Democracy:

Democracy Barometer for Established Democracies. National Center of Competence

Research, Working Paper #10.Bunce, Valerie and Sharon L. Wolchik. 2010. Defeating Dictators: Electoral Change and Stability

in Competitive Authoritarian Regimes. World Politics 62 (1): 43-86.Carbone, Giovanni. 2009. The Consequences of Democratization. Journal of Democracy 20 (2):

123-137.Carr, Adam. Nd. Adam Carrs Election Archive. http://psephos.adam-carr.net/Coppedge, Michael. 2001. Political Darwinism in Latin Americas Lost Decade. In Political

Parties and Democracy, eds. Larry Diamond and Richard Gunther. Baltimore: JohnsHopkins University Press, 173-205.

Coppedge, Michael and Wolfgang H. Reinicke. 1991. Measuring Polyarchy. In On MeasuringDemocracy: Its Consequences and Concomitants, ed. Alex Inkeles. New Brunswick:Transaction Press, 47-68.

Dahl, Robert. 1982. Dilemmas of Pluralist Democracy: Autonomy v. Control. New Haven: Yale

University Press.. 1971. Polyarchy: Participation and Opposition. New Haven: Yale University Press.Economist Intelligence Unit. 2010. Democracy Index 2010: Democracy in Retreat.

http://graphics.eiu.com/PDF/Democracy_Index_2010_web.pdfFish, M. Steven and Matthew Kroenig. 2009. The Handbook of National Legislatures: A Global

Survey. Cambridge: Cambridge University Press.Hadenius, Axel. 1992. Democracy and Development. Cambridge: Cambridge University Press.Hadenius, Axel and Jan Teorell. 2007. Pathways from Authoritarianism. Journal of Democracy

18 (1): 143-156.International IDEA. Nd. Voter Turnout. http://www.idea.int/vt/


15/20

Centellas 14

Levitsky, Steven and Lucan A. Way. 2002. The Rise of Competitive Authoritarianism. Journalof Democracy 13 (2): 51-65.

Lipset, Seymour M. 2000. The Indispensability of Political Parties. Journal of Democracy 11(1): 48-55.

Maravall, Jos Mara and Adam Przeworski. 2003. Democracy and the Rule of Law. Cambridge:Cambridge University Press.

Mashall, Monty G., Ted Robert Gurr, and Keith Jaggers. 2009. Political Regime Characteristicsand Transitions, 1800-2009: Dataset Users Manual.http://www.systemicpeace.org/inscr/p4manualv2009.pdf

Munck, Gerardo L. and Jay Verkuilen. 2002. Conceptualizing and Measuring Democracy:Evaluating Alternative Indices. Comparative Political Studies 35 (1): 5-34.

Nohlen, Dieter. 2005. Elections in the Americas: A Data Handbook. 2 vols. Oxford: OxfordUniversity Press.

ODonnell, Guillermo. 2004. Why the Rule of Law Matters. Journal of Democracy 15 (4): 32-46.

Pedersen, Mogens N. 1979. The Dynamics of European Party Systems: Changing Patterns ofElectoral Volatility. European Journal of Political Research 7 (1): 1-26.

Pemstein, Daniel, Stephen A. Meserve, and James Melton. 2010. Democratic Compromise: ALatent Variable Analysis of Ten Measures of Regime Type. Political Analysis 18 (4): 426-449.

Rueschemeyer, Dietrich. 2004. Addressing Inequality. Journal of Democracy 15 (4): 76-90.Schedler, Andreas. 2006. Electoral Authoritarianism: The Dynamics of Unfree Competition.Boulder: Lynne Rienner.

Schmitter, Philippe and Terry Lynn Karl. 1991. What Democracy Is and Is Not. Journal ofDemocracy 2 (3): 75-88.

Schumpeter, Joseph A. 1975. Capitalism, Socialism, and Democracy. New York: Harper & Row.Smith, Peter H and Melissa R. Zeigler. 2008. Liberal and Illiberal Democracy in Latin America.

Latin American Politics & Society 50 (1): 31-57.Srensen, Georg. 2008. Democracy and Democratization: Process and Prospect in a Changing

World, 3rd ed. Boulder: Westview Press.Vanhanen, Tatu. 2000. A New Dataset for Measuring Democracy, 1810-1998. Journal of Peace

Research 37 (2): 251-265.


16/20

Centellas 15

Table 1. Overview of cases included in the study

Country Year ofTransition toDemocracy

Federal orUnitary State

MandatoryVoting Laws

ElectoralSystem(Legislature)

ElectionsYearsIncluded

Bolivia 1982 Unitary Yes MMP 1997, 2002,

2005, 2009Brazil 1985 Federal Yes List PR 1998, 2002,

2006, 2010

Chile 1990 Unitary Yes Binominal 1997, 2001,2005, 2009

Colombia 1958 Unitary None List PR 1998, 2002,2006, 2010

Costa Rica 1949 Unitary None List PR 1998, 2002,2006, 2010

Mexico 2000 Federal None MMP (1997), 2000,(2003), 2006,

(2009)

Nicaragua 1990 Unitary None List PR 1990, 1996,2001, 2006

Peru 1979 Unitary Yes List PR 1995, 2000,2001, 2006

Venezuela 1958 Federal None MMP 1998, 2000,2005, (2010)

Years in parentheses are midterm elections.

0.00

0.25

0.50

0.75

1.00

0.00 0.25 0.50 0.75 1.00

EffectiveDemocraticParticipation

EffectiveDemocraticCompetitiveness

Figure 1: Competition & participation in selected Latin American cases

Brazil

Venezuela

Peru

Mexico

Costa Rica

NicaraguaChile

Bolivia

Colombia


17/20

Centellas 16

Figure 2: Bivariate relationship between proposed measures and Freedom House scores

EDP & FH Scores EDC & FH Scores

Pearsons r= 0.492; p > 0.0001 Pearsons r= 0.648; p > 0.0001

Figure 3: Bivariate relationship between proposed measures and Polity IV scores

EDP & Polity Scores EDC & Polity Scores

Pearsons r= 0.465; p > 0.0001 Pearsons r= 0.539; p > 0.0001

3

4

5

6

7

0.00 0.25 0.50 0.75 1.003

4

5

6

7

0.00 0.25 0.50 0.75 1.00

0

2

4

6

8

10

0.00 0.25 0.50 0.75 1.00

0

2

4

6

8

10

0.00 0.25 0.50 0.75 1.00


18/20

Centellas 17

0.00

0.25

0.50

0.75

1.00

1997 1999 2001 2003 2005 2007 2009

Figure 4. Effective democratic competition: Brazil, Colombia, Mexico

Brazil

Colombia

Mexico

0.00

0.25

0.50

0.75

1.00

1995 1997 1999 2001 2003 2005 2007 2009

Figure 5. Effective democratic competition: Bolivia, Peru, Venezuela

Bolivia

Peru

Venezuela


19/20

Centellas 18

Figure 6. Relationship between factored effective democracy and Freedom House scores

Pearsons r= 0.71104; p > 0.0001

3

4

5

6

7

0.00 0.25 0.50 0.75 1.00

Brazil

Venezuela

Peru

Mexico

Costa Rica

Nicaragua

Chile

Bolivia

Colombia3

4

5

6

7

0.00 0.25 0.50 0.75 1.00


20/20

Centellas 19

Appendix

Country ElectionYear

FH(inverted)

EDP EDC Vote/VAP Volatility

Bolivia 1997 6.0 0.61 0.37 0.645 0.285

2002 5.5 0.67 0.38 0.668 0.5672005 5.0 0.63 0.19 0.634 0.698

2009 5.0 0.22 0.407

Brazil 1998 4.5 0.65 o.46 0.810 0.202

2002 5.0 0.77 0.82 0.835 0.152

2006 6.0 0.61 0.68 0.684 0.104

2010 6.0 0.64 0.153

Chile 1997 5.5 0.60 0.69 0.731 0.119

2001 6.0 0.60 0.79 0.692 0.142

2005 7.0 0.61 0.79 0.665 0.069

2009 7.0 0.75 0.142

Colombia 1998 4.5 0.36 0.28 0.405 0.4742002 4.0 0.35 0.56 0.406 0.404

2006 5.0 0.33 0.39 0.394 0.568

2010 4.5 0.42 0.292

Costa Rica 1998 6.5 0.71 0.72 0.737 0.138

2002 6.5 0.65 0.88 0.670 0.037

2006 7.0 0.52 0.74 0.540 0.031

2010 7.0 0.63 0.128

Mexico 1997 4.5 0.53 0.63 0.544 0.162

2000 5.5 0.47 0.77 0.482 0.159

2003 6.0 0.42 0.80 0.434 0.119

2006 6.0 0.62 0.68 0.636 0.205

2009 5.5 0.71 0.241

Nicaragua 1990 5.0 0.69 0.39 0.733 0.493

1996 5.0 0.72 0.32 0.763 0.613

2001 5.0 0.28 0.612

2006 5.0 0.49 0.53 0.808 0.359

Peru 1995 3.5 0.31 0.13 0.577 0.719

2000 4.5 0.65 0.41 0.786 0.415

2001 6.0 0.62 0.41 0.786 0.464

2006 5.5 0.62 0.47 0.841 0.506

Venezuela 1998 5.5 0.37 0.57 0.427 0.4242000 4.0 0.32 0.36 0.465 0.325

2005 4.0 0.23 0.02 0.239 0.477

2010 3.0 0.44 0.320

revisiting assessing the quality of democracy: measuring democratic competition and participation in...

Documents