certain problems in election survey methodology

American Association for Public Opinion Research

Certain Problems in Election Survey MethodologyAuthor(s): Paul PerrySource: The Public Opinion Quarterly, Vol. 43, No. 3 (Autumn, 1979), pp. 312-325Published by: Oxford University Press on behalf of the American Association for Public OpinionResearchStable URL: http://www.jstor.org/stable/2748227 .

Accessed: 05/12/2014 19:22

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

American Association for Public Opinion Research and Oxford University Press are collaborating with JSTORto digitize, preserve and extend access to The Public Opinion Quarterly.

http://www.jstor.org

This content downloaded from 128.235.251.160 on Fri, 5 Dec 2014 19:22:46 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=oup

http://www.jstor.org/action/showPublisher?publisherCode=aapor

http://www.jstor.org/action/showPublisher?publisherCode=aapor

http://www.jstor.org/stable/2748227?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Certain Problems in Election Survey Methodology

PAUL PERRY

I HEORETICALLY, there should be no great difficulty with election surveys. That is to say, there should be no great difficulty in producing accurate estimates of how people will vote in an election based on a sample of the electorate. All one has to do is use a properly drawn sample of the electorate large enough to minimize random sampling error, get honest answers from everyone, do the questioning close enough to the time of voting to minimize changes in voting intentions, anticipate how the undecided will vote, and, finally, distinguish between voters and nonvoters in the electorate. If one can do these things reasonably well, the error of the estimate will be acceptably small, assuming, of course, a large sample. Therefore, why all the fuss about it, one might ask? Well, the fuss arises from the fact that it is not really easy to do any of these things, contrary to some recent opinion on the subject. As will be recalled, the SSRC ad hoc commit- tee's report on the 1948 pre-election polls included a statement concerning the severity of the test of survey methodology posed by pre-election surveys (Mosteller et al., 1949:6).

Some Historical Background

The use of sampling theory in the measurement of voting preferences was probably first adopted knowingly in the 1930s. The so- called straw polls had been conducted earlier by newspapers and

Abstract This article describes some solutions to common problems in pre-election surveys drawing upon Gallup Poll experience. It touches upon problems in sampling, estimation, response validity, the undecided, measuring likelihood to vote, and measuring late trends in voter preference. In conclusion it cites demonstrable gains in accuracy that have followed application of the solutions described.

Paul Perry is Vice Chairman, The Gallup Organization, Inc.

Public Opinion Quarterly ? 1979 by The Trustees of Columbia University Published by Elsevier North-Holland, Inc. 0033-362X/79/0043-312/$1.75



PROBLEMS IN ELECTION SURVEY METHODOLOGY 313

magazines, among others, but the methods used were relatively primi- tive. The earliest straw poll I have heard of was in 1824. However, it would not surprise me to discover that the Greeks and the Romans were active in the field.

In the 1930s Gallup, Roper, and Crossley began sampling the electorate using procedures which avoided the worst flaws in the earlier polls. The Gallup Poll's first effort in 1936 was based upon samples solicited by mail supplemented by personal interviews. While the Poll's final pre-election estimate showed Roosevelt winning the major party vote with a large majority, his actual majority was seven points greater. The size of the error clearly revealed a bias in the method, and in 1938 Gallup Poll surveys to measure preferences in the congressional elections were conducted entirely by personal interview. The error in 1938 was 3 percentage points. At the same time a Gallup Poll survey in New York State using the same sampling procedures and using personal interviews to measure preferences in the Lehman-Dewey gubernatorial election produced results within 1 percentage point of the election result. In these early Gallup surveys a purposive sample design (Neyman, 1934) was used in the selection of cities, towns, and rural areas, and the quota method was used for the selection of respondents within a sampling area.

The next major change in Gallup Poll methods occurred after weaknesses in sample design had become apparent, the necessity of measuring opinion as near to election day as feasible became clear, and the problem created by nonvoting had emerged. Weaknesses in sample design indicated by certain biases in survey results were largely the outgrowth of dependency on quota samples of respondents within sampling areas and the purposive rather than probability selection of sampling areas. A paper by Cantril and Harding (1943) concerning the 1942 elections pointed to the effect of nonvoting, and people at the Gallup Poll were well aware in 1944 of the possibility of changes in voting preferences during a campaign. In fact, they conducted a survey in the final days prior to the election of 1944 which was reported from the field by telegraph and telephone. In 1948 the same mechanism was in place but was not used because of the widespread belief that opinion had solidified. The final pre-election survey in that year was in mid-October. A relatively large error in 1948 ensued.

In 1950 preliminary changes in the overall design of the election survey method used by the Gallup Poll included the adoption of election precincts as the primary sampling units, provisions for a survey as near to election day as practicable, and the use of a turnout scale to select respondents most likely to vote (Perry, 1960; 1965).



314 PAUL PERRY

The Sample

While it is now relatively simple to draw a sample of the electorate in the United States, it is not at all easy to reach such a sample, particularly within acceptable time limits. In fact, it is more difficult today than it was 20 years ago because of higher rates of refusal. One cannot expect to achieve in surveys an accuracy consistent with the theory of simple random sampling. The biases introduced by refusals and not-at-homes, the variance added by using a cluster sample, and the factor of response variance prevent it. The importance of response variance is indicated by the experience of the Bureau of the Census. Hansen and Tepping (1969:10) have reported that for many items in a complete census taken by enumerators response variance was a significant factor. In fact, for some items the variability of the complete census estimates was as large, on the average, as if only a 25 percent sample had been taken in the absence of nonsampling variability. Therefore, implementing a sample design requires a great deal of effort, and allowances have to be made for failures in comple- tion and for other errors in execution.

One source of sampling error in a survey is the random error arising from the selection of locations in, say, a sample drawn in two stages in which locations are drawn at the first stage and respondents drawn from within the locations at the second stage. In election surveys sampling error can be controlled to some degree by using election precincts at the first stage (Perry, 1960). For example, if the mean percent Republican or Democratic of the actual vote in a previous election for the sample of precincts diverges from the true mean (which is equivalent to the percent Republican or Democratic of the total vote in the previous election) a correction can be made. The divergence of the mean of the sample of precincts from the true mean is random error if they are properly drawn. In any given sample, however, the divergence, if any, for all practical purposes is bias. The effect of this divergence can be corrected, at least to some degree, by a ratio adjustment or a regression adjustment in forming the estimates (Yates, 1960:155-67). In my experience the product-moment correlation among precincts between two presidential elections has been of the order of .7 to .8. Using a regression estimate such a correlation reduces the expected random error of the mean of a sample of areas, in this case precincts, by about 30 percent.

One part of any deviation of the precinct mean can be considered purely bias, and that is the part that arises from a difference in the ratio of voting participation among areas. For example, in most Gal- lup Poll election surveys the basic sample of locations is a general




purpose sample and is drawn in terms of population. Within these locations the precinct is drawn in terms of its total vote in a previous election. This makes partial allowance for differences in voting participation. However, it does not allow for a difference such as, for example, that between New York City and the rest of New York state. The voting age population of New York City comprises about 43 percent of the state; the vote of New York City in the 1976 presidential election comprised 33 percent of the total state vote.

Thus, a sample drawn in terms of population may well be biased in terms of total vote. Because of the differing political nature of large cities where voting participation is characteristically relatively low as compared with other areas, this is likely to be a bias in the direction of the Democratic party. Therefore, a weighting is introduced to make allowance for the larger differences of this nature to be found throughout the nation.

The average error of Gallup Poll final pre-election estimates as reported in 15 national elections from 1950 through 1978, congressional and presidential combined, reveals a small residual bias of about .5 of a percentage point in the direction of the Democratic party.

Another source of error is the typical cluster or design effect on sampling error in area samples with clusters of 5 or 10 interviews. Estimates computed for these surveys using a replicated sample indicate factors of 30 to 40 percent applicable to standard errors computed assuming a simple random sample. Part of this increase can be attributed to response variance as well as cluster effect. Kish reported similar results in a 1957 paper. Attempts to minimize the cluster effect are made by using what is essentially a single-stage sample of locations and within the precinct scattering the cases throughout the precinct. In recent elections the precincts have been divided into 10 segments, and one respondent has been selected in each segment to obtain maximum scatter.

Response Validity and the Secret Ballot

As a practical matter, in measuring political attitudes and preference between parties and candidates, one must assume evasion on the part of some among the sample of persons interviewed. In the past, a common basis for questioning the validity of the results of political surveys was the supposed lack of candor one would encounter in an interview. When mail surveys were commonly used, this was less of a problem because of the anonymity of respondents in such surveys. As early as 1936, however, the Gallup Poll was using a ballot box in its



316 PAUL PERRY

personal interview election surveys in which the respondent dropped the ballot after checking a response privately. In this early use of the secret ballot the interviewers approached the respondent with a one- or two-item questionnaire without any preliminaries other than the usual brief introductions. The refusal rate was somewhat greater than with the open and nonsecret approach. Interviewers reported that the secret ballot was more likely to arouse suspicion than to dispel it. As a result the secret ballot fell into disuse until Sidney Goldish, then director of the Minnesota Poll, developed a method of making the secret ballot on the election choices a part of the standard nonsecret interview and placed the question well into the interview so that it would be presented only after some rapport between respondent and interviewer had been established. We adopted Goldish's method in 1956 on the a priori grounds that it was a better simulator of the voting situation than the direct question, and it has worked well for us ever since. In all elections since then both the secret and nonsecret questioning techniques have been used during the campaign in con- junction with interpenetrating or split samples so that effects of the two methods could be contrasted. In the final surveys, however, only the secret ballot is used.

Our experience with the so-called secret ballot has been that apparently all but a small percentage of the sample do give honest answers, which is to say that differences in aggregate responses produced by secret and nonsecret questioning techniques are usually small. How- ever, in some elections differences are such as to indicate a response bias large enough to contribute two or three percentage points of error using a nonsecret question. This was particularly true in 1964 and 1972. In 1964 a comparison of nonsecret and secret responses revealed a hidden vote for Goldwater in nonsecret responses. Goldwater was stronger in the half of the sample using the secret ballot technique than in the half of the sample using the nonsecret method. The difference in his favor was of the order of 2 or 3 percentage points. In 1972 the same comparison revealed a significant hidden preference for McGovern. In 1956 there was a small hidden preference for Steven- son in nonsecret responses. In 1976 Ford was stronger on the secret ballot than the nonsecret in the South. The difference was significant at the two sigma level. Outside the South the difference was not significant.

Elizabeth Noelle-Neumann's "spiral of silence" concept is in part supported by the evidence of the secret-nonsecret comparisons. The first stage in the "spiral of silence" is when the supporters of a candidate or party begin to perceive themselves as in the minority (Noelle-Neumann, 1977). At this stage some of them will become less




likely to voice their support publicly. In the instances cited above the candidates who were weaker on the nonsecret ballot as compared with the secret were the losing candidates in elections in which the winning candidate had a wide lead. In 1956 Eisenhower defeated Stevenson with 58 percent of the Eisenhower-Stevenson vote; in 1964 Johnson defeated Goldwater with 61 percent of the Johnson- Goldwater vote; in 1972 Nixon defeated McGovern with 62 percent of the Nixon-McGovern vote; in 1976 Carter defeated Ford in the South with 59 percent of the Carter-Ford vote.

The Undecided and the Secret Ballot

Much has been made of the undecided in every election that I can remember. In earlier years relatively large proportions of undecided were reported. As late as 1960 reports citing 20 percent undecided were not uncommon, and in 1976 occasional reports of 15 to 20 percent undecided appeared. Of course, the size of the percent undecided depends on one' s definition of undecided. Some have maximized undecided deliberately for analytical purposes. Our objec- tive has been to minimize undecided on the theory that part of undecided is likely to be reluctance to take a position, reluctance to refuse the interviewer, and a desire to hide under the label of undecided. This notion has been supported by experience with the secret ballot. Using a secret ballot the undecided has been from one-third to one-fourth as large a group on the average as when using the nonsecret question. In the 1976 election, for example, using interpenetrating or split samples in surveys taken in September and October the experience of the combined samples was as follows: In the nonsecret version before using a "leaning" question, the undecided was 17 percent. After the "leaning" question the residual undecided was 9 percent; using the secret ballot the undecided was 6 percent. The secret and nonsecret samples each included 1,700 respondents. The relationship shown has been a consistent result in the six presidential elections in which this comparison has been made. It suggests that the relatively high undecided percentages sometimes reported in general elections are artifacts of the method of questioning and do not repre- sent true undecided. It is interesting to note in this regard the com- parative results of secret and nonsecret questionnaires in two studies in 1940 conducted by the Gallup Poll in Lisbon Township, Maine (Benson, 1941).

Another variable that is related to undecided is the likelihood of voting. For example, in the final 1976 survey the undecided using the total sample was 5.0 percent. Among the "likely voter" group (in our



318 PAUL PERRY

system the share of the sample most likely to vote which corresponds to the turnout ratio') the undecided was 3.7 percent. Therefore, a share of undecided, if no allowance is made for nonvoting, can be attributed to likely nonvoters.

We have not yet found an entirely satisfactory method of determin- ing how the residual undecided are likely to vote by indirect means, as, for example, by attitudinal or behavioral questions related to voting preferences. A reasonable and intuitively sensible method has seemed to be to compare the matched samples of secret and nonsecret and note how preferences change as the undecided group becomes a smaller percentage on the secret. For example, as the undecided declines from nonsecret to secret, how does the preference change? Assume the change in preference concomitant with the undecided reduction is largely the undecideds committing themselves. Project such change to the residual undecideds obtained using the secret ballot. While no theory can be adduced for such a projection, it is based on a reasonable assumption. However, it is not an entirely valid assumption. The differences in preference noted between samples using the nonsecret method and samples using the secret method in 1972 and 1976 were larger than the reduction in undecided. This is to say that the difference between nonsecret and secret could not all be attributed to the reduction in undecided and refusals on the question. Apparently in 1972, for example, some who said they preferred Nixon in answer to the nonsecret question would have opted for McGovern on the secret ballot. Since we were using independent samples for each method, it is impossible to document this in the way one could using each method on the same respondent. However, it seems to be a reasonable deduction. Therefore, part of the difference between the secret and nonsecret samples could be attributed (apart from sampling error) to undecided and refusals committing themselves and part to respondents voting differently with a secret ballot. That such would be the case was not unexpected, of course, but the degree to which it was operative was not anticipated, and the method was abandoned.

In 1974 and 1976 the "leaning" question on the nonsecret was used as an indicator of how the residual undecided would divide. The possible contribution to error of the undecided obtained using a secret ballot has not been large in national elections. It has been of the order of 4 percent among likely voters, and postelection studies have shown that about half of this 4 percent does not vote. This leaves some 2 percent to be allocated.

I The total vote for president divided by the total resident population 18 and older excluding residents of military bases. In the 1976 presidential election this ratio was .545.




Direct Questioning

Another approach considered for dealing with the problems of validity has been the use of corollary questions related to voter preference, deriving from such an approach an estimate of how respondents are likely to vote. The possibilities here range from elabo- rate unstructured depth interviews to lengthy structured questionnaires asking questions which probe attitudes toward the candidates, issues, and parties. A Guttman scale analysis can be applied to a series of questions related to candidate preference and the intensity component used to determine the cutting point which would in theory divide respondents into those for candidate A and those for candidate B. To assess the validity of such a method would require considerable experimentation. However, the unexplained variance is likely to be large enough to make any estimate subject to relatively large errors.

Experience has shown that a confidential voting choice in a survey gives every indication of producing valid responses from all but a negligible number of respondents. The evidence is not definitive; we have no experimental evidence that the vote by an individual in the voting booth is the same as in the survey. Election surveys, however, using the method have yielded results of considerable accuracy with a consistency that argues for the simple choice method.

In 1952, prior to the election, concern existed that in view of Eisenhower's considerable personal prestige, survey voting preference in his favor would include a hidden "halo" effect that would be magnified by Stevenson's relatively weak level of identification among many voters. The thought was that such a factor would tend to vanish when it came time to vote, and loyalty to party would cancel out most of the inflation. Therefore, two voting preference questions were asked in surveys throughout the campaign. One question asked which candidate for president the respondent would vote for "today," linking candidate to party;2 the other asked which party the respondent would like to see win the presidency.3 A third question-Which political party do you think is best for people like yourself?-was also included. To examine the relative accuracy ofthe three questions, results obtained were related to the vote in 73 precincts for which election results were available. The simple correlations between the

2 If the presidential election were being held today, which candidate would you vote for-Stevenson, the Democratic candidate, or Eisenhower, the Republican candidate? If "undecided," ask: As of today, do you lean a little more to Stevenson or a little more to Eisenhower?

3 If the presidential election were being held today, which political party would you like to see win-the Democratic party or the Republican party? If "undecided," ask: As of today, do you lean a little more to the Democratic party or a little more to the Republican party?



320 PAUL PERRY

election result and the results of each question were as follows: r between the candidate preference question and the election result was .90 for an r2 of .81; r between party preference and the election was .83 for an r2 of .69; r between party best for self and the election was .73 for an r2 of .54.4 The multiple R between the election and results of all three questions was .90 or the same as for the candidate question alone. In the regression equation the candidate question carried a weight of .82, the party preference question a weight of .02, and the party best for self question a slight minus weight of .08. Individually the corollary questions had some predictive power as indicated by the r2's for each, but the candidate question was the best. Also the candidate preference question produced very little bias, whereas the other two questions were biased toward the Democratic party as one might expect in that particular election with a strong personality as the Republican candidate.

Measuring Likelihood to Vote

It is now widely recognized that distinguishing between voters and nonvoters in election surveys is particularly important in U.S. elections (Perry, 1973). The turnout ratio in 1972 was low enough to warrant considerable comment in the press during the 1976 campaign concerning the likely effect of a low turnout on the election outcome. In the last 15 national elections the Gallup Poll has used a turnout scale to derive from the population sample a sample of likely voters. The scale is based on nine questions which are related to voting intention.5 This is not a precision instrument, but it has demonstrated

4 The average of the individual precinct sample N's was 49, and the range was from 32 to 61 cases.

5"Plan to vote" question: Do you, yourself, plan to vote in the election this November, or not? How certain are you that you will vote-absolutely certain, fairly certain, or not certain?

Registration questions: Are you now registered so that you can vote in the election this November? Do you plan to register so that you can vote in the November election?

Ladder scale question: Here is a picture of a ladder. Suppose we say the top of the ladder marked 10 represents a person who definitely will vote in the election this November, and the bottom of the ladder marked zero represents a person who definitely will not vote in the election. How far up or down the ladder would you place yourself?

Frequency of voting question: How often would you say you vote-always, nearly always, part of the time, or seldom?

Question on voting in a previous election: In the election in [month and year] when candidate A ran against candidate B-did things come up which kept you from voting, or did you happen to vote? For whom?

Location of voting place question: Where do people who live in this neighborhood go to vote?

Precinct voting question: Have you ever voted in this precinct or district?




a validity that enables us to improve our final estimates and provide greater accuracy than simply using the total sample of registered voters.

The first effort involved a scale using five questions (in 1950). Subsequently, questions were eliminated and others added based upon the results of evaluation or validation studies after each election. Finally a Guttman scale analysis was employed to choose the best questions and to derive a scoring method. Some measure of the efficacy of the scale as well as its limitations is the proportion by scale positions who vote. In the top scale position-i.e., the group most likely to vote-in the 11 national elections, both presidential and congressional, from 1950 through 1970 an average of 87 percent voted. In the lowest scale position an average of 8 percent voted.

Using data from 3 of the 12 postelection validation studies, 2 x 2 contingency tables were formed in which the dependent variable is whether the respondent did or did not vote, and the independent variable consists of the answer categories of each of the questions or items from which the turnout scale is constructed (where required collapsing to two categories). Using the Percentage Difference score as a measure of association, which in a 2 x 2 table is the equivalent of Somers' D coefficient, the turnout questions ranked as shown in Table 1.

In presidential elections the gain in accuracy using the turnout scale over using a sample of the total electorate is usually not large; in congressional elections, however, making no allowance for nonvoting can introduce an important source of error. For example, using the total samples with no exclusions in the seven presidential elections from 1952 through 1976, the results have averaged about one to two points more Democratic than the elections on a major party basis, and about one to two points more Democratic than likely voters as defined by the scale and cutoff point. In the seven midterm congressional elections 1950 through 1974, the total sample results have averaged about 4 points more Democratic than the elections and 4 points more Democratic than likely voters.

In 1976, in our final survey, the division of the vote in the total sample with no exclusions was Carter 48 percent, Ford 43 percent, McCarthy and other 4 percent, undecided and refusal 5 percent. In

Interest in politics question: Generally speaking, how much interest would you say you have in politics-a great deal, a fair amount, only a little, or no interest at all?

Interest in coming election: How much thought have you given to the coming November elections-quite a lot, or only a little?

Intensity of preference: Right now, how strongly do you feel about your choice- very strongly, fairly strongly, or not strongly at all?



322 PAUL PERRY

Table 1. Relative Efficiency of 9 Items as Measures of Likelihood of Voting Using Somers' D Scores

1966 1970 1976 Type of Item Congressional Congressional Presidential Mean

D D D D Registration 76.0 72.0 72.6 73.5 Plan to vote 79.7 63.7 71.2 71.5 Ladder scale 76.0 62.4 70.8 69.7 Frequency of voting 62.7 49.4 57.6 56.6 Awareness of where to vote 55.3 49.4 60.7 55.1 Voted in a previous

presidential election 59.2 46.3 53.0 52.8 Voted in precinct 47.3 52.0 49.1 49.5 Interest in politics 42.2 42.9 56.8 47.3 Interest in coming election 27.1 23.0 26.7 25.6 Intensity of preferencea 22.6 24.1 12.7 19.8

(N) (515) (404) (451)

a Although it is not a part of the scale, the association between a measure of intensity of preference and the act of voting is often of interest and for that reason is shown for comparison with the scale items.

the share of sample matching the turnout ratio the division of the vote was Carter 48 percent, Ford 46 percent, McCarthy and other 2 percent, undecided and refusal 4 percent. Carter, then, among those most likely to vote had the same percentage as he had among all those of voting age, and Ford was 3 percentage points stronger among the most likely voters. McCarthy and other candidates were two percentage points weaker among likely voters than in the total sample.

Part of the task of coping with the nonvoter problem is devising a method that will provide a reasonably accurate projection of the turnout ratio. For this purpose we have used only two of the nine questions in the turnout scale. The relationship between a composite of these two questions and the turnout ratio in the last 14 national elections has been close enough to enable us to project the turnout ratio in the 1976 presidential election almost exactly. It has been acceptably close, however, in most other elections. The average error in this projection has been 2.2 percentage points with the largest error six points. Deviations of this order can contribute very little to the error in the estimated vote distribution-perhaps no more than .2 or .3 of a percentage point.

Measuring Late Pre-election Trends

Given a satisfactory sample and a valid response, last-minute shifts can negate all the care expended in that direction. Therefore, in recent elections some have resorted to doing surveys on election day




among people coming from the polls. Others have made it a practice to poll through the last weekend prior to election day. From 1950 to 1976 the final estimate of voter preference by the Gallup Poll has been based on a system of two surveys, one taken in early October and a final survey centering in time on the Friday before election day. The first survey results have been processed using the complete estimating procedure, which includes the scale derived from nine questions designed to identify respondents most likely to vote. The second survey has been tabulated by the interviewers and telegraphed or telephoned to the central office. Both surveys have been done in the same precincts. A simplified breakdown of the data by likely voters and nonvoters has been employed in the second survey to minimize tabulating and transmission errors by the interviewers. They have tabulated the voting preference only among those who say they plan to vote.6 The trend, if any, in the "plan to vote" group has then been applied to the results of the first survey which have been processed using the full estimating procedure including the nine question turnout scale. The trend in the "plan to vote" group has been nearly the same as the trend among "likely voters," as determined by the scale, in every election except 1976. In 1976 there was a difference, and it contributed to the error. Using the simplified trend measurement resulted in a final estimate showing Ford one point ahead of Carter. Applying the full estimation procedure to the final survey data produced an estimate with Carter two points ahead of Ford. However, this estimate could not be made until all the completed questionnaires had been returned and processed the week following the election. As a result of this experience, in 1980 each and every case in the final survey will be telephoned in so that the complete procedure can be applied in detail for the final release.

Conclusion

The various technical changes in method have had a measurable effect in error reduction. In the seven national elections from 1936 through 1948 the mean error was 4.0 percentage points; in the five elections from 1950 through 1958 the mean error was 1.7 points; in the five elections from 1960 through 1968 the mean error was 1.5 points; in the five elections from 1970 through 1978 the mean error has been 1.0 points. In the 15 elections beginning with the 1950 congressional elections 7 of the 15 final estimates had a deviation of less than 1

6 Based on "yes" responses to the question: "Do you, yourself, plan to vote in the election this November, or not?"



324 PAUL PERRY

percentage point, 11 had a deviation of less than 2 percentage points, and 14 had a deviation of less than 3 points.

It is of interest to note the theoretical error possibilities contributed by the number of primary sampling units. For example, we use approximately 350 precincts in our final pre-election sample. Even if we had a 100 percent sample of accurately defined voters within each of these precincts, the expected average or mean error for repeated samplings would be 0.8, one standard error would be 1.0 percentage points, two standard errors would be 1.9.7 Doubling the number of the areas or precincts as the case may be would not help much either. The expected mean error with 700 precincts would be 0.6. One standard error would be 0.7 of a point and two standard errors 1.4. With these expected theoretical errors in the optimum case of a 100 percent sample in each precinct or sample location, in surveys with samples of 5 or 10 in each location as is typically the case, one can expect that average errors in national general election surveys will certainly be greater than 1 point, and errors of 2 points or greater will not be uncommon.

Add to the theoretical sampling error attributable to sample size alone the other sources of error inherent in the methodological problems discussed earlier, and one could say that in the long run errors greater than 2 percentage points are likely to be a common occurrence in election surveys, at least in the United States, even when such surveys are conducted on the eve of an election.

References

Benson, Lawrence E. 1941 "Studies in secret ballot technique." Public Opinion Quarterly

5:79-82. Cantril, Hadley, and John Harding

1943 "The 1942 elections: a case study in political psychology." Public Opinion Quarterly 7:222-41.

Hansen, Morris H., and Benjamin J. Tepping 1969 "Progress and problems in survey methods and theory illustrated by

the work of the United States Bureau of the Census," in New Developments in Survey Sampling. New York: Wiley.

Kish, Leslie 1957 "Confidence intervals for clustered samples." American Sociologi-

cal Review 22:154-65.

7 S.E. (y) = s vnH where 5y = mean % Democratic or Republican; s = the standard deviation of the % Democratic or Republican of the Democratic-Republican vote by precincts which is estimated to be about 18 percentage points based upon the samples of precincts drawn for Gallup Poll surveys in the elections of 1950 through 1970; n = number of precincts.




Mosteller, Frederick, Herbert Hyman, Philip McCarthy, Eli Marks, and David Truman

1949 The Pre-election Polls of 1948. New York: Social Science Research Council.

Neyman, J. 1934 "On the two different aspects of the representative method: the

method of stratified sampling and the method of purposive selection." Journal of the Royal Statistical Society 97:558-625.

Noelle-Neumann, Elisabeth 1977 "Turbulences in the climate of opinion: methodological applications

of the spiral of silence theory." Public Opinion Quarterly 41:143-58. Perry, Paul

1960 "Election survey procedures of the Gallup Poll." Public Opinion Quarterly 24:531-42.

1965 "Election survey methods." Gallup Political Index, Report 7, Spe- cial Supplement, pp. i-xiii, December 1965.

1973 "A comparison of the voting preferences of likely voters and likely nonvoters." Public Opinion Quarterly 37:99-109.

Yates, Frank 1960 Sampling Methods for Censuses and Surveys, 3rd ed. London: Grif-

fin.



certain problems in election survey methodology

Documents