the 2000 canadian election and poll reporting under the new elections act

Canadian Public Policy

The 2000 Canadian Election and Poll Reporting under the New Elections ActAuthor(s): Claire DurandSource: Canadian Public Policy / Analyse de Politiques, Vol. 28, No. 4 (Dec., 2002), pp. 539-545Published by: University of Toronto Press on behalf of Canadian Public PolicyStable URL: http://www.jstor.org/stable/3552213 .

Accessed: 14/06/2014 18:54

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

University of Toronto Press and Canadian Public Policy are collaborating with JSTOR to digitize, preserveand extend access to Canadian Public Policy / Analyse de Politiques.

http://www.jstor.org

This content downloaded from 195.34.79.49 on Sat, 14 Jun 2014 18:54:09 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=utp

http://www.jstor.org/action/showPublisher?publisherCode=cpp

http://www.jstor.org/stable/3552213?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


The 2000 Canadian Election and

Poll Reporting under the New

Elections Act

CLAIRE DURAND Department of Sociology University of Montreal Montreal, Quebec

Juste avant la campagne 6lectorale de l'an 2000, une nouvelle loi a 6t6 votee, rdglementant la publication des

sondages dans les mddias, pendant les campagnes 6lectorales. Cette loi exigeait que les journaux publient une information de base sur la fagon dont 6taient conduits les sondages qu'ils pr6sentaient, le libell des questions et les moyens par lesquels pouvait &tre obtenu un rapport methodologique. En outre, la loi specifiait quelles informations le rapport methodologique devait contenir. Notre article examine dans quelle mesure la loi a dt6

respectee. Notre recherche montre que les deux premiers points n'ont pas et6 problematiques, bien que l'information n'ait pas toujours 6t6 facilement accessible. Toutefois, les rapports methodologiques n'ont gdndralement pas fourni toute l'information requise. En particulier, des informations d'une importance cruciale n'ont et6 que rarement fournies : le taux de reponses, le taux de refus, la m6thode de sondage utilisde. 11 semblerait aussi qu'on n'ait pas verifi6 si la loi avait 6te ou non respect6e. (<Elections Canada>> comptait sur le public pour- mettre des plaintes et les medias comptaient sur les sondeurs pour fournir l'information. Notre etude conclut premierement que les

ambiguites de la loi quant 'i savoir exactement quelle information est requise devraient etre clarifiees; deuxiemement, que l'information methodologique devrait etre rassemble dans un paragraphe sdpare et identifi6, comme c'est le cas dans la plupart des rapports fournis par les medias; troisiemement, que <<Elections Canada >> devrait fournir un modile du rapport detaill6 (326-3) de sorte que les medias et les sondeurs presentent tous l'information sous une forme standard facilitant la comparaison.

Just before the 2000 Canadian electoral campaign, a new law was passed regulating the publication of polls in the media during electoral campaigns. The law required newspapers to publish basic information about the way the

polls they presented were conducted, and the wording of questions and the means by which a methodological report could be obtained. In addition, the law specified which information the methodological report must contain. This article examines the extent to which the law was complied with. The research shows that the first two points were not problematic, though sometimes the information was not easily available. However, methodological reports did not generally provide all the requested information. In particular, crucial information such as response rates, refusal rates and the sampling method used were rarely provided. It would also appear that compliance with the law was not verified. Elections Canada relied on the public to issue complaints, and the media relied on pollsters to provide the information. The study concludes first, that ambiguities in the Act concerning exactly what information is required should be clarified; second, that methodological information should be placed together in a separate and identified subsection, as was the case with most media reports; and third, that Elections Canada should provide a model of the detailed report (326 -3) so that the media and pollsters all present the information in a standard, easy-to-compare fashion.

CANADIAN PUBLIC POLICY - ANALYSE DE POLITIQUES, VOL. XXVIII, NO. 4 2002



540 Claire Durand

INTRODUCTION

Just prior to the 2000 Canadian electoral campaign, a new law was passed that regulated the

publication of polls in the media during electoral

campaigns. The Canadian Elections Act, Part 16, article 326, states that "The first person who trans- mits the results of an election survey ... must provide the following together with the results: the name of the sponsor ... the name of the person or organiza- tion that conducted the survey; ... the period during which the survey was conducted; the population from which the sample of respondents was drawn; the number of people who were contacted ... and, if applicable [author's italics], the margin of error."' In addition, the law states that "the wording of the

survey questions [for non-broadcast presentation only] ... and the means by which a report ... may be obtained" must be presented. Finally, section (3) of article 326 states that the methodological report shall include the following information about how the survey was conducted: the sampling method; the

population from which the sample was drawn; the size of the initial sample; the number of individuals who were asked to participate in the survey; the numbers and respective percentages of those who

participated in the survey, refused to participate, or were ineligible to participate in the survey; the dates and time of day of the interviews; the method used to recalculate data to take non-response into account; and the weighting or adjustment factors that were used. This law was in effect for the first time in the 2000 federal election. The goal of this paper is to examine to what extent the media, which sponsored the polls and were responsible for providing the information, complied with the law. More specifically, the article examines first, whether the media provided the required information in articles presenting poll results; second, whether detailed methodological reports were made available; and third, whether the available reports contained all the required information.

STUDY RESULTS

The research focuses on the print and electronic media for two reasons. First, articles 326(2) and 326(3) apply only to these media and not to broadcasters; and second, all the polls sponsored by broadcasters were co-sponsored by newspapers. Consequently, the information provided in the print and electronic media constitutes all the available information. The study thus collected all newspaper articles that presented first-hand poll data in the national media in English and French, and consulted the pollsters' Web sites, to which the media referred for detailed reports. The polls conducted by Environics and Zogby which, though not sponsored, were published on their respective Web sites and referred to in the media, were included in the analysis. After the election, the media outlets, which had published sponsored polls, were contacted by letter, informed of the missing information required under the Act, and requested to provide it or ensure that the pollster provided it. Except for one newspaper, which phoned to say that the required information was private information and would not be provided, none of the letters were replied to.

Table 1 presents the information that was made available in the newspaper articles or on the Web sites (in the case of Environics and Zogby). It shows that most media provided all the required information. In one case - a Decima poll conducted for Global TV and presented in The Globe and Mail -

only the information requested for broadcast presentation was made available. Finally, the two

pollsters who conducted unsponsored polls also provided most of the basic information. Moreover, in some cases, additional information such as percentages of undecided respondents, allocation of non-disclosers, weighting and response rates, which are required to be provided in the detailed reports, was willingly provided.

However, if one examines the quality of the information provided, the verdict is somewhat less




z z 0 z

co

0

0 ("3

0

z

0

>.

0 0

r•

ml

mvl

?

x:

'zl

< ?

h,,=

TABLE 1 Methodological Information Provided in the Print Media

Sponsor Pollster Number Name of Period during Population from No. of people Margin Wording Means by which Additional information provided of sponsor which survey which sample "contacted" of a report may be surveys and pollster was conducted was drawn respondents error obtained

Toronto Globe and Ipsos-Reid 4 / ,/ Adult 1502, 2500, / / /(1,2,3) (1,3,4) weighted by Census 96 region, Mail Canadians 2552, 4102 (paper) age, sex (2) percent undecided Global TV (reported Decima 1 / / Canadian 2585 / NA NA in Toronto Globe citizens 18+ and Mail) National Post Compas 3 / / (1) voters 1600, 1275, / (2) / / Website percent undecided

(2,3) people 1032 La Presse - Ekos 4 (1 / / respondents 2265, 2455, / / / Website ref. to refusal and other Radio Canada Ontario 1595 (Ont.), + paper measurement errors

only) 3910 Journal de LUger 4 (1 / / Canadians 18+, 2139, 3614, / (2,3,4) / / Website (1) weighted census region, sex, Montreal - TVA Marketing Quebec speaking French 2213, 2514 home lang., no. of calls, (1,2,3)

only) or English response rates (2) initial N Montreal Gazette Som 1 / / Quebec voters 976 / / - allocation of non disclosers - Le Soleil (QC only) Le Devoir Sondagem 2 / / Adults 1008, 912 / / / paper Initial N, response rate, allocation (QC only) of undecided, sampling method

Environics* 4 NA / Eligible voters 1339, 1857, / / NA type of poll: tracking, nightly 1835, 1815

Zogby* 2 NA / Decided voters 899, 899 / / NA time of day of calls

Notes: / : complete information -: no information provided NA: Not applicable *: not sponsored () : survey(s) for which the information was provided

0

6"

0

0

q3

q3

<3

(h

4•I



542 Claire Durand

positive. The Act is ambiguous in that it requires the number of people contacted to be provided in the report but not the number of respondents. The

newspapers generally interpreted this requirement as referring to the number of respondents and provided only this information. In addition, details about the reference population were rather scant; for example, reports referred to "people," "respondents," "adult Canadians," or "voters." Only the Journal de Montreal (L6ger) provided a specific definition: Canadians aged 18 and older, speaking French or English. Even so, it is possible that some firms screened for registered voters, Canadian citizens, or even respondents who intended to vote.

Newspapers provided no specific information on this

question.

In order to compare the 2000 situation with that of the previous election, the same analysis of the

newspaper articles presenting first-hand poll results was performed for the May/June 1997 electoral cam-

paign (data not presented). The comparison leads one to conclude that not much change has occurred. In 1997, most English-speaking media outlets did not provide the wording of the questions asked

(which they did in 2000), and one outlet did not provide the name of the pollster. All the other basic information required by 326(1) was provided in 1997. In an analysis of poll reporting during the 1997 election, Andersen (2000) states that in newspaper reports in which poll results were the main focus

during the 1997 electoral campaign (N=691), 64

percent reported the survey dates; 33 percent, the

sample size; 25 percent, the margin of error; 21 percent, the wording of the question; and 14 percent, the percentage of undecided respondents. However, Andersen examined all articles, not just those re-

porting first-hand poll results. This leads to the conclusion that the basic methodological information was provided only in the articles presenting first-hand poll results and not subsequently.

The second section of article 326 requires that newspapers indicate the means by which a detailed methodological report may be obtained. Table 1

shows that most newspapers complied with this re-

quest, many referring to the pollster's Web site.

Article 326 (3) stipulates the information that must be provided in the methodological report. Gen-

erally, poll sponsors referred readers either to the

pollsters' Web sites or included a postal address where requests could be sent. Some incorporated part of the information into the articles themselves. However, two co-sponsors did not comply with this

part of the Act, neither mentioning that a methodo-

logical report was available nor providing additional information when it was requested. Table 2 shows the extent to which the requested information was available, whatever the means used.

The only information provided by most outlets was the data on undecided and non-disclosers. All the pollsters but one provided the percentage of undecided respondents or non-disclosers, the latter term usually including all those who would not indicate a vote intention (undecided, refusal, will not

vote). The standard method of dealing with non- disclosers was to exclude them (akin to proportional allocation). In two cases, this was likely the method used, but it was not stated clearly in the document

provided on the site. The proportion of undecided

respondents reported by two pollsters was particu- larly low, namely L6ger at 6 percent and Ipsos-Reid at 11 percent. This suggests that these two firms used a different methodology or a more restrictive definition of undecided voters.

The second most useful reported information is the basis used for weighting in order to adjust for stratification; this was provided by five out of nine firms. Among the four other firms, however, some

may have used a general unstratified random sam-

ple and not adjusted further, but this information was not clearly stated in the consulted texts.

By contrast, only three pollsters provided information about the results of the data collection phase (i.e., response rates, refusal rates, and ineligibility rates), although this information is crucial in order




z

z

r- ob o r

z

r-"

0 t0

rn

D'1

o

c"

0

0

r

z 0

0 0

o

.,<

TABLE 2 Additional Methodological Information Provided in the Reports

Sponsor Pollster Sampling Size of Response Refusals and Time of % non Allocation Adjustment & weighting method initial rate ineligibles day of disclosers** of non

sample (N & %) interviews disclosers

Toronto Globe and Ipsos-Reid "random 11% (3-11 Nov.) region, age, sex, Mail selection" Census 96

Global TV (reported Decima "phone - - - / 19% (19-23 Nov.) / region, age, sex, mother in Globe & Mail) bank" tongue, Census 96

National Post Compas - - - - - 19% (11-14 Nov.) / -

La Presse Ekos - - - - - 25% (20-22 Nov.) / region, age, sex, census

Journal de I Leger - / / - - 6% (19-23 Nov.) region, sex, language spoken Montreal Marketing at home

Montreal Gazette - Som - - - - - 18% (19-21 Nov.) -

Le Soleil (QC only)

Le Devoir (QC only) Sondagem / / / - - 19% (17-23 Nov) / -

Environics* / / / / / 19% (16-22 Nov.) -

Zogby* - - - - / - - region, age, sex, voting population

Notes: / : complete information -:no information provided *: not sponsored **: for the most recent survey where the information was provided (): survey(s) for which the information was provided

Th

t% 0

., 0

0-

23

0

0S •b

c?

4•



544 Claire- Durand

to evaluate surveys results (Vachon, Durand and Blais 1999) and is required by law. L6ger and

Sondagem provided information about their initial

samples and response rates, while only Environics

provided information about ineligibles and refusals. Specific information about the sampling method was provided by two firms (Sondagem and

Environics), which described the selection process, the stratification used, and the sampling frame. Minimal information (use of random selection and a phone bank) was given by two other firms (Decima and Ipsos-Reid). At least some of the firms used

quotas, but information on the use of and basis for

quotas was not provided. The time of day of interviews was provided by three out of nine firms.

Finally, information was not always easily avail-

able, in some instances merely taking the form of a comment by the pollster's spokesperson in the text of the article. If one compares the information provided under the Elections Act with the information

provided by the media and pollsters in 1997, not much change can be reported.

DIscusSION

A thorough comparison with the preceding election shows that any improvement between 1997 and 2000 is so small that the Act might as well never have been passed. The main difference is the inclusion of both the wording of the vote intention question in the English-speaking media and also a note stat-

ing that a detailed methodological report is available

upon request. However, the information provided in these reports varies and has more to do with the general practices of each firm than with compliance to the law. The media outlets responsible for the polls clearly did not ensure that the pollsters provided the required information, nor did they reply to formal requests to provide the missing information.

Who, then, is responsible for applying the law? The Act says that anyone who contravenes sub-

sections 326 (1), (2), or (3) is guilty of an offence and is liable to a fine of $1,000 or imprisonment for a term of not more than three months (500-1); if the offence is committed wilfully, the fine can reach

$25,000 (500-4). The Act also states that the guilty person can be directed to perform any obligation, the non-performance of which gave rise to the offence (501c). It is the duty of the Commissioner of Canada Elections to enforce compliance with the Act; however charges must be laid within 18 months after the day on which the offence was committed. The commissioner can also apply for an injunction during the electoral campaign. Clearly, however, nothing was done to ensure, even minimally, that the Act was respected.

Following the 2000 election, the report of the CEO of Elections Canada had only one recommen- dation pertaining to election polls: that subsection

326(2) be amended so that "broadcast media regulated by the CRTC [be] subject to the same

requirements that broadcast media not regulated by the CRTC (e.g., the Internet) must [currently] meet with respect to disclosure of the wording of the questions used in political opinion surveys and the means

by which further details about such surveys may be obtained" (Elections Canada 2001, p. 74).

CONCLUSION

The 2000 Canadian election was the first to be held under the new legislation aimed at allowing a closer examination of the quality of the polls published during federal electoral campaigns. It may be con- cluded that the new law has given Canadians only slightly better information about the polls published in the media. In contrast with the 1997 election, all the media reports provided the exact wording of the

question. Some provided important pieces of information in their articles, pertaining to the sampling method, response rates, weighting, and adjustment.




The 2000 Canadian Election and Poll Reporting under the New Elections Act 545

However, generally speaking, the media relied on

pollsters to provide the specific methodological information required under the Act. They do not appear to have checked that the pollsters did in fact

provide such information, whether of their own vo- lition or upon request. Nor did pollsters generally modify their usual practices. Those that usually provided detailed information continued to do so, and those that did not generally provide information also continued their usual practices. Finally, the CEO of Elections Canada appeared to rely solely on com-

plaints and did not initiate any inquiries.

It would be easier for the media and pollsters to

comply with the law if first, the Act clarified ambi-

guities with respect to the exact information

required; second, methodological information was

required to be placed together in a separate and identified subsection, as most media outlets did; and third, a model for the detailed report (326 -3) was

provided by Elections Canada so that the media and

pollsters could present the required information in a standard, easily comparable manner.

NOTES

The author wishes to thank the reviewers for their very useful comments. The research benefited from the FCAR fund.

'The margin of error is applicable only when random sampling is used at all phases of the selection process, that is, selection within households as well as selection of households (phone numbers). However, the practice of reporting a margin of error whenever a survey is conducted is generalized, though not always justified.

REFERENCES

Andersen, R.M. 2000. "Reporting Public Opinion: The Media and the 1997 Canadian Election," International Journal of Public Opinion Research 12(3):285-98.

Elections Canada. 2001. Modernizing the Electoral Pro- cess: Recommendations from the Chief Electoral

Officer of Canada following the 37th general election. Ottawa: Elections Canada.

Vachon, S., C. Durand and A. Blais. 1999. "Les sondages moins rigoureux sont-ils moins fiables?" Canadian Public Policy/Analyse de Politiques 25(4):557-61.




the 2000 canadian election and poll reporting under the new elections act

Documents