© 2010 GfK NOP

Exit Polls – the Whys and Wherefores (and more importantly the Hows)

Nick Moon, GfK NOP

2

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

My own experience

Extensive research to develop an exit polling technique for the BBC

Ten Parliamentary by-elections

1990 European elections

Four GB general elections – 1992 – 2005

One Greek general election

On-site observation of US Presidential Election 2000

The most recent Ukrainian Presidential Election

3

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

Opinion Polls and Exit Polls

Methods for conducting opinion polls can differ enormously from one country to another

•Face to face/telephone/online

•Random/quota

Methods for conducting exit polls are much the same the world over

•Ukraine is a vast country (with even worse weather) but there exit poll was almost exactly the same as ours

4

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

Cut to the Chase, Nick

Don’t just make excuses about Ukraine being a big country – tell us how you got on

5

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

While the Triumph is Still Fresh

Result Poll ErrorYanukovych 35.3 34.5 -0.8Tymoshenko 25.1 25.6 +0.5Tigipko 13.1 13.8 +0.7Yatsenyuk 7.0 7.0 0Yuschenko 5.5 5.6 +0.1

6

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

What is an Exit Poll?

A very expensive way of getting an idea of an election result a few hours before you know for sure

A major contribution to understanding of the electoral process

A way to tell if the election was conducted fairly

• (or is it?)

7

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

Types of Exit Poll

Prediction Poll

•Who is going to win and by how much?

Analysis Poll

•Why did voters vote the way they did?

8

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

An exit poll may be more than a poll

Interviewing voters

•This is all we do in the UK

Collection of precinct-level results as soon as they are announced

•A representative sample can predict the whole, quickly

– In some European countries this is all that is done

– In the US this is done alongside a voter survey

Analysis of partially aggregated actual results

•Used to fine-tune projections made using the above

9

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

Exit Poll Problems - issues

Interviewing has to be clustered by polling station

•Cost is a function of number of stations, not number of interviews

Voting behaviour tends to be very clustered at the polling station level

•Several cases in the past of >90% support for one party

– And several of 100% in Ukraine

•Far more interviews are conducted than are actually needed, just because the interviewer might as well do as many interviews as are easily achievable

Actual results at polling station level not always available, which makes sampling less reliable

•US and Greece yes, UK and Ukraine no

10

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

Exit Poll Problems – issues (UK)

We are really predicting the result of over 600 separate elections, and can’t cover each one

The only local level information available on actual voting is at ward level, and only for local elections

•Local level turnout is far lower and behaviour often very different

•This makes stratification of the sample by voting is virtually impossible

11

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

Exit Poll Problems – UK solutions

Use as many polling stations as possible

Use the same polling stations as the previous election

•Can look at change rather than simple totals

•Eliminates effect of bias IF bias is constant (Big IF)

12

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The 2005 GB Exit Poll

Conducted jointly by NOP and MORI

Conducted on behalf of both BBC and ITN

120 polling stations

•Most used previously by NOP or MORI

Approx 16,500 interviews in all

Just measuring claimed current vote

13

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The Practicalities (1)

Polling runs from 7am to 10pm

Split into two equal shifts

Each has an interviewer and a “clicker”

“Clicker” uses preordained sampling interval to select respondent

Interviewer estimates voting behaviour, then asks voter to fill in an exit poll ballot

No replacement for refusals

14

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The Practicalities (2)

Ballot designed to replicate actual one as closely as possible

•Has candidate names and parties

•In alpha order as on real ballot

Ballots collected every hour

Individual results phoned back to team of telephone interviewers

Results weighted to take into account any discrepancy between intended and actual number of interviews in each hour

15

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The Practicalities (3)

Aggregate data at level of time by polling station produced at regular intervals

•Along with individual level data file

Collected by the BBC/ITN analysts

Run through various algorithms

Initial projection of result at 10pm

•Final projection at 11pm

16

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

And How Good Was It?

17

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

In case you’ve forgotten

NOP Actual result

36 36

33 33

23 23

18

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The BBC/ITN exit poll

Forecast majority 66 seats

Actual majority 66 seats

19

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The BBC/ITN exit poll

Jeremy Paxman (paraphrased)

“I don’t know why we spend all this money on the exit poll, we should just do without it”

Tony King (paraphrased)

“These results suggest our exit poll is overestimating Labour”

20

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The BBC/ITN exit poll

Predicted majority based on exit poll

66 seats

Predicted majority based on early actual results

80 seats

Final majority

66 seats

21

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The BBC/ITN exit poll

Sorry Jeremy

The BBC should carry on paying for the exit poll, and stop paying for everything else

22

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

The 2010 General Election Exit Poll

Conducted jointly by GfK NOP and Ipsos MORI

BBC and ITN now joined by Sky

As close as possible to the 2005 design

•Changed polling places where the constituent voters or the place itself had changed significantly

•Replaced these with an eye to filling gaps in current design and also looking to the next election

23

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

Are exit polls measures of election fixing?

Ukraine 2004 Round 2

•But many other indicators of fraud

Venezuela

•Questions raised about exit poll

US 2000

•There almost certainly was some fraud but the exit poll is not a smoking gun

24

GfK NOP Exit Polls Whys and Wherefores Nick Moon 20 January 2010

© 2010 GfK NOP

To conclude

Exit polls are a good way to give a good advance idea of who has won, and why, and properly conducted exit polls can act as some sort of check on electoral fraud, but as President Carter said, they should not be used as sole arbiters of the fairness of an election