Concern re- exit polls in the U.S. --

[snot]

they are the obvious next target for rigging, since they are a main basis for questioning rigged vote results.

In fact, I remember reading that even for the 2004 elections, the exit poll system had supposedly been overhauled and reformed in order to avoid the kind of discrepancy that occurred in 2000! Been wondering lately just what kind of overhaul occurred.

Is anyone looking at who is or will be in control of the exit polls and what's being done to them, for 2006 and beyond?

[EST]

1. Expect it.

The 2004 exit poll results were fudged, hour by hour, to allow and conform to the rigged voting machine results. Crooks and liars prevail, but we fight back as best we can.

[dchill]

2. They were already rigged...

RETROACTIVELY in 2004, so - you're obviously right. As for who will be in control of the exit polls in 2006 & beyond - might as well go with who they know - Edison/Mitofsky. They're tried and true. :sarcasm:

[Lasher]

3. Exit pollsters were on a short leash in 2004, probably 2000 also

Remember what they did when the vote counts strayed from the exit poll results? The exit pollsters capitulated so fast, it's a wonder they didn't all get whiplash. They even changed their exit poll results to match what was going on with the vote.

I had hoped that international observers might come in and make a difference, but remember seeing nothing from them, one way or another.

[Febble]

7. Please check your

FAQs

I mean, feel free to consider that the exit poll projections are simply an attempt to project the actual count, not an attempt to audit the election.

Feel free to assume they are useless for this purpose.

But what was done in 2004 was what is done every year, because the purpose of the exit poll is to project the vote-count, not audit it, as you can see from the FAQ, which was available well before the election. I certainly read it.

If you want an audit of your elections, then push for an audit.

If you want exit polls to do this, then design your exit polls for the purpose, perhaps on the lines of the BYU exit poll conducted each year in Utah, which nailed the Utah result.

But it will be extremely expensive, and not nearly as good as any decent random audit of your vote counts. Polls are lousy tools for an audit. They are subject to bias, and there is no way of eliminating it; what is worse, is that you have no way of knowing whether your poll is biased or not, unless you get a 100% response rate and you can be sure that you have truly randomly sampled your voters.

Neither assumption holds for the 2004 exit poll.

(edited for typo)

[Lasher]

8. Exit poll/election result discrepancies should be ignored?

Is that what you're saying?

Since exit polls are conducted to project election results, are you saying it is wrong to use them for any other purpose? If so, I don't agree. An unexplainable and significant exit poll/election result discrepancy can and should be considered as a possible indication that the election result was not recorded as voters intended.

All decent exit polls account for the bias you mention by projecting a margin of error. Within this parameter, they have a track record of being pretty accurate, on average. I don't agree with your assertion that audits and specifically designed exit polls are the only legitimate resources to be considered in this respect.

[Febble]

9. Did I say that?

No, I didn't. I agree that "an unexplainable and significant exit poll/election result discrepancy can and should be considered as a possible indication that the election result was not recorded as voters intended."

But "All decent exit polls account for the bias you mention by projecting a margin of error" is not true. The margin of error allows for sampling error - variations due to chance in a random sample. This can be computed mathematically. It cannot allow for bias, because there is no good way of computing bias.

Exit polls do not have a particularly good track record for being accurate. The US ones have a fairly good record for projecting the results, but that is because they also use the vote-returns.

The "raw data" is usually substantially biased. This can be partially compensated for by collecting age/race/sex data on non-respondents, but that only compensates for bias by visible characteristics. Invisible characteristics (like who they voted for) cannot be observed, and therefore cannot be compensated for. So the UK polls compensate for it by reference to the vote count.

Exit polls that do not do this (UK ones for example) are often out by quite a few points, and yet our voting system is pretty transparent and secure. Moreover, exit polls suffer from the problem that they are conducted face-to-face, something that UK research suggests increases bias.

Sure, the count could have been the cause of the discrepancy (for a number of good reasons I don't think it was, largely) but just as easily, it could have been bias in the poll. Polls can be biased. It has nothing at all to do with the MoE.

[Febble]

14. Correction

I meant, of course,

"So the US polls compensate for it by reference to the vote count".

In the UK, because we just have one time zone, we get the exits at poll close (10.00pm), then nothing else until the first results come in, so it is obvious when the projections start including the vote count. On the BBC, Peter Snow has a swingometer that projects the final result, based first on exits, then on returns.

Essentially, the US polls do the same, but it would be a lot clearer if you had a swingometer (and Peter Snow makes the whole process a lot clearer than CNN did) because you can watch it change as the results come in, which of course it does. It swings wildly at first, then gradually settles down as more and more results arrive.

So no-one in the UK has much faith in the exits, because we know they are not reliable. We can see that they aren't. The swingometer always ends up in a different place at the end of the night from the place where it started (when it was measuring the exits only). Peter Snow' catch phrase for the swingometer is "it's all a bit of fun...."

[southwood]

22. Count?

"...feel free to consider that the exit poll projections are simply an attempt to project the actual count..."

Seems to me they are trying to predict election outcome, which may or may not correspond to the actual count... (depending on how decent the election is run as a process).

I do seem to remember quite clearly that the last general election in Britain was predicted most accurately in the exit poll.

With E/M in the U.S., basically the following happenend. Their initial projections were way off in comparison to election outcome. We all waited for their data to be published. Turned out they didn't publish them so that precincts could be identified, using some nonsensical privacy argument for an excuse. Much later, apparently (after I quit reading this thread for many months) it even turned out they had only published part of their data. All highly suspcious, from a broader point of view.

Seems to me there are two choices. Either E/M screwed up completely, and they should publicy admit so and refuse to ever do an exit poll again, or the election was a fraud.

In combination with the reports on challenging voters during the election, pre-challenging voters (i.e. before the election, without their knowledge), the activities of the so-called Texas strike force, removal of the votes of an entire black liberal arts college, and so on, and so on, in combination with the initial exit poll results, chances that GWB actually won the election seem to be smaller than chances that he lost. IMHO.

[Febble]

25. Sure, the exit pollsters got the UK

just about right last year, but they certainly don't always.

You write:


With E/M in the U.S., basically the following happenend. Their initial projections were way off in comparison to election outcome.

This is true, except that these initial projections were not deemed "significant" by their own criteria. What I think you mean is that the exit poll responses that contributed to these projections (which included pre-election polls) were "way off" - which indeed they were, as they reported in their January evaluation. I am not just picking nits here: it is important. What is clear is that the reason that the early projections favoured Kerry was not because of errors in precinct selection, or other errors in their system for making predictions from their raw data, but because the actual responses, at precinct level (the "Within Precinct Error") was unusually large. This narrowed down the cause of the error to some form of bias in the sampling of voters (selection bias or non-response bias) - or fraud.

We all waited for their data to be published. Turned out they didn't publish them so that precincts could be identified, using some nonsensical privacy argument for an excuse.

They made available what they always make available, in January: responses to every question on every questionnaire that was entered into their database on election day (a subsample of the total responses). This includes age/race/sex of each respondent. Precinct identifiers are never given as this would violate the confidentiality of the respondent - it would, at least in some cases, allow respondents, to be identified together with detailed confidential information regarding not only their vote but a large range of their views on all kinds of other things. This would be an appalling violation of privacy. To describe it as "nonsensical" seems bizarre. There may be an argument for releasing aggregated data in some form; indeed a "blurred" dataset (vote shares for each precinct are replaced by the mean vote shares for a similar band of precincts to prevent identification of precincts from the vote-share values) was prepared for Ohio and analysed by Election Science Institute. I understand further datasets for other states may be similarly prepared.

Much later, apparently (after I quit reading this thread for many months) it even turned out they had only published part of their data.

They published all the full questionnaire data that was used. In addition, total responses for the presidential vote question were tallied from all questionnaires, by the interviewers, and phoned in as totals, and used for the early state projections. What, therefore, you are asking for, is for these totals, together with vote-share totals from each precinct. Essentially, this is what ESI were given in "blurred" form.

Yes, what they made publicly available was only "part of their data" but it was the part they used for the National Survey, and all crosstab analyses. The part they did not make available was the part that was only entered into the database as total tallies for the presidential vote question, without accompanying questionnaire data.

All highly suspcious, from a broader point of view.

Not particularly, because exactly what is done every year. Unusually, this year, they also issued a report in which they analysed the WPE in considerable detail. Not enough for some (including me) but far more than they normally produce.

Seems to me there are two choices. Either E/M screwed up completely, and they should publicy admit so and refuse to ever do an exit poll again, or the election was a fraud.

Here is the nub of the problem. E-M did not run a competent audit of the election. However: this was not what they were commissioned to do. They were commissioned to project the counted results. This they did, competently, and they did it the way they always do it, by incorporating vote-returns into their projections. The networks got what they wanted, and what they paid for.

If you want an audit of the election, you either have to design the poll differently (which would be prohibitively expensive) or you conduct proper audits of the ballots. The first will still be subject to polling bias, because all polls are. The second will probably work. I suggest the second.

In lieu of the second, for 2004, we can try to reverse-engineer the exit poll as a post hoc audit. Unfortunately, the exercise is seriously confounded by the fact that it was not what the poll was designed to do.

[Chi]

27. Question

In the often cited UK exit poll, that resulted in the 'reluctant torry' (sic?) conclusion....
Did the pre-election phone polls match the exit polls, or the vote results?

[Febble]

28. Pre-election polls

had Labour inching ahead. The exit polls were depressing, but not as depressing as the vote result.

Shy Tories (Conservatives) were cited as being the problem.

It is all burned into my memory. A few days before the election, Labour thought it was doing so well, it held a triumphal party in Sheffield, with balloons and all. We honestly thought the end of the black night of Thatcherism was finally coming to an end, although I cringed (as many of us did) at the party and crossed both fingers and toes. In the end it was another five miserable years before the Tories finally limped into oblivion.

I actually emigrated to Canada. We made the decision the day after the election. The thought of another five years of Tory rule was just too much.

[Chi]

31. What were the percentages, from your recollection?

Pre-election -
Exit polls -
Results -

[Chi]

32. Never mind...I found it.

(average of all the companies' polls conducted during the final week) was Conservative 31%, Labour 45% and Liberal Democrats 18%. Both close to the final result (32.7%:42.0%:18.8%), and within the standard 3% margin of error for all parties

[Febble]

33. That can't be right

The Tories won. Let me check the numbers.

In the mean time, have a look this if you are interested:

http://www.bbc.co.uk/election97/background/pastelec/92keyiss.htm

[Febble]

34. OK, I've found something

The Scotsman

In 1992, the opinion polls, which had normally predicted election results fairly accurately, were not just wrong but spectacularly so. The five main UK polls published on the morning of the general election predicted a Labour lead of 0.8 %, which would have ensured a hung parliament with Labour as the largest party. The BBC and ITV exit polls suggested a Conservative lead of 4%, which would have resulted in a hung parliament with the Conservatives as the largest party. In fact, the Conservatives were 7.5 % ahead and John Major was able to form a Conservative government with an overall majority of 21 seats.

That'll teach me not to rely on memory in future. The exit polls understated the Tory lead by 3.5% points, but they did, it turns out, predict a hung party with a Tory plurality. The pre-election polls had predicted a hung parliament with a Labour plurality. I don't know the MoEs. They should be smaller for the exit polls than for the pre-election polls, because the N will be larger. The pre-election polls were outside their MoEs but the between-poll difference meant that no-one (apart from the people at the Sheffield party) was banking on a Labour win, just hoping.

ICM for the Guardian were closest, but they tend to weight the Tories up; since then, I believe, more pollsters have done so, although at the last election there was some evidence of shy Labour I think. I'm not sure I would have wanted to admit to voting for Blair's party (though I did, and it got me banned from PI....)

I'll keep looking for more stuff.

[Chi]

35. So that means the pre-election polls actually exaggerated

the response bias, compared to the exit poll.

I would think, that a response bias would show up equally or greater in a phone poll,
compared to an exit poll.

Is that what you would expect as well?

[OnTheOtherHand]

36. totally different circumstances

What makes a voter, say, steer around a 20-something interviewer on the afternoon of Election Day wouldn't necessarily predispose the voter to hang up on a telephone interviewer before the election.

It's interesting to conjecture that if the exit polls and pre-election polls both disagreed with the official count, that would evince "consistent" response bias -- not fraud. Possibly true in the Palestinian election (I dunno, lots of data I haven't seen yet). Might even be partly true in the U.S. polls. But the exit polls appear to be a lot more volatile than the pre-election polls.

[Chi]

39. I would imagine whatever caused the bias would determine if

if it would affect both.
But whatever the cause, it appears to be the case in this instance,
the bias caused both polls to be inaccurate in the same way.
Unless you would like to propose a different cause of the response
bias in the phone poll caused the same effect...which makes little
sense to me.

[Febble]

40. I think, in the UK in 1992

the pre-election polls were genuinely closing, probably as the "undecideds" decided in favour of the status quo. It was a miserable time in the UK, the economy was in a mess, but no-one really trusted Labour (I have to say, with good reason, their previous term in government had been disastrous) to fix it. People had too much to lose, with mortgages larger than the plummeting value of their property. I think it was a question of the devil you know being better than the devil you don't know.

So the margin in the pre-election polls may have been correct in terms of decided voters but not in terms of the undecideds.

The exit polls nonetheless still underestimated the size of the Tory win. So even in a poll in which undecideds are, by definition, excluded, there was still a pro-Labour bias. I am still trying to find out the MoE of the exit poll. But shy, or embarassed, Tories were certainly cited as part of the problem, not just sampling error.

And it must have been the poll, not the count, because, in our picturesque medieval way, we still cast paper ballots, and count them by hand, under the scrutiny of bipartisan volunteers, the general public, and television cameras!

[Chi]

43. Do you think it was the same bias, or a different bias,

that caused the phone polls to be inaccurate in the same direction?

Would you consider that to be an expected pattern (phone polls showing
the same bias as exit polls), considering the magnitude of the bias?

[Febble]

47. I am not a political scientist

and I am just recalling what the papers said after the election. There seemed to be a consensus that both exit polls and pre-election polls had somehow under-sampled Tory voters, dubbed "shy Tories" although it has not been established whether the problem was people saying they would vote Labour (which at that data would have been seen as the "acceptable" answer, with an unpopular government) and actually voting Tory, or whether it was wrong allocation of undecideds, or whether it was actual non-response bias - more Tory voters refusing to answer. As I said, I did read that at one point people thought telephone polls were less subject to "shy Tory" bias than face-to-face, but I'm afraid I can't quote chapter and verse.

British politics is very different, as you will know. I wrote this thing for DKos if you are interested - not about polls, but about the way our government works, which might explain ways in which the UK experience can, and can't, be extrapolated to the States.

http://www.dailykos.com/storyonly/2005/3/20/6453/39287

It was done for last year's election.

[Chi]

51. So you have no idea whether exit poll response bias would show

up on phone polls?

OK, whatever...I'll leave it there.
Thanks for your time

[Febble]

52. My guess

and the guess in the press at the time, was that "shy Tories" contributed to both.

There was some research, and I'm still trying to find it for you. Maybe OTOH will get there first.

Cheers

Lizzie

[OnTheOtherHand]

54. I'm a very poor Anglophile

although I do like to hum to myself, "SPAM, spam, spam, spam, SPAM, spam, spam, spam...."

In 1992, if I understand rightly, it was argued that "shy Tories" contributed to errors in both pre-election and exit polls. Many pollsters, but not all, adopted an adjustment for this propensity in their pre-election polls. I am not aware that any adjustment was made in the exit polls. So I am not at all certain that the UK experience generally supports the idea that non-response bias in pre-election polls and in exit polls covaries. But I don't know.

Similarly, it is plausible that "shy Republicans" contributed to errors in both pre-election and exit polls in the 1992 U.S. election. But that account certainly wouldn't work for 2000. (Well, if anything can be certain given all the uncontrolled variables.)

[OnTheOtherHand]

44. as Febble says, trend confuses the issue

in the UK in 1992 as in the Palestinian election we've just seen. In general, I would expect more bias in an exit poll because it involves face-to-face contact (even though people generally fill out 'secret ballots'). But it is hard to compare pre-election polls to exit polls if there is a marked trend before the election. And the problem is rather intractable because in order to minimize bias in a pre-election poll, you want to be able to try to convert no-replies over several days -- so you can't just do a massive poll the day before the election, or at any rate, the results may not be ideal if you do.

It's actually not that hard for me to imagine circumstances in which a pre-election poll and an exit poll might be biased in the same direction for different reasons -- but as to whether any of them might apply to the UK in 1992, I won't claim to be entitled to an opinion. In the Palestinian election, the pre-election polls may have had a pro-incumbent bias somewhat distinct from a shy-Hamas bias. (Two distinct processes: first people have to decide whether to vote for the challenger; then they have to decide whether to admit to it.) I have no idea whether this is the least bit relevant to your question, much less to the OP!

[Chi]

46. If you have any historical basis for saying the two polls would not

demonstrate the same bias, at or near the magnitude of UK92, or US 04
I would be interested in it.
I don't consider a country with such a radically different culture, and with
first time elections, of any real relevance.

[OnTheOtherHand]

48. sorry, I'm not sure I understand the question

Actually, that's euphemistic: I'm sure I don't. Are you asking for historical basis for saying that the exit polls don't have to agree with the pre-election polls? I consider 2004 U.S. itself an example, since the result closely agreed with the pre-election polls -- this is the mainstream poli sci position, TIA notwithstanding. In 2000, the pre-election polls tended to slightly overstate Bush support; the exit polls had no clear trend but, judging from the WPE statistics, slightly favored Gore. In 1996, the pre-election polls tended to overstate Clinton support, by an average of some 4 points on the margin; the exit polls favored Clinton by a smaller margin, again based on WPE statistics. In 1992, the pre-election polls and exit polls both overstated Clinton support, the exit polls by considerably more (especially if one discounts the Gallup/CNN/USA Today pre-election result, which was an outlier). In 1988, the pre-election polls tended to overstate Bush support; the exit polls... umm, we're not entirely sure, although at least one of them clearly overstated Dukakis support.

Is the answer to your question somewhere in there?

I agree that Palestine isn't a very close analogy to anything (as I've said elsewhere).

[Chi]

50. OK...I'll try again.

I'm looking for a possible pattern.
When exit polls are way off the mark, and response bias is said to be the cause,
do the pre-election phone polls demonstrate the bias as well.

With the UK poll I asked Febble about, the phone polls not only reflect the bias,
but seem to magnify it.
As you know, this was not the case in the US 04 phone polls.

To my way of reasoning, taking the most erroneous exit polls from the US and/or
the UK, and seeing if the phone polls followed the same pattern, would be a way to test that theory.

I told you if you had examples of this theoretical pattern being wrong, I would be interested in it.

Better?

[OnTheOtherHand]

53. OK, so I guess my answer was

that arguably, every U.S. election I have data for evinces a disjuncture between pre-election poll "bias" and exit poll bias. In 2000, the pre-election and exit polls appear to err in opposite directions -- although it would be a stretch (not implausible, but at best uncertain) to posit non-response bias in the pre-election poll.

Not to be contentious, but we seem to be on the brink of endless circularity. If both the pre-election polls and the exit polls had shown Kerry winning by 10 points, even I would have thought that those results at least hinted at fraud. It would not have crossed my mind that on the contrary, the pre-election poll result tended to buttress the hypothesis of non-response bias in the exit polls. Or even if it had crossed my mind, I would have feared for my virtual life if I had actually ventured that argument here.

Nevertheless, I'm not saying that your reasoning is wrong. It is quite plausible that non-response bias in exit polls should covary with non-response bias in pre-election polls to some extent, although I don't think that could stand as a "test" of non-response bias. (If we posited that Bush voters were ashamed of having voted for Bush, that indeed probably should evoke non-response bias in both contexts -- but I have never believed that premise.) The specifics are probably analytically intractable given current knowledge. It seems clear in the 2004 case that non-response bias in exit polls varies from precinct to precinct, apparently as a function of precinct and interviewer characteristics; pre-election poll non-response bias may also vary as a function of interviewer characteristics. Then there is the problem that pre-election polls come before people can know for sure whether and how they will vote; and pre-election polls make disparate assumptions about likely voters; etc. And as I've said, exit polls just aren't like telephone polls.

My hunch is that pre-election and exit poll errors, averaged nationally per election and compared over time, should not be strongly correlated. But I'm not sure that this hunch is susceptible to a crucial test given available data. Even if it is, I'm not sure it has much bearing on the fraud debate. Really, I don't object to your question, but I think anyone would have to tell you that it probably can't be answered generally.

[applegrove]

4. Exit polls were off in places other than diebold. Exit polls have been

off in elections where bush has run. Including texas. I think exit polls should be dumped.

[snot]

5. Why dump exit polls when they appear more accurate than the vote tally? nt

[applegrove]

6. Tomato, tomay-to. We don't know.

[Febble]

10. What makes you think they are more accurate?

I wouldn't trust either, myself. I think you need transparent elections with proper audits.

[Kip Humphrey]

30. Seems like exit polls are least accurate when GW runs

Perhaps that's because Bush has a special appeal for habitual Liars who (on a larger scale) subsequently give misleading responses to exit pollsters (the dBr effect - Dishonest Bush Responders).

Or, maybe just maybe the exit polls' accuracy has remained relatively constant and

the ELECTIONS WERE STOLEN!


Of coarse, we'll never "prove" it. There is only the preponderance of election anomalies to go by:

Clint Curtis hired to deliver a vote switching demonstration program designed to run on DREs

A reluctant RNC employee claiming that on election night he manipulated the vote residing on state tabulators in 6 states using a vote switching algorithm

1 million + estimated minority voters disenfranchised

50,000+ documented reports of voting incidents and anomalies which favored GW by 98%

Reported incidents from around the country of electronic voting machines failing to register a vote for Kerry, registering votes for Bush or 3rd party candidates when Kerry (or Democratic Slate) was selected, including reports of voters actually watching their vote switch from Kerry to Bush.

Voting and vote processing software designed to facilitate unlimited vote switching without leaving a trace.

A Bush campaign state chairman and Secretary of State who had in his office on election day a fully functioning central tabulator capable of two-way data communications with the state's county voting systems

Anyone care to add a few more?

[YankeeFan]

11. As I Have a Part Time Job As A Poll Watcher...

You seem to have forgotten an Elemental Truth:

What you say in an Exit Poll can be 100% Different from what you actually Did vote. You can write in the Exit Poll that you voted straight Repunk when you actually voted straight Democrat. If you get put on some TV or radio talking head you can say that you voted for Snoopy when you voted for *Kerry.

In fact, during the 2004 Presidential Election I saw several people (sorry, but I'm not going to tell the amount) fill out the questionnaires who did not vote. Understand me? They did not vote at all, but according to the Exit Poll's they filled out they actually DID vote.

One more time people: Exit Polls do not accurately reflect who people actually voted FOR! In other words, if you live in a heavily Repunk Neighborhood (for whatever reason) you can vote straight Democrat and fill out an Exit Poll that said you voted straight Repunk! There is no law that says you have to fill out an Exit Poll the same way you voted.

Got it?

Oh, yeah. How did they fill out the Exit Polls and not vote? They were sons and daughters and nieces and nephews of elderly people who needed an arm to lean on while they walked into the Polls. There was one teen who grabbed at least half a dozen Exit Poll forms. I don't how he voted on the forms; as he didn't vote it didn't matter to me. As only about 10% of all the voters actually filled out the forms I'm not surprised at all that the Exit Polls don't match the Actual Voting numbers.

*told ya Dean should have been selected

[OnTheOtherHand]

13. interesting

I tend to downplay the role of misreporting in the exit poll because the evidence is fuzzier than for other sources of error. A couple of well-placed Teenagers for Kerry certainly could have 'blown up' a few precinct results, although I tend to doubt that misreporting was the main problem overall. Dunno.

[applegrove]

17. What about a plan for younger church member to drop of elders

at the door, and pick them up, at the door. Exit pollsters have to stand 100 feet from the door - no?

Don't know. But I have seen no reason why the exit polls would be wrong in districts where there was no diebold. And then right on in districts where there is diebold. Especially when exit polls at a national level were quire exating until 2000.

Like I said - we don't know. We don't know why the exit polls or diebold and sometimes mutually exclusively - either were off. Someday - perhaps we will.

[OnTheOtherHand]

19. not exactly

The restrictions on exit poll interviewers vary from jurisdiction to jurisdiction. In some places they were that far away. The farther away they are, the worse the results (on average).

I've seen no real evidence, even anecdotal, that anyone deliberately set out to manipulate the exit poll results. Not saying it didn't happen, just that there is much less evidence for that than for other explanations.

I don't think the exit polls were obviously more or less accurate where there was Diebold -- although E/M didn't give us a table by manufacturer, only by general voting type. But I've seen no evidence for a Diebold effect in any direction.

The exit polls at a national level arguably were more accurate in 2000 than they had been in '88, '92, or '96 -- despite the Florida FUBAR. We really don't know enough to generalize reliably about differences across elections.

As you say, there's a lot we don't know.

[applegrove]

20. I thought Miller said they were more accurate. You are right. I just

think of that whole afternoon on election day - when Kerry voters thought he had it. And what a long time that was. For overconfidence.

[southwood]

23. The thing is, pollsters

are supposed to know all this, and still arrive at a dependable result! If they don't, they're no good at their job.

[applegrove]

24. And they have not been. And they are understandably sheepish

about talking about the 2004 election. We didn't see many polls in the Nov 2005 elections.

[OnTheOtherHand]

37. you are almost making an important point

This is exactly what Mitofsky has been trying to tell people: that he knows the interviews are subject to error, and that is why the projection models combine interviews and vote counts in a manner that usually manages to avoid mistaken calls based on bias in the interview results.

It would be nice if pollsters had some way of "knowing all this" and actually managing to make the interviews accurate anyway. But that's not being good at their job, that's being omnipotent, which is not in the job description.

(Of course, this is a problem if the vote counts are wrong. But that is a separate matter from whether the exit poll researchers are doing their job right.)

[OnTheOtherHand]

12. overhauls after 2000

The facts about the exit polls are not as interesting as the speculations. For one thing, the 2000 exit poll results were mostly pretty accurate. But in Florida, the analysts underestimated the proportion of absentee voters, and made an unlucky choice of model race (the past election returns to which they compared the incoming results). Those mistakes made the exit poll data look more conclusive than they were, although still not enough to call the race. But the analysts got lulled into making the call (the first one, for Gore) based on insufficient vote data. Based on what we know at this point, Florida never should have been called either way on election night.

So for 2004, they tried to get better absentee estimates, changed the system so they wouldn't rely on a single past race, and changed their criteria for projecting a winner. I don't know that Edison/Mitofsky ever said exactly what their criteria would be (the networks have the ability to make their own projections independently), but we do know what the lead researchers recommended back in 2001. Simply put, in any state where the exit poll data weren't Pretty Darn Conclusive (probably 20-25 states in 2004), they would base their projections on vote count data alone.

That decision could certainly serve the purposes of vote-stealers, but there is nothing inherently suspicious about it. The changes were designed to prevent a repeat of what went wrong with the projections in Florida 2000, which isn't the same as what went wrong with the election.

[eomer]

26. But the vote count used in the projection was one of the main problems.

The projection at 2AM of Bush as the Forida winner (and therefore the electoral college winner) was in part due to a Diebold "glitch" in Volusia County. A memory card for Volusia County precinct 216 gave Gore minus 16,022 votes. This vote difference was one of the things that caused the errorneous call during the night. The "glitch" was later discovered and corrected but in the meantime had contributed to the feeling that Bush was the presumptive winner. The memory card with the negative Gore votes mysteriously couldn't be found during later investigations.

Another "glitch" in the errorneous call also involved VNS. The 2AM call of Florida to Bush was first made by Fox News (and then other networks quickly jumped on the wagon). The person at Fox News who made the call was Bush's cousin, John Ellis. And it's not like Ellis was some estranged cousin who had a family connection but not a real connection. He was on the phone to Austin during the night relaying data that came from VNS over to the Bush team.

I'm not sure why VNS and successor would have looked at these facts and concluded that the fix was to base their projections on vote count data alone. It was vote count data that caused the 2AM error and contributed to the wrong person occupying the WH.

Maybe what they should really do is wait a bit longer on election night so Diebold can finish their "adjustments" first. :hide:

[OnTheOtherHand]

38. different blown call, different problem

IIRC, VNS didn't actually make the second wrong call, for Bush, but all the networks did. (I think everyone made the first wrong call, for Gore.) You can read the Konner et al. evaluation report and get the network take on the second blown call as well as the first, if you like.
http://archives.cnn.com/2001/ALLPOLITICS/stories/02/02/cnn.report/cnn.pdf

To deal with the second blown call, Mitofsky and Lenski proposed not to make any calls with a margin of less than 1%, even if all precincts were reporting. That would've handled Volusia County. (Of course, it wouldn't have guaranteed a correct count!) In 2004, Ohio was not called on Election Night, period (at least not by E/M -- I can't swear that no one else called it).

[eomer]

41. Thanks for the link, I hadn't seen this report.

I checked and the report says, as you do, that VNS and AP didn't make the second wrong call. At CNN the second wrong call was made by Mitofsky and Lenski, who comprised the CNN/CBS Decision Team.

But Mitofsky and Lenski mostly lay the blame back on VNS, other than to say that they should not have relied on VNS so heavily:

Mitofsky and Lenski blame their mistaken calls primarily on the data provided by
VNS. As noted earlier, they believe that they would have been derelict in their duties had
they not made the two calls on Florida, based on the data supplied to them.

Their complaints include the same ones that have been identified by VNS —
particularly the mistakes in Duval and Volusia counties and the underestimates in the
outstanding votes in Palm Beach and other counties. The Decision Team first alerted
VNS to the Duval error, and the team complains that VNS did not promptly alert it to the
Volusia error once it was found and processed into the VNS system.

The team conceded that it made a significant error in relying on VNS data nearly
alone. Particularly with regard to the Bush call, following the mistaken Gore call earlier
in the evening, “in retrospect” the team should have consulted AP and secretary of state
vote counts, Mitofsky said.

“We could have checked every county vote ourselves among the three sources to
find the discrepancies, like we did after 3:00 a.m.,” Mitofsky and Lenski said in a written
report to the two networks. “Again, that is not really our job, but without massive
improvements in the quality control of vote data input at VNS it may have to be part of
our quality control in the future.”

The report, on the other hand, is somewhat more critical of Mitofsky/Lenski, not buying all of what they say:

Nothing in our committee’s review leads us to dispute the Decision Team’s
criticisms of VNS, but we believe that the Decision Team’s failure to consult other
available sources of data was a major mistake.

We largely discount the Decision Team’s insistence that time pressures were not a
problem in calling Florida prematurely. Time pressures are the whole reason for the use
of exit polls and other devices in calling the winners of states before the actual
computation of complete returns is known. If no one were in a hurry or competing to be
out front, everyone would wait until results were known with certainty.

It is a bit difficult to believe Mitofsky/Lenski's claim that they weren't influenced by time pressures given this piece of the timeline:

2:15 a.m.: Fox News calls Florida for Bush, giving Bush the presidency.

2:16 a.m: NBC calls Florida and the presidency for Bush.

2:17 a.m.: The {CNN/CBS} Decision Team completes its review of the Florida data and the
apparent Bush lead of some 51,000. However, it bases its calculations on information
from VNS and is not checking on vote counts of the AP, which is showing Bush with a
smaller lead and losing ground, or with the vote counts from the Florida secretary of
state. AP has just caught the Volusia County error and has restored votes to Gore, so that
AP now shows a Bush statewide lead of only about 30,000.

The Decision Team, unaware of this, recommends to CNN and CBS that they call
Florida for Bush, and they do so.

Regarding the fixes they've put in place, the problem is that the problem is not a detail here or there -- it is systemic. What CNN and the other networks need to do is to start gathering their own data, each one independently, so that we don't have to rely on one single source (and such a secretive black box of one at that) for such a crucial piece of information. Multiple sources is a basic tenet of journalism -- why do they abandon that tenet when reporting out the most important story of all?

The need for more diverse sources of information for calling elections is critically important not just because of the extreme level of interest in the result but even more so because of the "presumptive winner" problem. From the report:

The unanimous network calls for Bush created a premature impression that Bush
was the winner in Florida. That characterization carried through the post-election
challenge. Gore was perceived as the challenger and labeled a “sore loser” for trying to
steal the election.

The unreliable reporting of this vital information can actually (and actually did) result in changing the winner of the most powerful office in the history of the earth. That is truly scary.


_{Edit: typos}

[OnTheOtherHand]

42. fair points

I defend these folks against unsubstantiated accusations of complicity in election theft, but it's hard to say that they did a great job on Election Night 2000. And many of us would prefer to see more diverse sources of information (of course!).

I don't think one can single out the decision teams for the bad reporting after election day. On election day/night, there were two terribly blown calls (or sets of calls), each of which lasted for about two hours (or less). In a parallel universe close to this one, but without the butterfly ballot, the second call was never made, Gore won Florida by a small margin, and Republicans are angrily complaining about the first call (as indeed they did). Or maybe in that parallel universe, the Bushies found another way to win. At any rate, bad calls notwithstanding, the journalists who mouthed Bush campaign talking points in the weeks after the election can't very well blame it on events between 2 and 4 am of election night, in my view.

[eomer]

45. How far away is that parallel universe you mention?

That's where I want to live.

You say it's close to this one but I'm fearful it is light years away.

[OnTheOtherHand]

49. somewhere over the rainbow?

I really have no way of knowing. I don't think the butterfly ballot itself was a deliberate dirty trick, regardless of Theresa LePore's partisan affiliation. Regardless of whether I'm right about that, I'm really not positioned to evaluate what would have happened if Gore had ended the night several thousand votes ahead instead of several thousand votes behind.

That particular parallel universe might as well be light years away, at this point. But I've seen nothing to convince me that Republican control of the voting system will prevent Democrats from winning national races. I think the Dems have other problems. That said, I'm not dismissing your fears.

[autorank]

55. Careful, you're being too accurate! ;) n/t

[truckin]

15. Has any voting group like Verified Voting discussed hiring

someone like Zogby to perform exit polls in key races in 2006? This could be funded by concerned citizens of all parties and the methodology of the polls and details of the results could be made public unlike the current version of polls. If this could somehow happen it would be a great tool to determine if the 2006 elections are legit or not and may be a way to push for fair, legitimate audits after the election and get the press involved. It would be important for a non-partisan grassroot effort to start this process to give these exit polls credibility. This is probably a pipe dream but it could be a valuable tool to push for election reform if it were to happen.

[applegrove]

16. I think the MSM were so burned by the Kerry early results in the exit

polls, that ran all day, and gave him the win,... that you likely will see fewer polls. Nobody wants to be involved in something bad.

[applegrove]

18. What about exit poll monitors. Why not?

[liam_laddie]

21. some older exit-poll history

I'm not touting this book as gospel truth, but there are nonetheless
interesting observations about exit-polls in the book "Votescam."
The authors' view is that polls are mainly an arm of TV networks'
need to both draw audience and perhaps "influence" the outcome.

Chapter 15 of the book addresses this topic. I learned that Warren
Mitofsky replaced Lou Harris (considered father of exit-polling here)
in 1966 as "chief of CBS' exit-polling division." In 1989, the major
networks admitted they had sometime earlier formed a consortium
to conduct exit polls, after they'd competed for a few years to be
"first" with the predictions. This venture was named "Voter Research
and Surveys" (VRS), headed by Mitofsky. He's been in the polling
mix for nearly forty years! VRS morphed into Edison-Mitofsky, etc.

There are interesting examples of suspicious activity, denials, and
unexplained concurrence of poll results with reported ballot totals,
including "surprisingly" accurate poll results posted far too early to be
legitimate, such as certain results posted "in error" a day early!
Jes' sayin' - this book is based on their over twenty years' (1970-1992)
of direct experience with the "establishment powers." I'd start with
Chapter 11 to end, pp 203-360. Then read the early chapters if you're
intrigued. I believe the book's content is posted online...but it would be
noble to support the publisher at www.votescam.com. with a purchase.
Disclaimer - I have no connection to the bookseller.

[organik]

29. The real main basis for questioning the results is the grinning assface

that got "re-elected". He's a joke of a man, and a bigger joke of a president. He makes me cringe just to see his vacant mumblings on the TV.

I'm done with this exit poll nonsense, although I do believe the fact that Vegas odds-makers were giving the election to Kerry 2 to 1 says something. You don't screw up that bad when there's that much money involved.

I think there's not a chance in hell Bush actually won. I'm pretty sure there are thousands, if not millions who agree with me.

How do we make sure it doesn't happen again? Media won't report it. Democrats don't seem to care.

Maybe the only answer is to hit the streets. Demand hand counted paper ballots, with random audits, or at least a paper trail with audits, and if we don't get it for 2006, stage a massive protest during the election (while still voting of course - that's the worst consequence of questioning elections - discouraging voting).

Who's in?

[autorank]

56. Me and it's great to see your reality-based username and avatar!

[kster]

57. So long as there is SILENCE from honest Repugs and Dem's

We can let all them bastards have fun with there corrupt vote counting machines I'll vote, But as long as there is Silence on the election theft machine (from both sides) nothing makes sense. I like a good contest WIN OR LOSE but it has got to be a true contset.