2016 Postmortem
Related: About this forumStatistical Primer- The Infamous M O E
Just because a candidate's lead is inside the Margin Of Error doesn't mean he or she isn't really ahead, ergo:
... does not make it a statistical dead heat - if you think it does, you have been listening to too many talking heads on TV.
...
First of all the the MoE works both ways. With a 4.0 MoE the results could be Sanders +1 (least likely), A tie, Clinton +1, Clinton +2, Clinton +3 (most likely), Clinton +4, Clinton +5, Clinton +6, Clinton +7 (least likely). Note that with most of the possible outcomes, including the most likely outcome (Clinton +3), Clinton wins.
In addition, as you move one step at a time away from the most likely outcome (Clinton +3) towards the edges (Sanders +1 and Clinton +7) the chances of those results happening get less and less. The least likely outcomes by far are Sanders +1 and Clinton +7.
If you want to get technical, there is a 95% chance with this poll that the result will be within the MoE of 4.0, and a 5% chance that it will be off more than 4.0%. Since the graph of this poll's possible results is the area under a bell shaped curve, there is actually just a 2.5% chance that Sanders could win by more than +1 and 2.5% that Clinton could win by more than +7, so such results are highly unlikely.
-Cajun Blazer
NurseJackie
(42,862 posts)DemocratSinceBirth
(99,711 posts)I did know the MOE covers a theoretical range of outcomes but it certainly doesn't mean the candidates are tied.
Last night I aggregated the last five polls and HRC had a lead of 3.3% with a MOE of 1.73%.
Cajun Blazer can do a theoretical range of outcomes with it.
Jarqui
(10,130 posts)If the turnout is heavy, usually due to youth and enthusiasm, Bernie probably wins.
If the turnout is light, Hillary will not only win but might hit that +7 you're talking about.
I think she tried to poll enthusiasm. If one looks at average turnout to the campaign events, actions speak louder than words or poll numbers on enthusiasm. Bernie has that - particularly with the youth.
alcibiades_mystery
(36,437 posts)Iowa will show us more than just who wins Iowa. It will show us the accuracy of the Sanders' camp claims that it can win and actually institute policy due to a ground swell of new voters, young voters, independent voters, non-traditional voters, and the like. If that doesn't materialize in Iowa, then that claim becomes far more dubious, not only for the general election, but for all of the claims about policy change stemming from a transformed, engaged political landscape.
It's a very interesting challenge / promise / problem / opportunity. A big win for Bernie that includes lots of new, enthusiastic participants essentially confirms Sanders' narrative and lends support to his arguments about how he can win the general and govern effectively. A loss resulting from a failure to mobilize a new, enthusiastic participant pool is not only a loss of one state, but a refutation of his entire argument.
Tom Rinaldo
(22,913 posts)I am not knocking the latest poll - just pointing that out. When polls go wrong for any reason the MOE they initially assumed becomes moot.
DemocratSinceBirth
(99,711 posts)Aggregate polling in 00 showed Bush with a slight popular vote lead.
But my point and the point Cajun Blazer made so well is that the MOE represents a theoretical range of outcomes, from the most likely to the least likely, and it certainly doesn't mean that a candidate whose lead is within the Margin Of Error isn't really ahead.
Yes the polling can be wrong, but it isn't the pollster's opinion (at least if the pollster is not a hack it isn't) but a scientifically designed method to estimate a parameter within a desired range of accuracy.
DemocratSinceBirth
(99,711 posts)Others are better at explaining the minutiae of polling than I am but the turnout her poll is based on is a function of the percentage of respondents who said they would actually be caucusing.
She didn't guess.
Paulie
(8,462 posts)With average turnout the number is X. A few percent more turnout and its Y.
One more day and we'll know for sure.
Tom Rinaldo
(22,913 posts)I agree that the poll wasn't representing an opinion. That was poor shorthand. A more accurate comment might be "based on that polls design and the weighing of variables. Plus as pollsters always admit, polls are a snapshot in time and things can move fast at the end.
I only note this because there is something about a MOE that is seductive, making this all seem like more of fool proof predictive science than it really is. Exit polls, on the other hand, are a slightly different breed. They are a snapshot of the finish line, within a MOE, if the sampling is representative of the electorate.
JackRiddler
(24,979 posts)The MOE is a fig leaf and this discussion is pollster fantasy, or self-promotion. What matters is methodology, sample size, weighting, definition of "likely," etc., all of which varies greatly between polls (all of which claim similar MOEs as a matter of promotion). And of course what's actually being measured. This isn't a general election or mass-scale consumer behavior, where prediction can work well. Primary polls are already dodgy, but a caucus? It's all about motivated minorities, and the motivation varies right up until the day. Whichever pollster gets the exact numbers of the actual results in a caucus is winning a lottery, but will claim to have divine powers.
DemocratSinceBirth
(99,711 posts)I suspect if you polled one thousand Statistics professors all one thousand of them would say Cajun Blazer's description of the MOE is the technically correct one.
I would literally wager on it because that is what I and every other student who ever took a Statistics course was taught.
Now, a discussion of the vagaries of polling a caucus is a whole different ball of wax.
NCTraveler
(30,481 posts)DemocratSinceBirth
(99,711 posts)Last night I aggregated the last five polls and HRC had a lead of 3.3% with a MOE of 1.73%.
Cajun Blazer can do a theoretical range of outcomes with it.
morningfog
(18,115 posts)However, in addition to the normal unknowns (turnout, new voters, weather, etc) it is a caucus and their is a the is candidate who will likely not meet the 15% threshold in most precincts. Even with the reliability of the poll, Bernie is in a GREAT spot.
From down 30 to down 3 with developments between this poll and the caucus.
Bjorn Against
(12,041 posts)The margin of error does not just mean that one candidates numbers might be off by 4%, it means that 4% of a candidate's totals could belong to a different candidate. this means that Bernie could actually be +5 and still be within the poll's margin of error. Now this would be an unlikely scenario if the poll was conducted well, but +1 is not an unlikely scenario as Cajun Blazer suggests. All a +1 margin would mean is that the pollster over represented Clinton supporters by 2% and that 2% was actually for Bernie which would put him slightly in the lead.
DemocratSinceBirth
(99,711 posts)The margin of error represents a theoretical range of outcomes from the most likely to the least likely, all the obscurantism in the world notwithstanding, and the further you get from the reported values the less likely it is to occur, ergo:
However, the estimate a poll gets is in fact the likeliest true value, and the likelihood decreases as we move toward the extreme ends of sampling error.
-Gary Langer
http://blogs.abcnews.com/thenumbers/2007/12/moe-and-mojo.html
Langer has won two Emmy awards and received nine Emmy nominations including the first and only to cite public opinion polls and was honored with the 2010 Policy Impact Award of the American Association for Public Opinion Research for a series of surveys in Afghanistan and Iraq, described in AAPORs citation as a stellar example of high-impact public opinion polling at its finest. Hes a two-time winner of the University of Iowa-Gallup Award for Excellent Journalism Using Polls, produced a pair of ABC News polls recognized by the Excellence in Media Coverage of Polls Award from the National Council on Public Polls and shared a DuPont-Columbia Award for ABCs 9/11 coverage. Most recently, Langer and his colleagues shared a David R. Ogilvy Award for Excellence in Advertising Research with ESPN and its partner research firms for their work on fan interest in the College Football Playoffs.
Langer created ABCs industry-leading poll standards and vetting procedures and has promoted disclosure initiatives through various professional organizations. Hes authored or co-authored nearly 30 scholarly papers and given scores of invited presentations on the meaning and measurement of public attitudes.
http://www.langerresearch.com/our-depth/gary-langer/
Bjorn Against
(12,041 posts)If a poll overrepresents one candidate's numbers those numbers do not generally just disappear, they often go to another candidate which means the actual range of variance is double the margin of error.
Any statistics professor would tell you that Cajun Blazer is wrong in claiming that +1 is the best Bernie could do within the margin of error, +5 would still be in the margin of error.
DemocratSinceBirth
(99,711 posts)It is my understanding that the MOE applies equally to all values or in this instance candidates.
HRC has 45
SBS has 42
With a MOE of 4 HRC can be as high as 49 and as low as 41
With a MOE of 4 SBS can be as low as 38 and as high as 46
That being said, the reported values are most likely the correct one and the further removed from the reported values the less likely they are to occur. You can reduce the MOE by increasing the sample but landline/cellphone polls are really expensive.
Maybe, all three of us are saying the same thing differently...
Where I suggest the obscurantism comes in is when one calls a lead within the margin of error a tie.
Bjorn Against
(12,041 posts)The Selzer poll shows a real dead heat especially when you consider that 10% are still undecided. Those 10% could break either way which means the actual results could be outside the margin of error of this poll and the poll would still not necessarily be wrong.
DemocratSinceBirth
(99,711 posts)It wouldn't have been a top line number though.
If Clinton loses Selzer loses in the eyes of the media who have made her the oracle of polling, regardless of the fine points of her craft...
There is always a "hot" pollster... Zogby was once a "hot" pollster when he nailed the 96 general almost to the decimal point and 00 when his poll showed Gore winning the pop vote. It wasn't that a Martian couldn't have picked Clinton in 96 it was most polls showed Clinton winning by a larger margin. He won by 8.3%. Most polls showed him with a double digit lead.
NurseJackie
(42,862 posts)Here's a famous MOE:
mcar
(42,373 posts)I learned something new today.
DemocratSinceBirth
(99,711 posts)Depaysement
(1,835 posts)She deeply discounted the percentage of new caucus participants compared to 2008. Those participants break heavily for Sanders. To Selzer's credit, she admits this. It's a huge variable, along with general turnout ratios, that you are completely ignoring
Why do you assume the percentages within the MOE on a bell curve? Plus, there are a whole host of other statistical anomalies that could occur: coverage error, non-response error, weighting error and so on.
Kilgore
(1,733 posts)The average tv viewer could not either comprehend it or take the time to try. The "dead heat" short hand is media's way of handling it.
Gothmog
(145,558 posts)Jim Lane
(11,175 posts)If there's a 4% MOE in each candidate's share of the vote, what's the MOE for the gap between them? I'm pretty sure it's greater than 4%.
For example, let's say the poll result is Clinton 45%, Sanders 42%. It's correct to say that there's no more than a 5% chance that Clinton is outside the band 41% to 49%.
But she doesn't have to be outside that band for Sanders to win. If each number is off by just 1.5% (in the direction favorable to Sanders), then the two candidates actually do tie even with neither number being off by anywhere close to 4%. Any difference slightly more favorable to Sanders means that he wins.
The result is that, if you want to continue to use MOE to refer to the 95% confidence interval, then the MOE for the gap between the two numbers is greater than 4%. For a Sanders win, it's not necessary that actual results exhibit either of the unlikely events (i.e., an individual result that's outside the eight-point band of the MOE).
On the other hand, you can't just double the MOE of an individual number to get the MOE for the gap. That would mean that the poll result for the gap was Clinton by 3% plus or minus 8%. But "minus 8%" would mean Sanders winning by 5% instead of losing by 3%. That would happen only if both candidate totals fell outside the 95% confidence interval. If the chance that Clinton will be outside the range of 41 to 49 is 5%, then the chance that she's outside her range AND that Sanders is outside his range (of 38 to 46) is obviously less than 5%. Therefore, you can narrow the stated MOE (make it less than 8%) and still be within the 95% confidence interval.
All this applies to calculating the MOE for the difference between two numbers in two different polls. For example, if two polls with the same MOE of 4% give Sanders 42% in Iowa and 53% in New Hampshire, what's the MOE for the margin by which Sanders in New Hampshire is outperforming Sanders in Iowa? I think it's more than 4% but less than 8%. If instead you ask the typically more interesting question of comparing two candidates' results in the same poll, the calculation would probably be different because there's a covariance between the two numbers.
DemocratSinceBirth
(99,711 posts)Jim Lane
(11,175 posts)I agree with you that the MOE of 4% applies to the Clinton number and to the Sanders number. For the reasons I stated, though, I don't think it applies to the (Clinton minus Sanders) number.
Recursion
(56,582 posts)That is, if the polling firm's model of voter turnout is correct, then it is 95% likely that the outcome will be within the MoE of the poll. There's actually two elements of uncertainty here, and only one can be quantified.