NYT: The Secret Lives of Pollsters
By MARK BLUMENTHAL
Published: February 7, 2008
....As the remaining states prepare to cast their decisive votes in this campaign, how are voters supposed to make sense of all the conflicting data? Unfortunately, when the differences are as severe as they were in California, we can’t. Despite 22 years of experience as a Democratic pollster, I can only speculate about what might be going wrong. Why? Because so many pollsters fail to disclose basic facts about their methods.
Very few, for instance, describe how they determine likely voters. Did they select voters based on their self-reported history of voting, their knowledge of voting procedures, their professed intent to vote or interest in the campaign? Did they use actual voting history gleaned from official lists of registered voters? Fewer still report the percentage of eligible adults that their samples of likely voters are supposed to represent. This is a crucial statistic, given the relatively low percentage of eligible adults who participate in party primaries. (In California, for example, turnout surged in 2008 but still amounted to about 30 percent of the state’s eligible adults.)
Incredibly, some organizations routinely report results without any indication of whether a live interviewer or a recorded voice asked the questions.
In California, two pollsters — including the one showing a huge, erroneous lead by Mr. Obama — failed to disclose the demographic characteristics of their samples. Only a handful of pollsters that conduct statewide surveys routinely provide this data, like the percentage of the sample that is male, or African-American, or under 30 or college-educated....The ethical codes of organizations like the American Association for Public Opinion Research mandate the disclosure of these sorts of methodological details upon request. But when I asked all of the public pollsters surveying the Iowa caucuses last fall for their data, five pollsters refused to respond. Others provided answers that were incomplete or severely delayed.
Greater transparency might make some polls more accurate, by making pollsters less willing to cut corners that might otherwise go unnoticed. But the real benefit would be to journalists, political professionals and voters, who would be better equipped to analyze all the polls that confront us. If pollsters disclosed more about how their polls were conducted, we would be in a better position to know which polls are likely to be right, and which ones can be safely ignored.
(Mark Blumenthal is the editor and publisher of Pollster.com and a polling analyst for National Journal.)
http://www.nytimes.com/2008/02/07/opinion/07blumenthal.html
Printer-friendly format
Email this thread to a friend
Bookmark this thread
(1000+ posts)
Send PM |
Profile |
Ignore
