About online primary polling - is it valid?

[kristopher]

The emergence and evolution of online polling has left in its wake a sense of disdain by many informed people for polls emerging from that platform.
While that feeling is a legitimate response to the traditional internet polling conducted alongside some local news clickbait piece of sensationalistic 'journalism', there has emerged serious methods that deserve serious consideration as we move forward.
We've seen a fair amount of this polling already this primary season and we will undoubtedly be seeing a tremendous amount in the months ahead. This information is provided to help DUers place the results of that polling in its context. You might want to bookmark this for dealing with the inevitable disagreements about validity.

From Google:

Comparing Google Consumer Surveys to Existing Probability and Non-Probability Based Internet Surveys
Paul McDonald, Matt Mohebbi, Brett Slatkin Google Inc.
Abstract
This study compares the responses of a probability based Internet panel, a non-probability based Internet panel and Google Consumer Surveys against several media consumption and health benchmarks. The Consumer Surveys results were found to be more accurate than both the probability and non-probability based Internet panels in three separate measures: average absolute error (distance from the benchmark), largest absolute error, and percent of responses within 3.5 percentage points of the benchmark. These results suggest that despite differences in survey methodology, Consumer Surveys can be used in place of more traditional Internet based panels without sacrificing accuracy.
http://www.google.com/insights/consumersurveys/static/consumer_surveys_whitepaper.pdf

From Pew Research Center:

A Comparison of Results from Surveys by the Pew Research Center and Google Consumer Surveys
NOVEMBER 7, 2012
<snip>
Pew Research and Google Comparisons

From May to October, 2012, the Pew Research Center compared results for more than 40 questions asked in dual frame telephone surveys to those obtained using Google Consumer Surveys. Questions across a variety of subject areas were tested, including: demographic characteristics, technology use, political attitudes and behavior, domestic and foreign policy and civic engagement. Across these various types of questions, the median difference between 43 results obtained from Pew Research surveys and using Google Consumer Surveys was 3 percentage points. The mean difference was 6 points, which was a result of several sizeable differences that ranged from 10-21 points and served to increase the mean difference.

Differences between the Pew Research surveys and Google results occur for a number of reasons. Given that Google Consumer Surveys does not use a true probability sampling method, and its sampling frame is not of the general public, differences in the composition of the sample are potentially of greatest concern. A comparison of several demographic questions asked by Pew Research indicates that the Google Consumer Surveys sample appears to conform closely to the demographic composition of the overall internet population. Communication device ownership and internet use also aligns well for most, though not all, questions. In addition, there is little evidence so far that the Google Consumer Surveys sample is biased toward heavy internet users.

Some of the differences between results obtained from the two methodologies can be attributed to variations in how the questions were structured and administered. During the evaluation period, we typically tried to match the question wording and format. However, some exceptions had to be made since many of the questions were part of longstanding Pew Research trends and had to be modified to fit within the Google Consumer Surveys limits and the different mode of administration (online self-administered vs. interview-administered by telephone).

The context in which questions are asked could also explain some of the differences; questions in Pew Research surveys are asked as part of a larger survey in which earlier questions may influence those asked later in the survey. By contrast, only one or two questions are administered at a time to the same respondents in the Google Consumer Surveys method.

The Google Consumer Surveys method is a work in progress and the Pew Research Center’s evaluation began shortly after its inception and continued for six months. The testing is ongoing, and we will continue to evaluate their methodology.
<snip>
http://www.people-press.org/2012/11/07/a-comparison-of-results-from-surveys-by-the-pew-research-center-and-google-consumer-surveys/

List of some recent Google research on their survey methods and analysis.
http://research.google.com/pubs/MarioCallegaro.html

[kristopher]

1. Kick for post debate reference

[dsc]

2. those aren't the same kind of polls

that are being discussed with the debates.

This report examines Google Consumer Surveys, a new tool developed by Google that interviews a stratified sample of internet users from a diverse group of about 80 publisher sites who allow Google to ask one or two questions of selected visitors as they seek to view content on the site. The sample is stratified on age, gender and location; these demographic characteristics are inferred based on the types of websites the users visit, as recorded in their DoubleClick advertising cookie and their computer’s internet address, and then is weighted by these same characteristics to parameters for all internet users from the Current Population Survey. It is neither an “opt in” survey nor a recruited panel but does not constitute a probability sample of all internet users.

That is not what happens with the internet 'polls' after a debate. Those are opt in, not randomized, nor are they stratified and finally they aren't using all internet users as their population. Polls can be properly done over the internet. There is nothing magical about phones. But they have to be run like polls and their population might be different than the population as a whole.

[kristopher]

3. You are only partially correct

The distinction you note between types of internet polling is accurate (and covered in the OP material) but the polling that is hitting the news includes the Google Survey.

http://www.democraticunderground.com/1251814204
And reposted here:
http://www.democraticunderground.com/1251815267