Understanding What Went Wrong with 2016 Polls Will Take Time

Trying to grapple with the failure of polling to predict the Electoral College victory of President-elect Donald Trump, I wrote a short piece for the Castleton student newspaper, The Spartan. In that piece, I argue that we need examine the systematic omission of a segment of the population who are not unreachable, but rather who refuse to participate in polling as respondents. As response rates declined over the past two decades, it was not only a result of those that we could not reach, but we also saw a rise in refusals—those who we could reach but who refused to participate in any polls. Figure 1 shows that this segment of the public is actually greater than the proportion that we cannot reach at all. While we know very little about the unreachable segment of the population, we have some, but limited, information about the refusals. We need to employ that metadata to understand as much as we can about this subpopulation.

The American Association for Public Opinion Research has put together a taskforce to examine the polling from 2016; this is something to watch closely. What I believe that I know now is this:

  1. Any explanation that employs one factor to explain the polling errors is wrong. There are many factors in play.
  2. Most of the early attempts to explain the errors are also wrong; we need a thoughtful and deep examination of the methodology, which will take time and peer discussion.

More to come.

pewresponserates
Figure 1. Pew’s response rates, 1997 – 2012.