Deception in Polling

Recently, articles were trending with the finding that a particular candidate’s group of supporters would bomb a fictional city [for an example article: Click here].

Survey researchers tend to immediately seek out the methodological details of the poll when we see headlines like these. In this case, very few methodological details about the poll are available on the company’s website [PPP Poll Release] and certainly not enough detail to meet the basic AAPOR Transparency Initiative’s requirements (of which the company is not a member).

Without much methodological information, I do not claim to dispute the findings of this particular poll, but I would like to spend a moment considering the potential meaning and impact of this finding. Responding anything other than the “not sure” option to this question isn’t necessarily an indication that a respondent believes the city from Aladdin is an actual place, but rather a likely indication of general sentiment concerning military action in the Middle East.

Respondents don’t expect polls to intentionally deceive them. Those responding trust the researcher to ask valid and fair questions. Unfortunately headlines and poll questions like these erode the public’s trust in the field of survey research and further damages our ability to gather public opinion, behavioral, and other social science data for purposes far more important than sensational headlines.

It is one thing to carefully and respectfully design methodological question-wording experiments to advance the science of survey research to help us to write questions that are better at measuring public opinion, and quite another to use deception to collect data with the intention of being click-bait. In all likelihood, it is the misuse of methodological research that informed the creators of this poll that they would be likely to find a sensational, headline-making result by asking this question.

As part of our commitment to our respondents and our profession, Castleton Polling Institute will not undertake work that aims to intentionally deceive our respondents for the purposes attention-grabbing headlines. Our intentions when we ask to you to participate in research are not to trick you or ask unfair questions, but rather to be able to report on the public’s true opinion.

Christmas: Secular or Religious Holiday?

In 2013, a CNN/ORC International poll found that 94 percent of respondents in a general population polls said that they celebrate Christmas. Another poll by CNN and ORC International (December 2014) a year later asked respondents, “Do you think about Christmas more as a religious holiday, or more as a family holiday?” Half of all respondents said that they see Christmas as a family holiday, while 48 percent said they saw Christmas as a religious holiday.

The nation is divided nearly evenly on this question. Conservatives see Christmas more as a religious holiday (64 percent) whereas liberals see it more as a family holiday (69 percent) than as a religious holiday (28 percent). Just as we find ideology differs across the four Census Regions, views of Christmas differ as well. Respondents from the South were most likely to see Christmas as a religious holiday (58 percent), while those from the Northeast were the least likely (35 percent). [Cable News Network. CNN/ORC International Poll, Dec, 2014. USORC.122414.R46. ORC International. Storrs, CT:Roper Center for Public Opinion Research, iPOLL, accessed Dec-18-2015.]

Is there a war on Christmas in America? If so, the battle is over the extent to which the holiday is seen in secular terms instead of religious terms. This battle often plays out in the greetings used around the holiday.

A 2013 poll conducted by SSRS for the Public Religion Research Institute asked, “Do you think stores and businesses should greet their customers with ‘Happy Holidays’ or ‘Seasons Greetings’ instead of ‘Merry Christmas’ out of respect for people of different faiths, or not?” Again, there was almost an even split in responses, with 49 percent supporting the use of a secular greeting and 43 percent opposing the secular greeting, favoring the “Merry Christmas” greeting. A strong majority of Republicans (61 percent) opposed the use of the secular greeting, while majorities of Democrats (58 percent) and Independents (56 percent) favored a secular greeting. The secular greeting was most highly favored in the Northeast (57 percent); even a slim majority in the South (51 percent) favored the secular greeting, contrary to regional stereotypes. The region most opposed to the secular greetings was the North Central where 48 percent favor the use of “Merry Christmas” to any secular alternative.

With that in mind, I hope all of our readers have a Merry Christmas, Happy Holidays, or just a peaceful break during this holiday season!

Figure 1: Responses to “Do you think stores and businesses should greet their customers with Happy Holidays or Seasons Greetings instead of Merry Christmas out of respect for people of different faiths, or not?” by party identification [PRRI, Religion News Service. PRRI/RNS Religion News Survey, Dec, 2013 [survey question]. USPRRI.121713.R05. Public Religion Research Institute. Storrs, CT:Roper Center for Public Opinion Research, iPOLL, accessed Dec-18-2015.]

War on Xmas

The survey results reported here were obtained from searches of the iPOLL Databank and other resources provided by the Roper Center for Public Opinion Research.

A look back at Vermonters’ views on gun control

Friday, December 14, 2015, marks the third anniversary of the horrible shooting at Sandy Hook Elementary School, in Newtown, Connecticut. After the shooting, Vice President Joe Biden led a taskforce to examine what lawmakers in the nation’s capital could do to avert tragedies such as the Sandy Hook shooting, and the VP made recommendations for legislation. In February 2013, the Castleton Polling Institute asked Vermonters’ opinions on some of those recommendations, and this is what we learned.

In February 2013, majorities of adults in Vermont supported requiring background checks for all guns sales (75 percent), banning the sale of high capacity ammunition magazines or clips (66 percent), banning further sales of assault weapons (61 percent), and making it illegal to own an assault weapon (54 percent). Of course, like any poll, the results should be seen as a snapshot of public opinion in that time, reflecting the mood following one of the most horrific mass shooting the nation had seen.

What was interesting about these data were Vermonters’ reporting that the Sandy Hook tragedy had not caused them to change their views on gun control; only 19 percent of respondents said that their views had changed sin the Sandy Hook shooting. Even if social desirability bias is a factor—pushing some respondents to say that the event only two months earlier had not changed their views in order to sound more firm in their position—it is hard to imagine that social desirability bias could account for the overwhelming majority (79 percent) who said that their views had not been altered. Drilling deeper into the data, it is apparent that there is no statistical relationship between reporting that one’s views have changed and their support for background checks.

On the other hand, there is a statistically significant relationship between one’s views on banning future sales of assault weapons and whether or not one’s views have changed since the Sandy Hook shooting; those who said that their views had changed were much more likely to support banning future sales of assault weapons (p < 0.001) and making it illegal to own an assault weapon (p < 0.001). Additionally, those who said that their views had changed since Sandy Hook also were much more likely to support requiring strict reporting by mental health professionals to the national instant check system.

Figure 1. Support for banning the further sale of assault weapons, by whether views on gun control have changed since the Sandy Hook shootings (Feb. 2013)


In sum, although the February 2013 poll by the Castleton Polling Institute only represented a snap shot of views at that time, many of the views expressed cannot be dismissed simply as a reaction to a horrific event; 79 percent of Vermonters told pollsters that their views were set before the event occurred.

The 2016 Presidential Election and the Hispanic Vote

The Marist Poll recently published results of a number of head-to-head matchups for the 2016 presidential race, putting Hillary Clinton (the presumptive Democratic nominee) against several of the possible Republican nominees. In each case, Clinton has an edge over her Republican opponent.

While the head-to-head matchups are interesting to me, it is important to keep in mind that these are a snapshot of the current political zeitgeist and not a prediction of the November 2016 outcome. To read these results as predictive would be to discount the entire process of the presidential campaign, where the parties, the candidates, interest groups, and PACs will spend more than a billion dollars to influence voters—a complete waste of money if we were to presume that voters have already decided.

What is clear from the Marist Poll, however, is that those who identify as Democrat are very likely (89 percent) to support Clinton in November if she is the nominee, and those who identify as Republican are even more likely (94 percent) to support the Republican candidate. The campaign that follows is not about changing the minds of these partisans but rather exciting the base and making sure that they get out to vote in November.

While a majority of registered voters in the Marist Poll identify as either Democrat (34 percent) or Republican (27 percent), a slight plurality identify as Independent (37 percent). The presidential campaign of 2016 will be a battle to win the Independents in the states (a) where Independents do not lean heavily toward one of the two major parties and (b) where Independents can control the outcome.

The Marist Poll, a collaboration with MSNBC and Telemundo, had a large sample (n=3,121) in order to capture a sufficient number of Latinos (n=437, the survey offered a Spanish language option) to make subgroup comparisons. The most revealing results of the Marist Poll, in my opinion, is the degree to which Latino/Latina respondents are soured on Republican candidates, specifically but not exclusively Donald Trump. This is particularly notable since the Republican Party has two potential nominees with Hispanic roots: Ted Cruz and Marco Rubio, both with Cuban heritage. Granted that Cuban-Americans have never been the weather vane for the general Hispanic voter, still the allure of a Hispanic candidate is not appealing enough to Hispanic voters to overcome the damage that the Republican primary campaigns have thus far done to the Republican image among Hispanics in America.

All this being said, the views of Hispanics only matter to the campaigns if Hispanics vote in November. If the Hispanic turnout increases over the 2012 election—where a Pew Research report put the Hispanic turnout at 48 percent of eligible voters—then the Democrats will have a huge advantage, not just for 2016 but for several election cycles to follow.