By: Tsahi Halyo

Even with information and data collection the easiest it has ever been, our collective society should not consider polling as an exact predictor of election results.

Few things are misunderstood in the political sphere as much as polls. While their numbers tempt us with the siren call of mathematical certainty, a close examination reveals them to be far more limited and down-to-earth in their claims. Polls suffer from many flaws that should condition one’s faith in them so generally speaking, no news organization should pretend any polling result exists with 99% percent confidence. This article will characterize the efficacy of political polls by describing three facts that stymie perfect polling today.

Fact 1: Polls aren’t as accurate as most people think

                                                               Average Polling Error

	Presidential		State-Level
Cycle	Primary	General	Governor	U.S. Senate	U.S. House	Combined
2017-18	—	—	5.2	6.0	4.1	5.1
2015-16	10.1	4.8	5.4	5.0	5.5	6.8
2013-14	—	—	4.4	5.4	6.7	5.4
2011-12	8.9	3.6	4.8	4.7	4.7	5.1
2009-10	—	—	4.9	4.8	6.9	5.7
2007-08	7.4	3.6	4.1	4.7	5.7	5.4
2005-06	—	—	5.0	4.2	6.5	5.3
2003-04	7.1	3.2	6.1	5.6	5.4	4.8
2001-02	—	—	5.2	4.9	5.4	5.2
1999-2000	7.6	4.4	4.9	6.1	4.4	5.5
1998	—	—	8.1	7.4	6.8	7.5
All years	8.7	4.0	5.4	5.4	6.2	5.9

Figure courtesy of FiveThirtyEight: https://fivethirtyeight.com/features/the-polls-are-all-right/

There is a widespread belief among members of the media and the general public that polling is usually quite accurate. This led many of them to feel betrayed when polls predicted a Clinton presidency and were in turn granted a Trump one. What was supposedly meant to begin a new era of Democratic political hegemony instead ushered in an era in which the GOP would cement its control of all three branches of government. However, this shouldn’t have been such surprise. While the media reports polling numbers it rarely reports the polling error. The above table presents the average deviation polls conducted in the final 3 weeks of the campaign had from election results. It shows that on average, polling of Presidential elections is off by 4%; that of Senate races if off by 5.4%; and that that of U.S House elections is off by a whopping 6.2%. Trump’s winning the election should not have been as surprising as many perceived it to be. In the key states of Ohio, Florida and North Carolina his polling numbers trailed Clinton’s by numbers smaller than pollsters’ average error in Presidential races. The same can be said about that election cycle’s Senate races. Polls aren’t a crystal ball and the public should not expect them to act like one.

Fact 2: Republicans are underrepresented in poll results.

Pollster	Average GOP Bias
ABC	-1.4%
CBS	-1.75%
CNN	-2.6%
FOX	-4.1%
Pew	-4%

Another reason to take polling results with a grain of salt is the fact that polls systemically undercount GOP support. When comparing polls, conducted the week before congressional elections and with election results a consistent, it becomes clear that Republicans have consistently outperformed the polls on election night. Though many reasons have been offered to explain this phenomena this article will only explore one. It’s conjectured that some Republican voters are embarrassed to admit they back a Republican to pollsters who, practically speaking, are strangers over the phone. They may feel that there is a stigma attached to being a Republican (e.g as a bigot; racist; homophobe, etc.) which leads them to tell pollsters that they back a 3^rd party or remain undecided. Evidence for this theory can be found in the fact that undecided voters in Congressional elections disproportionately vote for Republicans. Further evidence for this can be found in the case of David Duke’s, a Klan leader’s, Senate bid. Though pollsters found that less than 5% backed him, he won 20% of the vote. Who could admit they planned to vote for a despicable man like Duke over the phone?

Fact 3: Polls temporarily dip in accuracy 10-6 months before the election.

The plot to the left displays the correlation polls have with election results (r-value) as a function of time. (Note: Higher correlation doesn’t mean that the polls are more accurate themselves but rather that they have a stronger linear relationship with that year’s election results.) Basically, it tells us how well we can predict election results based on current polling. In the case of midterm elections there is an astounding dip in predictive power 10 months before voting day that is only ameliorated 3 months prior to the vote. It is difficult to explain this phenomena. While it’s possible that it occurs due to the natural fluctuations of the political pendulum the polling doesn’t suggest that a clear oscillation against the leading party is the answer. It may be that before our hyper-news era of the Trump presidency, most voters had made up their minds who to back in the year before the election far before intense political news coverage began to cover political campaigns. Whether this trend will survive in our current environment remains to be seen.

It’s tempting after seeing polling’s flaws to discredit the entire enterprise. In fact, after the 2016 election many news services published articles expressing such a view. However, it’s better to see political polls as a tool of limited use. It can predict a blowouts going either way, but sometimes  can only tell you that an election will be close. There has been much hype surrounding Democrats’ chances of winning control of a chamber of Congress. Hopefully this fall we won’t repeat the mistake again of treating polls like a crystal ball.

 

Sources:

(Polling data was retrieved from pollingreport.com, analyses of such data are my own)

TABLE OF CONTENTS (n.d.) Retrieved from http://www.pollingreport.com/Contents.htm

(n.d.). (2018,May,31). The Polls Are All Right. Retrieved from https://fivethirtyeight.com/features/the-polls-are-all-right/

(n.d.). Retrieved from http://www.marketoracle.co.uk/images/2016/Nov/