The Accuracy of Election Polls

The accuracy of election polls has been under fire for quite some time, but more so during the 2016 presidential election. Results from election polls were part of our daily news diet during this period.

Many institutions, from universities to media outlets, jumped on the polling wagon. However, made it very difficult to decide which one to believe.

Mentions of results from election polls usually included information about sample size and margin of error. Nonetheless, that’s not enough to evaluate their quality.

Comparing election polls is nearly impossible without knowing the methodology used in terms of population universe selected, screening criteria, and question formulations. Many of them were flawed, and some were simply not comparable.

The Business of Predicting Elections

Predicting presidential elections is a risky business. Even Gallup, which for many years was considered the country’s gold standard for horse-race election polling, decided to give up predictions it failed to predict Obama’s win in 2012.

The 2016 US general election proved to be another defeat for pollsters. The infographic below shows the results from the final poll conducted by Gallup before each Presidential election since 1976. In most years, the polls made a correct prediction of the winner, except for 1976 and 2012.

For 2016, in the absence of Gallup numbers, we used the average across all polls compiled by RealClearPolitics.com during the last few days before 11/8/2016. This method smoothed out the influence of outliers, pegging Hillary Clinton ahead by a few points, although within the margin of error. Although many questioned the accuracy of election polls, the results matched pollsters’ predictions. She won the popular vote but lost the electorate vote. As a result, she became the fifth presidential candidate joining this unfortunate club.

The Ultimate Mistake

It seems the media and pollsters fell victim of the confirmation bias, declaring Hillary Clinton the winner. We all expected her to win. This was based on many polls pointing in her direction. However, most results were within the margin of error.

The sheer number of polls with similar results, despite a lack of statistically significant differences, blinded everybody.  Overlaying directional differences on the electorate map without digging deeper in the nature of the electorate, was the ultimate mistake.