What Can Pollsters Learn from 2015’s Errors?

What Can Pollsters Learn from 2015’s Errors?

Much has been said about the failings of the pollsters in the run up to the 2015 general election. Explanations include poor sampling[1] and the ‘lazy –Labour’ voters who never showed up at the ballot[2]. However, emerging awareness about the nature of ‘implicit preferences’ suggests that polling may be problematically plagued by the gap that exist between what people report and how they actually act. I am not referring to a case of dishonesty, as the ‘shy-Tory’[3] hypothesis suggests, but rather to the complexities that human thought and decision-making inevitably entail.

Nobel Laureate Daniel Kahneman’s ‘Thinking, Fast and Slow’, and the emergent findings from Harvard’s Project Implicit[4] are casting doubt upon the validity of what people communicate in relation to what they actually do. Both advance the idea that we make more reasoned decisions under certain circumstances (when thinking ‘slow’, or ‘explicitly’), and rely upon gut instincts in other circumstances (thinking ‘fast’ or ‘implicitly’).

Project Implicit has approached the theory through social prejudices. For instance, whilst few people would report that they have a preference for white people over black people, or for thin people over fat people, their ‘Implicit’ testing reveals that people have non-conscious negative associations with some groups and positive associations with others. Researchers study these implicit preferences by asking participants to categorise both thin and fat people, and positive and negative words at speed: when people make errors, their implicit preferences start to be identified.

Implicit preferences are more instinctual gut reactions, shaped by our past experiences and cultural associations. The Implicit Preferences revealed in the Harvard studies show strong correlations between negative implicit associations and groups who have been portrayed more negatively in society and the media.

How does this relate to polling? Chief Executive of ICM, Martin Boon, questioned the validity of online polls, given that respondents are paid for their participation and may rush through the poll without thinking carefully [5]. From our point of view, these online polls could be measuring ‘implicit preferences’. When people vote on Election Day, however, they may be more likely to reason through their decisions and rely less upon initial gut reaction. In short, they may be expressing their ‘explicit preferences’. ‘Thinking, Fast and Slow’ highlighted that depending on the intensity and stakes of a situation, people draw upon different ways of thinking. For polling and prediction, this means that people may use one level of thought to answer on the rushed, low-stakes YouGov polls, and then use a different level when it comes to making the big decision. If this is the case, pollsters need to be doing more to measure ‘explicit preferences’ when they are aiming to make predictions.

Indeed, whilst the mainstream polls were wrong in 2015, the internal polls of the Parties were much closer to the mark[6]. Rather than asking voters which party they’d vote for, these researchers asked people what their values were and then correlated this with the values that each party was campaigning about. The predictions based on values were much more accurate. Is it possible that this is because the value polls were measuring ‘explicit preferences’; those which would be drawn upon on Election Day?

Whilst being asked what your core values are, you are thinking a little harder thus being more cognitive. When asked if you prefer ‘the red team’ or ‘the blue team’, your reaction may be more shaped by non- conscious loyalties to miner grandads, or hidden instincts of individualistic self-preservation. A further example is the historical accuracy of betting odds in calling election results[7]. When you place a bet being right matters. This leads to more ‘explicit’ reasoning, which we suggest is the thinking that determines elections. Interestingly, before opinion polls became common practice, betting odds were remarkably accurate, and their predictive power actually began to decline when scientific polling was introduced and the gamblers began to follow them.  Again, the methodology resting upon careful thought appears advantageous compared to the auto-pilot responses which polls may sometimes evoke.

Why though, would the polling have been so wrong in 2015, if the truth is that polling surveys and the ballot simply tap in to different parts of the brain? It may be the case that in a world of increased political pluralism and ever-extending sources of information, we really have to think harder than we did before. Compounded by the increase of U-turns and political side-switching, on Election Day we have to think harder about what matters most to us. We may now be more ‘explicitly’ thinking voters. Whether this is set to be a growing trend, or was just a product of the particular circumstances of the election, is yet to be seen.

This of course brings us to the question of what will happen next. Will the EU referendum result come as another pollster-thwarting shock, or will they be right this time? If the complexities of ‘implicit’ and ‘explicit’ thinking are forces at play within polling, we are left with two considerations. On one hand, we will have to start actively considering the ways in which questions are asked, and the sort of attitudes they are measuring.

In the Insights department at Portland, we will certainly be paying attention to any polls attempting to factor in personal values, and more complex questions than simply ‘Brexit or Bremain?’. On the other hand, we can be slightly more astute in the directions in which our polling reservations take us. We have heard the Leave campaign touted as ‘Project Fear’ as opposed to ‘Project Fact’. We have also heard about the 7.5 million swing voters that fall in to the ‘Heads vs. Hearts’ categories’. If this holds, and we apply our theory that on Election Day people think more carefully and logically about their decisions, then we can expect that the instinctual ‘Heart’ aversions to Europe may be overcome by the ‘explicit’ reasoning of the ‘Head’. It is being able to understand how the often influential undecided voter will act that is the most attractive power of the implicit-explicit theory.


[1] Guardian . 2016. General election opinion poll failure down to not reaching Tory voters. [ONLINE] Available at: http://www.theguardian.com/politics/2016/jan/19/general-election-opinion-poll-failure-down-to-not-reaching-tory-voters. [Accessed 25 March 16].

[2] The Spectator. 2016. Were ‘Lazy Labour’ voters, not ‘Shy Tories’, responsible for the election result?. [ONLINE] Available at: http://blogs.spectator.co.uk/2015/07/lazy-labour-voters-not-shy-tories-responsible-election-result/. [Accessed 25 March 16].

[3] Guardian . 2016. How ‘shy Tories’ confounded the polls and gave David Cameron victory. [ONLINE] Available at: http://www.theguardian.com/politics/2015/may/08/election-2015-how-shy-tories-confounded-polls-cameron-victory. [Accessed 25 March 16].

[4] Project Implicit . 2011. PROJECT IMPLICIT SOCIAL ATTITUDES. [ONLINE] Available at:https://implicit.harvard.edu/implicit/index.jsp. [Accessed 25 March 16].

[5] Guardian . 2015. Why did the election pollsters get it so wrong?. [ONLINE] Available at:http://www.theguardian.com/politics/2015/may/14/why-did-the-election-pollsters-get-it-so-wrong. [Accessed 25 March 16].

[6] Guardian . 2015. Why did the election pollsters get it so wrong?. [ONLINE] Available at:http://www.theguardian.com/politics/2015/may/14/why-did-the-election-pollsters-get-it-so-wrong. [Accessed 25 March 16].

[7] Robert S. Erikson & Christopher Wlezien. 2012. Markets vs. polls as election predictors: An historical assessment. [ONLINE] Available at: http://www.utexas.edu/cola/government/_files/wlezien-web/EriksonandWlezienElectoralStudies2012published.pdf. [Accessed 25 March 16].

Back to thoughts