We’re saying it: Hillary Clinton wasn’t the only loser last night. The polling industry took a massive nose-dive as well.
— The New York Times (@nytimes) November 9, 2016
So how did everyone get it so wrong?
There was never meant to be any doubt about who would win the Presidency. Up until 8:22pm last night New York Times polling data showed one clear winner. Except it was the wrong one. The consequences of over-relying on online polling proved to be a disastrous blunder, with people waking up to a quite different future from the one sold to them by credible pollsters, pundits, and analysts.
Funnily enough, online polls failed to predict the outcomes of other recent elections too, including the UK’s Brexit vote, yet we continue to rely on them. Some suggest this is because polling companies usually have an online panel they can repeatedly survey. This select population is then taken to be representative of all eligible voters. Complex data analysis is then added to the mix to offset inaccuracies, and research firms think they’ve got it in the bag – except they haven’t.
According to the New Yorker, survey response rates have plummeted. In the 1930s, it was over 90 percent. In 2012, it was just 9 percent.
“The US population has grown 2.5 times larger since the 30s so the overall participation rate (the percentage of the total population that ends up actually completing a survey) has fallen even faster.” The Guardian
Online election survey methodology faces two increasingly hard problems. One is the rapid decline in people willing to answer online surveys. And the second is the correlation between Internet use and, age and voting habits.
“While all but 3 percent of those ages 18 to 29 use the Internet, they made up just 13 percent of the 2014 electorate, according to the exit poll conducted by Edison Research. Some 40 percent of those 65 and older do not use the Internet, but they made up 22 percent of voters.” The New York Times
Why is in-person better than online?
1) People act differently in person
Online polling is inexpensive and convenient, however, without discipline, they can be wildly ineffective. People are usually doing something else which means they’re not actually present and invested in the task in front of them. Anonymity also lets respondents provide evasive, misleading answers. By contrast, face to face interviewing, when done right, feels more like a conversation rather than an interrogation. Respondents are more invested and therefore more willing to give more honest answers.
2) Engagement is higher
In-person encourages greater participation because respondents are stimulated by what they see and are motivated to take part, especially in the presence of others. Skilled researchers and interviewers also play a huge role in driving engagement as they can 1) provide an introduction to the nature and benefit of the study and 2) eliminate gray areas and room for misunderstanding by answering questions and providing assistance when required. Once sufficient rapport is built, a skilled interviewer can quite easily probe the respondent until true emotions and attitudes are revealed.
3) Reach is better
Internet users tend to be urban dwellers and have above average incomes and are therefore unrepresentative of the population as a whole. In-person polls can be conducted anywhere, by multiple interviewers in multiple locations, and can reach hard-to-reach respondent groups who do not have internet access or are less likely to participate in online polling. Language and timing also play a crucial role – not only can you reach your target population at the best times – you can put them at ease by adopting their language.
Trump’s shock victory proves that current polling methods are failing at the very thing they’re meant to do. Our suggestion is to target better and combine online and in-person methodologies to get the best of both worlds. This just might be the solution the election polling industry needs to recover from its latest embarrassment.
Also published on Medium.