If the election had gone the way pollsters thought it would, by now Joe Biden would have been named president-elect. He would have won Texas and Florida easily, and would have delivered his victory speech in Delaware.
But, instead, more than a day after the polls closed, a winner has not been confirmed. As the votes were counted, the comfortable lead polling firms projected for Biden shrunk dramatically. Donald Trump outperformed pollster FiveThirtyEight’s average in every swing state but Arizona.
The US presidential election has turned out to be much tighter than political commentators could have ever imagined – and that is mostly down to polling firms. Many experts argue that pollsters would have done better had they more widely embraced data on social media.
"Traditional polling is missing a little bit around the edges," says John Murphy, a Professor in Digital Media Business Strategies at the University of Connecticut. "I believe when you look at the social media conversations and all the social media data, that fills in the blanks. It’s like another piece to the pie."
Polling has become a matter of much derision in the US, with the industry described as "a train wreck", irrevocably broken, and "done".
"Traditional polls are instruments that measure political contests in metres but these elections are being decided by centimetres," says Republican digital strategist Eric Wilson, who worked on Marco Rubio’s 2016 White House run.
"So they have their place but they’re not giving the resolution that voters have come to expect."
Instead pollsters have been hamstrung by their over-reliance on landline phones. Back in 2004, more than 90pc of adults in the US lived in households with a working landline – now that number has dropped to less than 40pc, according to data from the Centers for Disease Control and Prevention.
Although many firms have integrated online strategies in their methods – YouGov carries out questionnaires online and US pollster Frank Luntz told the BBC he uses email to reach voters – the industry’s litany of election misfires is turning pollsters into a target for frustration and mistrust.
CNN’s political commentator Ana Navarro said pollsters should be "tarred and feathered" while Fox News host Carlson Tucker said they should be fired so they can "do something useful like hang dry wall or learn to paint".
With political analysts rapidly losing faith in polling, many are looking to social media data as a more reliable barometer of public political mood.
What happens next in the US election? Key dates to watch
Lindsay Gorman, a policy expert from the Alliance for Securing Democracy, says that while “we still have to be patient to see how close the race actually comes once all the votes get counted in, before jumping to too many conclusions, I think the general point that the polls seem to be way off as they also were in 2016, obviously, is a valid one”.
“People have been looking at Facebook, but obviously not doing that same level of granularity and really using it as a tool to assess voter opinion,” she says. “We may be, in fact, entering a new era on how we use technology to understand voter opinions.”
Some businesses have already started to do this, conscripting firms that use artificial intelligence to "listen" to social media users en masse to gauge attitudes or to quickly identify and snuff out PR disasters before they escalate. In the UK, such work is becoming more mainstream, with AI firm Faculty working with the Government on its Covid-19 response using Twitter posts to chart a course.
There have already been successes here on polling. BrandsEye, for example, is a tool which looks at people’s tweets, and which correctly predicted both the vote to leave the EU in June’s referendum and a Trump victory in the 2016 US election.
Expert.ai is another firm that analyses online language. While the Italian company usually works with brands, police forces or researchers, it also claims a series of successes in forecasting elections.
Chief technology officer Marco Varone says by focusing mostly on Twitter posts, the company’s AI predicted a narrow lead for Joe Biden. "It was very similar to what we are seeing in the popular election," he says, adding the company used the same techniques to predict the outcome of the Brexit referendum.
He believes this kind of data analysis has an edge on polls because it can analyse a huge amount of information again and again, easily catching changing sentiment. "And the third element is that you need to have the technology that can really understand the nuances of the text, the language of expression," he says.
Professor Murphy credited similar tools for helping him predict that Trump would win in Florida.
University of Connecticut used social media analysis on election night
"I was not surprised," he says. "Our data showed Donald Trump winning Florida easily. And he did, whereas the polls had Biden up five [points] in Florida."
"We could see in Florida, a huge surge of positive feelings about Donald Trump, both in volume which is the numbers overall and a percentage."
Murphy runs the university’s Social Media Analytics Command Center which uses a variety of free and paid-for analytics tools to "listen" to voters across a range of social media platforms, including sites like Facebook and Twitter as well as forum Reddit and 4Chan.
"We think it’s better than polling," Murphy adds. "We really see how people feel, we see their passion. We can tell their sentiment. We can tell who they’re talking about. And we can tell if they’re engaging in a certain presidential candidate or Senate candidate. We can see if they’re engaging in their content, which tells us something too."
There remains debate about whether social media platforms is really not be a true reflection of the state of play. Reports from Crowdtangle, which assesses which posts are proving most popular on Facebook, suggest Right-wing commentators have the largest reach on the site and account for the most-read posts.
On November 3, three out of the top ten posts were by Dan Bongino, a Fox News commentator. Experts have suggested that this was a sign of the numbers of so-called “silent Trump voters”, who would share posts online but would shy away from disclosing their political preferences to official pollsters.
However, Finnish firm Yle Uutiset says what this really shows is that “polarising content gains visibility easily” on the site. It just so happens that the most polarising content online comes from Right-wing voices, perhaps because there are “just more of those conspiracy theories” on that side, policy expert Lindsay Gorman says. “I would be speculating, but there might be more trust in media outlets on the liberal side.”
Judging the outcome of an election based on Facebook data could, in fact, skew more towards extreme opinions, and miss much of the “middle ground” of voters who do not hold such strong opinions.
Republican strategist Wilson says this is why a blended approach might be a better way forward, with traditional polling mixed with things such as listening to social media, surveys, focus groups and data modelling. This would be drawing on similar sources used by campaign staff themselves. "We can’t do away with [polls] entirely," he says.
There are some who believe the recent criticism of the polling industry has been, however, overblown. Nate Silver, the founder of polling aggregator FiveThirtyEight, wrote on Twitter that “in the end, polling averages will probably ‘call’ the winners of all but 1-3 states correctly, along with the winner of the popular vote, which should wind up at Biden +4/+5”.
“That’s not great. It’s better to look at *margins* and some of the margins were off. But some of the perceptions were formulated because people were taking results in states like PA [Pennsylvania] at face value (Wow, Biden’s losing by 15 points!) when there weren’t any mail votes reported. In the final accounting, the polls will have done mediocrely, but not terribly.”