The 2020 presidential election was one of the most anticipated races in recent memory. In the days and months leading up to Election Day, pundits and pollsters grew increasingly confident in a Joe Biden victory — many going so far as to predict a landslide victory. However, similar to four years ago, the polls were wrong and the race was far closer than expected.
After back-to-back presidential elections with wildly inaccurate polling, it is clear that an element is missing from the equation. We feel that element can be found via social media.
In an era fueled by digital and social media, pollsters continue to shirk the unstructured information on these platforms for structured data aggregated from antiquated phone and online surveys.
While neither data source is free from bias, there is a rich set of data that we can extract from social media that paints a more complete picture of voter intent than traditional survey polling can. Our data proved it once again.
The Story Behind Voter Intent
According to today’s Associated Press tally, the 2020 popular vote indicates a 50.8%-to-47.5% lead in favor of Joe Biden over incumbent President Donald Trump. While this lead equates to a difference of more than 5 million votes (no small margin), it is far narrower than the 8.1% average lead Biden held in the final national polls.
Similar to previous elections and votes, we extracted hundreds of thousands of tweets that referenced candidate-specific keywords and hashtags, then identified and labeled the underlying emotions (from 84 different categories) within the text. By analyzing both voter sentiment around and the digital space occupied by each candidate via our natural language technology, we formulated a weighted equation to gauge voter intent.
From this accumulation of data, we, together with Sociometrica President Antonio Preiti, created the People Emotions Index: a synthetic index of the emotional strength of each candidate. Our initial results, shared on October 12, 2020, showed Joe Biden a 50.2% to 47.3% lead over President Trump — a near mirror image of the final popular vote.
While the People Emotions Index is by no means flawless — it is neither nationally representative nor without bias — it has continuously proven to be a reliable barometer of human intent. This election was no exception, proving once again that there is plenty of predictability in language
Finding Value in Natural Language Analysis
While NLU and sentiment analysis may not be a pure replacement for polling, numerous outlets, including Forbes, Wall Street Journal and Fortune, are putting stock in the AI-powered approach to better reflect behavioral intent.
Said Heidi Messer, chairman of New York-based Collective[i]: “Data reflecting behavior is much harder to amass but infinitely more useful in dynamically identifying the patterns and correlations that fuel probabilistic predictions.”
We have leveraged our sentiment analysis approach on numerous other occasions and realized similar results:
- In 2016, we performed a similar analysis asking, “Should the United Kingdom remain a member of or leave the European Union?” While UK polls indicated a split decision, our data showed that nearly 65% of people actually backed the UK’s exit from the EU.
- In 2017, we analyzed consumer intent around Black Friday shopping. From our data, we were able to highlight the items most coveted by shoppers and the retailers from which they intended to make their purchases.
How NLU Can Help You
There is plenty of value to be found in sentiment analysis, but it is only a small part of natural language technology. NL can provide so much more value to you by understanding the exact meaning of each word in context, just as humans do.
At expert.ai, we leverage a deep linguistic analysis engine and rich domain-independent knowledge graph (which includes terms and their meanings in English and 11 additional languages) to shorten your development time of applications and address complex use cases within search and knowledge discovery as well as intelligent process automation.
So whether you’re looking for voter intent around the runoff elections in Georgia, comparing contracts, analyzing email content to improve customer service or something entirely different, there is an NLU application that speaks to you. It’s time to listen!
