Getty Images
Days after President Joe Biden announced he would not be seeking re-election, and endorsed Vice President Kamala Harris, polling organization Siena College Research Institute sought to learn how “persuadable” voters were feeling about Harris.
In their survey, a 37-year-old Republican explained that they generally favored Trump for his ability to “get [things] done one way or another.”
“Who do you think cares about people like you? How do they compare in terms of caring about people like you?” the pollster asked.
“That’s where I think Harris wins, I lost a lot of faith in Trump when he didn’t even contact the family of the supporter who died at his rally,” the 37-year-old said.
Pollsters pressed this participant and others across the political spectrum to further explain their stances, and examine the nuance behind choosing a candidate. The researchers saw in real time how voters may sway depending on the issue, and asked follow-up questions about their belief systems.
But the “persuadable” voters weren’t talking to a human pollster. They were conversing with an AI chatbot called Engage.
The speed in which election cycles move, coupled with a steep drop of people participating in regular phone or door-to-door polls, have caused pollsters to turn to artificial intelligence for insights, both asking the questions and sometimes even answering them
Why do we poll?
The history of polling voters in presidential races goes back 200 years, to the 1824 race which ultimately landed John Quincy Adams in the White House. White men began polling each other at events leading up to the election, and newspapers began reporting the results, though they didn’t frame the results as predictive of the outcome of the election.
In modern times, polling for public opinion has become a business. Research centers, academic institutions and news conglomerates themselves have been conducting polls during election season for decades. Though their accuracy has limitations, the practice is one of the only ways to gauge how Americans may be thinking before they vote.
Polling plays a different role for different groups, said Rachel Cobb, an assistant professor of political science and legal studies at Suffolk University. For campaign workers, polling groups of voters helps provide insight into the issues people care about the most right now, and informs how candidates talk about those issues. It’s why questions at a presidential debate usually aren’t a surprise to candidates — moderators tend to ask questions about the highest-polling topics that week.
For news outlets, polls help give context to current events and give anchors numbers to illustrate a story. Constant polling also helps keep a 24-hour news cycle going.
And for regular Americans, poll results help them gauge where the race is, and either activate or calm their nerves, depending on if their candidate is polling favorably.
But Cobb said she, like many of her political science colleagues, has observed a drop in responses to more traditional style of polling. It’s much harder and more expensive for pollsters to do their job, because people aren’t answering their phones or their front doors.
“The time invested in getting the appropriate kind of balance of people that you need in order to determine accuracy has gotten greater and so and they’ve had to come up with more creative ways to get them,” Cobb said. “At the same time, our technological capacity has increased.”
How AI is assisting in polling?
The speed of information has increased exponentially with social media and 24-hour news cycles, and polls have had to keep up, too. Though they bring value in showing insights for a certain group of people, their validity is fleeting because of that speed, Cobb said. Results are truly only representative of that moment in time, because one breaking news story could quickly change public opinion.
That means pollsters have to work quickly, or train artificial intelligence to keep up.
Leib Litman, co-CEO and chief research officer of CloudResearch, which created the chatbot tool Engage, said AI has allowed them to collect answers so much faster than before.
“We’re able to interview thousands of people within a matter of a couple hours, and then all of that data that we get, all those conversations, we’re also able to analyze it, and derive the insights very, very quickly,” he said.
Engage was developed about a year ago and can be used in any industry where you need to conduct market research via interviews. But it’s become especially useful in this election cycle as campaigns attempt to learn how Americans are feeling at any given moment. The goal isn’t to replace human responses with AI, rather to use AI to reach more people, Litman said.
But some polling companies are skipping interviewing and instead relying on something called “sentiment analysis AI” to analyze publically available data and opinions. Think tank Heartland Forward recently worked with AI-powered polling group Aaru to determine the public perception of artificial intelligence.
The prediction AI company uses geographical and demographic data of an area and scrapes publicly available information, like tweets or voting records, to simulate respondents of a poll. The algorithm uses all this information to make assertions about how a certain demographic group may vote or how they may answer questions about political issues.
This type of poll was a first for Heartland Forward, and its executive vice president Angie Cooper said they paired the AI-conducted poll with in-person gatherings where they conducted more traditional polls.
“When we commissioned the poll, we didn’t know what the results were going to yield,” she said. “What we heard in person closely mirrored the poll results.”
Sentiment analysis
The Aaru poll is an example of sentiment analysis AI, which uses machine learning and large language models to analyze the meaning and tone behind text. It includes training an algorithm to not just understand literally what’s in a body of text, but also to seek out hidden messaging or context, like humans do in conversation.
The general public started interacting with this type of AI in about 2010, said Zohaib Ahmed, founder of Resemble AI, which specializes in voice generation AI. Sentiment analysis AI is the foundation behind search engines that can read a request and make recommendations, or to get your Alexa device to fulfill a command.
Between 2010 and 2020, though, the amount of information collected on the internet has increased exponentially. There’s so much more data for AI models to process and learn from, and technologists have taught it to process contextual, “between-the-lines” information.
The concept behind sentiment analysis is already well understood by pollsters, says Bruce Schneier, a security technologist and lecturer at Harvard University’s Kennedy School. In June, Schneier and other researchers published a look into how AI was playing a role in political polling.
Most people think polling is just asking people questions and recording their answers, Schneier said, but there’s a lot of “math” between the questions people answer and the poll results.
“All of the work in polling is turning the answers that humans give into usable data,” Schneier said.
You have to account for a few things: people lie to pollsters, certain groups may have been left out of a poll, and response rates are overall low. You’re also applying polling statistics to the answers to come up with consumable data. All of this is work that humans have had to do themselves before technology and computing helped speed up the process.
In the Harvard research, Schneier and the other authors say they believe AI will get better at anticipating human responses, and knowing when it needs human intervention for more accurate context. Currently, they said, humans are our primary respondents to polls, and computers fill in the gaps. In the future, though, we’ll likely see AI filling out surveys and humans filling in the gaps.
“I think AI should be another tool in the pollsters mathematical toolbox, which has been getting more complex for the past several decades,” Schneier said.
Pros and cons of AI-assisted polling
AI polling methods bring pollsters more access and opportunity to gauge public reaction. Those who have begun using it in their methodology said that they’ve struggled to get responses from humans organically, or they don’t have the time and resources to conduct in-person or telephone polling.
Being interviewed by an anonymous chatbot may also provide more transparent answers for controversial political topics. Litman said personal, private issues such as health care or abortion access are where their chatbot “really shines.” Women, in particular, have reported that they feel more comfortable sharing their true feelings about these topics when talking to a chatbot, he said.
But, like all methodology around polling, it’s possible to build flaws into AI-assisted polling.
The Harvard researchers ran their own experiment asking ChatGPT 3.5 questions about the political climate, and found shortcomings when it asked about U.S. intervention in the Ukraine war. Because the AI model only had access to data up through 2021, the answers missed all of the current context about Russia’s invasion beginning in 2022.
Sentiment analysis AI may also struggle with text that’s ambiguous, and it can’t be counted on for reviewing developing information, Ahmed said. For example, the X timeline following one of the two assassination attempts of Trump probably included favorable or supportive messages from politicians across the aisle. An AI algorithm might read the situation and conclude that all of those people are very pro-Trump.
“But it doesn’t necessarily mean they’re navigating towards Donald Trump,” Ahmed said. “It just means, you know, there’s sympathy towards an event that’s happened, right? But that event is completely missed by the AI. It has no context of that event occurring, per se.”
Just like phone-call polling, AI-assisted polling can also potentially leave whole groups of people out of surveys, Cobb said. Those who aren’t comfortable using a chatbot, or aren’t very active online will be excluded from public opinion polls if pollsters move most of their methods online.
“It’s very nuanced,” Ahmed said of AI polling. “I think it can give you a pretty decent, high-level look at what’s happening, and I guarantee that it’s being used by election teams to understand their position in the race, but we have to remember we exist in bubbles, and it can be misleading.”
Both the political and technology experts agreed that as with most other facets of our lives, AI has found its way into polling and we likely won’t look back. Technologists should aim to further train AI models to understand human sentiment, they say, and pollsters should continue to pair it with human responses for a fuller scope of public opinion.
“Science of polling is huge and complicated,” Schneier said. “And adding AI to the mix is another tiny step down a pathway we’ve been walking for a long time using, you know, fancy math combined with human data.”