Post-Mortem On KFF 2020 Election Polling

Digging Back Into The Sun Belt Voices Project

In early fall 2020, KFF in collaboration with the Cook Political Report conducted the Sun Belt Voices Project which included interviews with a random sample of 3,479 registered voters in three Sun Belt states (1,298 in Arizona, 1,009 in Florida, and 1,172 in North Carolina). Relying on an innovative probability-based approach of contacting registered voters sampled from voter registration files and allowing respondents to participate online or over the telephone, the project had two main goals, 1) to provide insights into the attitudes and experiences of voters in three key states for the 2020 presidential election; and 2) examine the issues that resonate with and motivate voters in state s in which shifting population characteristics are resulting in increasing Democratic vote shares in more traditionally Republican region of the U.S..

The original report released on September 17, 2020 found most voters had made up their minds about who they were going to vote for with more than six weeks before the official Election Day and that President Trump was a motivating force for both voters who intended to vote for the former President as well as those who were planning on voting for now-President Biden.

Now, more than nine months after we conducted the survey and six months after the 2020 presidential election, we are looking back into the data we collected and matching it with actual voter records from Election Day to better understand the demographics of voters who cast ballots during this election. By matching the original sample file from L2, including those who participated in the survey as well as those who were contacted to participate but did not, with the updated voting records from the 2020 election, this new analysis examines whether voters’ planned intentions to vote match their actual turnout in the 2020 presidential election. In addition, it does a deep dive into better understanding the demographics and views of the new 2020 voters who did not vote in the 2016 election. These voters fall into two categories: low propensity voters (those who were eligible to vote in 2016 but chose not to) and newly eligible voters, for whom the 2020 election is the first general election for which they are at least 18 years old.

Finally, this analysis examines whether the sampling methodology used in this project can help researchers better understand how the polling field can adapt to reach voters missed by traditional polling methodologies, a problem that contributed to underestimates of President Trump’s vote margins across key states in pre-election polling.

Executive Summary

The 2020 presidential election was unlike any other election in U.S. history. It was held in the middle of a global pandemic as the nation grappled with issues around the coronavirus, police violence against black Americans, and an uncertain economy. In an effort to address concerns about voting in-person, many states expanded access to early voting and absentee (mail-in) ballots and after all of the votes were counted, it was the highest voter turnout in more than a decade.

One of the major narratives out of the 2020 election, especially in the absence of a decision on Election Night, was that pre-election polls incorrectly estimated the support for President Trump in key battleground states. While the Sun Belt Voices Project was conducted more than six weeks prior to the election and was not designed to estimate support for the presidential candidates, our goal in this analysis was to examine whether the polls we conducted in Arizona, Florida, and North Carolina misgauged voters’ opinions and experiences leading up to Election Day. By linking the data we collected from the survey with voter records, we can better understand which voters chose to cast ballots in the 2020 election and whether their views and experiences were different than those who chose not to vote.

Several theories have arisen to explain why both the national polling average and state polling averages in the weeks leading up to the presidential election underestimated President Trump’s vote share. The first is turnout. In a high turnout election, it is more difficult for pollsters to predict who is going to show up to vote as you can’t rely on past voting behavior. Our analysis finds one-third of 2020 general election voters in our sample are “new” voters, that is they did not vote in the 2016 presidential election. This includes one quarter (26%) of total voters who were low propensity voters – voters who had been eligible in 2016 (at least 22 years old in 2020) but did not vote in 2016, and an additional 6% who were newly eligible to vote.

Tied to turnout is the role of Hispanic voters in the 2020 election. Our analysis finds a disproportionate share of Hispanic voters are in the low propensity voter group as over one-third of Hispanic voters who were eligible in 2016, but chose not to vote, chose to vote in 2020 (an additional 12% of Hispanic voters were newly eligible for this historic election). This may be due to increased access to absentee (mail voting) in Florida and North Carolina. Overall, a slightly larger share of new 2020 voters said they voted absentee (through the mail) with seven in ten (72%) low propensity voters taking advantage of absentee voting, and an additional one in five voted early in person. Our analysis finds neither political party had an advantage among the low propensity voters, but the Democratic Party did have a slight advantage among the newly eligible voters.

In the 2016 election, there was some evidence that many late deciding voters went disproportionately for President Trump in key battleground states. Overwhelmingly, self-reported turnout and enthusiasm matched what voters ended up doing in the 2020 election. The vast majority (91%) of voters who voted in the 2020 election reported they were “absolutely certain to vote” back in September and many Sun Belt voters who decided not to vote in the 2020 election had previously reported they were less motivated to vote and were “undecided” about their vote choice. While some of these voters did choose to cast a ballot, our analysis finds they were not disproportionately Republican or viewed President Trump more favorably. In fact, most of them identified as political independents.

What is clear in our analysis and others’ is that polls are missing a certain segment of voters who disproportionately supported President Trump. There has not been any evidence of “shy Trump voters” (voters who support President Trump may not be honest about their support for him either to purposely mislead researchers or because of a social desirability desire) in analyses of both the 2016 election and the 2020 election, and our results are similar. Voters who participated in the 2020 KFF/Cook Political Report Sun Belt Voices are representative of the demographics of all voters who voted in the 2020 election (in terms of education, race and ethnicity, and political partisanship), and those who chose to complete the survey online were no more likely to state support for President Trump. Yet, we strongly believe there is more work to do in better understand how polling’s failure to include or appropriately weight for a certain group of voters produced error that could explain a large portion of the polling error during the 2020 election and could have implications in future polls, both pre-election polls and others.

Key Findings

• One-third (32%) of 2020 general election voters in Florida, North Carolina and Arizona are “new” voters, that is they did not vote in the 2016 presidential election. This includes one quarter (26%) of total voters who were low propensity voters—voters who had been eligible in 2016 (at least 22 years old in 2020) but did not vote in 2016, and an additional 6% who were newly eligible to vote.
• Half (47%) of the Hispanic voters in Arizona, Florida, or North Carolina who cast their ballots in the 2020 election did not vote in 2016. This includes one-third of Hispanic voters who were eligible in 2016, but chose not to vote, and an additional 12% who were newly eligible for this historic election. One in three White (28%) and Black (30%) voters and in the three states had not voted in 2016.
• Increased access to absentee or mail voting as well as early in-person voting was a key element in the 2020 election and may have both mitigated some concerns about exposure to coronavirus while voting and modestly expanded access to some groups that had not previously voted. Two-thirds of 2020 voters who participated in the Sun Belt Voices Project cast absentee ballots while the remaining shares either voted early in-person (22%) or voted in-person on Election Day (11%). A slightly larger share of new 2020 voters said they voted absentee (through the mail) with seven in ten (72%) low propensity voters taking advantage of absentee voting, and an additional one in five voted early in person.
• Unlike the 2016 election, no major news event drastically impacted voters’ motivations or intentions during the last weeks of the 2020 presidential campaign. Many Sun Belt voters who decided not to vote in the 2020 election had previously reported they were “undecided” about who they were planning to vote for in the presidential election, and two in ten (22%) of them reported being “less motivated” to vote in 2020 compared to 2016 (compared to 6% of voters who voted in 2020), as well as said their chances of voting in the 2020 election were 50-50 or less. On the flip side, 91% of voters who voted in the 2020 election reported they were “absolutely certain to vote” back in September.
• There were some late deciders in the KFF/Cook Political Report Sun Belt Voices pollingMore than three-fourths of voters who reported being “undecided” about who they would cast their ballot for in their responses to the Sun Belt Voices Projectconducted in early September ended up voting in the 2020 presidential election. While we do not know which candidate they chose to vote for, our analysis finds that half of these undecided voters identify as political independents (53%), 27% identify as Republicans or Republican-leaning independents and 20% identify as Democrats or Democratic-leaning independents. In addition, nearly six in ten (59%) held favorable views of Democratic candidate Biden at the time of the poll compared to 46% who held favorable views of President Trump. One-fourth of these voters had favorable views of both candidates.
• The voters who participated in the 2020 the KFF/Cook Political Report Sun Belt Voices are representative of the demographics of all voters who voted in the 2020 election. However in each state, the poll underestimated support for President Trump. This analysis provides some insight into better mining of voter files to better understand who we missed in our polling and how to better reach these voters in all of KFF polling going forward.

Who Actually Voted?

According to the United States Election Project, 2020 was a high turnout election in each of the three states included in the Sun Belt Voices Project with 65.3% of voters in Arizona voting in the presidential election, 71.2% of Florida voters, and 71.2% of North Carolina voters. Participating in a pre-election poll is strongly correlated with actual voting in an election and this holds true in this project. Ninety-three percent of survey respondents voted in the 2020 election including 93% of Arizona participants, 94% of Florida participants, and 94% of North Carolina voters. If we expand this analysis to our entire sample (including those who were contacted but did not participate in the survey), three-fourths of all voters who were contacted to participate voted in the 2020 election.¹

Who Were The New Voters In 2020?

The 2020 election saw a surge of new voters, with a record breaking 159 million Americans voting in the general election compared to 138 million votes just four years earlier. This surge can likely be attributed to a combination of things, including the increased access with many states allowing ballots to be cast by mail for the first time and waves of political activism. This analysis of our Sun Belt Voices Project look at one segment of new voters – those voters who stayed home in 2016, despite being eligible to vote, but chose to cast a ballot in 2020. One quarter (26%) of the voters in the 2020 general election in Florida, North Carolina and Arizona were low propensity voters—voters who had been eligible in 2016 (at least 22 years old in 2020) but did not vote in 2016.²

A lot of attention around the 2020 election was given to Hispanic voter turnout, as some polls underestimated Hispanic support for incumbent Donald Trump. According to our analysis, one-third of Hispanic 2020 voters were low propensity voters, meaning they turned out to vote in 2020 but did not in 2016. If we include those who were newly eligible due to age, half of Hispanic voters (47%) who voted in 2020 in Arizona, Florida, or North Carolina had not voted in the previous presidential election. About one quarter White (24%) and Black (25%) 2020 voters were low propensity voters. About half (53%) of Hispanic voters voted in both the 2020 and 2016 election, compared to seven in ten Black (70%) and White (72%) 2020 voters.

According to this analysis, neither political party had the advantage among this group of new 2020 voters, with about one quarter Democratic voters and one in five Republican voters being in this low propensity category, however the Democratic Party did have a slight advantage among newly eligible 2020 voters. Eight percent of 2020 Democratic voters were newly eligible, meaning they recently turned 18 and were able to vote in their first presidential election, compared to 4 percent of 2020 Republican voters.

Three in ten (28%) of younger adults who had been eligible to vote in 2016 (ages 22 to 29) voted for the first time in 2020. Those ages 65 and older were least likely to be low propensity voters, with about one in six (17%) of these voters not voting in 2016 as well.

In Arizona and North Carolina, around four in ten low propensity 2020 voters (who had been previously eligible to vote but had not) had said they were “definitely going to vote” for President Biden, compared to about three in ten who said they same about Donald Trump. In Florida, these low propensity voters were divided in their support for each candidate. Leading up to the election, our polling similarly found Hispanic voters in Florida were more divided in candidate support.

How Did Voters cast their Ballots?

While voting by mail in Arizona was a widely used practice before the 2020 election, it was less common in Florida and only become available in North Carolina leading up to the 2020 election. Two-thirds of 2020 voters who participated in the Sun Belt Voices Project cast absentee ballots while the remaining shares either voted early in-person (22%) or vote in-person on Election Day (11%).

One of the major focuses of 2020 pre-election polls was to gauge how voters planned on casting their ballots during a global pandemic. Our analysis found that for many voters, their actual voting method did not exactly match their self-reported plans on voting method back in September. For example, while four in ten of those who ended up voting absentee said this was their intention, one in five had planned to either vote early in-person (19%) or had planned to vote in-person on Election Day (22%). Three-fourths of those who voted in-person had planned to do so back in September (77%) while those who voted early in-person had previously intended to vote various ways including in-person on Election Day (32%), by absentee (20%), or as they did – early in-person (38%).

While we do not know why many voters did not vote in the way they had planned to back in September, our analysis finds larger shares of voters across both political party and within different racial and ethnic groups decided to vote absentee than they had originally planned. Half (48%) of Democratic voters and one in four (23%) Republican voters had said they would vote absentee, while three in four Democrats (74%) and six in ten (62%) of Republicans actually voted using an absentee ballot. In addition, considerably larger shares of White voters, Black voters, and Hispanic voters voted absentee than originally planned to back in September.

This increase in the share of voters who voted absentee from who intended to vote absentee back in September may be due to an increase in trust in mail-in voting over the course of the 2020 election as well as increased access to absentee voting in the Florida and North Carolina.

Nearly seven in ten (69%) of new 2020 voters voted absentee (through the mail). Seven in ten (72%) low propensity voters took advantage of absentee voting, and an additional one in five voted early in person. Eight percent of low propensity 2020 voters voted in person on election day. Compared to low propensity voters, a slightly smaller share of total voters voted absentee. Early and absentee voting may have expanded accessibility, leading to an increase in voting among groups that had not previously voted.

Motivation And Intention Of Voters, New Voters and Non-Voters

Our analysis allows us to provide some insights into the views of voters who decided not to vote in the 2020 election, and it does not appear that anything happened in the last weeks of the 2020 presidential campaign that drastically impacted voters’ motivations or intentions. When we look at the 229 voters who participated in the Sun Belt Voices survey but who ended up not voting in the 2020 election, we find six weeks prior to Election Day about one-fifth (22%) of them had said they were “undecided” about who they were planning to vote for in the presidential election, compared to 9% of those who voted in the 2020 election. In addition, 22% of non-voters also said they were “less motivated” to vote in 2020 compared to 2016 (compared to 6% of voters who voted in 2020), as well as said their chances of voting in the 2020 election were 50-50 or less. One in four (26%) of these 2020 non-voters voted in 2016 compared to 68% of 2020 voters. Majorities of both groups (voters who voted and those who did not) said they were voting for a specific candidate and were enthusiastic about their vote choice.

On the other hand, 91% of voters who voted in the 2020 election reported they were “absolutely certain to vote” back in September and 78% of them reported being enthusiastic about their vote choice for either President Biden or President Trump.

Worries about voter suppression, voter fraud, or technical problems were not strong factors in voters’ decisions on whether or not to vote. In addition, it appears concerns around the coronavirus did not drastically impact decisions to vote, maybe due to the expanded access to mail voting, with similar shares of voters and non-voters saying they were worried they may be exposed to coronavirus while voting.

Looking again at low propensity voters who ended up voting in the 2020 general election, one reason for this decision can be attributed to enthusiasm about their vote choice (86%) and higher motivation to vote compared to 2016 levels (66%).

Who Were the Late Deciders?

Yet, while motivation to vote for a specific candidate is a strong predictor of the likelihood that a voter would cast a ballot, more than three-fourths of voters who reported being “undecided” in their responses to the Sun Belt Voices Project conducted in early September ended up voting in the 2020 presidential election.

While we do not know which candidate they chose to vote for, our analysis finds that half of these undecided voters identify as political independents (53%), one quarter (27%) identify as Republicans or Republican-leaning independents and one in five (20%) identify as Democrats or Democratic-leaning independents. In addition, among the group of initial undecided voters who ended up casting a vote in the 2020 presidential election, nearly six in ten (59%) held favorable views of Democratic candidate Biden at the time of the poll compared to 46% who held favorable views of President Trump. One-fourth of these voters approved of both President Trump and now-President Biden back in September.

What Motivated Voters In 2020?

When you examine the motivations and attitudes of voters who voted in the 2020 election, the top issues for voters are the same as we reported in the initial Sun Belt Voices Project analysis. The economy was the most important issue in deciding their vote for president, followed by criminal justice and policing, the coronavirus outbreak, health care, race relations, and immigration.

Similarly, low propensity voters said that economy was the most important issue in deciding their vote for president (32%), followed by the coronavirus outbreak (17%), criminal justice and policing (15%), health care (14%), race relations (13%), and immigration (5%).

This analysis also finds that the candidates’ advantages on key electoral issues among voters match their advantages in our pre-election polling as well as our analysis of the AP VoteCast data. Across most issues leading up to the 2020 general election, President Biden had the advantage among both voters who cast their ballot for the first time and overall 2020 voters; the economy is the exception with President Trump having the advantage on this issue with both sets of voters (+6 percentage points among low propensity voters and +8 percentage points among all 2020 voters).

Low propensity voters trusted Biden by larger margins, saying they trust Joe Biden to handle health care (+11 percentage points, +8 percentage points among all 2020 voters), race relations (+21 percentage points, +15 percentage points among all 2020 voters), the coronavirus outbreak (+16 percentage points, +9 percentage points among all 2020 voters), immigration (+9 percentage points, +2 percentage points among all 2020 voters) and criminal justice and policing (+5 percentage points, +2 percentage points among all 2020 voters).

How Did Our Methodology Do In Representing Voters?

One of the main narratives out of the 2020 election cycle was that public opinion polls underestimated support for President Trump in key states, including the three states included in the Sun Belt Voices Project. While the KFF/Cook Political Report partnership project was conducted more than a month prior to the election and was not designed to determine which candidate was leading in each of the key states, it is also our intention to ensure that the voters included in our sample are representative of the voters who ended up voting in the 2020 election.

Our methodology was designed in an effort to account for many of the deficits of polling from the 2016 presidential election. We started by generating a probability-based sample among all registered voters in each of the three states, contacting 31,500 voters (10,500 in each state) to participate in this project. We added extra incentives and outreach for key voting groups who had historically been underrepresented in election polls including Republicans, white voters without a college degree, and Hispanic voters. We allowed voters to participate in the survey using their preferred mode (online or over the phone), collecting responses from 1,298 voters in Arizona, 1,009 voters in Florida, and 1,172 voters in North Carolina. Finally, during the weighting stage, we adjusted the data so that Republicans and Democrats were represented in proportion to their share of the voter population as well as both 2016 and 2018 voter turnout in order to account for any additional partisan nonresponse bias.

Our analysis finds that these extra efforts as well as post-stratification weighting did a good job at estimating partisan turnout, turnout among Black and Hispanic voters, and correctly gauging the share of non-college educated white voters who voted in 2020. When we limit the analysis to just those who voted in the 2020 election (3,250 voters who participated in the survey and 24,468 voters who we contacted to participate), we find no meaningful differences by race/ethnicity, education, or partisanship among our weighted results to the entire sample of Sun Belt Voices voters.

In addition to this, our analysis finds both the mode and the sampling frame didn’t result in demographically different groups of voters. Samples from both the online completes and listed phone numbers in the Sun Belt Voices Project were similar to voters overall, with one third being Democrats or Republicans, and similarly distributed across education levels and race and ethnicity. Despite this, in each of the three states, our original Sun Belt Voices data overestimated Biden’s advantage in the states ranging from 3 percentage point difference in North Carolina to 5 percentage points in Arizona underestimating voter preference for President Trump. This is regardless of whether we limit our analysis to just voters who turned out in the 2020 election, those who participated in the survey online, or those who had listed phone numbers in their voter file records.

As pollsters prepare for the 2022 election cycle, more work is needed to better understand how polling’s failure to include or appropriately weight for a certain group of voters produced error that could explain a large portion of the polling error during the 2020 election. A lot of this work is already being done and was presented at the 2021 AAPOR Conference last week but as of now, there are still more lingering questions to be answered.