Pollsters: 'Impossible' to say what went wrong in 2020

Polls from the 2020 general election and state-level races were the worst they have been in decades, according to a new report, but the task force that conducted the review said it is “impossible” to determine what went wrong based on available data.

The report, commissioned by the American Association for Public Opinion Research, found that the polling error in surveys estimating the 2020 national popular vote was the highest it has been in 40 years. Similarly, state-level predictions in 2020 for presidential, senatorial and gubernatorial contests were the worst they have been in at least 20 years, according to the report.

Of the polls conducted in the final two weeks before Election Day, the average error on the margin in either direction for national popular vote polls was 4.5 points, and 5.1 points for state-level presidential polls.

ADVERTISEMENT

The report determined that the discrepancies were “much more likely” to favor then-candidate Joe BidenJoe BidenThe Hill's Morning Report - Presented by Facebook - White House, Dems play blame game over evictions GOP skepticism looms over bipartisan spending deal Biden vaccine rule sets stage for onslaught of lawsuits MORE over then-President TrumpDonald TrumpThe Hill's Morning Report - Presented by Facebook - White House, Dems play blame game over evictions The Memo: Left pins hopes on Nina Turner in Ohio after recent defeats Biden administration to keep Trump-era rule of turning away migrants during pandemic MORE. Among the polls put in the field in the last two weeks of the election, the average error between the two candidates favored Biden by 3.9 percentage points in national polls and by 4.3 percentage points in statewide presidential polls.

The report found that the polls conducted during the election on average overestimated Biden’s support and underestimated Trump’s backing. Biden’s average topline support was overshot by an average of one percentage point, while that of Trump’s was underrated by 3.3 percentage points.

The polling error for the presidential election was found to have been stable throughout the election, with the average discrepancy matching in surveys conducted at different points of the campaign season.

The report ruled out a number of factors that do not explain the polling error, including late-deciding voters swinging for Republican candidates, respondents’ reluctance to tell interviewers that they supported Trump and a failure to weigh education, which was suspected to be a contributor to the polling inaccuracies from the 2016 election, among other elements.

The report did not, however, give firm reasoning to what caused such large, historic discrepancies in the 2020 presidential polls, concluding that it “appears to be impossible” to determine what went wrong based on available information.

ADVERTISEMENT

“Some explanations of polling error can be ruled out according to the patterns found in the polls, but identifying conclusively why polls overstated the Democratic-Republican margin relative to the certified vote appears to be impossible with the available data,” the report says.

“Reliable information is lacking on the demographics, opinions, and vote choice of those not included in polls (either because they were excluded from the sampling frame or else because they chose not to participate), making it impossible to compare voters who responded to polls with voters who did not. Some educated guesses are possible based on patterns emerging from available data but conclusive statements are impossible. It cannot be ruled out that there is a multitude of overlapping explanations for the pattern of polling error,” the report added.

The report did find, however, that the form of polling conducted did not influence sampling error, concluding that “No mode of interviewing was unambiguously more accurate.”

Each method overstated the Democratic-Republican margin relative to the final certified vote margin.

Additionally, it concluded that the error in the Democratic-Republican margin in polls was, on average, larger in senatorial and gubernatorial races compared to the presidential contest.

The report found that senatorial and gubernatorial races combined on average favored Democratic candidates by 6 percentage points.

Within the same state, the discrepancies were often greater in senate contests compared to the presidential race.

The group looked at 2,858 polls, including 529 that looked at the national popular vote, 1,572 that examined the presidential contest from the state-level, and 757 senatorial-gubernatorial surveys.