The polls are changing. As we enter the home stretch, public pollsters are transitioning their samples from registered voters to so-called “likely voters.” The general impression this conveys is that polls are suddenly becoming more accurate. Readers should be skeptical. The science behind the selection of “likely” voters is soft. And with dueling polls using different methodologies to identify the most likely voters, it’s becoming evident that caution is advised.
The top public pollsters employ different systems for gauging likelihood of voting. Some pollsters won’t even reveal their “secret sauce” recipe. But most all rely on batteries of questions asked of each voter. Only voters who give the right responses are deemed likely to vote.
The most common question simply asks: Are you almost certain to vote, will you probably vote, are the chances 50-50 or don’t you think you’ll vote? Seems straightforward. If you want to know whether someone will vote, just ask . But this doesn’t work very well. A recent Kennedy School of Government study, looking at more than 10,000 pre-election interviews and actual turnout, determined independently from election records, demonstrates that many who say they’ll vote don’t. And even more surprising is many who say they won’t vote eventually do. In this study, 13 percent of those “almost certain to vote” didn’t. But more disturbing is that of voters who self-reported only a 50-50 chance of voting, a category most pollsters dismiss, 67 percent voted. Even more disconcerting is that 55 percent of those who said they probably wouldn’t vote eventually did. Almost no pollsters using likely-voter methodology would have kept these respondents in their samples. But they voted.
The issue is whether we can trust voters to predict their own behavior. The answer seems unequivocally no, yet many pundits treat a self-professed likely-voter poll like an unimpeachable source. There are all sorts of reasons why voters cannot be relied upon. First, to be fair to voters, prediction is hard to do. Can you tell me what you’ll be doing two weeks from Tuesday? There are too many variables in play. I once took to adding a preface to my question: “The election will be held on a Tuesday, a workday on which you may also have family or child care responsibilities, and you have to vote between 7 a.m. and 7 p.m.; given that, are you certain to vote, will you probably vote,” etc.
Remind them that it’s sometimes a challenge. I have asked them whether they know where their polling place is. I have reminded them many people don’t vote. All this is done to get a realistic take on their probability of voting. I’m not sure it helped.
Another set of problems in assessing likelihood of voting is rooted in voter psychology. Some voters exaggerate their enthusiasm for voting because participation is perceived to be a socially desirable act. You don’t want the interviewer to think you are shirking civic duty. There is also the problem of response style. Persons with lower levels of education are highly likely to choose extreme response options. So someone with a grade-school education is more likely to say he’s “certain” to vote than someone with a master’s degree, whose educational attainment creates constraints, so that she merely claims she’ll “probably” vote. The same goes for reports of attentiveness to campaigns and intensity of support for a candidate that supposedly boosts turnout. This “extremity response set” tendency of less-educated persons is well-known in the academic literature, probably plumping Democrat numbers in these alleged likely-voter polls, but no one in public polling seems to take heed. They should.
David Hill is a pollster that has worked for Republican candidates and causes since 1984.